2024-04-02T04:27:07.566 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T04:27:07.566 DEBUG:teuthology.run:Teuthology command: teuthology --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --name teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848 --verbose --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848/orig.config.yaml 2024-04-02T04:27:07.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T04:27:07.714 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848 branch: squid description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7634848' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 6530 sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca sleep_before_teardown: 0 subset: 52/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTkL4ZL+yC5FNyyWTjrdfuZYVoGk7EaJZA39/WFvdE+eqGSYhicJI2UtPOePseT8KykcS58+rzjW2bN71aEYT0= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlWYit8l5EfwMwMS/3uqYosTV4uwy7+LKuVzpWajwMvjfgo8DnqOoHUNERPi0WhRTLuiQ+pBYy7eicOYYRc8NM= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-01_21:08:08 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-02T04:27:07.715 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa; will attempt to use it 2024-04-02T04:27:07.715 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks 2024-04-02T04:27:07.715 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T04:27:07.717 INFO:teuthology.task.internal:Checking packages... 2024-04-02T04:27:07.740 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6cec793d9073f0594e2364b9b687ecaf22424cca' 2024-04-02T04:27:07.740 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T04:27:07.740 INFO:teuthology.packaging:ref: None 2024-04-02T04:27:07.740 INFO:teuthology.packaging:tag: None 2024-04-02T04:27:07.740 INFO:teuthology.packaging:branch: squid 2024-04-02T04:27:07.740 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:27:07.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-02T04:27:07.918 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1979.g6cec793d 2024-04-02T04:27:07.919 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T04:27:07.928 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T04:27:07.928 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T04:27:07.954 INFO:teuthology.task.internal:Saving configuration 2024-04-02T04:27:07.974 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T04:27:07.979 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T04:27:08.004 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 04:21:44.465608', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTkL4ZL+yC5FNyyWTjrdfuZYVoGk7EaJZA39/WFvdE+eqGSYhicJI2UtPOePseT8KykcS58+rzjW2bN71aEYT0='} 2024-04-02T04:27:08.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 04:21:44.467300', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlWYit8l5EfwMwMS/3uqYosTV4uwy7+LKuVzpWajwMvjfgo8DnqOoHUNERPi0WhRTLuiQ+pBYy7eicOYYRc8NM='} 2024-04-02T04:27:08.029 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T04:27:08.035 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-02T04:27:08.035 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-02T04:27:08.035 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T04:27:08.135 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faef9c730d0>, signals=[15]) 2024-04-02T04:27:08.135 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T04:27:08.142 INFO:teuthology.task.internal:Opening connections... 2024-04-02T04:27:08.142 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:27:08.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:27:08.237 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-04-02T04:27:08.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:27:08.329 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T04:27:08.334 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-04-02T04:27:08.351 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-04-02T04:27:08.352 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T04:27:08.409 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-04-02T04:27:08.410 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T04:27:08.410 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T04:27:08.410 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T04:27:08.410 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-04-02T04:27:08.434 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-04-02T04:27:08.453 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-04-02T04:27:08.453 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-04-02T04:27:08.510 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T04:27:08.511 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T04:27:08.512 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T04:27:08.512 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-04-02T04:27:08.534 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T04:27:08.550 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T04:27:08.563 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T04:27:08.563 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-04-02T04:27:08.566 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-04-02T04:27:08.581 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T04:27:08.586 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T04:27:08.587 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-04-02T04:27:08.623 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-04-02T04:27:08.727 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T04:27:08.781 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T04:27:08.795 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-02T04:27:08.795 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-02T04:27:08.795 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T04:27:08.796 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T04:27:08.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:27:08.812 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-04-02T04:27:08.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:27:08.813 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-04-02T04:27:08.869 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-361.el9.x86_64 2024-04-02T04:27:08.870 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-361.el9.x86_64 2024-04-02T04:27:08.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-04-02T04:27:08.871 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-361.el9.x86_64 2024-04-02T04:27:08.871 INFO:teuthology.task.kernel:Running kernel on smithi179: 5.14.0-361.el9.x86_64 2024-04-02T04:27:08.871 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum install -y kernel 2024-04-02T04:27:10.949 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-02T04:27:11.018 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-02T04:27:12.668 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-04-02T04:27:12.699 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - BaseOS 3.1 MB/s | 2.0 MB 00:00 2024-04-02T04:27:13.409 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 22 MB/s | 8.0 MB 00:00 2024-04-02T04:27:15.289 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.0 MB 00:02 2024-04-02T04:27:17.910 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-04-02T04:27:18.323 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 19 MB 00:01 2024-04-02T04:27:23.015 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.1 MB 00:01 2024-04-02T04:27:24.483 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-04-02T04:27:25.903 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-02T04:27:30.983 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-02T04:27:31.626 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 616 kB/s | 6.1 MB 00:10 2024-04-02T04:27:33.240 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T04:27:33.240 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T04:27:33.327 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-04-02T04:27:33.329 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:27:33.334 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:27:33.334 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-02T04:27:33.334 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:27:33.334 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:27:33.334 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T04:27:33.334 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:27:33.335 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T04:27:33.335 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T04:27:33.335 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T04:27:33.335 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:27:33.335 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:27:33.335 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:27:33.335 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-04-02T04:27:33.335 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:27:33.336 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 76 M 2024-04-02T04:27:33.336 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 123 M 2024-04-02T04:27:33.336 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:27:33.484 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-04-02T04:27:34.093 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 20 MB/s | 15 MB 00:00 2024-04-02T04:27:34.735 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 24 MB/s | 34 MB 00:01 2024-04-02T04:27:34.974 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-02T04:27:35.135 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 17 MB/s | 28 MB 00:01 2024-04-02T04:27:35.138 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:27:35.138 INFO:teuthology.orchestra.run.smithi070.stdout:Total 42 MB/s | 76 MB 00:01 2024-04-02T04:27:35.138 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:27:35.507 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:27:35.507 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:27:36.331 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:27:36.331 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:27:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:27:39.476 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T04:27:39.577 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:27:40.021 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-02T04:27:42.287 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T04:27:42.287 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T04:27:42.343 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:27:42.389 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:27:42.394 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:27:42.394 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-02T04:27:42.394 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:27:42.394 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:27:42.394 INFO:teuthology.orchestra.run.smithi179.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T04:27:42.394 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:27:42.394 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T04:27:42.395 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T04:27:42.395 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T04:27:42.395 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:27:42.395 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:27:42.395 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:27:42.395 INFO:teuthology.orchestra.run.smithi179.stdout:Install 4 Packages 2024-04-02T04:27:42.395 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:27:42.395 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 76 M 2024-04-02T04:27:42.396 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 123 M 2024-04-02T04:27:42.396 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:27:42.478 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:27:42.548 INFO:teuthology.orchestra.run.smithi179.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T04:27:43.056 INFO:teuthology.orchestra.run.smithi179.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-04-02T04:27:43.632 INFO:teuthology.orchestra.run.smithi179.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 27 MB/s | 34 MB 00:01 2024-04-02T04:27:44.132 INFO:teuthology.orchestra.run.smithi179.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 17 MB/s | 28 MB 00:01 2024-04-02T04:27:44.135 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:27:44.135 INFO:teuthology.orchestra.run.smithi179.stdout:Total 44 MB/s | 76 MB 00:01 2024-04-02T04:27:44.136 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:27:44.510 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:27:44.511 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:27:45.347 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:27:45.347 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:27:47.188 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:27:47.312 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:27:47.654 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:27:48.250 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T04:27:48.336 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:27:51.056 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:27:51.211 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:27:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:27:55.903 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:27:55.944 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:00.686 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:21.323 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:21.324 INFO:teuthology.orchestra.run.smithi070.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T04:28:21.324 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:28:21.366 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:21.717 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:21.717 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T04:28:21.717 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:28:21.717 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:28:22.147 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:28:22.479 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T04:28:22.577 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T04:28:22.578 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T04:28:22.578 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T04:28:22.578 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T04:28:22.578 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T04:28:22.578 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-04-02T04:28:22.578 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T04:28:22.579 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T04:28:22.658 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T04:28:22.658 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T04:28:22.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-04-02T04:28:22.692 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 68.el9 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8264254 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T04:28:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:28:22.694 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T04:28:22.695 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T04:28:22.695 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 5.14.0-434.el9.x86_64 2024-04-02T04:28:22.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T04:28:23.305 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-04-02T04:28:24.737 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T04:28:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-04-02T04:28:24.761 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-04-02T04:28:24.785 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.LVwfQJtanm 2024-04-02T04:28:24.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LVwfQJtanm 2024-04-02T04:28:24.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.LVwfQJtanm 2024-04-02T04:28:24.995 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.LVwfQJtanm is 6KB 2024-04-02T04:28:25.045 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.LVwfQJtanm 2024-04-02T04:28:25.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T04:28:25.129 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T04:28:25.129 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T04:28:25.129 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T04:28:25.129 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T04:28:25.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T04:28:25.258 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-04-02T04:28:30.054 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:30.054 INFO:teuthology.orchestra.run.smithi179.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T04:28:30.054 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:28:30.098 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:30.551 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:30.712 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T04:28:30.712 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:28:30.712 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:28:30.994 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:28:30.994 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:28:30.995 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:28:30.995 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T04:28:30.995 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T04:28:30.995 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T04:28:30.995 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T04:28:30.995 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:28:30.995 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:28:31.325 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T04:28:31.421 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T04:28:31.421 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T04:28:31.421 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T04:28:31.421 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T04:28:31.421 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T04:28:31.421 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-04-02T04:28:31.421 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi179.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T04:28:31.422 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T04:28:31.502 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T04:28:31.503 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T04:28:31.503 DEBUG:teuthology.orchestra.run.smithi179:> sudo rpm -qi grub2-tools 2024-04-02T04:28:31.544 INFO:teuthology.orchestra.run.smithi179.stdout:Name : grub2-tools 2024-04-02T04:28:31.544 INFO:teuthology.orchestra.run.smithi179.stdout:Epoch : 1 2024-04-02T04:28:31.544 INFO:teuthology.orchestra.run.smithi179.stdout:Version : 2.06 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Release : 68.el9 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Architecture: x86_64 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Group : Unspecified 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Size : 8264254 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:License : GPLv3+ 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Packager : builder@centos.org 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Vendor : CentOS 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T04:28:31.545 INFO:teuthology.orchestra.run.smithi179.stdout:Summary : Support tools for GRUB. 2024-04-02T04:28:31.546 INFO:teuthology.orchestra.run.smithi179.stdout:Description : 2024-04-02T04:28:31.546 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:28:31.546 INFO:teuthology.orchestra.run.smithi179.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T04:28:31.546 INFO:teuthology.orchestra.run.smithi179.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T04:28:31.546 INFO:teuthology.orchestra.run.smithi179.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T04:28:31.546 INFO:teuthology.orchestra.run.smithi179.stdout:hardware devices. 2024-04-02T04:28:31.546 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:28:31.546 INFO:teuthology.orchestra.run.smithi179.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T04:28:31.547 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T04:28:31.547 INFO:teuthology.task.kernel:Updating grub on smithi179 to boot 5.14.0-434.el9.x86_64 2024-04-02T04:28:31.547 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T04:28:32.195 INFO:teuthology.orchestra.run.smithi179.stderr:Generating grub configuration file ... 2024-04-02T04:28:33.528 INFO:teuthology.orchestra.run.smithi179.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T04:28:33.570 INFO:teuthology.orchestra.run.smithi179.stderr:done 2024-04-02T04:28:33.572 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-04-02T04:28:33.587 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.zStJ17CCuG 2024-04-02T04:28:33.588 DEBUG:teuthology.orchestra.run.smithi179:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zStJ17CCuG 2024-04-02T04:28:33.656 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.zStJ17CCuG 2024-04-02T04:28:33.801 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.zStJ17CCuG is 6KB 2024-04-02T04:28:33.851 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.zStJ17CCuG 2024-04-02T04:28:33.867 DEBUG:teuthology.orchestra.run.smithi179:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T04:28:33.931 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T04:28:33.931 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T04:28:33.932 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T04:28:33.932 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T04:28:33.933 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T04:28:34.067 DEBUG:teuthology.orchestra.run.smithi179:> sudo shutdown -r now 2024-04-02T04:28:55.282 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-04-02T04:28:55.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-02T04:28:55.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:29:04.078 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-04-02T04:29:04.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-02T04:29:04.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:29:13.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-04-02T04:29:22.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-04-02T04:29:22.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-02T04:29:22.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:29:25.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-04-02T04:29:31.594 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-02T04:29:31.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:29:34.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-04-02T04:29:37.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-02T04:29:37.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:29:46.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-02T04:29:46.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:30:05.087 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-04-02T04:30:09.554 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-04-02T04:30:10.028 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-02T04:30:10.028 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-434.el9.x86_64"... 2024-04-02T04:30:10.028 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-04-02T04:30:10.042 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-434.el9.x86_64 2024-04-02T04:30:10.043 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T04:30:10.043 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T04:30:10.043 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T04:30:11.043 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-02T04:30:11.044 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T04:30:11.169 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-04-02T04:30:11.219 DEBUG:teuthology.parallel:result is None 2024-04-02T04:30:20.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-02T04:30:20.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:30:20.488 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-04-02T04:30:20.903 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-02T04:30:20.903 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-434.el9.x86_64"... 2024-04-02T04:30:20.904 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-04-02T04:30:20.917 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-434.el9.x86_64 2024-04-02T04:30:20.918 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T04:30:20.918 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T04:30:20.918 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T04:30:21.918 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-02T04:30:21.918 DEBUG:teuthology.orchestra.run.smithi179:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T04:30:22.008 INFO:teuthology.orchestra.run.smithi179.stdout:ttyS1 2024-04-02T04:30:22.056 DEBUG:teuthology.parallel:result is None 2024-04-02T04:30:22.057 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T04:30:22.065 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T04:30:22.065 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T04:30:22.068 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T04:30:22.083 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T04:30:22.113 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T04:30:22.137 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T04:30:22.137 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T04:30:22.140 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T04:30:22.209 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T04:30:22.217 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T04:30:22.218 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T04:30:22.220 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T04:30:22.247 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T04:30:22.254 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T04:30:22.260 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T04:30:22.268 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T04:30:22.270 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T04:30:22.280 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T04:30:22.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T04:30:22.313 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T04:30:22.341 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T04:30:22.352 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T04:30:22.352 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T04:30:22.384 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T04:30:22.399 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T04:30:22.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T04:30:22.531 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T04:30:22.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T04:30:22.657 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:30:22.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T04:30:22.725 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T04:30:22.767 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T04:30:22.832 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T04:30:22.916 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T04:30:22.962 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:30:22.962 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T04:30:23.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-04-02T04:30:23.035 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-04-02T04:30:23.090 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T04:30:23.237 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T04:30:23.699 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T04:30:24.205 INFO:teuthology.task.internal:Starting timer... 2024-04-02T04:30:24.205 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T04:30:24.289 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T04:30:24.304 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-02T04:30:24.305 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-04-02T04:30:24.376 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-04-02T04:30:24.378 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-04-02T04:30:24.429 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-04-02T04:30:24.431 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T04:30:24.431 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-04-02T04:30:24.457 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-04-02T04:30:24.457 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-04-02T04:30:24.474 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-04-02T04:30:24.475 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-04-02T04:30:24.475 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T04:30:24.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:30:24.526 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T04:30:24.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:30:24.555 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T04:30:24.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-04-02T04:30:24.600 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-04-02T04:30:24.644 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T04:30:24.654 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T04:30:24.671 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T04:30:24.673 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T04:34:39.340 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-04-02T04:34:39.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-02T04:34:39.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:34:39.430 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-04-02T04:34:39.512 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-02T04:34:39.513 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-02T04:34:39.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:34:39.602 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-04-02T04:34:39.676 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-02T04:34:39.677 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T04:34:39.688 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T04:34:39.688 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T04:34:39.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T04:34:39.691 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T04:34:39.691 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T04:34:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T04:34:39.736 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T04:34:39.748 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T04:34:39.765 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T04:34:39.765 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-04-02T04:34:39.779 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-04-02T04:34:39.794 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-04-02T04:34:39.795 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T04:34:39.807 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-04-02T04:34:39.811 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T04:34:39.824 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T04:34:39.840 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T04:34:39.865 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-04-02T04:34:39.870 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T04:34:39.870 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-04-02T04:34:39.870 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:34:39.870 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:34:39.870 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:34:39.870 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:34:39.901 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-04-02T04:34:39.905 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T04:34:39.905 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-04-02T04:34:39.905 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:34:39.905 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:34:39.905 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:34:39.905 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:34:39.906 INFO:teuthology.run_tasks:Running task install... 2024-04-02T04:34:39.913 DEBUG:teuthology.task.install:project ceph 2024-04-02T04:34:39.914 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca'}} 2024-04-02T04:34:39.914 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca'} 2024-04-02T04:34:39.914 INFO:teuthology.task.install:Using flavor: default 2024-04-02T04:34:39.919 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T04:34:39.919 INFO:teuthology.task.install:extra packages: [] 2024-04-02T04:34:39.920 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-04-02T04:34:39.920 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-02T04:34:39.920 INFO:teuthology.packaging:ref: None 2024-04-02T04:34:39.920 INFO:teuthology.packaging:tag: v18.2.1 2024-04-02T04:34:39.920 INFO:teuthology.packaging:branch: None 2024-04-02T04:34:39.920 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:34:42.478 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-02T04:34:42.478 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-02T04:34:42.481 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-04-02T04:34:42.482 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-02T04:34:42.482 INFO:teuthology.packaging:ref: None 2024-04-02T04:34:42.482 INFO:teuthology.packaging:tag: v18.2.1 2024-04-02T04:34:42.482 INFO:teuthology.packaging:branch: None 2024-04-02T04:34:42.482 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:34:42.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-02T04:34:42.673 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-04-02T04:34:42.673 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-04-02T04:34:42.679 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-04-02T04:34:42.679 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-04-02T04:34:42.721 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T04:34:42.722 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:34:42.722 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T04:34:42.728 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T04:34:42.729 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:34:42.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T04:34:42.755 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-02T04:34:42.755 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-02T04:34:42.755 INFO:teuthology.packaging:ref: None 2024-04-02T04:34:42.755 INFO:teuthology.packaging:tag: v18.2.1 2024-04-02T04:34:42.755 INFO:teuthology.packaging:branch: None 2024-04-02T04:34:42.755 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:34:42.755 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T04:34:42.761 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-02T04:34:42.761 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-02T04:34:42.761 INFO:teuthology.packaging:ref: None 2024-04-02T04:34:42.761 INFO:teuthology.packaging:tag: v18.2.1 2024-04-02T04:34:42.762 INFO:teuthology.packaging:branch: None 2024-04-02T04:34:42.762 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:34:42.762 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T04:34:42.827 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T04:34:42.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T04:34:42.911 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T04:34:42.915 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T04:34:42.979 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-04-02T04:34:42.981 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-04-02T04:34:42.983 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-04-02T04:34:42.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-04-02T04:34:43.226 INFO:teuthology.orchestra.run.smithi179.stdout:74 files removed 2024-04-02T04:34:43.232 INFO:teuthology.orchestra.run.smithi070.stdout:74 files removed 2024-04-02T04:34:43.252 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw 2024-04-02T04:34:43.258 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw 2024-04-02T04:34:43.760 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 369 kB/s | 76 kB 00:00 2024-04-02T04:34:43.774 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 349 kB/s | 76 kB 00:00 2024-04-02T04:34:43.960 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 63 kB/s | 9.4 kB 00:00 2024-04-02T04:34:43.983 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 59 kB/s | 9.4 kB 00:00 2024-04-02T04:34:44.152 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 14 kB/s | 2.2 kB 00:00 2024-04-02T04:34:44.175 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 14 kB/s | 2.2 kB 00:00 2024-04-02T04:34:44.579 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-04-02T04:34:44.911 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - AppStream 9.6 MB/s | 6.7 MB 00:00 2024-04-02T04:34:45.980 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-04-02T04:34:46.344 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-04-02T04:34:46.712 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-02T04:34:48.545 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-02T04:34:48.930 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-02T04:35:02.297 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 637 kB/s | 6.1 MB 00:09 2024-04-02T04:35:03.772 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-02T04:35:05.260 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-02T04:35:10.199 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-02T04:35:12.827 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:35:12.837 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T04:35:12.838 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T04:35:12.839 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:12.840 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:35:12.840 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:12.840 INFO:teuthology.orchestra.run.smithi179.stdout:Install 28 Packages 2024-04-02T04:35:12.840 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 2 Packages 2024-04-02T04:35:12.840 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:12.840 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 53 M 2024-04-02T04:35:12.840 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:35:13.208 INFO:teuthology.orchestra.run.smithi179.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 31 MB/s | 5.2 MB 00:00 2024-04-02T04:35:13.233 INFO:teuthology.orchestra.run.smithi179.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 963 kB/s | 24 kB 00:00 2024-04-02T04:35:13.275 INFO:teuthology.orchestra.run.smithi179.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 16 MB/s | 658 kB 00:00 2024-04-02T04:35:13.309 INFO:teuthology.orchestra.run.smithi179.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-04-02T04:35:13.409 INFO:teuthology.orchestra.run.smithi179.stdout:(5/30): librgw2-18.2.1-0.el9.x86_64.rpm 45 MB/s | 4.5 MB 00:00 2024-04-02T04:35:13.434 INFO:teuthology.orchestra.run.smithi179.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-04-02T04:35:13.527 INFO:teuthology.orchestra.run.smithi179.stdout:(7/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 16 MB/s | 7.7 MB 00:00 2024-04-02T04:35:13.552 INFO:teuthology.orchestra.run.smithi179.stdout:(8/30): python3-ceph-common-18.2.1-0.el9.x86_64 1.0 MB/s | 124 kB 00:00 2024-04-02T04:35:13.577 INFO:teuthology.orchestra.run.smithi179.stdout:(9/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 3.1 MB/s | 161 kB 00:00 2024-04-02T04:35:13.602 INFO:teuthology.orchestra.run.smithi179.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 6.2 MB/s | 321 kB 00:00 2024-04-02T04:35:13.628 INFO:teuthology.orchestra.run.smithi179.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 5.8 MB/s | 297 kB 00:00 2024-04-02T04:35:13.653 INFO:teuthology.orchestra.run.smithi179.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-04-02T04:35:13.904 INFO:teuthology.orchestra.run.smithi179.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-04-02T04:35:13.929 INFO:teuthology.orchestra.run.smithi179.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 352 kB/s | 106 kB 00:00 2024-04-02T04:35:13.954 INFO:teuthology.orchestra.run.smithi179.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 576 kB/s | 159 kB 00:00 2024-04-02T04:35:13.971 INFO:teuthology.orchestra.run.smithi179.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 664 kB/s | 44 kB 00:00 2024-04-02T04:35:14.054 INFO:teuthology.orchestra.run.smithi179.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 282 kB 00:00 2024-04-02T04:35:14.179 INFO:teuthology.orchestra.run.smithi179.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 661 kB 00:00 2024-04-02T04:35:14.205 INFO:teuthology.orchestra.run.smithi179.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 176 kB/s | 41 kB 00:00 2024-04-02T04:35:14.555 INFO:teuthology.orchestra.run.smithi179.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 221 kB/s | 111 kB 00:00 2024-04-02T04:35:14.656 INFO:teuthology.orchestra.run.smithi179.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-02T04:35:14.756 INFO:teuthology.orchestra.run.smithi179.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-02T04:35:14.789 INFO:teuthology.orchestra.run.smithi179.stdout:(23/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 507 kB/s | 309 kB 00:00 2024-04-02T04:35:14.856 INFO:teuthology.orchestra.run.smithi179.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-04-02T04:35:15.040 INFO:teuthology.orchestra.run.smithi179.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-04-02T04:35:15.073 INFO:teuthology.orchestra.run.smithi179.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-04-02T04:35:15.149 INFO:teuthology.orchestra.run.smithi179.stdout:(27/30): libarrow-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-04-02T04:35:15.224 INFO:teuthology.orchestra.run.smithi179.stdout:(28/30): librados2-18.2.1-0.el9.x86_64.rpm 22 MB/s | 3.3 MB 00:00 2024-04-02T04:35:15.274 INFO:teuthology.orchestra.run.smithi179.stdout:(29/30): librbd1-18.2.1-0.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-04-02T04:35:15.416 INFO:teuthology.orchestra.run.smithi179.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-04-02T04:35:15.421 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:35:15.421 INFO:teuthology.orchestra.run.smithi179.stdout:Total 21 MB/s | 53 MB 00:02 2024-04-02T04:35:15.421 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:35:15.457 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:35:15.457 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:35:15.894 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:35:15.894 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:35:16.049 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 732 kB/s | 19 MB 00:26 2024-04-02T04:35:17.282 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:35:17.364 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-02T04:35:17.404 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-02T04:35:17.454 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-02T04:35:17.838 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-02T04:35:17.870 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-04-02T04:35:17.971 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-04-02T04:35:18.050 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-04-02T04:35:18.096 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-04-02T04:35:18.142 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-04-02T04:35:18.171 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-02T04:35:18.215 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-04-02T04:35:18.280 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-04-02T04:35:18.307 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-04-02T04:35:18.356 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-04-02T04:35:18.463 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-02T04:35:18.498 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-02T04:35:18.538 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-02T04:35:18.579 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-02T04:35:18.606 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-02T04:35:18.642 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-02T04:35:18.674 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-02T04:35:18.982 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-02T04:35:19.013 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-04-02T04:35:19.103 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-04-02T04:35:19.134 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-04-02T04:35:19.170 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-02T04:35:19.576 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-02T04:35:19.598 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-04-02T04:35:19.670 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-04-02T04:35:19.745 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-04-02T04:35:19.775 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-04-02T04:35:20.125 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-04-02T04:35:21.743 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-02T04:35:21.872 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-02T04:35:21.906 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-02T04:35:22.450 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-02T04:35:22.542 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-04-02T04:35:22.671 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-04-02T04:35:22.854 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-04-02T04:35:22.854 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T04:35:22.854 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T04:35:22.854 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:22.894 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-04-02T04:35:27.577 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-02T04:35:30.206 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:35:30.217 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-04-02T04:35:30.218 INFO:teuthology.orchestra.run.smithi070.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T04:35:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout:Install 28 Packages 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-04-02T04:35:30.220 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:30.221 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 53 M 2024-04-02T04:35:30.221 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:35:30.453 INFO:teuthology.orchestra.run.smithi070.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 29 MB/s | 5.2 MB 00:00 2024-04-02T04:35:30.478 INFO:teuthology.orchestra.run.smithi070.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 977 kB/s | 24 kB 00:00 2024-04-02T04:35:30.511 INFO:teuthology.orchestra.run.smithi070.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-04-02T04:35:30.545 INFO:teuthology.orchestra.run.smithi070.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-04-02T04:35:30.654 INFO:teuthology.orchestra.run.smithi070.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 20 MB/s | 7.7 MB 00:00 2024-04-02T04:35:30.671 INFO:teuthology.orchestra.run.smithi070.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 2.7 MB/s | 45 kB 00:00 2024-04-02T04:35:30.696 INFO:teuthology.orchestra.run.smithi070.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-04-02T04:35:30.721 INFO:teuthology.orchestra.run.smithi070.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-04-02T04:35:30.755 INFO:teuthology.orchestra.run.smithi070.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 9.4 MB/s | 321 kB 00:00 2024-04-02T04:35:30.789 INFO:teuthology.orchestra.run.smithi070.stdout:(10/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 8.7 MB/s | 297 kB 00:00 2024-04-02T04:35:30.814 INFO:teuthology.orchestra.run.smithi070.stdout:(11/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-04-02T04:35:30.889 INFO:teuthology.orchestra.run.smithi070.stdout:(12/30): librgw2-18.2.1-0.el9.x86_64.rpm 13 MB/s | 4.5 MB 00:00 2024-04-02T04:35:31.148 INFO:teuthology.orchestra.run.smithi070.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-04-02T04:35:31.173 INFO:teuthology.orchestra.run.smithi070.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 561 kB/s | 159 kB 00:00 2024-04-02T04:35:31.198 INFO:teuthology.orchestra.run.smithi070.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 275 kB/s | 106 kB 00:00 2024-04-02T04:35:31.215 INFO:teuthology.orchestra.run.smithi070.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 664 kB/s | 44 kB 00:00 2024-04-02T04:35:31.374 INFO:teuthology.orchestra.run.smithi070.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 661 kB 00:00 2024-04-02T04:35:31.424 INFO:teuthology.orchestra.run.smithi070.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 282 kB 00:00 2024-04-02T04:35:31.441 INFO:teuthology.orchestra.run.smithi070.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 182 kB/s | 41 kB 00:00 2024-04-02T04:35:31.550 INFO:teuthology.orchestra.run.smithi070.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 632 kB/s | 111 kB 00:00 2024-04-02T04:35:31.600 INFO:teuthology.orchestra.run.smithi070.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.7 MB/s | 309 kB 00:00 2024-04-02T04:35:31.642 INFO:teuthology.orchestra.run.smithi070.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-02T04:35:31.675 INFO:teuthology.orchestra.run.smithi070.stdout:(23/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-04-02T04:35:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-02T04:35:31.776 INFO:teuthology.orchestra.run.smithi070.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-02T04:35:32.051 INFO:teuthology.orchestra.run.smithi070.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-04-02T04:35:32.118 INFO:teuthology.orchestra.run.smithi070.stdout:(27/30): librados2-18.2.1-0.el9.x86_64.rpm 49 MB/s | 3.3 MB 00:00 2024-04-02T04:35:32.168 INFO:teuthology.orchestra.run.smithi070.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-04-02T04:35:32.219 INFO:teuthology.orchestra.run.smithi070.stdout:(29/30): librbd1-18.2.1-0.el9.x86_64.rpm 30 MB/s | 3.0 MB 00:00 2024-04-02T04:35:32.268 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-04-02T04:35:32.269 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-04-02T04:35:32.269 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-04-02T04:35:32.269 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-04-02T04:35:32.269 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-04-02T04:35:32.269 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-04-02T04:35:32.269 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-04-02T04:35:32.269 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-04-02T04:35:32.269 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:32.669 INFO:teuthology.orchestra.run.smithi070.stdout:(30/30): libarrow-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-04-02T04:35:32.674 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:35:32.675 INFO:teuthology.orchestra.run.smithi070.stdout:Total 22 MB/s | 53 MB 00:02 2024-04-02T04:35:32.675 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:35:32.712 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:35:32.712 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:35:33.038 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-04-02T04:35:33.064 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-04-02T04:35:33.064 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:35:33.064 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T04:35:33.064 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T04:35:33.064 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T04:35:33.064 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:33.066 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-02T04:35:33.150 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-02T04:35:33.150 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:35:33.157 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:35:33.158 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-04-02T04:35:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-02T04:35:34.299 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-02T04:35:34.300 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-02T04:35:34.300 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-02T04:35:34.300 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-02T04:35:34.300 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-02T04:35:34.300 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-02T04:35:34.300 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-02T04:35:34.300 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-02T04:35:34.301 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-04-02T04:35:34.301 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-02T04:35:34.301 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-04-02T04:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:35:34.637 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-02T04:35:34.665 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-02T04:35:34.724 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-02T04:35:35.051 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-02T04:35:35.122 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-04-02T04:35:35.197 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-04-02T04:35:35.279 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-04-02T04:35:35.315 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-04-02T04:35:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-04-02T04:35:35.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:35:35.386 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:35.386 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T04:35:35.387 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.388 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:35.389 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T04:35:35.389 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T04:35:35.389 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:35.389 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:35:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-02T04:35:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-04-02T04:35:35.525 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-04-02T04:35:35.552 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-04-02T04:35:35.621 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-04-02T04:35:35.731 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-test 2024-04-02T04:35:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-02T04:35:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-02T04:35:35.826 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-02T04:35:35.865 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-02T04:35:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-02T04:35:35.927 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-02T04:35:35.960 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-02T04:35:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-02T04:35:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-04-02T04:35:36.303 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:26 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:35:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-04-02T04:35:36.420 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-04-02T04:35:36.453 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:35:36.454 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:36.454 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-02T04:35:36.454 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:36.454 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:35:36.455 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:36.456 INFO:teuthology.orchestra.run.smithi179.stdout:Install 6 Packages 2024-04-02T04:35:36.456 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:36.456 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 41 M 2024-04-02T04:35:36.456 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 167 M 2024-04-02T04:35:36.456 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:35:36.465 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-02T04:35:36.684 INFO:teuthology.orchestra.run.smithi179.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 165 kB 00:00 2024-04-02T04:35:36.808 INFO:teuthology.orchestra.run.smithi179.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 917 kB/s | 217 kB 00:00 2024-04-02T04:35:36.842 INFO:teuthology.orchestra.run.smithi179.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 300 kB 00:00 2024-04-02T04:35:36.868 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-02T04:35:36.875 INFO:teuthology.orchestra.run.smithi179.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 945 kB/s | 63 kB 00:00 2024-04-02T04:35:36.893 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-04-02T04:35:36.989 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-04-02T04:35:37.056 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-04-02T04:35:37.103 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-04-02T04:35:37.259 INFO:teuthology.orchestra.run.smithi179.stdout:(5/6): ceph-test-18.2.1-0.el9.x86_64.rpm 58 MB/s | 40 MB 00:00 2024-04-02T04:35:37.418 INFO:teuthology.orchestra.run.smithi179.stdout:(6/6): jq-1.6-16.el9.x86_64.rpm 325 kB/s | 187 kB 00:00 2024-04-02T04:35:37.421 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:35:37.421 INFO:teuthology.orchestra.run.smithi179.stdout:Total 42 MB/s | 41 MB 00:00 2024-04-02T04:35:37.436 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:35:37.457 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:35:37.457 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:35:37.820 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:35:37.820 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:35:38.338 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:35:38.419 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-02T04:35:38.462 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-02T04:35:38.492 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-02T04:35:38.537 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-02T04:35:38.562 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-04-02T04:35:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-02T04:35:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-02T04:35:39.784 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-02T04:35:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-04-02T04:35:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-04-02T04:35:39.911 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T04:35:39.912 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T04:35:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:39.953 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-04-02T04:35:42.297 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-04-02T04:35:42.379 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-04-02T04:35:44.455 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-04-02T04:35:44.455 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-04-02T04:35:44.456 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-04-02T04:35:44.456 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-02T04:35:44.456 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-04-02T04:35:44.456 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-04-02T04:35:44.915 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-04-02T04:35:44.915 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:44.915 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:35:44.915 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-04-02T04:35:44.915 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T04:35:44.915 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T04:35:44.915 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:44.915 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:35:45.190 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph 2024-04-02T04:35:45.757 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:35 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:35:45.914 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-04-02T04:35:45.917 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T04:35:45.918 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T04:35:45.919 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout:Install 33 Packages 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 30 M 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 110 M 2024-04-02T04:35:45.920 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:35:46.487 INFO:teuthology.orchestra.run.smithi179.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 64 kB/s | 6.4 kB 00:00 2024-04-02T04:35:46.536 INFO:teuthology.orchestra.run.smithi179.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-04-02T04:35:46.587 INFO:teuthology.orchestra.run.smithi179.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-04-02T04:35:46.621 INFO:teuthology.orchestra.run.smithi179.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 7.1 MB/s | 242 kB 00:00 2024-04-02T04:35:46.696 INFO:teuthology.orchestra.run.smithi179.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-02T04:35:46.813 INFO:teuthology.orchestra.run.smithi179.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 339 kB/s | 32 kB 00:00 2024-04-02T04:35:46.838 INFO:teuthology.orchestra.run.smithi179.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 664 kB/s | 144 kB 00:00 2024-04-02T04:35:46.863 INFO:teuthology.orchestra.run.smithi179.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 947 kB/s | 47 kB 00:00 2024-04-02T04:35:47.056 INFO:teuthology.orchestra.run.smithi179.stdout:(9/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 34 MB/s | 18 MB 00:00 2024-04-02T04:35:47.080 INFO:teuthology.orchestra.run.smithi179.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 241 kB 00:00 2024-04-02T04:35:47.106 INFO:teuthology.orchestra.run.smithi179.stdout:(11/33): python3-toml-0.10.2-6.el9.noarch.rpm 144 kB/s | 39 kB 00:00 2024-04-02T04:35:47.147 INFO:teuthology.orchestra.run.smithi179.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 13 MB/s | 1.1 MB 00:00 2024-04-02T04:35:47.173 INFO:teuthology.orchestra.run.smithi179.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-04-02T04:35:47.198 INFO:teuthology.orchestra.run.smithi179.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 1.3 MB/s | 124 kB 00:00 2024-04-02T04:35:47.224 INFO:teuthology.orchestra.run.smithi179.stdout:(15/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.6 MB/s | 187 kB 00:00 2024-04-02T04:35:47.340 INFO:teuthology.orchestra.run.smithi179.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 747 kB/s | 125 kB 00:00 2024-04-02T04:35:47.566 INFO:teuthology.orchestra.run.smithi179.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 118 kB/s | 43 kB 00:00 2024-04-02T04:35:47.666 INFO:teuthology.orchestra.run.smithi179.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-04-02T04:35:47.716 INFO:teuthology.orchestra.run.smithi179.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 349 kB/s | 172 kB 00:00 2024-04-02T04:35:47.775 INFO:teuthology.orchestra.run.smithi179.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-04-02T04:35:47.817 INFO:teuthology.orchestra.run.smithi179.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-04-02T04:35:47.892 INFO:teuthology.orchestra.run.smithi179.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 166 kB/s | 19 kB 00:00 2024-04-02T04:35:47.940 INFO:teuthology.orchestra.run.smithi179.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2024-04-02T04:35:47.951 INFO:teuthology.orchestra.run.smithi179.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 587 kB/s | 358 kB 00:00 2024-04-02T04:35:48.035 INFO:teuthology.orchestra.run.smithi179.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 326 kB/s | 46 kB 00:00 2024-04-02T04:35:48.060 INFO:teuthology.orchestra.run.smithi179.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 589 kB/s | 79 kB 00:00 2024-04-02T04:35:48.102 INFO:teuthology.orchestra.run.smithi179.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-02T04:35:48.152 INFO:teuthology.orchestra.run.smithi179.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 140 kB/s | 16 kB 00:00 2024-04-02T04:35:48.211 INFO:teuthology.orchestra.run.smithi179.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 598 kB/s | 90 kB 00:00 2024-04-02T04:35:48.236 INFO:teuthology.orchestra.run.smithi179.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 267 kB/s | 36 kB 00:00 2024-04-02T04:35:48.370 INFO:teuthology.orchestra.run.smithi179.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 150 kB/s | 20 kB 00:00 2024-04-02T04:35:48.478 INFO:teuthology.orchestra.run.smithi179.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 705 kB/s | 230 kB 00:00 2024-04-02T04:35:48.503 INFO:teuthology.orchestra.run.smithi179.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.4 MB/s | 427 kB 00:00 2024-04-02T04:35:48.508 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:35:48.508 INFO:teuthology.orchestra.run.smithi179.stdout:Total 12 MB/s | 30 MB 00:02 2024-04-02T04:35:48.522 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:35:48.562 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:35:48.562 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:35:48.921 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:35:48.921 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:35:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-04-02T04:35:49.652 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-04-02T04:35:49.652 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-04-02T04:35:49.652 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-04-02T04:35:49.652 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-04-02T04:35:49.652 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-04-02T04:35:49.652 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-04-02T04:35:49.653 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-04-02T04:35:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:50.133 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:35:50.189 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-02T04:35:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-02T04:35:50.270 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-02T04:35:50.362 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-02T04:35:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-04-02T04:35:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-04-02T04:35:50.434 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:35:50.435 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T04:35:50.435 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T04:35:50.435 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T04:35:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-02T04:35:50.448 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-02T04:35:50.490 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-02T04:35:50.513 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-02T04:35:50.513 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:35:50.561 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-02T04:35:50.615 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-04-02T04:35:50.658 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-04-02T04:35:50.711 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-04-02T04:35:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-04-02T04:35:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-04-02T04:35:51.033 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-04-02T04:35:51.070 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-04-02T04:35:51.107 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-04-02T04:35:51.137 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-02T04:35:51.212 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-04-02T04:35:51.247 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-02T04:35:51.281 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-04-02T04:35:51.321 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-04-02T04:35:51.356 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-04-02T04:35:51.390 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-04-02T04:35:51.479 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-04-02T04:35:51.521 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-04-02T04:35:51.596 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:35:51.596 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-04-02T04:35:51.596 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-02T04:35:51.597 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-04-02T04:35:51.598 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-02T04:35:51.599 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-04-02T04:35:51.601 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-04-02T04:35:51.736 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-04-02T04:35:51.853 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-02T04:35:52.009 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-04-02T04:35:52.035 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-04-02T04:35:52.059 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-04-02T04:35:52.059 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:35:52.059 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T04:35:52.059 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T04:35:52.059 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T04:35:52.059 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:52.785 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:35:52.785 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:52.785 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-04-02T04:35:52.785 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.785 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T04:35:52.786 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.787 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.788 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:52.788 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T04:35:52.788 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T04:35:52.788 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:52.788 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:35:53.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-test 2024-04-02T04:35:53.628 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-04-02T04:35:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-04-02T04:35:53.657 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:35:53.657 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T04:35:53.657 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T04:35:53.657 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T04:35:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:53.722 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:26 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:35:53.873 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:35:53.874 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:35:53.874 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-02T04:35:53.874 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-04-02T04:35:53.875 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:35:53.876 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 41 M 2024-04-02T04:35:53.876 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 167 M 2024-04-02T04:35:53.876 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:35:54.112 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-04-02T04:35:54.114 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 165 kB 00:00 2024-04-02T04:35:54.136 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-04-02T04:35:54.136 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:35:54.136 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T04:35:54.136 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T04:35:54.136 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T04:35:54.136 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:54.173 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-04-02T04:35:54.215 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.9 MB/s | 300 kB 00:00 2024-04-02T04:35:54.240 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 940 kB/s | 63 kB 00:00 2024-04-02T04:35:54.337 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-04-02T04:35:54.361 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-04-02T04:35:54.361 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:35:54.361 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T04:35:54.362 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T04:35:54.362 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T04:35:54.362 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:54.426 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-04-02T04:35:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): ceph-test-18.2.1-0.el9.x86_64.rpm 60 MB/s | 40 MB 00:00 2024-04-02T04:35:54.807 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): jq-1.6-16.el9.x86_64.rpm 316 kB/s | 187 kB 00:00 2024-04-02T04:35:54.810 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:35:54.811 INFO:teuthology.orchestra.run.smithi070.stdout:Total 43 MB/s | 41 MB 00:00 2024-04-02T04:35:54.826 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:35:54.847 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:35:54.847 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:35:55.204 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:35:55.204 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:35:55.728 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:35:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-02T04:35:55.866 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-02T04:35:55.898 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-02T04:35:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-02T04:35:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-04-02T04:35:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-04-02T04:35:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-04-02T04:35:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-04-02T04:35:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-02T04:35:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-02T04:35:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-02T04:35:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-02T04:35:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-02T04:35:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-02T04:35:57.291 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-02T04:35:57.291 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:57.291 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:35:57.291 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T04:35:57.292 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T04:35:57.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T04:35:57.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T04:35:57.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T04:35:57.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T04:35:57.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T04:35:57.294 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:57.294 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:35:57.660 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-base 2024-04-02T04:35:58.235 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:48 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:35:58.335 INFO:teuthology.orchestra.run.smithi179.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:35:58.388 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:35:58.388 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:35:58.389 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:35:58.462 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install cephadm 2024-04-02T04:35:59.028 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:49 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:35:59.178 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:35:59.178 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 221 k 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 213 k 2024-04-02T04:35:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:35:59.252 INFO:teuthology.orchestra.run.smithi179.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.0 MB/s | 221 kB 00:00 2024-04-02T04:35:59.253 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:35:59.253 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.9 MB/s | 221 kB 00:00 2024-04-02T04:35:59.254 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:35:59.258 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:35:59.258 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:35:59.261 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:35:59.261 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:35:59.332 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:35:59.694 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-04-02T04:35:59.772 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-02T04:35:59.776 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-04-02T04:35:59.857 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-02T04:36:00.233 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-02T04:36:00.519 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-02T04:36:00.519 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:00.520 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:00.520 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-04-02T04:36:00.520 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:00.520 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:00.757 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-immutable-object-cache 2024-04-02T04:36:01.326 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:51 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:01.482 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:01.483 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:01.483 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-04-02T04:36:01.483 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:01.483 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:01.483 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-04-02T04:36:01.483 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:01.484 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:01.484 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:01.484 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-02T04:36:01.484 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:01.484 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 142 k 2024-04-02T04:36:01.484 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 395 k 2024-04-02T04:36:01.484 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:01.543 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 2.4 MB/s | 142 kB 00:00 2024-04-02T04:36:01.544 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:01.544 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.3 MB/s | 142 kB 00:00 2024-04-02T04:36:01.544 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:01.551 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:01.552 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:01.603 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:01.603 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:01.741 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:01.794 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:01.819 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:01.819 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:36:01.819 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T04:36:01.819 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:02.012 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-04-02T04:36:02.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-04-02T04:36:02.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-04-02T04:36:02.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-02T04:36:02.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-04-02T04:36:02.012 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-04-02T04:36:02.361 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-04-02T04:36:02.361 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:02.361 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:02.361 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-04-02T04:36:02.361 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T04:36:02.362 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T04:36:02.362 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:02.362 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:02.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph 2024-04-02T04:36:02.794 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:02.794 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:02.794 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:02.794 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:02.794 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:02.794 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:03.071 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr 2024-04-02T04:36:03.239 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:36 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:03.390 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:03.392 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:03.392 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-02T04:36:03.392 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:03.392 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T04:36:03.393 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T04:36:03.394 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout:Install 33 Packages 2024-04-02T04:36:03.395 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:03.396 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 30 M 2024-04-02T04:36:03.396 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 110 M 2024-04-02T04:36:03.396 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:03.653 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:53 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:03.752 INFO:teuthology.orchestra.run.smithi179.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:03.804 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:03.805 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:36:03.805 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:03.873 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-dashboard 2024-04-02T04:36:03.926 INFO:teuthology.orchestra.run.smithi070.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 62 kB/s | 6.4 kB 00:00 2024-04-02T04:36:03.976 INFO:teuthology.orchestra.run.smithi070.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-04-02T04:36:04.034 INFO:teuthology.orchestra.run.smithi070.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 9.9 MB/s | 2.1 MB 00:00 2024-04-02T04:36:04.059 INFO:teuthology.orchestra.run.smithi070.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.6 MB/s | 242 kB 00:00 2024-04-02T04:36:04.144 INFO:teuthology.orchestra.run.smithi070.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-02T04:36:04.410 INFO:teuthology.orchestra.run.smithi070.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 41 MB/s | 18 MB 00:00 2024-04-02T04:36:04.438 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:54 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:04.452 INFO:teuthology.orchestra.run.smithi070.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 111 kB/s | 32 kB 00:00 2024-04-02T04:36:04.477 INFO:teuthology.orchestra.run.smithi070.stdout:(8/33): python3-mako-1.1.4-6.el9.noarch.rpm 345 kB/s | 144 kB 00:00 2024-04-02T04:36:04.494 INFO:teuthology.orchestra.run.smithi070.stdout:(9/33): python3-pytz-2021.1-5.el9.noarch.rpm 577 kB/s | 47 kB 00:00 2024-04-02T04:36:04.520 INFO:teuthology.orchestra.run.smithi070.stdout:(10/33): python3-toml-0.10.2-6.el9.noarch.rpm 578 kB/s | 39 kB 00:00 2024-04-02T04:36:04.589 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T04:36:04.590 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-04-02T04:36:04.591 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-02T04:36:04.591 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:04.591 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:04.591 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:04.591 INFO:teuthology.orchestra.run.smithi179.stdout:Install 7 Packages 2024-04-02T04:36:04.591 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:04.591 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 2.1 M 2024-04-02T04:36:04.592 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 8.5 M 2024-04-02T04:36:04.592 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:04.611 INFO:teuthology.orchestra.run.smithi070.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 241 kB 00:00 2024-04-02T04:36:04.645 INFO:teuthology.orchestra.run.smithi070.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 823 kB/s | 103 kB 00:00 2024-04-02T04:36:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 372 kB/s | 124 kB 00:00 2024-04-02T04:36:04.987 INFO:teuthology.orchestra.run.smithi070.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 547 kB/s | 187 kB 00:00 2024-04-02T04:36:05.021 INFO:teuthology.orchestra.run.smithi070.stdout:(15/33): python3-cryptography-36.0.1-4.el9.x86_ 2.3 MB/s | 1.1 MB 00:00 2024-04-02T04:36:05.055 INFO:teuthology.orchestra.run.smithi179.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 216 kB/s | 23 kB 00:00 2024-04-02T04:36:05.080 INFO:teuthology.orchestra.run.smithi179.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 111 kB/s | 15 kB 00:00 2024-04-02T04:36:05.139 INFO:teuthology.orchestra.run.smithi179.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 9.2 MB/s | 1.8 MB 00:00 2024-04-02T04:36:05.171 INFO:teuthology.orchestra.run.smithi070.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 554 kB/s | 125 kB 00:00 2024-04-02T04:36:05.197 INFO:teuthology.orchestra.run.smithi179.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 64 kB/s | 9.0 kB 00:00 2024-04-02T04:36:05.223 INFO:teuthology.orchestra.run.smithi179.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 368 kB/s | 31 kB 00:00 2024-04-02T04:36:05.248 INFO:teuthology.orchestra.run.smithi179.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 244 kB/s | 41 kB 00:00 2024-04-02T04:36:05.304 INFO:teuthology.orchestra.run.smithi070.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-04-02T04:36:05.307 INFO:teuthology.orchestra.run.smithi179.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.7 MB/s | 188 kB 00:00 2024-04-02T04:36:05.309 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:05.309 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.9 MB/s | 2.1 MB 00:00 2024-04-02T04:36:05.309 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:05.317 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:05.317 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:05.330 INFO:teuthology.orchestra.run.smithi070.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 557 kB/s | 172 kB 00:00 2024-04-02T04:36:05.381 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:05.381 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:05.464 INFO:teuthology.orchestra.run.smithi070.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 67 kB/s | 11 kB 00:00 2024-04-02T04:36:05.489 INFO:teuthology.orchestra.run.smithi070.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 111 kB/s | 18 kB 00:00 2024-04-02T04:36:05.556 INFO:teuthology.orchestra.run.smithi070.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 933 kB/s | 358 kB 00:00 2024-04-02T04:36:05.667 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:05.714 INFO:teuthology.orchestra.run.smithi070.stdout:(22/33): python3-jaraco-text-3.2.0-6.el9.noarch 124 kB/s | 20 kB 00:00 2024-04-02T04:36:05.733 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-02T04:36:05.739 INFO:teuthology.orchestra.run.smithi070.stdout:(23/33): python3-jaraco-collections-3.0.0-8.el9 84 kB/s | 23 kB 00:00 2024-04-02T04:36:05.767 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-02T04:36:05.790 INFO:teuthology.orchestra.run.smithi070.stdout:(24/33): python3-jaraco-functools-3.5.0-2.el9.n 65 kB/s | 19 kB 00:00 2024-04-02T04:36:05.822 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-02T04:36:05.862 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-02T04:36:05.891 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-04-02T04:36:05.982 INFO:teuthology.orchestra.run.smithi070.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 173 kB/s | 46 kB 00:00 2024-04-02T04:36:06.007 INFO:teuthology.orchestra.run.smithi070.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 364 kB/s | 79 kB 00:00 2024-04-02T04:36:06.131 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-04-02T04:36:06.161 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-04-02T04:36:06.182 INFO:teuthology.orchestra.run.smithi070.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 82 kB/s | 16 kB 00:00 2024-04-02T04:36:06.208 INFO:teuthology.orchestra.run.smithi070.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 653 kB/s | 272 kB 00:00 2024-04-02T04:36:06.299 INFO:teuthology.orchestra.run.smithi070.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 308 kB/s | 90 kB 00:00 2024-04-02T04:36:06.366 INFO:teuthology.orchestra.run.smithi070.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 194 kB/s | 36 kB 00:00 2024-04-02T04:36:06.542 INFO:teuthology.orchestra.run.smithi070.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 114 kB/s | 20 kB 00:00 2024-04-02T04:36:06.566 INFO:teuthology.orchestra.run.smithi070.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 640 kB/s | 230 kB 00:00 2024-04-02T04:36:06.675 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-04-02T04:36:06.675 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-04-02T04:36:06.675 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-04-02T04:36:06.675 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-04-02T04:36:06.675 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-02T04:36:06.675 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-02T04:36:06.675 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-02T04:36:06.850 INFO:teuthology.orchestra.run.smithi070.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 776 kB/s | 427 kB 00:00 2024-04-02T04:36:06.855 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:06.855 INFO:teuthology.orchestra.run.smithi070.stdout:Total 8.6 MB/s | 30 MB 00:03 2024-04-02T04:36:06.869 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:06.911 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:06.911 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:06.970 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-02T04:36:06.970 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:06.970 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:06.970 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-04-02T04:36:06.971 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-04-02T04:36:06.971 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-04-02T04:36:06.971 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-02T04:36:06.971 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-02T04:36:06.971 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T04:36:06.971 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T04:36:06.971 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:06.971 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:07.196 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-02T04:36:07.277 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:07.277 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:07.754 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:57 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:07.903 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T04:36:07.905 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout:Install 12 Packages 2024-04-02T04:36:07.906 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:07.907 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 37 M 2024-04-02T04:36:07.907 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 209 M 2024-04-02T04:36:07.907 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:08.213 INFO:teuthology.orchestra.run.smithi179.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 249 kB/s | 32 kB 00:00 2024-04-02T04:36:08.238 INFO:teuthology.orchestra.run.smithi179.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 697 kB/s | 17 kB 00:00 2024-04-02T04:36:08.339 INFO:teuthology.orchestra.run.smithi179.stdout:(3/12): ceph-mgr-diskprediction-local-18.2.1-0. 29 MB/s | 7.4 MB 00:00 2024-04-02T04:36:08.355 INFO:teuthology.orchestra.run.smithi179.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 292 kB/s | 34 kB 00:00 2024-04-02T04:36:08.506 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:08.522 INFO:teuthology.orchestra.run.smithi179.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2024-04-02T04:36:08.561 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-02T04:36:08.590 INFO:teuthology.orchestra.run.smithi179.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-04-02T04:36:08.595 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-02T04:36:08.628 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-02T04:36:08.640 INFO:teuthology.orchestra.run.smithi179.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.4 MB/s | 3.0 MB 00:00 2024-04-02T04:36:08.665 INFO:teuthology.orchestra.run.smithi179.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.5 MB/s | 217 kB 00:00 2024-04-02T04:36:08.721 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-02T04:36:08.799 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-02T04:36:08.839 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-02T04:36:08.911 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-02T04:36:08.965 INFO:teuthology.orchestra.run.smithi179.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-04-02T04:36:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-04-02T04:36:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-04-02T04:36:09.068 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-04-02T04:36:09.135 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-04-02T04:36:09.165 INFO:teuthology.orchestra.run.smithi179.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-04-02T04:36:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-04-02T04:36:09.366 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-04-02T04:36:09.418 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-04-02T04:36:09.456 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-04-02T04:36:09.486 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-02T04:36:09.563 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-04-02T04:36:09.596 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-02T04:36:09.629 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-04-02T04:36:09.670 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-04-02T04:36:09.713 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-04-02T04:36:09.756 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-04-02T04:36:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-04-02T04:36:09.882 INFO:teuthology.orchestra.run.smithi179.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 161 kB/s | 195 kB 00:01 2024-04-02T04:36:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-04-02T04:36:09.959 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-04-02T04:36:10.097 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-04-02T04:36:10.222 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-02T04:36:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-04-02T04:36:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-04-02T04:36:10.433 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-04-02T04:36:10.433 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:36:10.433 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T04:36:10.433 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T04:36:10.433 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T04:36:10.433 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:10.683 INFO:teuthology.orchestra.run.smithi179.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 398 kB/s | 803 kB 00:02 2024-04-02T04:36:10.687 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:10.687 INFO:teuthology.orchestra.run.smithi179.stdout:Total 13 MB/s | 37 MB 00:02 2024-04-02T04:36:10.730 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:10.755 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:10.756 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:11.196 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:11.197 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:11.893 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:11.943 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-04-02T04:36:11.969 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-04-02T04:36:11.969 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:36:11.970 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T04:36:11.970 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T04:36:11.970 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T04:36:11.970 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:11.972 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-04-02T04:36:12.012 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-02T04:36:12.114 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-04-02T04:36:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-02T04:36:12.393 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T04:36:12.418 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-04-02T04:36:12.422 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T04:36:12.442 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-04-02T04:36:12.442 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:36:12.442 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T04:36:12.442 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T04:36:12.442 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T04:36:12.442 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:12.504 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-02T04:36:12.645 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-04-02T04:36:12.668 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-04-02T04:36:12.668 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:36:12.668 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T04:36:12.668 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T04:36:12.668 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T04:36:12.668 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:12.734 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-04-02T04:36:13.150 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-02T04:36:13.260 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-02T04:36:14.525 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-02T04:36:14.542 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-04-02T04:36:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-02T04:36:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-02T04:36:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-02T04:36:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-02T04:36:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-02T04:36:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-02T04:36:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-02T04:36:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-02T04:36:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-02T04:36:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:15.345 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T04:36:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T04:36:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T04:36:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T04:36:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T04:36:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T04:36:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T04:36:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:15.348 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:15.556 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-02T04:36:15.580 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-04-02T04:36:15.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-base 2024-04-02T04:36:16.194 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:49 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:16.293 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:16.345 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:16.345 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:16.346 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:16.412 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install cephadm 2024-04-02T04:36:16.981 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:49 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:17.133 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:17.134 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:17.134 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 221 k 2024-04-02T04:36:17.135 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 213 k 2024-04-02T04:36:17.136 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:17.226 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.4 MB/s | 221 kB 00:00 2024-04-02T04:36:17.229 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:17.229 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.3 MB/s | 221 kB 00:00 2024-04-02T04:36:17.229 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:17.232 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:17.232 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:17.234 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:17.235 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:17.302 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:17.754 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-02T04:36:17.826 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-02T04:36:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-04-02T04:36:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-04-02T04:36:18.358 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-04-02T04:36:18.484 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-02T04:36:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:18.485 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-04-02T04:36:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:18.485 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:18.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-immutable-object-cache 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T04:36:18.771 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T04:36:18.772 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T04:36:18.772 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T04:36:18.772 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T04:36:18.772 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T04:36:18.772 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T04:36:18.772 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T04:36:18.772 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:18.772 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:19.159 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-rook 2024-04-02T04:36:19.280 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:52 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:19.444 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-04-02T04:36:19.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:19.446 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 142 k 2024-04-02T04:36:19.446 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 395 k 2024-04-02T04:36:19.446 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:19.520 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.9 MB/s | 142 kB 00:00 2024-04-02T04:36:19.520 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:19.521 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.9 MB/s | 142 kB 00:00 2024-04-02T04:36:19.521 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:19.529 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:19.529 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:19.584 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:19.584 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:19.731 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:19.732 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:09 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:19.804 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:19.830 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:19.830 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:36:19.830 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T04:36:19.830 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:19.890 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:19.890 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:19.890 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-02T04:36:19.890 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:19.890 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:19.890 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T04:36:19.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T04:36:19.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T04:36:19.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T04:36:19.892 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:19.892 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:19.892 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:19.892 INFO:teuthology.orchestra.run.smithi179.stdout:Install 13 Packages 2024-04-02T04:36:19.892 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:19.893 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 2.0 M 2024-04-02T04:36:19.893 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 26 M 2024-04-02T04:36:19.893 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:20.163 INFO:teuthology.orchestra.run.smithi179.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 447 kB/s | 50 kB 00:00 2024-04-02T04:36:20.188 INFO:teuthology.orchestra.run.smithi179.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 178 kB/s | 24 kB 00:00 2024-04-02T04:36:20.214 INFO:teuthology.orchestra.run.smithi179.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 111 kB/s | 18 kB 00:00 2024-04-02T04:36:20.231 INFO:teuthology.orchestra.run.smithi179.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-04-02T04:36:20.256 INFO:teuthology.orchestra.run.smithi179.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-04-02T04:36:20.298 INFO:teuthology.orchestra.run.smithi179.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 211 kB 00:00 2024-04-02T04:36:20.323 INFO:teuthology.orchestra.run.smithi179.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 466 kB/s | 43 kB 00:00 2024-04-02T04:36:20.432 INFO:teuthology.orchestra.run.smithi179.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 112 kB/s | 15 kB 00:00 2024-04-02T04:36:20.457 INFO:teuthology.orchestra.run.smithi179.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 203 kB/s | 32 kB 00:00 2024-04-02T04:36:20.549 INFO:teuthology.orchestra.run.smithi179.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 898 kB/s | 203 kB 00:00 2024-04-02T04:36:20.575 INFO:teuthology.orchestra.run.smithi179.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-04-02T04:36:20.616 INFO:teuthology.orchestra.run.smithi179.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-04-02T04:36:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:20.842 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:20.842 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:20.967 INFO:teuthology.orchestra.run.smithi179.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-04-02T04:36:20.969 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:20.969 INFO:teuthology.orchestra.run.smithi179.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2024-04-02T04:36:20.969 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:20.984 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:20.984 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:21.080 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:21.081 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:21.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr 2024-04-02T04:36:21.594 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:21.674 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:54 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:21.763 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-02T04:36:21.775 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:21.817 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-02T04:36:21.827 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:21.828 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:21.828 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:21.872 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-02T04:36:21.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-dashboard 2024-04-02T04:36:21.902 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-02T04:36:21.943 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-02T04:36:22.037 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-02T04:36:22.145 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-02T04:36:22.203 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-02T04:36:22.456 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:55 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:22.607 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:22.608 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:22.608 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-02T04:36:22.608 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:22.608 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:22.608 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-04-02T04:36:22.608 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:36:22.608 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-04-02T04:36:22.608 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout:Install 7 Packages 2024-04-02T04:36:22.609 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:22.610 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.1 M 2024-04-02T04:36:22.610 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 8.5 M 2024-04-02T04:36:22.610 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:22.775 INFO:teuthology.orchestra.run.smithi070.stdout:(1/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 131 kB/s | 15 kB 00:00 2024-04-02T04:36:22.791 INFO:teuthology.orchestra.run.smithi070.stdout:(2/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 180 kB/s | 23 kB 00:00 2024-04-02T04:36:22.833 INFO:teuthology.orchestra.run.smithi070.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 10 MB/s | 1.8 MB 00:00 2024-04-02T04:36:22.926 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-02T04:36:22.967 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-02T04:36:23.000 INFO:teuthology.orchestra.run.smithi070.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 40 kB/s | 9.0 kB 00:00 2024-04-02T04:36:23.001 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-02T04:36:23.048 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-02T04:36:23.070 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-04-02T04:36:23.109 INFO:teuthology.orchestra.run.smithi070.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 112 kB/s | 31 kB 00:00 2024-04-02T04:36:23.142 INFO:teuthology.orchestra.run.smithi070.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 117 kB/s | 41 kB 00:00 2024-04-02T04:36:23.384 INFO:teuthology.orchestra.run.smithi070.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 491 kB/s | 188 kB 00:00 2024-04-02T04:36:23.386 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:23.386 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.6 MB/s | 2.1 MB 00:00 2024-04-02T04:36:23.386 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:23.394 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:23.394 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:23.748 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:23.811 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-02T04:36:23.844 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-02T04:36:23.858 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-02T04:36:23.859 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-02T04:36:23.899 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-02T04:36:23.931 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-02T04:36:23.960 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-04-02T04:36:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-04-02T04:36:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-04-02T04:36:24.236 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-02T04:36:24.236 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:24.236 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:24.236 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-04-02T04:36:24.236 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T04:36:24.236 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T04:36:24.236 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T04:36:24.236 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:24.237 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:24.518 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-cephadm 2024-04-02T04:36:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-04-02T04:36:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-04-02T04:36:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-04-02T04:36:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-04-02T04:36:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-02T04:36:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-02T04:36:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-02T04:36:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T04:36:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T04:36:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:25.084 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:25.086 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:15 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:25.238 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T04:36:25.239 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T04:36:25.240 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T04:36:25.240 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:25.240 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:25.240 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:25.240 INFO:teuthology.orchestra.run.smithi179.stdout:Install 6 Packages 2024-04-02T04:36:25.240 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:25.240 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 6.8 M 2024-04-02T04:36:25.241 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 33 M 2024-04-02T04:36:25.241 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:25.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-02T04:36:25.627 INFO:teuthology.orchestra.run.smithi179.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-04-02T04:36:25.880 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:58 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:25.894 INFO:teuthology.orchestra.run.smithi179.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-04-02T04:36:25.944 INFO:teuthology.orchestra.run.smithi179.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 652 kB/s | 248 kB 00:00 2024-04-02T04:36:26.031 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:26.032 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:26.032 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-02T04:36:26.032 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:26.032 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T04:36:26.033 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T04:36:26.034 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T04:36:26.034 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T04:36:26.034 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:26.034 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:26.034 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:26.034 INFO:teuthology.orchestra.run.smithi070.stdout:Install 12 Packages 2024-04-02T04:36:26.034 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:26.035 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 37 M 2024-04-02T04:36:26.035 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 209 M 2024-04-02T04:36:26.035 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 247 kB/s | 58 kB 00:00 2024-04-02T04:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 244 kB/s | 51 kB 00:00 2024-04-02T04:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 991 kB/s | 548 kB 00:00 2024-04-02T04:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:Total 7.0 MB/s | 6.8 MB 00:00 2024-04-02T04:36:26.218 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:26.229 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:26.229 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:26.317 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:26.317 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:26.606 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:26.708 INFO:teuthology.orchestra.run.smithi070.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 251 kB/s | 32 kB 00:00 2024-04-02T04:36:26.776 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-02T04:36:26.808 INFO:teuthology.orchestra.run.smithi070.stdout:(2/12): ceph-mgr-diskprediction-local-18.2.1-0. 33 MB/s | 7.4 MB 00:00 2024-04-02T04:36:26.829 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-02T04:36:26.833 INFO:teuthology.orchestra.run.smithi070.stdout:(3/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 137 kB/s | 17 kB 00:00 2024-04-02T04:36:26.859 INFO:teuthology.orchestra.run.smithi070.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 682 kB/s | 34 kB 00:00 2024-04-02T04:36:27.109 INFO:teuthology.orchestra.run.smithi070.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2024-04-02T04:36:27.176 INFO:teuthology.orchestra.run.smithi070.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 4.8 MB 00:00 2024-04-02T04:36:27.201 INFO:teuthology.orchestra.run.smithi070.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-04-02T04:36:27.338 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-02T04:36:27.377 INFO:teuthology.orchestra.run.smithi070.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-02T04:36:27.416 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-02T04:36:27.443 INFO:teuthology.orchestra.run.smithi070.stdout:(9/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.5 MB/s | 3.0 MB 00:00 2024-04-02T04:36:27.476 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-02T04:36:27.477 INFO:teuthology.orchestra.run.smithi070.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 195 kB 00:00 2024-04-02T04:36:27.500 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-04-02T04:36:27.669 INFO:teuthology.orchestra.run.smithi070.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-04-02T04:36:27.778 INFO:teuthology.orchestra.run.smithi070.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 740 kB/s | 245 kB 00:00 2024-04-02T04:36:27.780 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:27.780 INFO:teuthology.orchestra.run.smithi070.stdout:Total 21 MB/s | 37 MB 00:01 2024-04-02T04:36:27.821 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:27.844 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:27.845 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:28.272 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:28.273 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:28.349 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-04-02T04:36:28.349 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-04-02T04:36:28.349 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-02T04:36:28.349 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-02T04:36:28.349 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-02T04:36:28.350 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-02T04:36:28.647 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-02T04:36:28.647 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:28.648 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:28.921 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-fuse 2024-04-02T04:36:28.981 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:29.059 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-04-02T04:36:29.122 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-02T04:36:29.226 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-04-02T04:36:29.268 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-02T04:36:29.449 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T04:36:29.476 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T04:36:29.478 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:19 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:29.558 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-02T04:36:29.629 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:29.630 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:29.630 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:29.630 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:29.630 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:29.630 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-04-02T04:36:29.630 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-02T04:36:29.630 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T04:36:29.630 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:29.631 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:29.631 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:29.631 INFO:teuthology.orchestra.run.smithi179.stdout:Install 2 Packages 2024-04-02T04:36:29.631 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:29.631 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 918 k 2024-04-02T04:36:29.631 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 2.7 M 2024-04-02T04:36:29.632 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:29.721 INFO:teuthology.orchestra.run.smithi179.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-04-02T04:36:29.813 INFO:teuthology.orchestra.run.smithi179.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 434 kB/s | 79 kB 00:00 2024-04-02T04:36:29.814 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:29.815 INFO:teuthology.orchestra.run.smithi179.stdout:Total 4.9 MB/s | 918 kB 00:00 2024-04-02T04:36:29.815 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:29.835 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:29.835 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:29.906 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:29.907 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:30.085 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:30.221 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-02T04:36:30.234 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-02T04:36:30.305 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-04-02T04:36:30.333 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-02T04:36:30.990 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-04-02T04:36:30.990 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-04-02T04:36:31.301 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-02T04:36:31.301 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:31.301 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:31.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-02T04:36:31.301 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:31.301 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:31.586 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librados-devel 2024-04-02T04:36:31.587 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-02T04:36:32.163 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:22 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:32.321 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:32.321 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:32.321 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 127 k 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 456 k 2024-04-02T04:36:32.322 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:32.395 INFO:teuthology.orchestra.run.smithi179.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.7 MB/s | 127 kB 00:00 2024-04-02T04:36:32.395 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:32.395 INFO:teuthology.orchestra.run.smithi179.stdout:Total 1.7 MB/s | 127 kB 00:00 2024-04-02T04:36:32.396 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:32.401 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:32.401 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:32.456 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:32.456 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:32.604 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:32.617 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-02T04:36:32.643 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-04-02T04:36:32.770 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:33.226 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:33.515 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:33.515 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:33.515 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:33.516 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:33.516 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:33.516 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:33.805 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install libcephfs2 2024-04-02T04:36:34.388 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:24 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:34.493 INFO:teuthology.orchestra.run.smithi179.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:34.545 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:34.546 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:36:34.546 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:34.630 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install libcephfs-devel 2024-04-02T04:36:35.202 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:25 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:35.355 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:35.356 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:35.356 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:35.356 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:35.356 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:35.356 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-04-02T04:36:35.356 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:35.356 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:35.357 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:35.357 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-02T04:36:35.357 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:35.357 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 31 k 2024-04-02T04:36:35.357 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 139 k 2024-04-02T04:36:35.357 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:35.423 INFO:teuthology.orchestra.run.smithi179.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 464 kB/s | 31 kB 00:00 2024-04-02T04:36:35.424 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:35.424 INFO:teuthology.orchestra.run.smithi179.stdout:Total 453 kB/s | 31 kB 00:00 2024-04-02T04:36:35.424 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:35.428 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:35.428 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-04-02T04:36:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-04-02T04:36:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-04-02T04:36:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-04-02T04:36:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-04-02T04:36:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T04:36:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T04:36:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-04-02T04:36:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-04-02T04:36:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-04-02T04:36:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-04-02T04:36:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-04-02T04:36:35.456 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:35.456 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:35.542 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:35.646 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:35.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T04:36:35.892 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T04:36:35.893 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:35.893 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:36.033 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:36.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-rook 2024-04-02T04:36:36.298 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:36.298 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:36.298 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:36.298 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:36.298 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:36.298 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:36.535 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librados2 2024-04-02T04:36:36.867 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:09 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:37.018 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:37.019 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:37.019 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-02T04:36:37.019 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:37.019 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:37.019 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T04:36:37.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:37.021 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:37.021 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:37.021 INFO:teuthology.orchestra.run.smithi070.stdout:Install 13 Packages 2024-04-02T04:36:37.021 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:37.021 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.0 M 2024-04-02T04:36:37.021 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 26 M 2024-04-02T04:36:37.022 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:37.104 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:27 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:37.202 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:37.256 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:37.257 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:36:37.257 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:37.340 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librbd1 2024-04-02T04:36:37.485 INFO:teuthology.orchestra.run.smithi070.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 509 kB/s | 50 kB 00:00 2024-04-02T04:36:37.510 INFO:teuthology.orchestra.run.smithi070.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 197 kB/s | 24 kB 00:00 2024-04-02T04:36:37.536 INFO:teuthology.orchestra.run.smithi070.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 121 kB/s | 18 kB 00:00 2024-04-02T04:36:37.561 INFO:teuthology.orchestra.run.smithi070.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-04-02T04:36:37.586 INFO:teuthology.orchestra.run.smithi070.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-02T04:36:37.611 INFO:teuthology.orchestra.run.smithi070.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2024-04-02T04:36:37.637 INFO:teuthology.orchestra.run.smithi070.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 570 kB/s | 43 kB 00:00 2024-04-02T04:36:37.722 INFO:teuthology.orchestra.run.smithi070.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 138 kB/s | 15 kB 00:00 2024-04-02T04:36:37.746 INFO:teuthology.orchestra.run.smithi070.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 240 kB/s | 32 kB 00:00 2024-04-02T04:36:37.812 INFO:teuthology.orchestra.run.smithi070.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 1.1 MB/s | 203 kB 00:00 2024-04-02T04:36:37.838 INFO:teuthology.orchestra.run.smithi070.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-02T04:36:37.880 INFO:teuthology.orchestra.run.smithi070.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-04-02T04:36:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:27 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:37.972 INFO:teuthology.orchestra.run.smithi070.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.1 MB/s | 1.0 MB 00:00 2024-04-02T04:36:37.973 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:37.974 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.1 MB/s | 2.0 MB 00:00 2024-04-02T04:36:37.974 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:37.986 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:37.986 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:38.054 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:38.055 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:36:38.055 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:38.075 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:38.076 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:38.130 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rados 2024-04-02T04:36:38.594 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:38.694 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:28 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:38.771 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-02T04:36:38.792 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:38.823 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-02T04:36:38.844 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:38.845 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:36:38.845 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:38.886 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-02T04:36:38.923 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-02T04:36:38.926 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rgw 2024-04-02T04:36:38.965 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-02T04:36:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-02T04:36:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-02T04:36:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-02T04:36:39.491 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:29 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:39.589 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:39.642 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:39.643 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:36:39.643 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:39.722 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-cephfs 2024-04-02T04:36:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-02T04:36:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-02T04:36:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-02T04:36:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-02T04:36:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-04-02T04:36:40.305 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:30 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:40.404 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:40.457 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:40.458 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:36:40.458 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:40.539 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rbd 2024-04-02T04:36:40.919 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-02T04:36:40.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-02T04:36:41.104 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:31 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:41.202 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:41.255 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:41.256 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-02T04:36:41.256 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T04:36:41.270 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T04:36:41.271 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T04:36:41.271 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T04:36:41.271 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T04:36:41.271 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T04:36:41.271 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T04:36:41.271 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T04:36:41.271 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:41.271 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:41.355 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-fuse 2024-04-02T04:36:41.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-cephadm 2024-04-02T04:36:41.923 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:31 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:42.074 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-02T04:36:42.075 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:42.076 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 86 k 2024-04-02T04:36:42.076 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 226 k 2024-04-02T04:36:42.076 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:42.081 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:15 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:42.143 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 86 kB 00:00 2024-04-02T04:36:42.143 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:42.144 INFO:teuthology.orchestra.run.smithi179.stdout:Total 1.2 MB/s | 86 kB 00:00 2024-04-02T04:36:42.144 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:42.151 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:42.151 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:42.206 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:42.206 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:42.235 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:42.236 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:42.236 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-02T04:36:42.236 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:42.236 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:42.236 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-04-02T04:36:42.237 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:42.238 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 6.8 M 2024-04-02T04:36:42.238 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 33 M 2024-04-02T04:36:42.238 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:42.343 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:42.441 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:42.732 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-04-02T04:36:42.799 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-04-02T04:36:42.842 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:42.983 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-04-02T04:36:43.137 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:43.137 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:43.137 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:43.137 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:43.137 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:43.137 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:43.150 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 165 kB/s | 58 kB 00:00 2024-04-02T04:36:43.350 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 139 kB/s | 51 kB 00:00 2024-04-02T04:36:43.430 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-mirror 2024-04-02T04:36:43.508 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 707 kB/s | 548 kB 00:00 2024-04-02T04:36:43.509 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:43.510 INFO:teuthology.orchestra.run.smithi070.stdout:Total 5.3 MB/s | 6.8 MB 00:01 2024-04-02T04:36:43.523 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:43.533 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:43.533 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:43.620 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:43.620 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:43.922 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:44.009 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:34 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:44.099 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-02T04:36:44.152 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-02T04:36:44.166 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:44.166 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:44.166 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:44.166 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:44.166 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:44.167 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-04-02T04:36:44.167 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:44.167 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:44.167 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:44.167 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-02T04:36:44.167 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:44.167 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 3.0 M 2024-04-02T04:36:44.168 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 12 M 2024-04-02T04:36:44.168 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:44.263 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 31 MB/s | 3.0 MB 00:00 2024-04-02T04:36:44.264 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:44.264 INFO:teuthology.orchestra.run.smithi179.stdout:Total 31 MB/s | 3.0 MB 00:00 2024-04-02T04:36:44.264 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:44.274 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:44.274 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:44.346 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:44.346 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-02T04:36:44.753 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:44.771 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-02T04:36:44.822 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:44.840 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-02T04:36:44.847 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:44.847 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:36:44.847 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T04:36:44.847 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T04:36:44.847 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T04:36:44.847 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:44.873 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-04-02T04:36:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-04-02T04:36:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-04-02T04:36:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-02T04:36:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-02T04:36:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-02T04:36:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-02T04:36:45.987 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:45.987 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:45.987 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:45.988 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:45.988 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:45.988 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-02T04:36:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:45.993 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-04-02T04:36:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T04:36:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T04:36:45.993 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T04:36:45.994 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T04:36:45.994 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T04:36:45.994 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:45.994 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:46.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-fuse 2024-04-02T04:36:46.244 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-nbd 2024-04-02T04:36:46.794 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:19 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:46.827 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:36 ago on Tue 02 Apr 2024 04:35:10 AM UTC. 2024-04-02T04:36:46.946 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:46.947 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:46.947 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:46.947 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:46.947 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:46.947 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-04-02T04:36:46.947 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-02T04:36:46.947 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T04:36:46.948 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:46.948 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:46.948 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:46.948 INFO:teuthology.orchestra.run.smithi070.stdout:Install 2 Packages 2024-04-02T04:36:46.948 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:46.948 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 918 k 2024-04-02T04:36:46.948 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 2.7 M 2024-04-02T04:36:46.948 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:46.979 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-02T04:36:46.980 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:46.980 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:46.980 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:46.980 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 171 k 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 494 k 2024-04-02T04:36:46.981 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-02T04:36:47.042 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.8 MB/s | 171 kB 00:00 2024-04-02T04:36:47.042 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:47.042 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.7 MB/s | 171 kB 00:00 2024-04-02T04:36:47.042 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-02T04:36:47.050 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-02T04:36:47.050 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-02T04:36:47.061 INFO:teuthology.orchestra.run.smithi070.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 7.3 MB/s | 839 kB 00:00 2024-04-02T04:36:47.107 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-02T04:36:47.108 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-02T04:36:47.119 INFO:teuthology.orchestra.run.smithi070.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 459 kB/s | 79 kB 00:00 2024-04-02T04:36:47.120 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:47.121 INFO:teuthology.orchestra.run.smithi070.stdout:Total 5.2 MB/s | 918 kB 00:00 2024-04-02T04:36:47.121 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:47.141 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:47.141 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:47.213 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:47.213 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:47.254 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-02T04:36:47.356 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:47.385 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:47.518 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-02T04:36:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-04-02T04:36:47.833 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:48.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:48.121 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:48.121 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-02T04:36:48.121 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:48.122 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:36:48.122 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-02T04:36:48.260 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-04-02T04:36:48.260 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-04-02T04:36:48.371 DEBUG:teuthology.parallel:result is None 2024-04-02T04:36:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-02T04:36:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:48.561 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-02T04:36:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:48.561 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:48.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados-devel 2024-04-02T04:36:49.415 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:22 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:49.573 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:49.574 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-04-02T04:36:49.574 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:49.574 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 127 k 2024-04-02T04:36:49.574 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 456 k 2024-04-02T04:36:49.574 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:49.632 INFO:teuthology.orchestra.run.smithi070.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.1 MB/s | 127 kB 00:00 2024-04-02T04:36:49.633 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:49.633 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.1 MB/s | 127 kB 00:00 2024-04-02T04:36:49.633 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:49.638 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:49.639 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:49.694 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:49.694 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:49.940 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:50.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:50.719 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:50.719 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:50.719 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:50.720 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:50.720 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:50.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs2 2024-04-02T04:36:51.574 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:24 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:51.730 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:51.731 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:51.731 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:51.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs-devel 2024-04-02T04:36:52.370 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:25 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:52.522 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:52.523 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:52.523 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:52.523 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-04-02T04:36:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:52.523 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:52.523 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:52.524 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-04-02T04:36:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:52.524 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 31 k 2024-04-02T04:36:52.524 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 139 k 2024-04-02T04:36:52.524 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:52.587 INFO:teuthology.orchestra.run.smithi070.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 490 kB/s | 31 kB 00:00 2024-04-02T04:36:52.588 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:52.588 INFO:teuthology.orchestra.run.smithi070.stdout:Total 481 kB/s | 31 kB 00:00 2024-04-02T04:36:52.588 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:52.591 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:52.592 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:52.629 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:52.629 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:52.710 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:52.827 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:53.202 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:53.494 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:53.495 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:53.495 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:36:53.495 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-04-02T04:36:53.495 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:53.495 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:53.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados2 2024-04-02T04:36:54.236 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:27 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:54.336 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:54.390 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:54.391 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:54.391 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:54.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librbd1 2024-04-02T04:36:55.013 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:28 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:55.111 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:55.163 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:55.164 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:55.164 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:55.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rados 2024-04-02T04:36:55.799 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:28 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:55.897 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:55.950 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:55.951 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:55.951 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:56.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rgw 2024-04-02T04:36:56.575 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:29 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:56.675 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:56.728 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:56.729 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:56.729 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:56.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-cephfs 2024-04-02T04:36:57.350 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:30 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:57.448 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:57.502 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:57.503 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:57.503 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:57.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rbd 2024-04-02T04:36:58.133 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:31 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:58.233 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-02T04:36:58.285 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:58.286 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-02T04:36:58.286 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:36:58.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-fuse 2024-04-02T04:36:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:31 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:36:59.073 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:36:59.074 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:59.074 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-02T04:36:59.074 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:59.074 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:36:59.074 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-04-02T04:36:59.075 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:59.075 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:36:59.075 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:36:59.075 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-04-02T04:36:59.075 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:36:59.075 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 86 k 2024-04-02T04:36:59.075 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 226 k 2024-04-02T04:36:59.075 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:36:59.140 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 86 kB 00:00 2024-04-02T04:36:59.141 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:36:59.141 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.3 MB/s | 86 kB 00:00 2024-04-02T04:36:59.141 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:36:59.148 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:36:59.148 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:36:59.201 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:36:59.202 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:36:59.334 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:36:59.438 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:36:59.808 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:37:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:37:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:00.098 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:37:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-04-02T04:37:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:00.098 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:37:00.324 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-mirror 2024-04-02T04:37:00.902 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:33 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:37:01.062 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:37:01.063 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:37:01.063 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-02T04:37:01.063 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:37:01.063 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:37:01.063 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-04-02T04:37:01.063 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:01.063 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:37:01.064 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:37:01.064 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-04-02T04:37:01.064 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:01.064 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 3.0 M 2024-04-02T04:37:01.064 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 12 M 2024-04-02T04:37:01.064 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:37:01.188 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-04-02T04:37:01.188 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:37:01.189 INFO:teuthology.orchestra.run.smithi070.stdout:Total 24 MB/s | 3.0 MB 00:00 2024-04-02T04:37:01.189 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:37:01.198 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:37:01.198 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:37:01.269 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:37:01.269 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:37:01.685 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:37:01.755 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:37:01.780 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:37:01.780 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:37:01.780 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T04:37:01.780 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T04:37:01.780 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T04:37:01.780 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:02.925 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:37:02.925 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:02.925 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:37:02.925 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-04-02T04:37:02.925 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:02.925 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:37:03.128 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-nbd 2024-04-02T04:37:03.697 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:36 ago on Tue 02 Apr 2024 04:35:27 AM UTC. 2024-04-02T04:37:03.847 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-02T04:37:03.847 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 171 k 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 494 k 2024-04-02T04:37:03.848 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-02T04:37:03.916 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-04-02T04:37:03.917 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:37:03.917 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.4 MB/s | 171 kB 00:00 2024-04-02T04:37:03.917 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-02T04:37:03.925 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-02T04:37:03.925 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-02T04:37:03.980 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-02T04:37:03.980 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-02T04:37:04.127 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-02T04:37:04.231 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:37:04.650 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:37:04.958 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-02T04:37:04.958 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:04.958 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-02T04:37:04.959 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-04-02T04:37:04.959 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:04.959 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-02T04:37:05.204 DEBUG:teuthology.parallel:result is None 2024-04-02T04:37:05.204 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-02T04:37:05.204 INFO:teuthology.packaging:ref: None 2024-04-02T04:37:05.204 INFO:teuthology.packaging:tag: v18.2.1 2024-04-02T04:37:05.204 INFO:teuthology.packaging:branch: None 2024-04-02T04:37:05.204 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:37:05.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-02T04:37:05.513 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T04:37:05.534 INFO:teuthology.orchestra.run.smithi070.stdout:18.2.1-0.el9 2024-04-02T04:37:05.535 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-04-02T04:37:05.535 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-04-02T04:37:05.537 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-02T04:37:05.537 INFO:teuthology.packaging:ref: None 2024-04-02T04:37:05.537 INFO:teuthology.packaging:tag: v18.2.1 2024-04-02T04:37:05.537 INFO:teuthology.packaging:branch: None 2024-04-02T04:37:05.537 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:37:05.537 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-02T04:37:05.776 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T04:37:05.801 INFO:teuthology.orchestra.run.smithi179.stdout:18.2.1-0.el9 2024-04-02T04:37:05.802 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-04-02T04:37:05.802 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-04-02T04:37:05.804 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T04:37:05.804 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:37:05.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T04:37:05.832 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:37:05.832 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T04:37:05.873 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T04:37:05.873 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:37:05.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T04:37:05.901 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T04:37:05.969 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:37:05.970 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T04:37:06.000 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T04:37:06.070 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T04:37:06.071 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:37:06.071 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T04:37:06.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T04:37:06.163 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:37:06.164 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T04:37:06.194 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T04:37:06.263 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T04:37:06.263 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:37:06.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T04:37:06.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T04:37:06.356 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:37:06.356 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T04:37:06.383 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T04:37:06.450 INFO:teuthology.run_tasks:Running task print... 2024-04-02T04:37:06.461 INFO:teuthology.task.print:**** done install task... 2024-04-02T04:37:06.461 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-02T04:37:06.571 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca'} 2024-04-02T04:37:06.571 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-04-02T04:37:06.572 INFO:tasks.cephadm:Cluster fsid is a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:06.572 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-02T04:37:06.572 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-02T04:37:06.572 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi070': '172.21.15.70', 'mon.smithi179': '172.21.15.179'} 2024-04-02T04:37:06.573 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-02T04:37:06.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname $(hostname -s) 2024-04-02T04:37:06.600 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-04-02T04:37:06.629 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-04-02T04:37:06.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:37:06.842 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54776/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1979-g6cec793d', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1979.g6cec793d'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/6cec793d9073f0594e2364b9b687ecaf22424cca/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-01 16:46:51.658725', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/6cec793d9073f0594e2364b9b687ecaf22424cca/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-02T04:37:07.103 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref reef, sha1 4d05f4cad464de0fcdd11a2856643e533aea34e8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-04-02T04:37:07.105 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/x86_64/flavors/default/cephadm 2024-04-02T04:37:07.105 DEBUG:teuthology.orchestra.run.smithi070:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T04:37:07.511 INFO:teuthology.orchestra.run.smithi070.stdout:-rw-r--r--. 1 ubuntu ubuntu 215333 Apr 2 04:37 /home/ubuntu/cephtest/cephadm 2024-04-02T04:37:07.511 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T04:37:08.079 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 215333 Apr 2 04:37 /home/ubuntu/cephtest/cephadm 2024-04-02T04:37:08.079 DEBUG:teuthology.orchestra.run.smithi070:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T04:37:08.098 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T04:37:08.125 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-04-02T04:37:08.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-04-02T04:37:08.141 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-04-02T04:37:08.933 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-02T04:37:09.125 INFO:teuthology.orchestra.run.smithi070.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-02T04:37:29.645 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-04-02T04:37:29.646 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-02T04:37:29.646 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-04-02T04:37:29.646 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-04-02T04:37:29.646 INFO:teuthology.orchestra.run.smithi179.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-04-02T04:37:29.646 INFO:teuthology.orchestra.run.smithi179.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-04-02T04:37:29.646 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-04-02T04:37:29.646 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-04-02T04:37:30.677 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:37:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-02T04:37:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-04-02T04:37:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: "repo_digests": [ 2024-04-02T04:37:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-04-02T04:37:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-04-02T04:37:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-04-02T04:37:30.677 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:37:30.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph 2024-04-02T04:37:30.758 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-04-02T04:37:30.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /etc/ceph 2024-04-02T04:37:30.819 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-04-02T04:37:30.864 INFO:tasks.cephadm:Writing seed config... 2024-04-02T04:37:30.865 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-02T04:37:30.865 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-02T04:37:30.865 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-02T04:37:30.866 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-02T04:37:30.867 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-02T04:37:30.867 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-02T04:37:30.867 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-02T04:37:30.867 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-02T04:37:30.867 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-02T04:37:30.867 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-02T04:37:30.867 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-02T04:37:30.867 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-02T04:37:30.868 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-02T04:37:30.869 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-02T04:37:30.869 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-02T04:37:30.869 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-02T04:37:30.869 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-02T04:37:30.869 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-02T04:37:30.869 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:37:30.870 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-02T04:37:30.884 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a9031cd8-f0aa-11ee-b647-cb9ed24678a4 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-02T04:37:30.885 DEBUG:teuthology.orchestra.run.smithi070:mon.smithi070> sudo journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070.service 2024-04-02T04:37:30.927 INFO:tasks.cephadm:Bootstrapping... 2024-04-02T04:37:30.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.70 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-02T04:37:31.082 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:37:31.106 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', 'a9031cd8-f0aa-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.70', '--skip-admin-label'] 2024-04-02T04:37:31.108 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T04:37:31.108 INFO:teuthology.orchestra.run.smithi070.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-02T04:37:31.108 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying podman|docker is present... 2024-04-02T04:37:31.131 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T04:37:31.131 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying lvm2 is present... 2024-04-02T04:37:31.131 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying time synchronization is in place... 2024-04-02T04:37:31.138 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T04:37:31.138 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T04:37:31.144 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T04:37:31.144 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-04-02T04:37:31.151 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2024-04-02T04:37:31.156 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2024-04-02T04:37:31.157 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2024-04-02T04:37:31.157 INFO:teuthology.orchestra.run.smithi070.stdout:Repeating the final host check... 2024-04-02T04:37:31.179 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T04:37:31.179 INFO:teuthology.orchestra.run.smithi070.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-02T04:37:31.179 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl is present 2024-04-02T04:37:31.179 INFO:teuthology.orchestra.run.smithi070.stdout:lvcreate is present 2024-04-02T04:37:31.185 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T04:37:31.186 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T04:37:31.191 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T04:37:31.191 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-04-02T04:37:31.197 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2024-04-02T04:37:31.203 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2024-04-02T04:37:31.203 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2024-04-02T04:37:31.203 INFO:teuthology.orchestra.run.smithi070.stdout:Host looks OK 2024-04-02T04:37:31.203 INFO:teuthology.orchestra.run.smithi070.stdout:Cluster fsid: a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:31.203 INFO:teuthology.orchestra.run.smithi070.stdout:Acquiring lock 140261271395008 on /run/cephadm/a9031cd8-f0aa-11ee-b647-cb9ed24678a4.lock 2024-04-02T04:37:31.204 INFO:teuthology.orchestra.run.smithi070.stdout:Lock 140261271395008 acquired on /run/cephadm/a9031cd8-f0aa-11ee-b647-cb9ed24678a4.lock 2024-04-02T04:37:31.204 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 3300 ... 2024-04-02T04:37:31.204 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 6789 ... 2024-04-02T04:37:31.204 INFO:teuthology.orchestra.run.smithi070.stdout:Base mon IP(s) is [172.21.15.70:3300, 172.21.15.70:6789], mon addrv is [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2024-04-02T04:37:31.207 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.70 metric 100 2024-04-02T04:37:31.208 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.70 metric 100 2024-04-02T04:37:31.210 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-02T04:37:31.210 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-02T04:37:31.210 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-04-02T04:37:31.212 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-02T04:37:31.213 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-02T04:37:31.213 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T04:37:31.213 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-02T04:37:31.213 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2024-04-02T04:37:31.213 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T04:37:31.213 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2024-04-02T04:37:31.213 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2024-04-02T04:37:31.214 INFO:teuthology.orchestra.run.smithi070.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-02T04:37:31.214 INFO:teuthology.orchestra.run.smithi070.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-02T04:37:31.215 INFO:teuthology.orchestra.run.smithi070.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-02T04:37:32.097 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-04-02T04:37:32.097 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-04-02T04:37:32.097 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Getting image source signatures 2024-04-02T04:37:32.097 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-04-02T04:37:32.097 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-04-02T04:37:32.098 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-04-02T04:37:32.098 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-02T04:37:33.017 INFO:teuthology.orchestra.run.smithi070.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-04-02T04:37:33.018 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-04-02T04:37:33.018 INFO:teuthology.orchestra.run.smithi070.stdout:Extracting ceph user uid/gid from container image... 2024-04-02T04:37:34.041 INFO:teuthology.orchestra.run.smithi070.stdout:stat: stdout 167 167 2024-04-02T04:37:34.041 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial keys... 2024-04-02T04:37:34.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQCOiwtmDVKoFBAACwxKy3ETTF7MrvGZL+2+lA== 2024-04-02T04:37:35.893 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQCPiwtmzZx7ERAAoBnTKZ7SqYrUxC4IrRO3Xw== 2024-04-02T04:37:36.941 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQCQiwtmw49zDhAAy52vKWcuUMiDg6LnxVOMTA== 2024-04-02T04:37:36.942 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial monmap... 2024-04-02T04:37:46.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T04:37:46.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-02T04:37:46.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:46.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T04:37:46.321 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool for smithi070 [v2:172.21.15.70:3300,v1:172.21.15.70:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T04:37:46.321 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = pacific 2024-04-02T04:37:46.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: set fsid to a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:46.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T04:37:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:37:46.322 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mon... 2024-04-02T04:37:48.583 INFO:teuthology.orchestra.run.smithi070.stdout:create mon.smithi070 on 2024-04-02T04:37:48.786 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-02T04:37:48.918 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-02T04:37:49.127 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4.target. 2024-04-02T04:37:49.127 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4.target. 2024-04-02T04:37:49.334 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070 2024-04-02T04:37:49.334 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070.service: Unit ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070.service not loaded. 2024-04-02T04:37:49.497 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4.target.wants/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070.service → /etc/systemd/system/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@.service. 2024-04-02T04:37:49.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:49 smithi070 podman[28808]: 2024-04-02 04:37:49.955832568 +0000 UTC m=+0.341295278 container start 345c9fc95a8cd057ed85237192893cc5b9377780670dc4e2726b1d6cd7e45330 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, ceph=True, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-02T04:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T04:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-04-02T04:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T04:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-04-02T04:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-04-02T04:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T04:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon to start... 2024-04-02T04:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon... 2024-04-02T04:37:50.272 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:49 smithi070 ceph-mon[28826]: mkfs a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:50.273 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:49 smithi070 ceph-mon[28826]: mon.smithi070 is new leader, mons smithi070 in quorum (ranks 0) 2024-04-02T04:37:50.273 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:50 smithi070 bash[28808]: 345c9fc95a8cd057ed85237192893cc5b9377780670dc4e2726b1d6cd7e45330 2024-04-02T04:37:50.273 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:50 smithi070 systemd[1]: Started Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:37:51.122 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout cluster: 2024-04-02T04:37:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout id: a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-02T04:37:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:37:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout services: 2024-04-02T04:37:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi070 (age 0.571999s) 2024-04-02T04:37:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-02T04:37:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-02T04:37:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout data: 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pgs: 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.484+0000 7f468fe8a700 1 Processor -- start 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.484+0000 7f468fe8a700 1 -- start start 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.484+0000 7f468fe8a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468807a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.484+0000 7f468fe8a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468807a890 con 0x7f468807be60 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.485+0000 7f468dc26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468807a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.485+0000 7f468dc26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468807a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57388/0 (socket says 172.21.15.70:57388) 2024-04-02T04:37:51.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.485+0000 7f468dc26700 1 -- 172.21.15.70:0/1072412952 learned_addr learned my addr 172.21.15.70:0/1072412952 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:51.131 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.485+0000 7f468dc26700 1 -- 172.21.15.70:0/1072412952 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468807a9d0 con 0x7f468807be60 2024-04-02T04:37:51.131 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.485+0000 7f468dc26700 1 --2- 172.21.15.70:0/1072412952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468807a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f46840098a0 tx=0x7f4684009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8a3f39725310e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:51.131 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.485+0000 7f468cc24700 1 -- 172.21.15.70:0/1072412952 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4684004180 con 0x7f468807be60 2024-04-02T04:37:51.131 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.485+0000 7f468cc24700 1 -- 172.21.15.70:0/1072412952 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840042e0 con 0x7f468807be60 2024-04-02T04:37:51.131 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.485+0000 7f468cc24700 1 -- 172.21.15.70:0/1072412952 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46840036a0 con 0x7f468807be60 2024-04-02T04:37:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.486+0000 7f468fe8a700 1 -- 172.21.15.70:0/1072412952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 msgr2=0x7f468807a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.486+0000 7f468fe8a700 1 --2- 172.21.15.70:0/1072412952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468807a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f46840098a0 tx=0x7f4684009bb0 comp rx=0 tx=0).stop 2024-04-02T04:37:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.486+0000 7f468fe8a700 1 -- 172.21.15.70:0/1072412952 shutdown_connections 2024-04-02T04:37:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.486+0000 7f468fe8a700 1 --2- 172.21.15.70:0/1072412952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468807a2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.486+0000 7f468fe8a700 1 -- 172.21.15.70:0/1072412952 >> 172.21.15.70:0/1072412952 conn(0x7f4688101b40 msgr2=0x7f4688103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.486+0000 7f468fe8a700 1 -- 172.21.15.70:0/1072412952 shutdown_connections 2024-04-02T04:37:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.486+0000 7f468fe8a700 1 -- 172.21.15.70:0/1072412952 wait complete. 2024-04-02T04:37:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.487+0000 7f468fe8a700 1 Processor -- start 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.487+0000 7f468fe8a700 1 -- start start 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.487+0000 7f468fe8a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.487+0000 7f468fe8a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468819c0c0 con 0x7f468807be60 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.487+0000 7f468dc26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.487+0000 7f468dc26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468819bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57394/0 (socket says 172.21.15.70:57394) 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.487+0000 7f468dc26700 1 -- 172.21.15.70:0/738863430 learned_addr learned my addr 172.21.15.70:0/738863430 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.487+0000 7f468dc26700 1 -- 172.21.15.70:0/738863430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4684009550 con 0x7f468807be60 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.488+0000 7f468dc26700 1 --2- 172.21.15.70:0/738863430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468819bb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f468807b330 tx=0x7f4684004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.488+0000 7f467effd700 1 -- 172.21.15.70:0/738863430 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4684003ce0 con 0x7f468807be60 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.488+0000 7f467effd700 1 -- 172.21.15.70:0/738863430 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4684020070 con 0x7f468807be60 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.488+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468819c2c0 con 0x7f468807be60 2024-04-02T04:37:51.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.488+0000 7f467effd700 1 -- 172.21.15.70:0/738863430 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4684011630 con 0x7f468807be60 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.488+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468819c760 con 0x7f468807be60 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.488+0000 7f467effd700 1 -- 172.21.15.70:0/738863430 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4684022030 con 0x7f468807be60 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.488+0000 7f467effd700 1 -- 172.21.15.70:0/738863430 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4684025d00 con 0x7f468807be60 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.489+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4688195b40 con 0x7f468807be60 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.491+0000 7f467effd700 1 -- 172.21.15.70:0/738863430 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f4684025530 con 0x7f468807be60 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.563+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f468802cee0 con 0x7f468807be60 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.563+0000 7f467effd700 1 -- 172.21.15.70:0/738863430 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f4684011b50 con 0x7f468807be60 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.564+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 msgr2=0x7f468819bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.564+0000 7f468fe8a700 1 --2- 172.21.15.70:0/738863430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468819bb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f468807b330 tx=0x7f4684004750 comp rx=0 tx=0).stop 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.564+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 shutdown_connections 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.564+0000 7f468fe8a700 1 --2- 172.21.15.70:0/738863430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f468807be60 0x7f468819bb80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:51.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.564+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 >> 172.21.15.70:0/738863430 conn(0x7f4688101b40 msgr2=0x7f4688103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:51.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.564+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 shutdown_connections 2024-04-02T04:37:51.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:50.564+0000 7f468fe8a700 1 -- 172.21.15.70:0/738863430 wait complete. 2024-04-02T04:37:51.136 INFO:teuthology.orchestra.run.smithi070.stdout:mon is available 2024-04-02T04:37:51.136 INFO:teuthology.orchestra.run.smithi070.stdout:Assimilating anything we can from ceph.conf... 2024-04-02T04:37:51.245 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:50 smithi070 ceph-mon[28826]: mon.smithi070 is new leader, mons smithi070 in quorum (ranks 0) 2024-04-02T04:37:51.245 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:50 smithi070 ceph-mon[28826]: monmap e1: 1 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-02T04:37:51.245 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:50 smithi070 ceph-mon[28826]: fsmap 2024-04-02T04:37:51.246 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:50 smithi070 ceph-mon[28826]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T04:37:51.246 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:50 smithi070 ceph-mon[28826]: mgrmap e1: no daemons active 2024-04-02T04:37:51.246 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:50 smithi070 ceph-mon[28826]: from='client.? 172.21.15.70:0/738863430' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-02T04:37:52.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:51 smithi070 ceph-mon[28826]: from='client.? 172.21.15.70:0/2333298876' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T04:37:52.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:51 smithi070 ceph-mon[28826]: from='client.? 172.21.15.70:0/2333298876' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-02T04:37:52.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:37:52.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T04:37:52.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:52.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2024-04-02T04:37:52.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T04:37:52.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T04:37:52.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T04:37:52.512 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.575+0000 7f50fbe48700 1 Processor -- start 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.575+0000 7f50fbe48700 1 -- start start 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.575+0000 7f50fbe48700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.575+0000 7f50fbe48700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f407a890 con 0x7f50f407be60 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.576+0000 7f50f9be4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.576+0000 7f50f9be4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57408/0 (socket says 172.21.15.70:57408) 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.576+0000 7f50f9be4700 1 -- 172.21.15.70:0/3835190714 learned_addr learned my addr 172.21.15.70:0/3835190714 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:52.513 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.576+0000 7f50f9be4700 1 -- 172.21.15.70:0/3835190714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f407a9d0 con 0x7f50f407be60 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.576+0000 7f50f9be4700 1 --2- 172.21.15.70:0/3835190714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f407a2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009e80 tx=0x7f50f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad120eb444f7784 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.576+0000 7f50f8be2700 1 -- 172.21.15.70:0/3835190714 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50f000e070 con 0x7f50f407be60 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.576+0000 7f50f8be2700 1 -- 172.21.15.70:0/3835190714 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0003a20 con 0x7f50f407be60 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.576+0000 7f50f8be2700 1 -- 172.21.15.70:0/3835190714 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50f0003bd0 con 0x7f50f407be60 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.577+0000 7f50fbe48700 1 -- 172.21.15.70:0/3835190714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 msgr2=0x7f50f407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.577+0000 7f50fbe48700 1 --2- 172.21.15.70:0/3835190714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f407a2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009e80 tx=0x7f50f000b330 comp rx=0 tx=0).stop 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.577+0000 7f50fbe48700 1 -- 172.21.15.70:0/3835190714 shutdown_connections 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.577+0000 7f50fbe48700 1 --2- 172.21.15.70:0/3835190714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f407a2c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.577+0000 7f50fbe48700 1 -- 172.21.15.70:0/3835190714 >> 172.21.15.70:0/3835190714 conn(0x7f50f4101b00 msgr2=0x7f50f4103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.577+0000 7f50fbe48700 1 -- 172.21.15.70:0/3835190714 shutdown_connections 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.577+0000 7f50fbe48700 1 -- 172.21.15.70:0/3835190714 wait complete. 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.577+0000 7f50fbe48700 1 Processor -- start 2024-04-02T04:37:52.514 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50fbe48700 1 -- start start 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50fbe48700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50fbe48700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f419c150 con 0x7f50f407be60 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50f9be4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50f9be4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57424/0 (socket says 172.21.15.70:57424) 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50f9be4700 1 -- 172.21.15.70:0/2333298876 learned_addr learned my addr 172.21.15.70:0/2333298876 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50f9be4700 1 -- 172.21.15.70:0/2333298876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0009b30 con 0x7f50f407be60 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50f9be4700 1 --2- 172.21.15.70:0/2333298876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f419bc10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f50f407b330 tx=0x7f50f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50eaffd700 1 -- 172.21.15.70:0/2333298876 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50f0004240 con 0x7f50f407be60 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50eaffd700 1 -- 172.21.15.70:0/2333298876 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f00043e0 con 0x7f50f407be60 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50eaffd700 1 -- 172.21.15.70:0/2333298876 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50f00046e0 con 0x7f50f407be60 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f419c350 con 0x7f50f407be60 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.578+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f419c7f0 con 0x7f50f407be60 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.579+0000 7f50eaffd700 1 -- 172.21.15.70:0/2333298876 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f50f0019050 con 0x7f50f407be60 2024-04-02T04:37:52.515 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.579+0000 7f50eaffd700 1 -- 172.21.15.70:0/2333298876 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f50f00257e0 con 0x7f50f407be60 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.579+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f4195b60 con 0x7f50f407be60 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.581+0000 7f50eaffd700 1 -- 172.21.15.70:0/2333298876 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f50f002a060 con 0x7f50f407be60 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.649+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f50f402cee0 con 0x7f50f407be60 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.652+0000 7f50eaffd700 1 -- 172.21.15.70:0/2333298876 <== mon.0 v2:172.21.15.70:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50f0012d30 con 0x7f50f407be60 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.653+0000 7f50eaffd700 1 -- 172.21.15.70:0/2333298876 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f50f0020030 con 0x7f50f407be60 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.653+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 msgr2=0x7f50f419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.653+0000 7f50fbe48700 1 --2- 172.21.15.70:0/2333298876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f419bc10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f50f407b330 tx=0x7f50f0004ad0 comp rx=0 tx=0).stop 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.653+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 shutdown_connections 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.653+0000 7f50fbe48700 1 --2- 172.21.15.70:0/2333298876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50f407be60 0x7f50f419bc10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.653+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 >> 172.21.15.70:0/2333298876 conn(0x7f50f4101b00 msgr2=0x7f50f4103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.654+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 shutdown_connections 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:51.654+0000 7f50fbe48700 1 -- 172.21.15.70:0/2333298876 wait complete. 2024-04-02T04:37:52.516 INFO:teuthology.orchestra.run.smithi070.stdout:Generating new minimal ceph.conf... 2024-04-02T04:37:53.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:53 smithi070 ceph-mon[28826]: from='client.? 172.21.15.70:0/2402839542' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:54.220 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.933+0000 7f068417b700 1 Processor -- start 2024-04-02T04:37:54.220 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.933+0000 7f068417b700 1 -- start start 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.933+0000 7f068417b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c106960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.933+0000 7f068417b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c106f30 con 0x7f067c106540 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.934+0000 7f0681f17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.934+0000 7f0681f17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57440/0 (socket says 172.21.15.70:57440) 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.934+0000 7f0681f17700 1 -- 172.21.15.70:0/1567086525 learned_addr learned my addr 172.21.15.70:0/1567086525 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.934+0000 7f0681f17700 1 -- 172.21.15.70:0/1567086525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c107740 con 0x7f067c106540 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.934+0000 7f0681f17700 1 --2- 172.21.15.70:0/1567086525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c106960 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f06780098a0 tx=0x7f0678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6563c467dc596ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.934+0000 7f0680f15700 1 -- 172.21.15.70:0/1567086525 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0678004180 con 0x7f067c106540 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.934+0000 7f0680f15700 1 -- 172.21.15.70:0/1567086525 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06780042e0 con 0x7f067c106540 2024-04-02T04:37:54.221 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.934+0000 7f0680f15700 1 -- 172.21.15.70:0/1567086525 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0678011a50 con 0x7f067c106540 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 -- 172.21.15.70:0/1567086525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 msgr2=0x7f067c106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 --2- 172.21.15.70:0/1567086525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c106960 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f06780098a0 tx=0x7f0678009bb0 comp rx=0 tx=0).stop 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 -- 172.21.15.70:0/1567086525 shutdown_connections 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 --2- 172.21.15.70:0/1567086525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c106960 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 -- 172.21.15.70:0/1567086525 >> 172.21.15.70:0/1567086525 conn(0x7f067c101b40 msgr2=0x7f067c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 -- 172.21.15.70:0/1567086525 shutdown_connections 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 -- 172.21.15.70:0/1567086525 wait complete. 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 Processor -- start 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.935+0000 7f068417b700 1 -- start start 2024-04-02T04:37:54.222 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f068417b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f068417b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c19a770 con 0x7f067c106540 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0681f17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0681f17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57444/0 (socket says 172.21.15.70:57444) 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0681f17700 1 -- 172.21.15.70:0/2402839542 learned_addr learned my addr 172.21.15.70:0/2402839542 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0681f17700 1 -- 172.21.15.70:0/2402839542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678009550 con 0x7f067c106540 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0681f17700 1 --2- 172.21.15.70:0/2402839542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c19a230 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0678009520 tx=0x7f0678011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0672ffd700 1 -- 172.21.15.70:0/2402839542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0678003a80 con 0x7f067c106540 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0672ffd700 1 -- 172.21.15.70:0/2402839542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0678003be0 con 0x7f067c106540 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c19a970 con 0x7f067c106540 2024-04-02T04:37:54.223 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0672ffd700 1 -- 172.21.15.70:0/2402839542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f06780197b0 con 0x7f067c106540 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c19ae10 con 0x7f067c106540 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0672ffd700 1 -- 172.21.15.70:0/2402839542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f067802d440 con 0x7f067c106540 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.936+0000 7f0672ffd700 1 -- 172.21.15.70:0/2402839542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f06780247c0 con 0x7f067c106540 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.937+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067c0623c0 con 0x7f067c106540 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:52.939+0000 7f0672ffd700 1 -- 172.21.15.70:0/2402839542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f067801ac30 con 0x7f067c106540 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.006+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f067c10a9f0 con 0x7f067c106540 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.006+0000 7f0672ffd700 1 -- 172.21.15.70:0/2402839542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f0678024a20 con 0x7f067c106540 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.007+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 msgr2=0x7f067c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.007+0000 7f068417b700 1 --2- 172.21.15.70:0/2402839542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c19a230 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0678009520 tx=0x7f0678011ab0 comp rx=0 tx=0).stop 2024-04-02T04:37:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.007+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 shutdown_connections 2024-04-02T04:37:54.225 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.007+0000 7f068417b700 1 --2- 172.21.15.70:0/2402839542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c106540 0x7f067c19a230 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:54.225 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.007+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 >> 172.21.15.70:0/2402839542 conn(0x7f067c101b40 msgr2=0x7f067c1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:54.225 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.007+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 shutdown_connections 2024-04-02T04:37:54.225 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:53.007+0000 7f068417b700 1 -- 172.21.15.70:0/2402839542 wait complete. 2024-04-02T04:37:54.225 INFO:teuthology.orchestra.run.smithi070.stdout:Restarting the monitor... 2024-04-02T04:37:54.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:54 smithi070 systemd[1]: Stopping Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:37:54.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:54 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[28822]: 2024-04-02T04:37:54.461+0000 7f47b8352700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi070 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:37:54.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:54 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[28822]: 2024-04-02T04:37:54.461+0000 7f47b8352700 -1 mon.smithi070@0(leader) e1 *** Got Signal Terminated *** 2024-04-02T04:37:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:54 smithi070 podman[29083]: 2024-04-02 04:37:54.472502558 +0000 UTC m=+0.094465676 container died 345c9fc95a8cd057ed85237192893cc5b9377780670dc4e2726b1d6cd7e45330 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, RELEASE=HEAD, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1) 2024-04-02T04:37:55.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:54 smithi070 podman[29083]: 2024-04-02 04:37:54.828538135 +0000 UTC m=+0.450501252 container cleanup 345c9fc95a8cd057ed85237192893cc5b9377780670dc4e2726b1d6cd7e45330 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:37:55.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:54 smithi070 bash[29083]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070 2024-04-02T04:37:55.449 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 podman[29097]: 2024-04-02 04:37:55.121077844 +0000 UTC m=+0.640910297 container remove 345c9fc95a8cd057ed85237192893cc5b9377780670dc4e2726b1d6cd7e45330 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, org.label-schema.license=GPLv2, ceph=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-02T04:37:55.449 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070.service: Deactivated successfully. 2024-04-02T04:37:55.449 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 systemd[1]: Stopped Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:37:55.449 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 systemd[1]: Starting Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:37:55.699 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 podman[29194]: 2024-04-02 04:37:55.446726307 +0000 UTC m=+0.106812752 container create 6fceaaf3da10741c2e3cc8c86ed5c9b2b69b77d6fa89d6c863411f0c3608a0ed (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1) 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 podman[29194]: 2024-04-02 04:37:55.357580102 +0000 UTC m=+0.017666566 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 podman[29194]: 2024-04-02 04:37:55.657785107 +0000 UTC m=+0.317871567 container init 6fceaaf3da10741c2e3cc8c86ed5c9b2b69b77d6fa89d6c863411f0c3608a0ed (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, ceph=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 podman[29194]: 2024-04-02 04:37:55.660883748 +0000 UTC m=+0.320970197 container start 6fceaaf3da10741c2e3cc8c86ed5c9b2b69b77d6fa89d6c863411f0c3608a0ed (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: pidfile_write: ignore empty --pid-file 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: load: jerasure load: lrc 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: RocksDB version: 7.9.2 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Git sha 0 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Compile date 2023-12-11 22:07:34 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: DB SUMMARY 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: DB Session ID: PEOEFKETFCX13ANWMKOI 2024-04-02T04:37:55.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: CURRENT file: CURRENT 2024-04-02T04:37:55.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T04:37:55.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-02T04:37:55.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi070/store.db dir, Total Num: 1, files: 000008.sst 2024-04-02T04:37:55.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi070/store.db: 000009.log size: 98359 ; 2024-04-02T04:37:55.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.error_if_exists: 0 2024-04-02T04:37:55.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.create_if_missing: 0 2024-04-02T04:37:55.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.paranoid_checks: 1 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.env: 0x557ba03e0720 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.info_log: 0x557ba2e712e0 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.statistics: (nil) 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.use_fsync: 0 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_log_file_size: 0 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T04:37:55.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.allow_fallocate: 1 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.use_direct_reads: 0 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.db_log_dir: 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.wal_dir: 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T04:37:55.704 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.write_buffer_manager: 0x557ba20e2140 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.unordered_write: 0 2024-04-02T04:37:55.705 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.row_cache: None 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.wal_filter: None 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.two_write_queues: 0 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.wal_compression: 0 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.atomic_flush: 0 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T04:37:55.706 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.log_readahead_size: 0 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_background_jobs: 2 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_background_compactions: -1 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_subcompactions: 1 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T04:37:55.707 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_open_files: -1 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_background_flushes: -1 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Compression algorithms supported: 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T04:37:55.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: kZSTD supported: 0 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: kXpressCompression supported: 0 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: kZlibCompression supported: 1 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: kSnappyCompression supported: 1 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: kLZ4Compression supported: 1 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: kBZip2Compression supported: 0 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000010 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.merge_operator: 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_filter: None 2024-04-02T04:37:55.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557ba2e713e0) 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_type: 0 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_index_type: 0 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_shortening: 1 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T04:37:55.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: checksum: 4 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: no_block_cache: 0 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache: 0x557ba2183350 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_options: 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: capacity : 536870912 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_shard_bits : 4 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: strict_capacity_limit : 0 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_compressed: (nil) 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: persistent_cache: (nil) 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size: 4096 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size_deviation: 10 2024-04-02T04:37:55.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_restart_interval: 16 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_block_restart_interval: 1 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_block_size: 4096 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: partition_filters: 0 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: use_delta_encoding: 1 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: filter_policy: bloomfilter 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: whole_key_filtering: 1 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: verify_compression: 0 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: format_version: 5 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_index_compression: 1 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_align: 0 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_auto_readahead_size: 262144 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: prepopulate_block_cache: 0 2024-04-02T04:37:55.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: initial_auto_readahead_size: 8192 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression: NoCompression 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.num_levels: 7 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T04:37:55.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T04:37:55.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T04:37:55.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T04:37:55.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.table_properties_collectors: 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.inplace_update_support: 0 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.bloom_locality: 0 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.max_successive_merges: 0 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T04:37:55.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.ttl: 2592000 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.enable_blob_files: false 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.min_blob_size: 0 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T04:37:55.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0c88179a-63b2-4520-a499-6be02674e2c2 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032675689537, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032675691025, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93603, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91625, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84234, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712032675, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0c88179a-63b2-4520-a499-6be02674e2c2", "db_session_id": "PEOEFKETFCX13ANWMKOI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032675691130, "job": 1, "event": "recovery_finished"} 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi070/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T04:37:55.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557ba2220000 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: DB pointer 0x557ba220c000 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** DB Stats ** 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:37:55.720 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L0 2/0 93.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Sum 2/0 93.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 97.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 97.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:37:55.721 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative compaction: 0.00 GB write, 21.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval compaction: 0.00 GB write, 21.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache BinnedLRUCache@0x557ba2183350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: starting mon.smithi070 rank 0 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi070 fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:55.722 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070@-1(???) e1 preinit fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070@-1(???).mds e1 new map 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070@-1(???).mds e1 print_map 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: e1 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: legacy client fscid: -1 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: No filesystems configured 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta expand map: {default=false} 2024-04-02T04:37:55.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta from 'false' to 'false' 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta expanded map: {default=false} 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta expand map: {default=info} 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta from 'info' to 'info' 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta expanded map: {default=info} 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta expand map: {default=daemon} 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta from 'daemon' to 'daemon' 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta expanded map: {default=daemon} 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta expand map: {default=debug} 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta from 'debug' to 'debug' 2024-04-02T04:37:55.724 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: expand_channel_meta expanded map: {default=debug} 2024-04-02T04:37:55.748 INFO:teuthology.orchestra.run.smithi070.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-02T04:37:55.954 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 bash[29194]: 6fceaaf3da10741c2e3cc8c86ed5c9b2b69b77d6fa89d6c863411f0c3608a0ed 2024-04-02T04:37:55.955 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 systemd[1]: Started Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:37:55.955 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mon.smithi070 is new leader, mons smithi070 in quorum (ranks 0) 2024-04-02T04:37:55.955 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: monmap e1: 1 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-02T04:37:55.955 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: fsmap 2024-04-02T04:37:55.955 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T04:37:55.955 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:55 smithi070 ceph-mon[29212]: mgrmap e1: no daemons active 2024-04-02T04:37:56.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.136+0000 7fa46549b700 1 Processor -- start 2024-04-02T04:37:56.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.136+0000 7fa46549b700 1 -- start start 2024-04-02T04:37:56.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.136+0000 7fa46549b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa460105850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:56.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.136+0000 7fa46549b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa460105e20 con 0x7fa460105430 2024-04-02T04:37:56.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.136+0000 7fa45effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa460105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:56.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.136+0000 7fa45effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa460105850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57452/0 (socket says 172.21.15.70:57452) 2024-04-02T04:37:56.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.136+0000 7fa45effd700 1 -- 172.21.15.70:0/2890824393 learned_addr learned my addr 172.21.15.70:0/2890824393 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:56.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.137+0000 7fa45effd700 1 -- 172.21.15.70:0/2890824393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa460106630 con 0x7fa460105430 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.137+0000 7fa45effd700 1 --2- 172.21.15.70:0/2890824393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa460105850 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4500098a0 tx=0x7fa450009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc81ba391fceb3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.137+0000 7fa45dffb700 1 -- 172.21.15.70:0/2890824393 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa450004180 con 0x7fa460105430 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.137+0000 7fa45dffb700 1 -- 172.21.15.70:0/2890824393 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4500042e0 con 0x7fa460105430 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.137+0000 7fa45dffb700 1 -- 172.21.15.70:0/2890824393 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa450011a50 con 0x7fa460105430 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 -- 172.21.15.70:0/2890824393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 msgr2=0x7fa460105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 --2- 172.21.15.70:0/2890824393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa460105850 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4500098a0 tx=0x7fa450009bb0 comp rx=0 tx=0).stop 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 -- 172.21.15.70:0/2890824393 shutdown_connections 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 --2- 172.21.15.70:0/2890824393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa460105850 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 -- 172.21.15.70:0/2890824393 >> 172.21.15.70:0/2890824393 conn(0x7fa4601009b0 msgr2=0x7fa460102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 -- 172.21.15.70:0/2890824393 shutdown_connections 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 -- 172.21.15.70:0/2890824393 wait complete. 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 Processor -- start 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.138+0000 7fa46549b700 1 -- start start 2024-04-02T04:37:56.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa46549b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa46019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa46549b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46019a710 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa45effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa46019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa45effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa46019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57468/0 (socket says 172.21.15.70:57468) 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa45effd700 1 -- 172.21.15.70:0/945285966 learned_addr learned my addr 172.21.15.70:0/945285966 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa45effd700 1 -- 172.21.15.70:0/945285966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa450009550 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa45effd700 1 --2- 172.21.15.70:0/945285966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa46019a1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa450000c00 tx=0x7fa450011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa447fff700 1 -- 172.21.15.70:0/945285966 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa450003b90 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa447fff700 1 -- 172.21.15.70:0/945285966 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4500248b0 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46019a910 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa447fff700 1 -- 172.21.15.70:0/945285966 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa45001a8e0 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.139+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46019adb0 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.140+0000 7fa447fff700 1 -- 172.21.15.70:0/945285966 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa45002d3f0 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.140+0000 7fa447fff700 1 -- 172.21.15.70:0/945285966 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa450019410 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.140+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4600623c0 con 0x7fa460105430 2024-04-02T04:37:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.142+0000 7fa447fff700 1 -- 172.21.15.70:0/945285966 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fa450024a20 con 0x7fa460105430 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.207+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fa4601098e0 con 0x7fa460105430 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.210+0000 7fa447fff700 1 -- 172.21.15.70:0/945285966 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa4500350c0 con 0x7fa460105430 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.211+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 msgr2=0x7fa46019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.211+0000 7fa46549b700 1 --2- 172.21.15.70:0/945285966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa46019a1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa450000c00 tx=0x7fa450011ab0 comp rx=0 tx=0).stop 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.211+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 shutdown_connections 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.211+0000 7fa46549b700 1 --2- 172.21.15.70:0/945285966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa460105430 0x7fa46019a1d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.211+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 >> 172.21.15.70:0/945285966 conn(0x7fa4601009b0 msgr2=0x7fa460079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.211+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 shutdown_connections 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:56.211+0000 7fa46549b700 1 -- 172.21.15.70:0/945285966 wait complete. 2024-04-02T04:37:56.810 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-02T04:37:56.832 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-02T04:37:56.832 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mgr... 2024-04-02T04:37:56.832 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-02T04:37:56.833 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-02T04:37:56.833 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:8443 ... 2024-04-02T04:37:57.137 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mgr.smithi070.igidvr 2024-04-02T04:37:57.138 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mgr.smithi070.igidvr.service: Unit ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mgr.smithi070.igidvr.service not loaded. 2024-04-02T04:37:57.383 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4.target.wants/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mgr.smithi070.igidvr.service → /etc/systemd/system/ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@.service. 2024-04-02T04:37:57.403 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:57 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/945285966' entity='client.admin' 2024-04-02T04:37:58.258 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T04:37:58.259 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-04-02T04:37:58.265 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T04:37:58.265 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-04-02T04:37:58.265 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-04-02T04:37:58.266 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T04:37:58.273 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T04:37:58.273 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-04-02T04:37:58.280 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T04:37:58.280 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-04-02T04:37:58.280 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-04-02T04:37:58.281 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-04-02T04:37:58.281 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr to start... 2024-04-02T04:37:58.281 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr... 2024-04-02T04:37:59.193 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:37:58 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1931006518' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T04:37:59.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:37:59.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:37:59.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "a9031cd8-f0aa-11ee-b647-cb9ed24678a4", 2024-04-02T04:37:59.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T04:37:59.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T04:37:59.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T04:37:59.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T04:37:59.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "smithi070" 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T04:37:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:37:59.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T04:37:59.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:37:59.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T04:37:59.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T04:37:59.551 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T04:37:59.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T04:37:59.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T04:37:49.994006+0000", 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T04:37:59.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.802+0000 7ff03fd41700 1 Processor -- start 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.803+0000 7ff03fd41700 1 -- start start 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.803+0000 7ff03fd41700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff038109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.803+0000 7ff03fd41700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038074640 con 0x7ff038107020 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.803+0000 7ff03dadd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff038109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.803+0000 7ff03dadd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff038109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56000/0 (socket says 172.21.15.70:56000) 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.803+0000 7ff03dadd700 1 -- 172.21.15.70:0/387207840 learned_addr learned my addr 172.21.15.70:0/387207840 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.803+0000 7ff03dadd700 1 -- 172.21.15.70:0/387207840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038109950 con 0x7ff038107020 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.804+0000 7ff03dadd700 1 --2- 172.21.15.70:0/387207840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff038109410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff0340098a0 tx=0x7ff034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcba0c15c234070 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.804+0000 7ff03cadb700 1 -- 172.21.15.70:0/387207840 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff034004180 con 0x7ff038107020 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.804+0000 7ff03cadb700 1 -- 172.21.15.70:0/387207840 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0340042e0 con 0x7ff038107020 2024-04-02T04:37:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.804+0000 7ff03cadb700 1 -- 172.21.15.70:0/387207840 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff034011a50 con 0x7ff038107020 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 -- 172.21.15.70:0/387207840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 msgr2=0x7ff038109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 --2- 172.21.15.70:0/387207840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff038109410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff0340098a0 tx=0x7ff034009bb0 comp rx=0 tx=0).stop 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 -- 172.21.15.70:0/387207840 shutdown_connections 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 --2- 172.21.15.70:0/387207840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff038109410 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 -- 172.21.15.70:0/387207840 >> 172.21.15.70:0/387207840 conn(0x7ff0381009f0 msgr2=0x7ff038102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 -- 172.21.15.70:0/387207840 shutdown_connections 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 -- 172.21.15.70:0/387207840 wait complete. 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 Processor -- start 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 -- start start 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff03819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.805+0000 7ff03fd41700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819a680 con 0x7ff038107020 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff03dadd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff03819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff03dadd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff03819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56004/0 (socket says 172.21.15.70:56004) 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff03dadd700 1 -- 172.21.15.70:0/1931006518 learned_addr learned my addr 172.21.15.70:0/1931006518 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff03dadd700 1 -- 172.21.15.70:0/1931006518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff034009550 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff03dadd700 1 --2- 172.21.15.70:0/1931006518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff03819a140 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff0340119e0 tx=0x7ff034011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff02effd700 1 -- 172.21.15.70:0/1931006518 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff034003b90 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff02effd700 1 -- 172.21.15.70:0/1931006518 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff03401a8a0 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03819a880 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff02effd700 1 -- 172.21.15.70:0/1931006518 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff034019930 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03819ad20 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.806+0000 7ff02effd700 1 -- 172.21.15.70:0/1931006518 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff03402d440 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.807+0000 7ff02effd700 1 -- 172.21.15.70:0/1931006518 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff0340248f0 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.807+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0380623c0 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.810+0000 7ff02effd700 1 -- 172.21.15.70:0/1931006518 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ff03401aa10 con 0x7ff038107020 2024-04-02T04:37:59.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.881+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff03819d6e0 con 0x7ff038107020 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.881+0000 7ff02effd700 1 -- 172.21.15.70:0/1931006518 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7ff03401abb0 con 0x7ff038107020 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.882+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 msgr2=0x7ff03819a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.882+0000 7ff03fd41700 1 --2- 172.21.15.70:0/1931006518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff03819a140 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff0340119e0 tx=0x7ff034011ac0 comp rx=0 tx=0).stop 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.882+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 shutdown_connections 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.882+0000 7ff03fd41700 1 --2- 172.21.15.70:0/1931006518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff038107020 0x7ff03819a140 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.882+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 >> 172.21.15.70:0/1931006518 conn(0x7ff0381009f0 msgr2=0x7ff038101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.882+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 shutdown_connections 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:37:58.882+0000 7ff03fd41700 1 -- 172.21.15.70:0/1931006518 wait complete. 2024-04-02T04:37:59.565 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (1/15)... 2024-04-02T04:38:02.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:02 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/235173527' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T04:38:02.696 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:38:02.696 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:38:02.696 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "a9031cd8-f0aa-11ee-b647-cb9ed24678a4", 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "smithi070" 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T04:38:02.697 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:02.698 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-02T04:38:02.698 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T04:38:02.698 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:02.698 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T04:38:02.698 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:02.698 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T04:38:02.698 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T04:38:02.698 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T04:38:02.706 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T04:38:02.707 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T04:37:49.994006+0000", 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.977+0000 7f164a3c3700 1 Processor -- start 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f164a3c3700 1 -- start start 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f164a3c3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f16441093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:02.708 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f164a3c3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644074680 con 0x7f1644106fe0 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f1643fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f16441093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f1643fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f16441093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56018/0 (socket says 172.21.15.70:56018) 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f1643fff700 1 -- 172.21.15.70:0/888564876 learned_addr learned my addr 172.21.15.70:0/888564876 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f1643fff700 1 -- 172.21.15.70:0/888564876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644109910 con 0x7f1644106fe0 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f1643fff700 1 --2- 172.21.15.70:0/888564876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f16441093d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16340098a0 tx=0x7f1634009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31adc7d4497e372f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.978+0000 7f16437fe700 1 -- 172.21.15.70:0/888564876 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1634004180 con 0x7f1644106fe0 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f16437fe700 1 -- 172.21.15.70:0/888564876 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16340042e0 con 0x7f1644106fe0 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f16437fe700 1 -- 172.21.15.70:0/888564876 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1634011a50 con 0x7f1644106fe0 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f164a3c3700 1 -- 172.21.15.70:0/888564876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 msgr2=0x7f16441093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f164a3c3700 1 --2- 172.21.15.70:0/888564876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f16441093d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16340098a0 tx=0x7f1634009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f164a3c3700 1 -- 172.21.15.70:0/888564876 shutdown_connections 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f164a3c3700 1 --2- 172.21.15.70:0/888564876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f16441093d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f164a3c3700 1 -- 172.21.15.70:0/888564876 >> 172.21.15.70:0/888564876 conn(0x7f16441009f0 msgr2=0x7f1644102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:02.709 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f164a3c3700 1 -- 172.21.15.70:0/888564876 shutdown_connections 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.979+0000 7f164a3c3700 1 -- 172.21.15.70:0/888564876 wait complete. 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.980+0000 7f164a3c3700 1 Processor -- start 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.980+0000 7f164a3c3700 1 -- start start 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.980+0000 7f164a3c3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f164419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.980+0000 7f164a3c3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164419c8c0 con 0x7f1644106fe0 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.980+0000 7f1643fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f164419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.980+0000 7f1643fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f164419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56026/0 (socket says 172.21.15.70:56026) 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.980+0000 7f1643fff700 1 -- 172.21.15.70:0/235173527 learned_addr learned my addr 172.21.15.70:0/235173527 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f1643fff700 1 -- 172.21.15.70:0/235173527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634009550 con 0x7f1644106fe0 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f1643fff700 1 --2- 172.21.15.70:0/235173527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f164419c380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1634000c00 tx=0x7f1634011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f1641ffb700 1 -- 172.21.15.70:0/235173527 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1634003b90 con 0x7f1644106fe0 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f1641ffb700 1 -- 172.21.15.70:0/235173527 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16340248b0 con 0x7f1644106fe0 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164419cac0 con 0x7f1644106fe0 2024-04-02T04:38:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f1641ffb700 1 -- 172.21.15.70:0/235173527 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f163401a8e0 con 0x7f1644106fe0 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164419cf60 con 0x7f1644106fe0 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f1641ffb700 1 -- 172.21.15.70:0/235173527 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f163402d3f0 con 0x7f1644106fe0 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.981+0000 7f1641ffb700 1 -- 172.21.15.70:0/235173527 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f16340194a0 con 0x7f1644106fe0 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.982+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16440623c0 con 0x7f1644106fe0 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:01.984+0000 7f1641ffb700 1 -- 172.21.15.70:0/235173527 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f1634024db0 con 0x7f1644106fe0 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.057+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f164419f8d0 con 0x7f1644106fe0 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.057+0000 7f1641ffb700 1 -- 172.21.15.70:0/235173527 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f1634035090 con 0x7f1644106fe0 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.058+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 msgr2=0x7f164419c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.058+0000 7f164a3c3700 1 --2- 172.21.15.70:0/235173527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f164419c380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1634000c00 tx=0x7f1634011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.058+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 shutdown_connections 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.058+0000 7f164a3c3700 1 --2- 172.21.15.70:0/235173527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1644106fe0 0x7f164419c380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.058+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 >> 172.21.15.70:0/235173527 conn(0x7f16441009f0 msgr2=0x7f1644101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.058+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 shutdown_connections 2024-04-02T04:38:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:02.058+0000 7f164a3c3700 1 -- 172.21.15.70:0/235173527 wait complete. 2024-04-02T04:38:02.712 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (2/15)... 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: Activating manager daemon smithi070.igidvr 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: mgrmap e2: smithi070.igidvr(active, starting, since 0.00299214s) 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: Manager daemon smithi070.igidvr is now available 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:38:05.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/trash_purge_schedule"}]: dispatch 2024-04-02T04:38:05.145 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:05.145 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:05.145 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:04 smithi070 ceph-mon[29212]: from='mgr.14100 172.21.15.70:0/1227882294' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "a9031cd8-f0aa-11ee-b647-cb9ed24678a4", 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "smithi070" 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T04:38:05.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T04:38:05.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T04:38:05.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T04:37:49.994006+0000", 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.161+0000 7f0ec4915700 1 Processor -- start 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.161+0000 7f0ec4915700 1 -- start start 2024-04-02T04:38:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.161+0000 7f0ec4915700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec01069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.161+0000 7f0ec4915700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0106f70 con 0x7f0ec0106580 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.161+0000 7f0ebe59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec01069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.161+0000 7f0ebe59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec01069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56108/0 (socket says 172.21.15.70:56108) 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.161+0000 7f0ebe59c700 1 -- 172.21.15.70:0/549221242 learned_addr learned my addr 172.21.15.70:0/549221242 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.161+0000 7f0ebe59c700 1 -- 172.21.15.70:0/549221242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0107780 con 0x7f0ec0106580 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ebe59c700 1 --2- 172.21.15.70:0/549221242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec01069a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009e80 tx=0x7f0eb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f691362b94b536e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ebd59a700 1 -- 172.21.15.70:0/549221242 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0eb800e070 con 0x7f0ec0106580 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ebd59a700 1 -- 172.21.15.70:0/549221242 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0eb8003e90 con 0x7f0ec0106580 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ebd59a700 1 -- 172.21.15.70:0/549221242 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0eb8004380 con 0x7f0ec0106580 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ec4915700 1 -- 172.21.15.70:0/549221242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 msgr2=0x7f0ec01069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ec4915700 1 --2- 172.21.15.70:0/549221242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec01069a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009e80 tx=0x7f0eb800b330 comp rx=0 tx=0).stop 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ec4915700 1 -- 172.21.15.70:0/549221242 shutdown_connections 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ec4915700 1 --2- 172.21.15.70:0/549221242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec01069a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:05.848 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ec4915700 1 -- 172.21.15.70:0/549221242 >> 172.21.15.70:0/549221242 conn(0x7f0ec0101b00 msgr2=0x7f0ec0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ec4915700 1 -- 172.21.15.70:0/549221242 shutdown_connections 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.162+0000 7f0ec4915700 1 -- 172.21.15.70:0/549221242 wait complete. 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.163+0000 7f0ec4915700 1 Processor -- start 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.163+0000 7f0ec4915700 1 -- start start 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.163+0000 7f0ec4915700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec007bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.163+0000 7f0ec4915700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec007c170 con 0x7f0ec0106580 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.163+0000 7f0ebe59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec007bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.163+0000 7f0ebe59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec007bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56120/0 (socket says 172.21.15.70:56120) 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.163+0000 7f0ebe59c700 1 -- 172.21.15.70:0/1588578924 learned_addr learned my addr 172.21.15.70:0/1588578924 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.164+0000 7f0ebe59c700 1 -- 172.21.15.70:0/1588578924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb8009b30 con 0x7f0ec0106580 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.164+0000 7f0ebe59c700 1 --2- 172.21.15.70:0/1588578924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec007bc30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009e50 tx=0x7f0eb8004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.164+0000 7f0eaf7fe700 1 -- 172.21.15.70:0/1588578924 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0eb800e040 con 0x7f0ec0106580 2024-04-02T04:38:05.849 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.164+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec007a280 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.164+0000 7f0eaf7fe700 1 -- 172.21.15.70:0/1588578924 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0eb8003cf0 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.164+0000 7f0eaf7fe700 1 -- 172.21.15.70:0/1588578924 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0eb801b770 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.164+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec007a780 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.165+0000 7f0eaf7fe700 1 -- 172.21.15.70:0/1588578924 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 2) v1 ==== 45039+0+0 (secure 0 0 0) 0x7f0eb8019040 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.165+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec00623c0 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.165+0000 7f0eaf7fe700 1 -- 172.21.15.70:0/1588578924 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0eb804b830 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.167+0000 7f0eaf7fe700 1 -- 172.21.15.70:0/1588578924 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f0eb801bad0 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.236+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0ec010aa00 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.236+0000 7f0eaf7fe700 1 -- 172.21.15.70:0/1588578924 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f0eb801f4f0 con 0x7f0ec0106580 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.237+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 msgr2=0x7f0ec007bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.237+0000 7f0ec4915700 1 --2- 172.21.15.70:0/1588578924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec007bc30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009e50 tx=0x7f0eb8004920 comp rx=0 tx=0).stop 2024-04-02T04:38:05.850 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.237+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 shutdown_connections 2024-04-02T04:38:05.851 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.237+0000 7f0ec4915700 1 --2- 172.21.15.70:0/1588578924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec0106580 0x7f0ec007bc30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:05.851 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.237+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 >> 172.21.15.70:0/1588578924 conn(0x7f0ec0101b00 msgr2=0x7f0ec01028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:05.851 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.237+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 shutdown_connections 2024-04-02T04:38:05.851 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:05.237+0000 7f0ec4915700 1 -- 172.21.15.70:0/1588578924 wait complete. 2024-04-02T04:38:05.851 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (3/15)... 2024-04-02T04:38:06.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:05 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1588578924' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T04:38:06.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:05 smithi070 ceph-mon[29212]: mgrmap e3: smithi070.igidvr(active, since 1.00689s) 2024-04-02T04:38:08.183 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:07 smithi070 ceph-mon[29212]: mgrmap e4: smithi070.igidvr(active, since 2s) 2024-04-02T04:38:09.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:08 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2846381812' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T04:38:09.173 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:38:09.173 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:38:09.173 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "a9031cd8-f0aa-11ee-b647-cb9ed24678a4", 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "smithi070" 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T04:38:09.175 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T04:38:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T04:38:09.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T04:37:49.994006+0000", 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:38:09.186 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.285+0000 7f1112e13700 1 Processor -- start 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.285+0000 7f1112e13700 1 -- start start 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.285+0000 7f1112e13700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.285+0000 7f1112e13700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c106f50 con 0x7f110c106560 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.286+0000 7f1110baf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.286+0000 7f1110baf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33042/0 (socket says 172.21.15.70:33042) 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.286+0000 7f1110baf700 1 -- 172.21.15.70:0/13149668 learned_addr learned my addr 172.21.15.70:0/13149668 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.286+0000 7f1110baf700 1 -- 172.21.15.70:0/13149668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c107760 con 0x7f110c106560 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.286+0000 7f1110baf700 1 --2- 172.21.15.70:0/13149668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c106980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f11080098a0 tx=0x7f1108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5c856b010825e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.286+0000 7f11037fe700 1 -- 172.21.15.70:0/13149668 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1108004180 con 0x7f110c106560 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.286+0000 7f11037fe700 1 -- 172.21.15.70:0/13149668 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11080042e0 con 0x7f110c106560 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.286+0000 7f11037fe700 1 -- 172.21.15.70:0/13149668 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1108011a50 con 0x7f110c106560 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 -- 172.21.15.70:0/13149668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 msgr2=0x7f110c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:09.187 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 --2- 172.21.15.70:0/13149668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c106980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f11080098a0 tx=0x7f1108009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 -- 172.21.15.70:0/13149668 shutdown_connections 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 --2- 172.21.15.70:0/13149668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c106980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 -- 172.21.15.70:0/13149668 >> 172.21.15.70:0/13149668 conn(0x7f110c101b40 msgr2=0x7f110c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 -- 172.21.15.70:0/13149668 shutdown_connections 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 -- 172.21.15.70:0/13149668 wait complete. 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 Processor -- start 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.287+0000 7f1112e13700 1 -- start start 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1112e13700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c07bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1112e13700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c07c170 con 0x7f110c106560 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1110baf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c07bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1110baf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c07bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33050/0 (socket says 172.21.15.70:33050) 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1110baf700 1 -- 172.21.15.70:0/2846381812 learned_addr learned my addr 172.21.15.70:0/2846381812 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1110baf700 1 -- 172.21.15.70:0/2846381812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1108009550 con 0x7f110c106560 2024-04-02T04:38:09.188 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1110baf700 1 --2- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c07bc30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1108003de0 tx=0x7f1108011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1101ffb700 1 -- 172.21.15.70:0/2846381812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1108003930 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1101ffb700 1 -- 172.21.15.70:0/2846381812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1108003a90 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c07a280 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.288+0000 7f1101ffb700 1 -- 172.21.15.70:0/2846381812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11080196c0 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.289+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c07a720 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.289+0000 7f1101ffb700 1 -- 172.21.15.70:0/2846381812 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f1108019930 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.289+0000 7f1101ffb700 1 --2- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f10f4038320 0x7f10f403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.290+0000 7f1101ffb700 1 -- 172.21.15.70:0/2846381812 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f110804d090 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.290+0000 7f1103fff700 1 --2- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f10f4038320 0x7f10f403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.290+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c0623c0 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.290+0000 7f1103fff700 1 --2- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f10f4038320 0x7f10f403a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f10fc006fd0 tx=0x7f10fc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.295+0000 7f1101ffb700 1 -- 172.21.15.70:0/2846381812 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1108019be0 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.562+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f110c07b130 con 0x7f110c106560 2024-04-02T04:38:09.189 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.562+0000 7f1101ffb700 1 -- 172.21.15.70:0/2846381812 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f1108010b70 con 0x7f110c106560 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f10f4038320 msgr2=0x7f10f403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 --2- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f10f4038320 0x7f10f403a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f10fc006fd0 tx=0x7f10fc006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 msgr2=0x7f110c07bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 --2- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c07bc30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1108003de0 tx=0x7f1108011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 shutdown_connections 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 --2- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f10f4038320 0x7f10f403a7e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 --2- 172.21.15.70:0/2846381812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c106560 0x7f110c07bc30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 >> 172.21.15.70:0/2846381812 conn(0x7f110c101b40 msgr2=0x7f110c102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 shutdown_connections 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:08.565+0000 7f1112e13700 1 -- 172.21.15.70:0/2846381812 wait complete. 2024-04-02T04:38:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:mgr is available 2024-04-02T04:38:10.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:09 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/826343232' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T04:38:10.384 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:38:10.384 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T04:38:10.384 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:38:10.384 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T04:38:10.384 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T04:38:10.384 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T04:38:10.384 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T04:38:10.384 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.588+0000 7f67b7de2700 1 Processor -- start 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b7de2700 1 -- start start 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b7de2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b0106960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b7de2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0106f30 con 0x7f67b0106540 2024-04-02T04:38:10.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b5b7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b0106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b5b7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b0106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33058/0 (socket says 172.21.15.70:33058) 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b5b7e700 1 -- 172.21.15.70:0/411379052 learned_addr learned my addr 172.21.15.70:0/411379052 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b5b7e700 1 -- 172.21.15.70:0/411379052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0107740 con 0x7f67b0106540 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b5b7e700 1 --2- 172.21.15.70:0/411379052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b0106960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0098a0 tx=0x7f67ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590ab9ee3a402c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.589+0000 7f67b4b7c700 1 -- 172.21.15.70:0/411379052 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67ac004180 con 0x7f67b0106540 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b4b7c700 1 -- 172.21.15.70:0/411379052 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67ac0042e0 con 0x7f67b0106540 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b4b7c700 1 -- 172.21.15.70:0/411379052 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67ac011a50 con 0x7f67b0106540 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b7de2700 1 -- 172.21.15.70:0/411379052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 msgr2=0x7f67b0106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b7de2700 1 --2- 172.21.15.70:0/411379052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b0106960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0098a0 tx=0x7f67ac009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b7de2700 1 -- 172.21.15.70:0/411379052 shutdown_connections 2024-04-02T04:38:10.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b7de2700 1 --2- 172.21.15.70:0/411379052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b0106960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b7de2700 1 -- 172.21.15.70:0/411379052 >> 172.21.15.70:0/411379052 conn(0x7f67b0101b40 msgr2=0x7f67b0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b7de2700 1 -- 172.21.15.70:0/411379052 shutdown_connections 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.590+0000 7f67b7de2700 1 -- 172.21.15.70:0/411379052 wait complete. 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b7de2700 1 Processor -- start 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b7de2700 1 -- start start 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b7de2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b007bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b7de2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b007c170 con 0x7f67b0106540 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b5b7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b007bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b5b7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b007bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33066/0 (socket says 172.21.15.70:33066) 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b5b7e700 1 -- 172.21.15.70:0/826343232 learned_addr learned my addr 172.21.15.70:0/826343232 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b5b7e700 1 -- 172.21.15.70:0/826343232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac009550 con 0x7f67b0106540 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.591+0000 7f67b5b7e700 1 --2- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b007bc30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009520 tx=0x7f67ac011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:10.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.592+0000 7f67a6ffd700 1 -- 172.21.15.70:0/826343232 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67ac003a80 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.592+0000 7f67a6ffd700 1 -- 172.21.15.70:0/826343232 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67ac003be0 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.592+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b007a280 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.592+0000 7f67a6ffd700 1 -- 172.21.15.70:0/826343232 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67ac0197b0 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.592+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b007a720 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.592+0000 7f67a6ffd700 1 -- 172.21.15.70:0/826343232 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f67ac019a70 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.593+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b00623c0 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.593+0000 7f67a6ffd700 1 --2- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f679c038280 0x7f679c03a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.593+0000 7f67a6ffd700 1 -- 172.21.15.70:0/826343232 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f67ac04c6c0 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.593+0000 7f67b537d700 1 --2- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f679c038280 0x7f679c03a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.593+0000 7f67b537d700 1 --2- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f679c038280 0x7f679c03a740 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.597+0000 7f67a6ffd700 1 -- 172.21.15.70:0/826343232 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f67ac019d20 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.785+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f67b010a9c0 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.786+0000 7f67a6ffd700 1 -- 172.21.15.70:0/826343232 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f67ac023390 con 0x7f67b0106540 2024-04-02T04:38:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f679c038280 msgr2=0x7f679c03a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 --2- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f679c038280 0x7f679c03a740 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 msgr2=0x7f67b007bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 --2- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b007bc30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009520 tx=0x7f67ac011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 shutdown_connections 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 --2- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f679c038280 0x7f679c03a740 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 --2- 172.21.15.70:0/826343232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b0106540 0x7f67b007bc30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 >> 172.21.15.70:0/826343232 conn(0x7f67b0101b40 msgr2=0x7f67b01031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 shutdown_connections 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:09.788+0000 7f67b7de2700 1 -- 172.21.15.70:0/826343232 wait complete. 2024-04-02T04:38:10.389 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling cephadm module... 2024-04-02T04:38:11.971 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:11 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/427315615' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-02T04:38:12.291 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.772+0000 7f6d59467700 1 Processor -- start 2024-04-02T04:38:12.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.772+0000 7f6d59467700 1 -- start start 2024-04-02T04:38:12.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.772+0000 7f6d59467700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d54105250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:12.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.772+0000 7f6d59467700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d54074620 con 0x7f6d54104e30 2024-04-02T04:38:12.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.772+0000 7f6d52ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d54105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:12.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.772+0000 7f6d52ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d54105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33078/0 (socket says 172.21.15.70:33078) 2024-04-02T04:38:12.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.772+0000 7f6d52ffd700 1 -- 172.21.15.70:0/1214943717 learned_addr learned my addr 172.21.15.70:0/1214943717 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:12.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.772+0000 7f6d52ffd700 1 -- 172.21.15.70:0/1214943717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d54105790 con 0x7f6d54104e30 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d52ffd700 1 --2- 172.21.15.70:0/1214943717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d54105250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6d440098a0 tx=0x7f6d44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184a4a1809640d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d51ffb700 1 -- 172.21.15.70:0/1214943717 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d44004180 con 0x7f6d54104e30 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d51ffb700 1 -- 172.21.15.70:0/1214943717 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6d440042e0 con 0x7f6d54104e30 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d51ffb700 1 -- 172.21.15.70:0/1214943717 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d44011a50 con 0x7f6d54104e30 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d59467700 1 -- 172.21.15.70:0/1214943717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 msgr2=0x7f6d54105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d59467700 1 --2- 172.21.15.70:0/1214943717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d54105250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6d440098a0 tx=0x7f6d44009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d59467700 1 -- 172.21.15.70:0/1214943717 shutdown_connections 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d59467700 1 --2- 172.21.15.70:0/1214943717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d54105250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d59467700 1 -- 172.21.15.70:0/1214943717 >> 172.21.15.70:0/1214943717 conn(0x7f6d541009f0 msgr2=0x7f6d54102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d59467700 1 -- 172.21.15.70:0/1214943717 shutdown_connections 2024-04-02T04:38:12.298 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.773+0000 7f6d59467700 1 -- 172.21.15.70:0/1214943717 wait complete. 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.774+0000 7f6d59467700 1 Processor -- start 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.774+0000 7f6d59467700 1 -- start start 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.774+0000 7f6d59467700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d5419c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.774+0000 7f6d59467700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5419c8b0 con 0x7f6d54104e30 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.774+0000 7f6d52ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d5419c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.774+0000 7f6d52ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d5419c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33092/0 (socket says 172.21.15.70:33092) 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.774+0000 7f6d52ffd700 1 -- 172.21.15.70:0/427315615 learned_addr learned my addr 172.21.15.70:0/427315615 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.775+0000 7f6d52ffd700 1 -- 172.21.15.70:0/427315615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44009550 con 0x7f6d54104e30 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.775+0000 7f6d52ffd700 1 --2- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d5419c370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6d44000c00 tx=0x7f6d44011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.775+0000 7f6d3bfff700 1 -- 172.21.15.70:0/427315615 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d44003b90 con 0x7f6d54104e30 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.775+0000 7f6d3bfff700 1 -- 172.21.15.70:0/427315615 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6d440248b0 con 0x7f6d54104e30 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.775+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5419cab0 con 0x7f6d54104e30 2024-04-02T04:38:12.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.775+0000 7f6d3bfff700 1 -- 172.21.15.70:0/427315615 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d4401a8e0 con 0x7f6d54104e30 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.775+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5419cf50 con 0x7f6d54104e30 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.776+0000 7f6d3bfff700 1 -- 172.21.15.70:0/427315615 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f6d4401aba0 con 0x7f6d54104e30 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.776+0000 7f6d3bfff700 1 --2- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f6d3c0382d0 0x7f6d3c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.776+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d54195be0 con 0x7f6d54104e30 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.776+0000 7f6d3bfff700 1 -- 172.21.15.70:0/427315615 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6d4404c090 con 0x7f6d54104e30 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.776+0000 7f6d527fc700 1 --2- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f6d3c0382d0 0x7f6d3c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.776+0000 7f6d527fc700 1 --2- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f6d3c0382d0 0x7f6d3c03a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d48006fd0 tx=0x7f6d48006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:10.781+0000 7f6d3bfff700 1 -- 172.21.15.70:0/427315615 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6d440233a0 con 0x7f6d54104e30 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.023+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6d5402cc70 con 0x7f6d54104e30 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.703+0000 7f6d3bfff700 1 -- 172.21.15.70:0/427315615 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f6d44018350 con 0x7f6d54104e30 2024-04-02T04:38:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.705+0000 7f6d3bfff700 1 -- 172.21.15.70:0/427315615 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6d4404bdf0 con 0x7f6d54104e30 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.707+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f6d3c0382d0 msgr2=0x7f6d3c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.707+0000 7f6d59467700 1 --2- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f6d3c0382d0 0x7f6d3c03a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d48006fd0 tx=0x7f6d48006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.707+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 msgr2=0x7f6d5419c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.707+0000 7f6d59467700 1 --2- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d5419c370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6d44000c00 tx=0x7f6d44011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.707+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 shutdown_connections 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.707+0000 7f6d59467700 1 --2- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7f6d3c0382d0 0x7f6d3c03a790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.707+0000 7f6d59467700 1 --2- 172.21.15.70:0/427315615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d54104e30 0x7f6d5419c370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.707+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 >> 172.21.15.70:0/427315615 conn(0x7f6d541009f0 msgr2=0x7f6d541909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.708+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 shutdown_connections 2024-04-02T04:38:12.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:11.708+0000 7f6d59467700 1 -- 172.21.15.70:0/427315615 wait complete. 2024-04-02T04:38:12.948 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:12 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/427315615' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-02T04:38:12.948 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:12 smithi070 ceph-mon[29212]: mgrmap e5: smithi070.igidvr(active, since 6s) 2024-04-02T04:38:13.637 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "smithi070.igidvr", 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.739+0000 7fb015741700 1 Processor -- start 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.739+0000 7fb015741700 1 -- start start 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.739+0000 7fb015741700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb010105250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.739+0000 7fb015741700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb010074620 con 0x7fb010104e30 2024-04-02T04:38:13.638 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.739+0000 7fb00effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb010105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.739+0000 7fb00effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb010105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33112/0 (socket says 172.21.15.70:33112) 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.739+0000 7fb00effd700 1 -- 172.21.15.70:0/607781812 learned_addr learned my addr 172.21.15.70:0/607781812 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.739+0000 7fb00effd700 1 -- 172.21.15.70:0/607781812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010105790 con 0x7fb010104e30 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb00effd700 1 --2- 172.21.15.70:0/607781812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb010105250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0000098a0 tx=0x7fb000009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9459cf74d720f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb00dffb700 1 -- 172.21.15.70:0/607781812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb000004180 con 0x7fb010104e30 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb00dffb700 1 -- 172.21.15.70:0/607781812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0000042e0 con 0x7fb010104e30 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb00dffb700 1 -- 172.21.15.70:0/607781812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb000011a50 con 0x7fb010104e30 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb015741700 1 -- 172.21.15.70:0/607781812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 msgr2=0x7fb010105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb015741700 1 --2- 172.21.15.70:0/607781812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb010105250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0000098a0 tx=0x7fb000009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb015741700 1 -- 172.21.15.70:0/607781812 shutdown_connections 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb015741700 1 --2- 172.21.15.70:0/607781812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb010105250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.740+0000 7fb015741700 1 -- 172.21.15.70:0/607781812 >> 172.21.15.70:0/607781812 conn(0x7fb0101009f0 msgr2=0x7fb010102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.741+0000 7fb015741700 1 -- 172.21.15.70:0/607781812 shutdown_connections 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.741+0000 7fb015741700 1 -- 172.21.15.70:0/607781812 wait complete. 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.741+0000 7fb015741700 1 Processor -- start 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7fb015741700 1 -- start start 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7fb015741700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb01019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7fb015741700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01019c8b0 con 0x7fb010104e30 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7fb00effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb01019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7fb00effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb01019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33122/0 (socket says 172.21.15.70:33122) 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7fb00effd700 1 -- 172.21.15.70:0/793131151 learned_addr learned my addr 172.21.15.70:0/793131151 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7fb00effd700 1 -- 172.21.15.70:0/793131151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000009550 con 0x7fb010104e30 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7fb00effd700 1 --2- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb01019c370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb0000119e0 tx=0x7fb000011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7faff7fff700 1 -- 172.21.15.70:0/793131151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb000003b90 con 0x7fb010104e30 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.742+0000 7faff7fff700 1 -- 172.21.15.70:0/793131151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb00001a8a0 con 0x7fb010104e30 2024-04-02T04:38:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.743+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01019cab0 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.743+0000 7faff7fff700 1 -- 172.21.15.70:0/793131151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb000019930 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.743+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01019cf50 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.744+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb010195be0 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.744+0000 7faff7fff700 1 -- 172.21.15.70:0/793131151 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb00001aa10 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.744+0000 7faff7fff700 1 --2- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7faff8038330 0x7faff803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.744+0000 7faff7fff700 1 -- 172.21.15.70:0/793131151 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb00004c6b0 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.744+0000 7fb00e7fc700 1 -- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7faff8038330 msgr2=0x7faff803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/1197360681 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.744+0000 7fb00e7fc700 1 --2- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7faff8038330 0x7faff803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.749+0000 7faff7fff700 1 -- 172.21.15.70:0/793131151 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb000018530 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.945+0000 7fb00e7fc700 1 -- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7faff8038330 msgr2=0x7faff803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/1197360681 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:12.945+0000 7fb00e7fc700 1 --2- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7faff8038330 0x7faff803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.032+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb01002cc70 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.032+0000 7faff7fff700 1 -- 172.21.15.70:0/793131151 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fb000018350 con 0x7fb010104e30 2024-04-02T04:38:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7faff8038330 msgr2=0x7faff803a7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:38:13.642 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 --2- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7faff8038330 0x7faff803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:13.642 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 msgr2=0x7fb01019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:13.642 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 --2- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb01019c370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb0000119e0 tx=0x7fb000011ac0 comp rx=0 tx=0).stop 2024-04-02T04:38:13.642 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 shutdown_connections 2024-04-02T04:38:13.642 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 --2- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7faff8038330 0x7faff803a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:13.642 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 --2- 172.21.15.70:0/793131151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb010104e30 0x7fb01019c370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:13.642 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 >> 172.21.15.70:0/793131151 conn(0x7fb0101009f0 msgr2=0x7fb0101909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:13.643 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 shutdown_connections 2024-04-02T04:38:13.643 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:13.035+0000 7fb015741700 1 -- 172.21.15.70:0/793131151 wait complete. 2024-04-02T04:38:13.643 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2024-04-02T04:38:13.643 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 5... 2024-04-02T04:38:13.885 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:13 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/793131151' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T04:38:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: Active manager daemon smithi070.igidvr restarted 2024-04-02T04:38:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: Activating manager daemon smithi070.igidvr 2024-04-02T04:38:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: osdmap e2: 0 total, 0 up, 0 in 2024-04-02T04:38:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: mgrmap e6: smithi070.igidvr(active, starting, since 0.00324314s) 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: Manager daemon smithi070.igidvr is now available 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:38:18.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:18 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/trash_purge_schedule"}]: dispatch 2024-04-02T04:38:19.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:19 smithi070 ceph-mon[29212]: Found migration_current of "None". Setting to last migration. 2024-04-02T04:38:19.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:19 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:19.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:19 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:19.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:19 smithi070 ceph-mon[29212]: mgrmap e7: smithi070.igidvr(active, since 1.00531s) 2024-04-02T04:38:19.709 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:19 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:19.978 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:38:19.978 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-02T04:38:19.978 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.113+0000 7fa3aa061700 1 Processor -- start 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.113+0000 7fa3aa061700 1 -- start start 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.113+0000 7fa3aa061700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a4105230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.113+0000 7fa3aa061700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4074640 con 0x7fa3a4104e10 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.114+0000 7fa3a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a4105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.114+0000 7fa3a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a4105230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33132/0 (socket says 172.21.15.70:33132) 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.114+0000 7fa3a37fe700 1 -- 172.21.15.70:0/1586948952 learned_addr learned my addr 172.21.15.70:0/1586948952 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.114+0000 7fa3a37fe700 1 -- 172.21.15.70:0/1586948952 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4105770 con 0x7fa3a4104e10 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.114+0000 7fa3a37fe700 1 --2- 172.21.15.70:0/1586948952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a4105230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa3980098a0 tx=0x7fa398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cc09d0f8027700 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.114+0000 7fa3a27fc700 1 -- 172.21.15.70:0/1586948952 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa398004180 con 0x7fa3a4104e10 2024-04-02T04:38:19.979 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.114+0000 7fa3a27fc700 1 -- 172.21.15.70:0/1586948952 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3980042e0 con 0x7fa3a4104e10 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.114+0000 7fa3a27fc700 1 -- 172.21.15.70:0/1586948952 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa398011a50 con 0x7fa3a4104e10 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.115+0000 7fa3aa061700 1 -- 172.21.15.70:0/1586948952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 msgr2=0x7fa3a4105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.115+0000 7fa3aa061700 1 --2- 172.21.15.70:0/1586948952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a4105230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa3980098a0 tx=0x7fa398009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.115+0000 7fa3aa061700 1 -- 172.21.15.70:0/1586948952 shutdown_connections 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.115+0000 7fa3aa061700 1 --2- 172.21.15.70:0/1586948952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a4105230 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.115+0000 7fa3aa061700 1 -- 172.21.15.70:0/1586948952 >> 172.21.15.70:0/1586948952 conn(0x7fa3a41009f0 msgr2=0x7fa3a4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.115+0000 7fa3aa061700 1 -- 172.21.15.70:0/1586948952 shutdown_connections 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.115+0000 7fa3aa061700 1 -- 172.21.15.70:0/1586948952 wait complete. 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.116+0000 7fa3aa061700 1 Processor -- start 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.116+0000 7fa3aa061700 1 -- start start 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.116+0000 7fa3aa061700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:19.980 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.116+0000 7fa3aa061700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a419c8c0 con 0x7fa3a4104e10 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.116+0000 7fa3a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.116+0000 7fa3a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33138/0 (socket says 172.21.15.70:33138) 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.116+0000 7fa3a37fe700 1 -- 172.21.15.70:0/3445757363 learned_addr learned my addr 172.21.15.70:0/3445757363 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.116+0000 7fa3a37fe700 1 -- 172.21.15.70:0/3445757363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa398009550 con 0x7fa3a4104e10 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.117+0000 7fa3a37fe700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a419c380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa398000c00 tx=0x7fa398011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.117+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa398003b90 con 0x7fa3a4104e10 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.117+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3980248b0 con 0x7fa3a4104e10 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.117+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a419cac0 con 0x7fa3a4104e10 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.117+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa39801a8e0 con 0x7fa3a4104e10 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.117+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a419cf60 con 0x7fa3a4104e10 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.118+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa39801aba0 con 0x7fa3a4104e10 2024-04-02T04:38:19.981 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.118+0000 7fa3a0ff9700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 0x7fa38c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.118+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 --> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3a4106590 con 0x7fa38c038320 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.118+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa39804c170 con 0x7fa3a4104e10 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.118+0000 7fa3a2ffd700 1 -- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 msgr2=0x7fa38c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/1197360681 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.118+0000 7fa3a2ffd700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 0x7fa38c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.319+0000 7fa3a2ffd700 1 -- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 msgr2=0x7fa38c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/1197360681 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.319+0000 7fa3a2ffd700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 0x7fa38c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.720+0000 7fa3a2ffd700 1 -- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 msgr2=0x7fa38c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/1197360681 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:14.720+0000 7fa3a2ffd700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 0x7fa38c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:15.521+0000 7fa3a2ffd700 1 -- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 msgr2=0x7fa38c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/1197360681 2024-04-02T04:38:19.982 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:15.521+0000 7fa3a2ffd700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 0x7fa38c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:17.122+0000 7fa3a2ffd700 1 -- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 msgr2=0x7fa38c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/1197360681 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:17.122+0000 7fa3a2ffd700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 0x7fa38c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:18.398+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7fa39802dc90 con 0x7fa3a4104e10 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:18.399+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 msgr2=0x7fa38c03a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:18.399+0000 7fa3a0ff9700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 0x7fa38c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.400+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7fa39804d970 con 0x7fa3a4104e10 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.400+0000 7fa3a0ff9700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fa38c03ba50 0x7fa38c03de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.400+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3a4106590 con 0x7fa38c03ba50 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.400+0000 7fa3a2ffd700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fa38c03ba50 0x7fa38c03de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.401+0000 7fa3a2ffd700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fa38c03ba50 0x7fa38c03de40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa394003a10 tx=0x7fa394009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.401+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fa3a4106590 con 0x7fa38c03ba50 2024-04-02T04:38:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.408+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa3a419d6c0 con 0x7fa38c03ba50 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.408+0000 7fa3a0ff9700 1 -- 172.21.15.70:0/3445757363 <== mgr.14120 v2:172.21.15.70:6800/2939436653 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa3a419d6c0 con 0x7fa38c03ba50 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fa38c03ba50 msgr2=0x7fa38c03de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fa38c03ba50 0x7fa38c03de40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa394003a10 tx=0x7fa394009290 comp rx=0 tx=0).stop 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 msgr2=0x7fa3a419c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a419c380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa398000c00 tx=0x7fa398011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 shutdown_connections 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3a4104e10 0x7fa3a419c380 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fa38c03ba50 0x7fa38c03de40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 --2- 172.21.15.70:0/3445757363 >> [v2:172.21.15.70:6800/1197360681,v1:172.21.15.70:6801/1197360681] conn(0x7fa38c038320 0x7fa38c03a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 >> 172.21.15.70:0/3445757363 conn(0x7fa3a41009f0 msgr2=0x7fa3a41909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 shutdown_connections 2024-04-02T04:38:19.984 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:19.409+0000 7fa3aa061700 1 -- 172.21.15.70:0/3445757363 wait complete. 2024-04-02T04:38:19.985 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 5 is available 2024-04-02T04:38:19.985 INFO:teuthology.orchestra.run.smithi070.stdout:Setting orchestrator backend to cephadm... 2024-04-02T04:38:20.955 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:20 smithi070 ceph-mon[29212]: [02/Apr/2024:04:38:19] ENGINE Bus STARTING 2024-04-02T04:38:20.956 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:20 smithi070 ceph-mon[29212]: [02/Apr/2024:04:38:19] ENGINE Serving on https://172.21.15.70:7150 2024-04-02T04:38:20.956 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:20 smithi070 ceph-mon[29212]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T04:38:20.956 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:20 smithi070 ceph-mon[29212]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T04:38:20.956 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:20 smithi070 ceph-mon[29212]: [02/Apr/2024:04:38:19] ENGINE Serving on http://172.21.15.70:8765 2024-04-02T04:38:20.956 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:20 smithi070 ceph-mon[29212]: [02/Apr/2024:04:38:19] ENGINE Bus STARTED 2024-04-02T04:38:21.255 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.381+0000 7fea684e3700 1 Processor -- start 2024-04-02T04:38:21.255 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.382+0000 7fea684e3700 1 -- start start 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.382+0000 7fea684e3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea60109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.382+0000 7fea684e3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea60079e30 con 0x7fea60107020 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.382+0000 7fea6627f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea60109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.382+0000 7fea6627f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea60109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54936/0 (socket says 172.21.15.70:54936) 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.382+0000 7fea6627f700 1 -- 172.21.15.70:0/3655004895 learned_addr learned my addr 172.21.15.70:0/3655004895 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.382+0000 7fea6627f700 1 -- 172.21.15.70:0/3655004895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea60109950 con 0x7fea60107020 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea6627f700 1 --2- 172.21.15.70:0/3655004895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea60109410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fea500098a0 tx=0x7fea50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acafe5a4ed5e8f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea6527d700 1 -- 172.21.15.70:0/3655004895 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea50004180 con 0x7fea60107020 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea6527d700 1 -- 172.21.15.70:0/3655004895 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea500042e0 con 0x7fea60107020 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea6527d700 1 -- 172.21.15.70:0/3655004895 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea50011a50 con 0x7fea60107020 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea684e3700 1 -- 172.21.15.70:0/3655004895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 msgr2=0x7fea60109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea684e3700 1 --2- 172.21.15.70:0/3655004895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea60109410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fea500098a0 tx=0x7fea50009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:21.256 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea684e3700 1 -- 172.21.15.70:0/3655004895 shutdown_connections 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea684e3700 1 --2- 172.21.15.70:0/3655004895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea60109410 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea684e3700 1 -- 172.21.15.70:0/3655004895 >> 172.21.15.70:0/3655004895 conn(0x7fea601009f0 msgr2=0x7fea60102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea684e3700 1 -- 172.21.15.70:0/3655004895 shutdown_connections 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.383+0000 7fea684e3700 1 -- 172.21.15.70:0/3655004895 wait complete. 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.384+0000 7fea684e3700 1 Processor -- start 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.384+0000 7fea684e3700 1 -- start start 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.384+0000 7fea684e3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea6019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.384+0000 7fea684e3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6019a710 con 0x7fea60107020 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.384+0000 7fea6627f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea6019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.384+0000 7fea6627f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea6019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54942/0 (socket says 172.21.15.70:54942) 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.384+0000 7fea6627f700 1 -- 172.21.15.70:0/4293335317 learned_addr learned my addr 172.21.15.70:0/4293335317 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.384+0000 7fea6627f700 1 -- 172.21.15.70:0/4293335317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea50009550 con 0x7fea60107020 2024-04-02T04:38:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.385+0000 7fea6627f700 1 --2- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea6019a1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fea50003de0 tx=0x7fea50011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.385+0000 7fea577fe700 1 -- 172.21.15.70:0/4293335317 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea50003930 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.385+0000 7fea577fe700 1 -- 172.21.15.70:0/4293335317 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea50003a90 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.385+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6019a910 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.385+0000 7fea577fe700 1 -- 172.21.15.70:0/4293335317 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea50019730 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.385+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6019adb0 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.386+0000 7fea577fe700 1 -- 172.21.15.70:0/4293335317 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7fea50019960 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.386+0000 7fea577fe700 1 --2- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fea4c038250 0x7fea4c03a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.386+0000 7fea577fe700 1 -- 172.21.15.70:0/4293335317 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fea5004c210 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.386+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6010a670 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.386+0000 7fea65a7e700 1 --2- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fea4c038250 0x7fea4c03a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.386+0000 7fea65a7e700 1 --2- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fea4c038250 0x7fea4c03a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea5c006fd0 tx=0x7fea5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.391+0000 7fea577fe700 1 -- 172.21.15.70:0/4293335317 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fea50019e40 con 0x7fea60107020 2024-04-02T04:38:21.258 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.590+0000 7fea577fe700 1 -- 172.21.15.70:0/4293335317 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fea5000fda0 con 0x7fea60107020 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.594+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fea6019db90 con 0x7fea4c038250 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.599+0000 7fea577fe700 1 -- 172.21.15.70:0/4293335317 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fea6019db90 con 0x7fea4c038250 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fea4c038250 msgr2=0x7fea4c03a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 --2- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fea4c038250 0x7fea4c03a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea5c006fd0 tx=0x7fea5c006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 msgr2=0x7fea6019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 --2- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea6019a1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fea50003de0 tx=0x7fea50011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 shutdown_connections 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 --2- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fea4c038250 0x7fea4c03a710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 --2- 172.21.15.70:0/4293335317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea60107020 0x7fea6019a1d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 >> 172.21.15.70:0/4293335317 conn(0x7fea601009f0 msgr2=0x7fea601017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 shutdown_connections 2024-04-02T04:38:21.259 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:20.602+0000 7fea684e3700 1 -- 172.21.15.70:0/4293335317 wait complete. 2024-04-02T04:38:21.764 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:21 smithi070 ceph-mon[29212]: mgrmap e8: smithi070.igidvr(active, since 2s) 2024-04-02T04:38:21.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:21 smithi070 ceph-mon[29212]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:21.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:21 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:21.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:21 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:22.519 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-02T04:38:22.519 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.722+0000 7f99bfbca700 1 Processor -- start 2024-04-02T04:38:22.519 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.722+0000 7f99bfbca700 1 -- start start 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.722+0000 7f99bfbca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b8109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.722+0000 7f99bfbca700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8079e30 con 0x7f99b8107020 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.722+0000 7f99bd966700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b8109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.722+0000 7f99bd966700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b8109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54950/0 (socket says 172.21.15.70:54950) 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.722+0000 7f99bd966700 1 -- 172.21.15.70:0/455860913 learned_addr learned my addr 172.21.15.70:0/455860913 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.722+0000 7f99bd966700 1 -- 172.21.15.70:0/455860913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8109950 con 0x7f99b8107020 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.723+0000 7f99bd966700 1 --2- 172.21.15.70:0/455860913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b8109410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098a0 tx=0x7f99b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324d96bc3e90bee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.723+0000 7f99bc964700 1 -- 172.21.15.70:0/455860913 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b4004180 con 0x7f99b8107020 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.723+0000 7f99bc964700 1 -- 172.21.15.70:0/455860913 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99b40042e0 con 0x7f99b8107020 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.723+0000 7f99bc964700 1 -- 172.21.15.70:0/455860913 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b4011a50 con 0x7f99b8107020 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.723+0000 7f99bfbca700 1 -- 172.21.15.70:0/455860913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 msgr2=0x7f99b8109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.723+0000 7f99bfbca700 1 --2- 172.21.15.70:0/455860913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b8109410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098a0 tx=0x7f99b4009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.520 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.723+0000 7f99bfbca700 1 -- 172.21.15.70:0/455860913 shutdown_connections 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.723+0000 7f99bfbca700 1 --2- 172.21.15.70:0/455860913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b8109410 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.724+0000 7f99bfbca700 1 -- 172.21.15.70:0/455860913 >> 172.21.15.70:0/455860913 conn(0x7f99b81009b0 msgr2=0x7f99b8102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.724+0000 7f99bfbca700 1 -- 172.21.15.70:0/455860913 shutdown_connections 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.724+0000 7f99bfbca700 1 -- 172.21.15.70:0/455860913 wait complete. 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.724+0000 7f99bfbca700 1 Processor -- start 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.724+0000 7f99bfbca700 1 -- start start 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.724+0000 7f99bfbca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.724+0000 7f99bfbca700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b819a750 con 0x7f99b8107020 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99bd966700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99bd966700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54956/0 (socket says 172.21.15.70:54956) 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99bd966700 1 -- 172.21.15.70:0/2621256828 learned_addr learned my addr 172.21.15.70:0/2621256828 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99bd966700 1 -- 172.21.15.70:0/2621256828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b4009550 con 0x7f99b8107020 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99bd966700 1 --2- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b819a210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99b4003f80 tx=0x7f99b4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:22.521 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99aeffd700 1 -- 172.21.15.70:0/2621256828 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b4003a80 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b819a950 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99aeffd700 1 -- 172.21.15.70:0/2621256828 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99b4003be0 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99aeffd700 1 -- 172.21.15.70:0/2621256828 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b4019820 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.725+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b819adf0 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.726+0000 7f99aeffd700 1 -- 172.21.15.70:0/2621256828 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f99b4019a90 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.726+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b810a670 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.726+0000 7f99aeffd700 1 --2- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f99a4038320 0x7f99a403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.726+0000 7f99aeffd700 1 -- 172.21.15.70:0/2621256828 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f99b404c540 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.727+0000 7f99bd165700 1 --2- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f99a4038320 0x7f99a403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.727+0000 7f99bd165700 1 --2- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f99a4038320 0x7f99a403a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006fd0 tx=0x7f99a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.735+0000 7f99aeffd700 1 -- 172.21.15.70:0/2621256828 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f99b402b750 con 0x7f99b8107020 2024-04-02T04:38:22.522 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.933+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f99b8102cc0 con 0x7f99a4038320 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.934+0000 7f99aeffd700 1 -- 172.21.15.70:0/2621256828 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f99b8102cc0 con 0x7f99a4038320 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f99a4038320 msgr2=0x7f99a403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 --2- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f99a4038320 0x7f99a403a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006fd0 tx=0x7f99a8006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 msgr2=0x7f99b819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 --2- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b819a210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99b4003f80 tx=0x7f99b4011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 shutdown_connections 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 --2- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f99a4038320 0x7f99a403a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 --2- 172.21.15.70:0/2621256828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b8107020 0x7f99b819a210 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 >> 172.21.15.70:0/2621256828 conn(0x7f99b81009b0 msgr2=0x7f99b8102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 shutdown_connections 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:21.936+0000 7f99bfbca700 1 -- 172.21.15.70:0/2621256828 wait complete. 2024-04-02T04:38:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:Generating ssh key... 2024-04-02T04:38:23.134 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:22 smithi070 ceph-mon[29212]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:23.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f367a3d7700 1 Processor -- start 2024-04-02T04:38:23.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f367a3d7700 1 -- start start 2024-04-02T04:38:23.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f367a3d7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f3674105220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:23.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f367a3d7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674079e50 con 0x7f3674104e00 2024-04-02T04:38:23.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f3673fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f3674105220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:23.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f3673fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f3674105220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54962/0 (socket says 172.21.15.70:54962) 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f3673fff700 1 -- 172.21.15.70:0/941181123 learned_addr learned my addr 172.21.15.70:0/941181123 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f3673fff700 1 -- 172.21.15.70:0/941181123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674105760 con 0x7f3674104e00 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.933+0000 7f3673fff700 1 --2- 172.21.15.70:0/941181123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f3674105220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f36680098a0 tx=0x7f3668009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce50fc7114a0576 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f3672ffd700 1 -- 172.21.15.70:0/941181123 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3668004180 con 0x7f3674104e00 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f3672ffd700 1 -- 172.21.15.70:0/941181123 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36680042e0 con 0x7f3674104e00 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f3672ffd700 1 -- 172.21.15.70:0/941181123 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3668011a50 con 0x7f3674104e00 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f367a3d7700 1 -- 172.21.15.70:0/941181123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 msgr2=0x7f3674105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f367a3d7700 1 --2- 172.21.15.70:0/941181123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f3674105220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f36680098a0 tx=0x7f3668009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f367a3d7700 1 -- 172.21.15.70:0/941181123 shutdown_connections 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f367a3d7700 1 --2- 172.21.15.70:0/941181123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f3674105220 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f367a3d7700 1 -- 172.21.15.70:0/941181123 >> 172.21.15.70:0/941181123 conn(0x7f3674100a20 msgr2=0x7f3674102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f367a3d7700 1 -- 172.21.15.70:0/941181123 shutdown_connections 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.934+0000 7f367a3d7700 1 -- 172.21.15.70:0/941181123 wait complete. 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.935+0000 7f367a3d7700 1 Processor -- start 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.935+0000 7f367a3d7700 1 -- start start 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.935+0000 7f367a3d7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f367419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.935+0000 7f367a3d7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367419c920 con 0x7f3674104e00 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.935+0000 7f3673fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f367419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.935+0000 7f3673fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f367419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54972/0 (socket says 172.21.15.70:54972) 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.935+0000 7f3673fff700 1 -- 172.21.15.70:0/2891483056 learned_addr learned my addr 172.21.15.70:0/2891483056 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.936+0000 7f3673fff700 1 -- 172.21.15.70:0/2891483056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3668009550 con 0x7f3674104e00 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.936+0000 7f3673fff700 1 --2- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f367419c3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3668003de0 tx=0x7f3668011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.936+0000 7f36717fa700 1 -- 172.21.15.70:0/2891483056 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3668003930 con 0x7f3674104e00 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.936+0000 7f36717fa700 1 -- 172.21.15.70:0/2891483056 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3668003a90 con 0x7f3674104e00 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.936+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367419cb20 con 0x7f3674104e00 2024-04-02T04:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.936+0000 7f36717fa700 1 -- 172.21.15.70:0/2891483056 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3668019730 con 0x7f3674104e00 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.936+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367419cfc0 con 0x7f3674104e00 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.937+0000 7f36717fa700 1 -- 172.21.15.70:0/2891483056 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f36680199b0 con 0x7f3674104e00 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.937+0000 7f36717fa700 1 --2- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f365c038320 0x7f365c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.937+0000 7f36717fa700 1 -- 172.21.15.70:0/2891483056 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f366804c430 con 0x7f3674104e00 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.937+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3674195cb0 con 0x7f3674104e00 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.937+0000 7f36737fe700 1 --2- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f365c038320 0x7f365c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.938+0000 7f36737fe700 1 --2- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f365c038320 0x7f365c03a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3664006fd0 tx=0x7f3664006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:22.942+0000 7f36717fa700 1 -- 172.21.15.70:0/2891483056 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3668019cb0 con 0x7f3674104e00 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.130+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f36740611d0 con 0x7f365c038320 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.382+0000 7f36717fa700 1 -- 172.21.15.70:0/2891483056 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f36740611d0 con 0x7f365c038320 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f365c038320 msgr2=0x7f365c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:23.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 --2- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f365c038320 0x7f365c03a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3664006fd0 tx=0x7f3664006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:23.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 msgr2=0x7f367419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:23.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 --2- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f367419c3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3668003de0 tx=0x7f3668011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:23.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 shutdown_connections 2024-04-02T04:38:23.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 --2- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f365c038320 0x7f365c03a7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:23.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 --2- 172.21.15.70:0/2891483056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674104e00 0x7f367419c3e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:23.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 >> 172.21.15.70:0/2891483056 conn(0x7f3674100a20 msgr2=0x7f36741909c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:23.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 shutdown_connections 2024-04-02T04:38:23.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:23.384+0000 7f367a3d7700 1 -- 172.21.15.70:0/2891483056 wait complete. 2024-04-02T04:38:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:24 smithi070 ceph-mon[29212]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:24 smithi070 ceph-mon[29212]: Generating ssh key... 2024-04-02T04:38:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:24 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:24 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:25.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL/eJTrtXsg+rXRF5Naum1qW6NUePVatDMqJngI7sYc7Rw1wvmlxQZ9SumtUkbHD80NTIqyFJH3/4e5x+b8mxpiIlWAHxDlvyGL2IBhLZKzayAtApfU5xyL50iq5egaoTuLl/VUY1sOBMKC/Ew6KuhgJpAmbjW2BZ/Hh+cHo6DlvhLciINX8+v04tVL5bFv5KdjHxd2V526wmuvTo3/YLpm2kvgEZKPAzQ3mGyYNyTxaby+WA2AGUcHL7s2512+BaepSauL3yCv+QQEwn7MZVdS0bGbRxg4Nd02bAf1d676hiPGClXfPfTtljRgSdW9klO9aRrjponaywk+uQs3+tNuBZ46T5U3E12ih3ijSB/JUvM2dXuxAYb8tWHTe3tEAgU/PMTef7qzXo0OUEhrG817FCG/jGJsYkbV+tInLI9mwPsuIYKRrQvNB1Dh5gwYHABqxwUvMJwEFYFjzEJTa7XdHYBBo45OOQGQzXFKG84TFkjKIbhPzw+bGY3bq1egX8= ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.339+0000 7f39640c5700 1 Processor -- start 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f39640c5700 1 -- start start 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f39640c5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c106960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f39640c5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c106f30 con 0x7f395c106540 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f3961e61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f3961e61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54986/0 (socket says 172.21.15.70:54986) 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f3961e61700 1 -- 172.21.15.70:0/2612482542 learned_addr learned my addr 172.21.15.70:0/2612482542 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f3961e61700 1 -- 172.21.15.70:0/2612482542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c107740 con 0x7f395c106540 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f3961e61700 1 --2- 172.21.15.70:0/2612482542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c106960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39540098a0 tx=0x7f3954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74f89a99072404e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:25.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f3960e5f700 1 -- 172.21.15.70:0/2612482542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3954004180 con 0x7f395c106540 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f3960e5f700 1 -- 172.21.15.70:0/2612482542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f39540042e0 con 0x7f395c106540 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.340+0000 7f3960e5f700 1 -- 172.21.15.70:0/2612482542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3954011a50 con 0x7f395c106540 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.341+0000 7f39640c5700 1 -- 172.21.15.70:0/2612482542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 msgr2=0x7f395c106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.341+0000 7f39640c5700 1 --2- 172.21.15.70:0/2612482542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c106960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39540098a0 tx=0x7f3954009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.341+0000 7f39640c5700 1 -- 172.21.15.70:0/2612482542 shutdown_connections 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.341+0000 7f39640c5700 1 --2- 172.21.15.70:0/2612482542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c106960 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.341+0000 7f39640c5700 1 -- 172.21.15.70:0/2612482542 >> 172.21.15.70:0/2612482542 conn(0x7f395c101b40 msgr2=0x7f395c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.341+0000 7f39640c5700 1 -- 172.21.15.70:0/2612482542 shutdown_connections 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.341+0000 7f39640c5700 1 -- 172.21.15.70:0/2612482542 wait complete. 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f39640c5700 1 Processor -- start 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f39640c5700 1 -- start start 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f39640c5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f39640c5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19a770 con 0x7f395c106540 2024-04-02T04:38:25.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f3961e61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f3961e61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54996/0 (socket says 172.21.15.70:54996) 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f3961e61700 1 -- 172.21.15.70:0/1261801633 learned_addr learned my addr 172.21.15.70:0/1261801633 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f3961e61700 1 -- 172.21.15.70:0/1261801633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954009550 con 0x7f395c106540 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f3961e61700 1 --2- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c19a230 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3954009520 tx=0x7f3954011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f395affd700 1 -- 172.21.15.70:0/1261801633 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3954003a80 con 0x7f395c106540 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c19a970 con 0x7f395c106540 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.342+0000 7f395affd700 1 -- 172.21.15.70:0/1261801633 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3954003be0 con 0x7f395c106540 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.343+0000 7f395affd700 1 -- 172.21.15.70:0/1261801633 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39540197b0 con 0x7f395c106540 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.343+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c19ae10 con 0x7f395c106540 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.343+0000 7f395affd700 1 -- 172.21.15.70:0/1261801633 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f395401a430 con 0x7f395c106540 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.343+0000 7f395affd700 1 --2- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f39440382d0 0x7f394403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.343+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c0623c0 con 0x7f395c106540 2024-04-02T04:38:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.343+0000 7f395affd700 1 -- 172.21.15.70:0/1261801633 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f395404c590 con 0x7f395c106540 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.344+0000 7f3961660700 1 --2- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f39440382d0 0x7f394403a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.344+0000 7f3961660700 1 --2- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f39440382d0 0x7f394403a790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3950006fd0 tx=0x7f3950006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.348+0000 7f395affd700 1 -- 172.21.15.70:0/1261801633 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f395401ac30 con 0x7f395c106540 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.542+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f395c103e00 con 0x7f39440382d0 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.543+0000 7f395affd700 1 -- 172.21.15.70:0/1261801633 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f395c103e00 con 0x7f39440382d0 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f39440382d0 msgr2=0x7f394403a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 --2- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f39440382d0 0x7f394403a790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3950006fd0 tx=0x7f3950006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 msgr2=0x7f395c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 --2- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c19a230 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3954009520 tx=0x7f3954011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 shutdown_connections 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 --2- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f39440382d0 0x7f394403a790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 --2- 172.21.15.70:0/1261801633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c106540 0x7f395c19a230 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 >> 172.21.15.70:0/1261801633 conn(0x7f395c101b40 msgr2=0x7f395c1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 shutdown_connections 2024-04-02T04:38:25.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:24.545+0000 7f39640c5700 1 -- 172.21.15.70:0/1261801633 wait complete. 2024-04-02T04:38:25.186 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-02T04:38:25.186 INFO:teuthology.orchestra.run.smithi070.stdout:Adding key to root@localhost authorized_keys... 2024-04-02T04:38:25.186 INFO:teuthology.orchestra.run.smithi070.stdout:Adding host smithi070... 2024-04-02T04:38:25.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:25 smithi070 ceph-mon[29212]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:27.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:26 smithi070 ceph-mon[29212]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi070", "addr": "172.21.15.70", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:27.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:26 smithi070 ceph-mon[29212]: Deploying cephadm binary to smithi070 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Added host 'smithi070' with addr '172.21.15.70' 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.580+0000 7f3ea13ef700 1 Processor -- start 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.580+0000 7f3ea13ef700 1 -- start start 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.580+0000 7f3ea13ef700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c104c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.580+0000 7f3ea13ef700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c105230 con 0x7f3e9c104840 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.580+0000 7f3e9affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.580+0000 7f3e9affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c104c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55006/0 (socket says 172.21.15.70:55006) 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.580+0000 7f3e9affd700 1 -- 172.21.15.70:0/2014350273 learned_addr learned my addr 172.21.15.70:0/2014350273 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3e9affd700 1 -- 172.21.15.70:0/2014350273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c1077d0 con 0x7f3e9c104840 2024-04-02T04:38:28.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3e9affd700 1 --2- 172.21.15.70:0/2014350273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c104c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0098a0 tx=0x7f3e8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a63525674050c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3e99ffb700 1 -- 172.21.15.70:0/2014350273 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e8c004180 con 0x7f3e9c104840 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3e99ffb700 1 -- 172.21.15.70:0/2014350273 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e8c0042e0 con 0x7f3e9c104840 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3e99ffb700 1 -- 172.21.15.70:0/2014350273 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e8c011a50 con 0x7f3e9c104840 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2014350273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 msgr2=0x7f3e9c104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3ea13ef700 1 --2- 172.21.15.70:0/2014350273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c104c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0098a0 tx=0x7f3e8c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2014350273 shutdown_connections 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3ea13ef700 1 --2- 172.21.15.70:0/2014350273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c104c60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.581+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2014350273 >> 172.21.15.70:0/2014350273 conn(0x7f3e9c100190 msgr2=0x7f3e9c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2014350273 shutdown_connections 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2014350273 wait complete. 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3ea13ef700 1 Processor -- start 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3ea13ef700 1 -- start start 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3ea13ef700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3ea13ef700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c19a740 con 0x7f3e9c104840 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3e9affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3e9affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55012/0 (socket says 172.21.15.70:55012) 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.582+0000 7f3e9affd700 1 -- 172.21.15.70:0/2437716756 learned_addr learned my addr 172.21.15.70:0/2437716756 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.583+0000 7f3e9affd700 1 -- 172.21.15.70:0/2437716756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c009550 con 0x7f3e9c104840 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.583+0000 7f3e9affd700 1 --2- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c19a200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009520 tx=0x7f3e8c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.583+0000 7f3e83fff700 1 -- 172.21.15.70:0/2437716756 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e8c003a80 con 0x7f3e9c104840 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.583+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c19a940 con 0x7f3e9c104840 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.583+0000 7f3e83fff700 1 -- 172.21.15.70:0/2437716756 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e8c003be0 con 0x7f3e9c104840 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.583+0000 7f3e83fff700 1 -- 172.21.15.70:0/2437716756 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e8c0197b0 con 0x7f3e9c104840 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.583+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c19ade0 con 0x7f3e9c104840 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.584+0000 7f3e83fff700 1 -- 172.21.15.70:0/2437716756 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f3e8c019a70 con 0x7f3e9c104840 2024-04-02T04:38:28.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.584+0000 7f3e83fff700 1 --2- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f3e840382d0 0x7f3e8403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.584+0000 7f3e83fff700 1 -- 172.21.15.70:0/2437716756 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3e8c04d4e0 con 0x7f3e9c104840 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.584+0000 7f3e9a7fc700 1 --2- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f3e840382d0 0x7f3e8403a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.584+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c0623c0 con 0x7f3e9c104840 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.584+0000 7f3e9a7fc700 1 --2- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f3e840382d0 0x7f3e8403a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006fd0 tx=0x7f3e90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.589+0000 7f3e83fff700 1 -- 172.21.15.70:0/2437716756 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3e8c02c3a0 con 0x7f3e9c104840 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:25.782+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi070", "addr": "172.21.15.70", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e9c1024a0 con 0x7f3e840382d0 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.299+0000 7f3e83fff700 1 -- 172.21.15.70:0/2437716756 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f3e9c1024a0 con 0x7f3e840382d0 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.301+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f3e840382d0 msgr2=0x7f3e8403a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.301+0000 7f3ea13ef700 1 --2- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f3e840382d0 0x7f3e8403a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006fd0 tx=0x7f3e90006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.301+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 msgr2=0x7f3e9c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.301+0000 7f3ea13ef700 1 --2- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c19a200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009520 tx=0x7f3e8c011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.301+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 shutdown_connections 2024-04-02T04:38:28.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.301+0000 7f3ea13ef700 1 --2- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f3e840382d0 0x7f3e8403a790 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.301+0000 7f3ea13ef700 1 --2- 172.21.15.70:0/2437716756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c104840 0x7f3e9c19a200 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.301+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 >> 172.21.15.70:0/2437716756 conn(0x7f3e9c100190 msgr2=0x7f3e9c101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:28.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.302+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 shutdown_connections 2024-04-02T04:38:28.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:28.302+0000 7f3ea13ef700 1 -- 172.21.15.70:0/2437716756 wait complete. 2024-04-02T04:38:28.968 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying mon service with default placement... 2024-04-02T04:38:29.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:29 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:29.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:29 smithi070 ceph-mon[29212]: Added host smithi070 2024-04-02T04:38:29.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:29 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:30.754 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:30 smithi070 ceph-mon[29212]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:30.754 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:30 smithi070 ceph-mon[29212]: Saving service mon spec with placement count:5 2024-04-02T04:38:30.755 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:30 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:30.755 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:30 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:31.025 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-02T04:38:31.025 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.379+0000 7f5a33b8a700 1 Processor -- start 2024-04-02T04:38:31.025 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.380+0000 7f5a33b8a700 1 -- start start 2024-04-02T04:38:31.025 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.380+0000 7f5a33b8a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c1093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:31.025 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.380+0000 7f5a33b8a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c079e90 con 0x7f5a2c106fe0 2024-04-02T04:38:31.025 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.380+0000 7f5a31926700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c1093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:31.025 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.380+0000 7f5a31926700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c1093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45430/0 (socket says 172.21.15.70:45430) 2024-04-02T04:38:31.025 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.380+0000 7f5a31926700 1 -- 172.21.15.70:0/2408319682 learned_addr learned my addr 172.21.15.70:0/2408319682 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.380+0000 7f5a31926700 1 -- 172.21.15.70:0/2408319682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c109910 con 0x7f5a2c106fe0 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a31926700 1 --2- 172.21.15.70:0/2408319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c1093d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5a280098a0 tx=0x7f5a28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbe74bcdacbc9ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a30924700 1 -- 172.21.15.70:0/2408319682 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a28004180 con 0x7f5a2c106fe0 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a30924700 1 -- 172.21.15.70:0/2408319682 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a280042e0 con 0x7f5a2c106fe0 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a30924700 1 -- 172.21.15.70:0/2408319682 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a28011a50 con 0x7f5a2c106fe0 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a33b8a700 1 -- 172.21.15.70:0/2408319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 msgr2=0x7f5a2c1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a33b8a700 1 --2- 172.21.15.70:0/2408319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c1093d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5a280098a0 tx=0x7f5a28009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a33b8a700 1 -- 172.21.15.70:0/2408319682 shutdown_connections 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a33b8a700 1 --2- 172.21.15.70:0/2408319682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c1093d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.381+0000 7f5a33b8a700 1 -- 172.21.15.70:0/2408319682 >> 172.21.15.70:0/2408319682 conn(0x7f5a2c1009f0 msgr2=0x7f5a2c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a33b8a700 1 -- 172.21.15.70:0/2408319682 shutdown_connections 2024-04-02T04:38:31.026 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a33b8a700 1 -- 172.21.15.70:0/2408319682 wait complete. 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a33b8a700 1 Processor -- start 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a33b8a700 1 -- start start 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a33b8a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c106940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a33b8a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c106e80 con 0x7f5a2c106fe0 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a31926700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a31926700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45434/0 (socket says 172.21.15.70:45434) 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.382+0000 7f5a31926700 1 -- 172.21.15.70:0/399844651 learned_addr learned my addr 172.21.15.70:0/399844651 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.383+0000 7f5a31926700 1 -- 172.21.15.70:0/399844651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a28009550 con 0x7f5a2c106fe0 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.383+0000 7f5a31926700 1 --2- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c106940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.383+0000 7f5a22ffd700 1 -- 172.21.15.70:0/399844651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a28003b90 con 0x7f5a2c106fe0 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.383+0000 7f5a22ffd700 1 -- 172.21.15.70:0/399844651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a280248b0 con 0x7f5a2c106fe0 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.383+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c104f90 con 0x7f5a2c106fe0 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.383+0000 7f5a22ffd700 1 -- 172.21.15.70:0/399844651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a2801a8e0 con 0x7f5a2c106fe0 2024-04-02T04:38:31.027 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.383+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c105430 con 0x7f5a2c106fe0 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.384+0000 7f5a22ffd700 1 -- 172.21.15.70:0/399844651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f5a2801ab50 con 0x7f5a2c106fe0 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.384+0000 7f5a22ffd700 1 --2- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5a18038370 0x7f5a1803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.384+0000 7f5a22ffd700 1 -- 172.21.15.70:0/399844651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5a2804c3c0 con 0x7f5a2c106fe0 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.384+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c10a690 con 0x7f5a2c106fe0 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.384+0000 7f5a31125700 1 --2- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5a18038370 0x7f5a1803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.384+0000 7f5a31125700 1 --2- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5a18038370 0x7f5a1803a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c006fd0 tx=0x7f5a1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.389+0000 7f5a22ffd700 1 -- 172.21.15.70:0/399844651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5a2801ae00 con 0x7f5a2c106fe0 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.592+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a2c101e20 con 0x7f5a18038370 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.596+0000 7f5a22ffd700 1 -- 172.21.15.70:0/399844651 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5a2c101e20 con 0x7f5a18038370 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.598+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5a18038370 msgr2=0x7f5a1803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.598+0000 7f5a33b8a700 1 --2- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5a18038370 0x7f5a1803a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c006fd0 tx=0x7f5a1c006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:31.028 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.598+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 msgr2=0x7f5a2c106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:31.029 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.598+0000 7f5a33b8a700 1 --2- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c106940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28011ab0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.029 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.598+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 shutdown_connections 2024-04-02T04:38:31.029 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.598+0000 7f5a33b8a700 1 --2- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5a18038370 0x7f5a1803a830 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.029 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.599+0000 7f5a33b8a700 1 --2- 172.21.15.70:0/399844651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a2c106fe0 0x7f5a2c106940 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.029 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.599+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 >> 172.21.15.70:0/399844651 conn(0x7f5a2c1009f0 msgr2=0x7f5a2c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:31.029 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.599+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 shutdown_connections 2024-04-02T04:38:31.029 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:29.599+0000 7f5a33b8a700 1 -- 172.21.15.70:0/399844651 wait complete. 2024-04-02T04:38:31.029 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying mgr service with default placement... 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.466+0000 7f1e07e09700 1 Processor -- start 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.467+0000 7f1e07e09700 1 -- start start 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.467+0000 7f1e07e09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e00106e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.467+0000 7f1e07e09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00107420 con 0x7f1e00106a30 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.467+0000 7f1e05ba5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e00106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.467+0000 7f1e05ba5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e00106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45444/0 (socket says 172.21.15.70:45444) 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.467+0000 7f1e05ba5700 1 -- 172.21.15.70:0/800169874 learned_addr learned my addr 172.21.15.70:0/800169874 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:32.342 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.467+0000 7f1e05ba5700 1 -- 172.21.15.70:0/800169874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e00105640 con 0x7f1e00106a30 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.467+0000 7f1e05ba5700 1 --2- 172.21.15.70:0/800169874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e00106e50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1df000bb40 tx=0x7f1df000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770f393cb4b28501 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e04ba3700 1 -- 172.21.15.70:0/800169874 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1df000dc00 con 0x7f1e00106a30 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e04ba3700 1 -- 172.21.15.70:0/800169874 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1df00044e0 con 0x7f1e00106a30 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e04ba3700 1 -- 172.21.15.70:0/800169874 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1df0010470 con 0x7f1e00106a30 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e07e09700 1 -- 172.21.15.70:0/800169874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 msgr2=0x7f1e00106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e07e09700 1 --2- 172.21.15.70:0/800169874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e00106e50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1df000bb40 tx=0x7f1df000be50 comp rx=0 tx=0).stop 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e07e09700 1 -- 172.21.15.70:0/800169874 shutdown_connections 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e07e09700 1 --2- 172.21.15.70:0/800169874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e00106e50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e07e09700 1 -- 172.21.15.70:0/800169874 >> 172.21.15.70:0/800169874 conn(0x7f1e00100190 msgr2=0x7f1e001025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:32.343 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e07e09700 1 -- 172.21.15.70:0/800169874 shutdown_connections 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.468+0000 7f1e07e09700 1 -- 172.21.15.70:0/800169874 wait complete. 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e07e09700 1 Processor -- start 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e07e09700 1 -- start start 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e07e09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e0019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e07e09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019c9c0 con 0x7f1e00106a30 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e05ba5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e0019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e05ba5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e0019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45454/0 (socket says 172.21.15.70:45454) 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e05ba5700 1 -- 172.21.15.70:0/440926014 learned_addr learned my addr 172.21.15.70:0/440926014 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e05ba5700 1 -- 172.21.15.70:0/440926014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df000b7f0 con 0x7f1e00106a30 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.469+0000 7f1e05ba5700 1 --2- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e0019c480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1df0003ba0 tx=0x7f1df0004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.470+0000 7f1df6ffd700 1 -- 172.21.15.70:0/440926014 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1df000de00 con 0x7f1e00106a30 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.470+0000 7f1df6ffd700 1 -- 172.21.15.70:0/440926014 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1df0010b90 con 0x7f1e00106a30 2024-04-02T04:38:32.344 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.470+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0019cbc0 con 0x7f1e00106a30 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.470+0000 7f1df6ffd700 1 -- 172.21.15.70:0/440926014 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1df0021a70 con 0x7f1e00106a30 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.470+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0019d000 con 0x7f1e00106a30 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.470+0000 7f1df6ffd700 1 -- 172.21.15.70:0/440926014 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1df0021d30 con 0x7f1e00106a30 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.471+0000 7f1df6ffd700 1 --2- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f1dec038320 0x7f1dec03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.471+0000 7f1df6ffd700 1 -- 172.21.15.70:0/440926014 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1df004c420 con 0x7f1e00106a30 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.471+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e00191ce0 con 0x7f1e00106a30 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.471+0000 7f1e053a4700 1 --2- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f1dec038320 0x7f1dec03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.471+0000 7f1e053a4700 1 --2- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f1dec038320 0x7f1dec03a7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc006fd0 tx=0x7f1dfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.476+0000 7f1df6ffd700 1 -- 172.21.15.70:0/440926014 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1df000e770 con 0x7f1e00106a30 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.686+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e000611d0 con 0x7f1dec038320 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.691+0000 7f1df6ffd700 1 -- 172.21.15.70:0/440926014 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1e000611d0 con 0x7f1dec038320 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f1dec038320 msgr2=0x7f1dec03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 --2- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f1dec038320 0x7f1dec03a7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc006fd0 tx=0x7f1dfc006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:32.345 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 msgr2=0x7f1e0019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:32.346 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 --2- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e0019c480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1df0003ba0 tx=0x7f1df0004070 comp rx=0 tx=0).stop 2024-04-02T04:38:32.346 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 shutdown_connections 2024-04-02T04:38:32.346 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 --2- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f1dec038320 0x7f1dec03a7e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:32.346 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 --2- 172.21.15.70:0/440926014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e00106a30 0x7f1e0019c480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:32.346 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 >> 172.21.15.70:0/440926014 conn(0x7f1e00100190 msgr2=0x7f1e0018ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:32.346 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 shutdown_connections 2024-04-02T04:38:32.346 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:31.693+0000 7f1e07e09700 1 -- 172.21.15.70:0/440926014 wait complete. 2024-04-02T04:38:32.346 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying crash service with default placement... 2024-04-02T04:38:32.976 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:32 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:32.976 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:32 smithi070 ceph-mon[29212]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:32.976 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:32 smithi070 ceph-mon[29212]: Saving service mgr spec with placement count:2 2024-04-02T04:38:32.976 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:32 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:34.149 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:33 smithi070 ceph-mon[29212]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:34.150 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:33 smithi070 ceph-mon[29212]: Saving service crash spec with placement * 2024-04-02T04:38:34.150 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:33 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:34.150 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:33 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.747+0000 7f729743c700 1 Processor -- start 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.747+0000 7f729743c700 1 -- start start 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.747+0000 7f729743c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290105d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.747+0000 7f729743c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290106360 con 0x7f7290105970 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.747+0000 7f72951d8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290105d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.747+0000 7f72951d8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290105d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45470/0 (socket says 172.21.15.70:45470) 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.747+0000 7f72951d8700 1 -- 172.21.15.70:0/3730975425 learned_addr learned my addr 172.21.15.70:0/3730975425 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.748+0000 7f72951d8700 1 -- 172.21.15.70:0/3730975425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290106690 con 0x7f7290105970 2024-04-02T04:38:34.157 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.748+0000 7f72951d8700 1 --2- 172.21.15.70:0/3730975425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290105d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f728000bb40 tx=0x7f728000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c20ecfd7af7fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.748+0000 7f7287fff700 1 -- 172.21.15.70:0/3730975425 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f728000dc00 con 0x7f7290105970 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.748+0000 7f7287fff700 1 -- 172.21.15.70:0/3730975425 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f72800044e0 con 0x7f7290105970 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.748+0000 7f7287fff700 1 -- 172.21.15.70:0/3730975425 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7280010470 con 0x7f7290105970 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 -- 172.21.15.70:0/3730975425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 msgr2=0x7f7290105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 --2- 172.21.15.70:0/3730975425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290105d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f728000bb40 tx=0x7f728000be50 comp rx=0 tx=0).stop 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 -- 172.21.15.70:0/3730975425 shutdown_connections 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 --2- 172.21.15.70:0/3730975425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290105d90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 -- 172.21.15.70:0/3730975425 >> 172.21.15.70:0/3730975425 conn(0x7f7290100190 msgr2=0x7f72901025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 -- 172.21.15.70:0/3730975425 shutdown_connections 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 -- 172.21.15.70:0/3730975425 wait complete. 2024-04-02T04:38:34.158 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 Processor -- start 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.749+0000 7f729743c700 1 -- start start 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f729743c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f729743c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72901984f0 con 0x7f7290105970 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f72951d8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f72951d8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45476/0 (socket says 172.21.15.70:45476) 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f72951d8700 1 -- 172.21.15.70:0/3286512242 learned_addr learned my addr 172.21.15.70:0/3286512242 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f72951d8700 1 -- 172.21.15.70:0/3286512242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728000b7f0 con 0x7f7290105970 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f72951d8700 1 --2- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290197fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7280003fc0 tx=0x7f72800040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f72867fc700 1 -- 172.21.15.70:0/3286512242 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f728001e070 con 0x7f7290105970 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72901986f0 con 0x7f7290105970 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f72867fc700 1 -- 172.21.15.70:0/3286512242 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f728000de20 con 0x7f7290105970 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f72867fc700 1 -- 172.21.15.70:0/3286512242 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7280022b10 con 0x7f7290105970 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.750+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7290198b90 con 0x7f7290105970 2024-04-02T04:38:34.159 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.751+0000 7f72867fc700 1 -- 172.21.15.70:0/3286512242 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f7280022d50 con 0x7f7290105970 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.751+0000 7f72867fc700 1 --2- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f727c0382d0 0x7f727c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.751+0000 7f72867fc700 1 -- 172.21.15.70:0/3286512242 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f728004f700 con 0x7f7290105970 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.751+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7290191d20 con 0x7f7290105970 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.751+0000 7f72949d7700 1 --2- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f727c0382d0 0x7f727c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.752+0000 7f72949d7700 1 --2- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f727c0382d0 0x7f727c03a790 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f728c006fd0 tx=0x7f728c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.756+0000 7f72867fc700 1 -- 172.21.15.70:0/3286512242 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7280027070 con 0x7f7290105970 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.971+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f72900611d0 con 0x7f727c0382d0 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.975+0000 7f72867fc700 1 -- 172.21.15.70:0/3286512242 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f72900611d0 con 0x7f727c0382d0 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.977+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f727c0382d0 msgr2=0x7f727c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.977+0000 7f729743c700 1 --2- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f727c0382d0 0x7f727c03a790 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f728c006fd0 tx=0x7f728c006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.977+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 msgr2=0x7f7290197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.977+0000 7f729743c700 1 --2- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290197fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7280003fc0 tx=0x7f72800040a0 comp rx=0 tx=0).stop 2024-04-02T04:38:34.160 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.977+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 shutdown_connections 2024-04-02T04:38:34.161 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.977+0000 7f729743c700 1 --2- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f727c0382d0 0x7f727c03a790 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:34.161 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.977+0000 7f729743c700 1 --2- 172.21.15.70:0/3286512242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7290105970 0x7f7290197fb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:34.161 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.977+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 >> 172.21.15.70:0/3286512242 conn(0x7f7290100190 msgr2=0x7f729018ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:34.161 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.978+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 shutdown_connections 2024-04-02T04:38:34.161 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:32.978+0000 7f729743c700 1 -- 172.21.15.70:0/3286512242 wait complete. 2024-04-02T04:38:34.161 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying ceph-exporter service with default placement... 2024-04-02T04:38:35.749 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:35 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:35.749 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:35 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:35.933 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.636+0000 7faab716f700 1 Processor -- start 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.636+0000 7faab716f700 1 -- start start 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.636+0000 7faab716f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab0106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.636+0000 7faab716f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab0106f50 con 0x7faab0106560 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.636+0000 7faab4f0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab0106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.636+0000 7faab4f0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab0106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45488/0 (socket says 172.21.15.70:45488) 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.636+0000 7faab4f0b700 1 -- 172.21.15.70:0/3224789412 learned_addr learned my addr 172.21.15.70:0/3224789412 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.637+0000 7faab4f0b700 1 -- 172.21.15.70:0/3224789412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0107760 con 0x7faab0106560 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.637+0000 7faab4f0b700 1 --2- 172.21.15.70:0/3224789412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab0106980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faaa40098a0 tx=0x7faaa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0f5bd52c7043b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.637+0000 7faaaf7fe700 1 -- 172.21.15.70:0/3224789412 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faaa4004180 con 0x7faab0106560 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.637+0000 7faaaf7fe700 1 -- 172.21.15.70:0/3224789412 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faaa40042e0 con 0x7faab0106560 2024-04-02T04:38:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.637+0000 7faaaf7fe700 1 -- 172.21.15.70:0/3224789412 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faaa4011af0 con 0x7faab0106560 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.637+0000 7faab716f700 1 -- 172.21.15.70:0/3224789412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 msgr2=0x7faab0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.637+0000 7faab716f700 1 --2- 172.21.15.70:0/3224789412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab0106980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faaa40098a0 tx=0x7faaa4009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 -- 172.21.15.70:0/3224789412 shutdown_connections 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 --2- 172.21.15.70:0/3224789412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab0106980 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 -- 172.21.15.70:0/3224789412 >> 172.21.15.70:0/3224789412 conn(0x7faab0101b20 msgr2=0x7faab0103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 -- 172.21.15.70:0/3224789412 shutdown_connections 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 -- 172.21.15.70:0/3224789412 wait complete. 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 Processor -- start 2024-04-02T04:38:35.935 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 -- start start 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab007ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab716f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab007b060 con 0x7faab0106560 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab4f0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab007ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab4f0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab007ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45504/0 (socket says 172.21.15.70:45504) 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.638+0000 7faab4f0b700 1 -- 172.21.15.70:0/759636281 learned_addr learned my addr 172.21.15.70:0/759636281 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.639+0000 7faab4f0b700 1 -- 172.21.15.70:0/759636281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa4009550 con 0x7faab0106560 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.639+0000 7faab4f0b700 1 --2- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab007ab20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faaa4006ba0 tx=0x7faaa4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.639+0000 7faaadffb700 1 -- 172.21.15.70:0/759636281 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faaa4003b00 con 0x7faab0106560 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.639+0000 7faaadffb700 1 -- 172.21.15.70:0/759636281 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faaa401a8e0 con 0x7faab0106560 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.639+0000 7faaadffb700 1 -- 172.21.15.70:0/759636281 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faaa4019990 con 0x7faab0106560 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.639+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab0079170 con 0x7faab0106560 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.639+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab0079610 con 0x7faab0106560 2024-04-02T04:38:35.936 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.640+0000 7faaadffb700 1 -- 172.21.15.70:0/759636281 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7faaa4019c00 con 0x7faab0106560 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.640+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab00623c0 con 0x7faab0106560 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.640+0000 7faaadffb700 1 --2- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7faaa0038320 0x7faaa003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.640+0000 7faaadffb700 1 -- 172.21.15.70:0/759636281 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faaa404c550 con 0x7faab0106560 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.640+0000 7faaaffff700 1 --2- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7faaa0038320 0x7faaa003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.641+0000 7faaaffff700 1 --2- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7faaa0038320 0x7faaa003a7e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006fd0 tx=0x7faa9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.645+0000 7faaadffb700 1 -- 172.21.15.70:0/759636281 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faaa401aa50 con 0x7faab0106560 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.859+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7faab0102f70 con 0x7faaa0038320 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.863+0000 7faaadffb700 1 -- 172.21.15.70:0/759636281 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7faab0102f70 con 0x7faaa0038320 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.865+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7faaa0038320 msgr2=0x7faaa003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.865+0000 7faab716f700 1 --2- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7faaa0038320 0x7faaa003a7e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006fd0 tx=0x7faa9c006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.865+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 msgr2=0x7faab007ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.865+0000 7faab716f700 1 --2- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab007ab20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faaa4006ba0 tx=0x7faaa4011b50 comp rx=0 tx=0).stop 2024-04-02T04:38:35.937 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.865+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 shutdown_connections 2024-04-02T04:38:35.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.865+0000 7faab716f700 1 --2- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7faaa0038320 0x7faaa003a7e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:35.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.865+0000 7faab716f700 1 --2- 172.21.15.70:0/759636281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faab0106560 0x7faab007ab20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:35.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.865+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 >> 172.21.15.70:0/759636281 conn(0x7faab0101b20 msgr2=0x7faab0102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:35.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.866+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 shutdown_connections 2024-04-02T04:38:35.938 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:34.866+0000 7faab716f700 1 -- 172.21.15.70:0/759636281 wait complete. 2024-04-02T04:38:35.938 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying prometheus service with default placement... 2024-04-02T04:38:37.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:36 smithi070 ceph-mon[29212]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:37.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:36 smithi070 ceph-mon[29212]: Saving service ceph-exporter spec with placement * 2024-04-02T04:38:37.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:36 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.337+0000 7fc0eef7c700 1 Processor -- start 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.337+0000 7fc0eef7c700 1 -- start start 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.337+0000 7fc0eef7c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e8106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.337+0000 7fc0eef7c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e8106f50 con 0x7fc0e8106560 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.338+0000 7fc0ecd18700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e8106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.338+0000 7fc0ecd18700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e8106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45508/0 (socket says 172.21.15.70:45508) 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.338+0000 7fc0ecd18700 1 -- 172.21.15.70:0/3602764754 learned_addr learned my addr 172.21.15.70:0/3602764754 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:37.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.338+0000 7fc0ecd18700 1 -- 172.21.15.70:0/3602764754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e8107760 con 0x7fc0e8106560 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.338+0000 7fc0ecd18700 1 --2- 172.21.15.70:0/3602764754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e8106980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0098a0 tx=0x7fc0dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fb57a87886c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.338+0000 7fc0e77fe700 1 -- 172.21.15.70:0/3602764754 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0dc004180 con 0x7fc0e8106560 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.338+0000 7fc0e77fe700 1 -- 172.21.15.70:0/3602764754 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc0042e0 con 0x7fc0e8106560 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.338+0000 7fc0e77fe700 1 -- 172.21.15.70:0/3602764754 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0dc011af0 con 0x7fc0e8106560 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 -- 172.21.15.70:0/3602764754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 msgr2=0x7fc0e8106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 --2- 172.21.15.70:0/3602764754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e8106980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0098a0 tx=0x7fc0dc009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 -- 172.21.15.70:0/3602764754 shutdown_connections 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 --2- 172.21.15.70:0/3602764754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e8106980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 -- 172.21.15.70:0/3602764754 >> 172.21.15.70:0/3602764754 conn(0x7fc0e8101b40 msgr2=0x7fc0e8103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 -- 172.21.15.70:0/3602764754 shutdown_connections 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 -- 172.21.15.70:0/3602764754 wait complete. 2024-04-02T04:38:37.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 Processor -- start 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 -- start start 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.339+0000 7fc0eef7c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e807bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0eef7c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e807c170 con 0x7fc0e8106560 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0ecd18700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e807bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0ecd18700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e807bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45518/0 (socket says 172.21.15.70:45518) 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0ecd18700 1 -- 172.21.15.70:0/1539012072 learned_addr learned my addr 172.21.15.70:0/1539012072 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0ecd18700 1 -- 172.21.15.70:0/1539012072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc009550 con 0x7fc0e8106560 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0ecd18700 1 --2- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e807bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc011670 tx=0x7fc0dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0e5ffb700 1 -- 172.21.15.70:0/1539012072 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0dc0039c0 con 0x7fc0e8106560 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0e5ffb700 1 -- 172.21.15.70:0/1539012072 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc003b20 con 0x7fc0e8106560 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e807a280 con 0x7fc0e8106560 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0e5ffb700 1 -- 172.21.15.70:0/1539012072 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0dc0247f0 con 0x7fc0e8106560 2024-04-02T04:38:37.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.340+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e807a720 con 0x7fc0e8106560 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.341+0000 7fc0e5ffb700 1 -- 172.21.15.70:0/1539012072 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc0dc01ac70 con 0x7fc0e8106560 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.341+0000 7fc0e5ffb700 1 --2- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc0d8038330 0x7fc0d803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.341+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e80623c0 con 0x7fc0e8106560 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.342+0000 7fc0e5ffb700 1 -- 172.21.15.70:0/1539012072 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc0dc04c270 con 0x7fc0e8106560 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.342+0000 7fc0e7fff700 1 --2- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc0d8038330 0x7fc0d803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.342+0000 7fc0e7fff700 1 --2- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc0d8038330 0x7fc0d803a7f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4006fd0 tx=0x7fc0d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.346+0000 7fc0e5ffb700 1 -- 172.21.15.70:0/1539012072 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc0dc02d3f0 con 0x7fc0e8106560 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.559+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0e810a960 con 0x7fc0d8038330 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.563+0000 7fc0e5ffb700 1 -- 172.21.15.70:0/1539012072 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fc0e810a960 con 0x7fc0d8038330 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc0d8038330 msgr2=0x7fc0d803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 --2- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc0d8038330 0x7fc0d803a7f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4006fd0 tx=0x7fc0d4006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 msgr2=0x7fc0e807bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 --2- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e807bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc011670 tx=0x7fc0dc011aa0 comp rx=0 tx=0).stop 2024-04-02T04:38:37.320 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 shutdown_connections 2024-04-02T04:38:37.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 --2- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc0d8038330 0x7fc0d803a7f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:37.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 --2- 172.21.15.70:0/1539012072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0e8106560 0x7fc0e807bc30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:37.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 >> 172.21.15.70:0/1539012072 conn(0x7fc0e8101b40 msgr2=0x7fc0e8102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:37.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 shutdown_connections 2024-04-02T04:38:37.321 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:36.565+0000 7fc0eef7c700 1 -- 172.21.15.70:0/1539012072 wait complete. 2024-04-02T04:38:37.321 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying grafana service with default placement... 2024-04-02T04:38:38.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:37 smithi070 ceph-mon[29212]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:38.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:37 smithi070 ceph-mon[29212]: Saving service prometheus spec with placement count:1 2024-04-02T04:38:38.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-02T04:38:38.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.710+0000 7f24bbd6f700 1 Processor -- start 2024-04-02T04:38:38.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.710+0000 7f24bbd6f700 1 -- start start 2024-04-02T04:38:38.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.710+0000 7f24bbd6f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b4105250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:38.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.710+0000 7f24bbd6f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b4074620 con 0x7f24b4104e30 2024-04-02T04:38:38.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.711+0000 7f24b9b0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b4105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:38.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.711+0000 7f24b9b0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b4105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57288/0 (socket says 172.21.15.70:57288) 2024-04-02T04:38:38.555 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.711+0000 7f24b9b0b700 1 -- 172.21.15.70:0/3695575260 learned_addr learned my addr 172.21.15.70:0/3695575260 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.711+0000 7f24b9b0b700 1 -- 172.21.15.70:0/3695575260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b4105790 con 0x7f24b4104e30 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.711+0000 7f24b9b0b700 1 --2- 172.21.15.70:0/3695575260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b4105250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009e80 tx=0x7f24ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3306c47936aa829 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.711+0000 7f24b8b09700 1 -- 172.21.15.70:0/3695575260 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24ac00e070 con 0x7f24b4104e30 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.711+0000 7f24b8b09700 1 -- 172.21.15.70:0/3695575260 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24ac00b8c0 con 0x7f24b4104e30 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.711+0000 7f24b8b09700 1 -- 172.21.15.70:0/3695575260 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24ac0044d0 con 0x7f24b4104e30 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3695575260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 msgr2=0x7f24b4105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 --2- 172.21.15.70:0/3695575260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b4105250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009e80 tx=0x7f24ac00b330 comp rx=0 tx=0).stop 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3695575260 shutdown_connections 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 --2- 172.21.15.70:0/3695575260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b4105250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3695575260 >> 172.21.15.70:0/3695575260 conn(0x7f24b41009f0 msgr2=0x7f24b4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3695575260 shutdown_connections 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3695575260 wait complete. 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 Processor -- start 2024-04-02T04:38:38.556 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 -- start start 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.712+0000 7f24bbd6f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24bbd6f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b419c880 con 0x7f24b4104e30 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24b9b0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24b9b0b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b419c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57292/0 (socket says 172.21.15.70:57292) 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24b9b0b700 1 -- 172.21.15.70:0/3638296565 learned_addr learned my addr 172.21.15.70:0/3638296565 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24b9b0b700 1 -- 172.21.15.70:0/3638296565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac009b30 con 0x7f24b4104e30 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24b9b0b700 1 --2- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b419c340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0072a0 tx=0x7f24ac0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24aaffd700 1 -- 172.21.15.70:0/3638296565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24ac00e040 con 0x7f24b4104e30 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24aaffd700 1 -- 172.21.15.70:0/3638296565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24ac00bd80 con 0x7f24b4104e30 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b419ca80 con 0x7f24b4104e30 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24aaffd700 1 -- 172.21.15.70:0/3638296565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24ac0126c0 con 0x7f24b4104e30 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.713+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b419cf80 con 0x7f24b4104e30 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.714+0000 7f24aaffd700 1 -- 172.21.15.70:0/3638296565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f24ac019040 con 0x7f24b4104e30 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.714+0000 7f24aaffd700 1 --2- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f24a00382e0 0x7f24a003a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:38.557 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.714+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b4195c60 con 0x7f24b4104e30 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.714+0000 7f24aaffd700 1 -- 172.21.15.70:0/3638296565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f24ac04bdc0 con 0x7f24b4104e30 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.714+0000 7f24b930a700 1 --2- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f24a00382e0 0x7f24a003a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.715+0000 7f24b930a700 1 --2- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f24a00382e0 0x7f24a003a7a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006fd0 tx=0x7f24a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.719+0000 7f24aaffd700 1 -- 172.21.15.70:0/3638296565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f24ac01f080 con 0x7f24b4104e30 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.920+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b40611d0 con 0x7f24a00382e0 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.924+0000 7f24aaffd700 1 -- 172.21.15.70:0/3638296565 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f24b40611d0 con 0x7f24a00382e0 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f24a00382e0 msgr2=0x7f24a003a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 --2- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f24a00382e0 0x7f24a003a7a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006fd0 tx=0x7f24a4006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 msgr2=0x7f24b419c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 --2- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b419c340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0072a0 tx=0x7f24ac0049f0 comp rx=0 tx=0).stop 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 shutdown_connections 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 --2- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f24a00382e0 0x7f24a003a7a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:38.558 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 --2- 172.21.15.70:0/3638296565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24b4104e30 0x7f24b419c340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:38.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 >> 172.21.15.70:0/3638296565 conn(0x7f24b41009f0 msgr2=0x7f24b41909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:38.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 shutdown_connections 2024-04-02T04:38:38.559 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:37.926+0000 7f24bbd6f700 1 -- 172.21.15.70:0/3638296565 wait complete. 2024-04-02T04:38:38.559 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying node-exporter service with default placement... 2024-04-02T04:38:39.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:38 smithi070 ceph-mon[29212]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:39.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:38 smithi070 ceph-mon[29212]: Saving service grafana spec with placement count:1 2024-04-02T04:38:39.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:38 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:39.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:38 smithi070 ceph-mon[29212]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:38:39.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-02T04:38:39.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78edca700 1 Processor -- start 2024-04-02T04:38:39.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78edca700 1 -- start start 2024-04-02T04:38:39.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78edca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe788109420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78edca700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788079e30 con 0x7fe788107030 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78cb66700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe788109420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78cb66700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe788109420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57300/0 (socket says 172.21.15.70:57300) 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78cb66700 1 -- 172.21.15.70:0/1578018896 learned_addr learned my addr 172.21.15.70:0/1578018896 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78cb66700 1 -- 172.21.15.70:0/1578018896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788109960 con 0x7fe788107030 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.965+0000 7fe78cb66700 1 --2- 172.21.15.70:0/1578018896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe788109420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0098a0 tx=0x7fe77c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ed17c2cbb1fc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe7877fe700 1 -- 172.21.15.70:0/1578018896 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe77c004180 con 0x7fe788107030 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe7877fe700 1 -- 172.21.15.70:0/1578018896 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe77c0042e0 con 0x7fe788107030 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe7877fe700 1 -- 172.21.15.70:0/1578018896 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe77c011af0 con 0x7fe788107030 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe78edca700 1 -- 172.21.15.70:0/1578018896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 msgr2=0x7fe788109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe78edca700 1 --2- 172.21.15.70:0/1578018896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe788109420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0098a0 tx=0x7fe77c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe78edca700 1 -- 172.21.15.70:0/1578018896 shutdown_connections 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe78edca700 1 --2- 172.21.15.70:0/1578018896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe788109420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe78edca700 1 -- 172.21.15.70:0/1578018896 >> 172.21.15.70:0/1578018896 conn(0x7fe788100a20 msgr2=0x7fe788102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe78edca700 1 -- 172.21.15.70:0/1578018896 shutdown_connections 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.966+0000 7fe78edca700 1 -- 172.21.15.70:0/1578018896 wait complete. 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78edca700 1 Processor -- start 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78edca700 1 -- start start 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78edca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe78819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78edca700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78819a780 con 0x7fe788107030 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78cb66700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe78819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78cb66700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe78819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57316/0 (socket says 172.21.15.70:57316) 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78cb66700 1 -- 172.21.15.70:0/1687658891 learned_addr learned my addr 172.21.15.70:0/1687658891 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78cb66700 1 -- 172.21.15.70:0/1687658891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c009550 con 0x7fe788107030 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.967+0000 7fe78cb66700 1 --2- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe78819a240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe77c011670 tx=0x7fe77c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.968+0000 7fe785ffb700 1 -- 172.21.15.70:0/1687658891 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe77c003a40 con 0x7fe788107030 2024-04-02T04:38:39.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.968+0000 7fe785ffb700 1 -- 172.21.15.70:0/1687658891 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe77c003ba0 con 0x7fe788107030 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.968+0000 7fe785ffb700 1 -- 172.21.15.70:0/1687658891 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe77c024790 con 0x7fe788107030 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.968+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78819a980 con 0x7fe788107030 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.968+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78819ae20 con 0x7fe788107030 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.969+0000 7fe785ffb700 1 -- 172.21.15.70:0/1687658891 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe77c01a430 con 0x7fe788107030 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.969+0000 7fe785ffb700 1 --2- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe778038370 0x7fe77803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.969+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78810a6f0 con 0x7fe788107030 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.969+0000 7fe785ffb700 1 -- 172.21.15.70:0/1687658891 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe77c04c440 con 0x7fe788107030 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.969+0000 7fe787fff700 1 --2- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe778038370 0x7fe77803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.969+0000 7fe787fff700 1 --2- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe778038370 0x7fe77803a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe774006fd0 tx=0x7fe774006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:38.974+0000 7fe785ffb700 1 -- 172.21.15.70:0/1687658891 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe77c010380 con 0x7fe788107030 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.173+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fe788101ed0 con 0x7fe778038370 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.177+0000 7fe785ffb700 1 -- 172.21.15.70:0/1687658891 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fe788101ed0 con 0x7fe778038370 2024-04-02T04:38:39.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.179+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe778038370 msgr2=0x7fe77803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.179+0000 7fe78edca700 1 --2- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe778038370 0x7fe77803a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe774006fd0 tx=0x7fe774006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.179+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 msgr2=0x7fe78819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.179+0000 7fe78edca700 1 --2- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe78819a240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe77c011670 tx=0x7fe77c011b50 comp rx=0 tx=0).stop 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.179+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 shutdown_connections 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.179+0000 7fe78edca700 1 --2- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe778038370 0x7fe77803a830 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.179+0000 7fe78edca700 1 --2- 172.21.15.70:0/1687658891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe788107030 0x7fe78819a240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.179+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 >> 172.21.15.70:0/1687658891 conn(0x7fe788100a20 msgr2=0x7fe7881017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.180+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 shutdown_connections 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:39.180+0000 7fe78edca700 1 -- 172.21.15.70:0/1687658891 wait complete. 2024-04-02T04:38:39.722 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying alertmanager service with default placement... 2024-04-02T04:38:40.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:40 smithi070 ceph-mon[29212]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:40.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:40 smithi070 ceph-mon[29212]: Saving service node-exporter spec with placement * 2024-04-02T04:38:40.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:40 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:40.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-02T04:38:40.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb252835700 1 Processor -- start 2024-04-02T04:38:40.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb252835700 1 -- start start 2024-04-02T04:38:40.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb252835700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c07bea0 0x7fb24c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:40.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb252835700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c07a8d0 con 0x7fb24c07bea0 2024-04-02T04:38:40.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c07bea0 0x7fb24c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c07bea0 0x7fb24c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57320/0 (socket says 172.21.15.70:57320) 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb24bfff700 1 -- 172.21.15.70:0/783384882 learned_addr learned my addr 172.21.15.70:0/783384882 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb24bfff700 1 -- 172.21.15.70:0/783384882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c07aa10 con 0x7fb24c07bea0 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.175+0000 7fb24bfff700 1 --2- 172.21.15.70:0/783384882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c07bea0 0x7fb24c07a300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0098a0 tx=0x7fb23c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357f47689855d7e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb24affd700 1 -- 172.21.15.70:0/783384882 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb23c004180 con 0x7fb24c07bea0 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb24affd700 1 -- 172.21.15.70:0/783384882 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb23c0042e0 con 0x7fb24c07bea0 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb24affd700 1 -- 172.21.15.70:0/783384882 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb23c011af0 con 0x7fb24c07bea0 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb252835700 1 -- 172.21.15.70:0/783384882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c07bea0 msgr2=0x7fb24c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb252835700 1 --2- 172.21.15.70:0/783384882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c07bea0 0x7fb24c07a300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0098a0 tx=0x7fb23c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb252835700 1 -- 172.21.15.70:0/783384882 shutdown_connections 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb252835700 1 --2- 172.21.15.70:0/783384882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c07bea0 0x7fb24c07a300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb252835700 1 -- 172.21.15.70:0/783384882 >> 172.21.15.70:0/783384882 conn(0x7fb24c101b40 msgr2=0x7fb24c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb252835700 1 -- 172.21.15.70:0/783384882 shutdown_connections 2024-04-02T04:38:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.176+0000 7fb252835700 1 -- 172.21.15.70:0/783384882 wait complete. 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.177+0000 7fb252835700 1 Processor -- start 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.177+0000 7fb252835700 1 -- start start 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.177+0000 7fb252835700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c19c650 0x7fb24c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.177+0000 7fb252835700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c19cfb0 con 0x7fb24c19c650 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.177+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c19c650 0x7fb24c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.177+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c19c650 0x7fb24c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57336/0 (socket says 172.21.15.70:57336) 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.177+0000 7fb24bfff700 1 -- 172.21.15.70:0/1358704647 learned_addr learned my addr 172.21.15.70:0/1358704647 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.178+0000 7fb24bfff700 1 -- 172.21.15.70:0/1358704647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c009550 con 0x7fb24c19c650 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.178+0000 7fb24bfff700 1 --2- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c19c650 0x7fb24c19ca70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0098a0 tx=0x7fb23c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.178+0000 7fb2497fa700 1 -- 172.21.15.70:0/1358704647 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb23c011ba0 con 0x7fb24c19c650 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.178+0000 7fb2497fa700 1 -- 172.21.15.70:0/1358704647 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb23c01f070 con 0x7fb24c19c650 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.178+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24c19d1b0 con 0x7fb24c19c650 2024-04-02T04:38:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.178+0000 7fb2497fa700 1 -- 172.21.15.70:0/1358704647 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb23c01ad90 con 0x7fb24c19c650 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.178+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24c19fe00 con 0x7fb24c19c650 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.179+0000 7fb2497fa700 1 -- 172.21.15.70:0/1358704647 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb23c021030 con 0x7fb24c19c650 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.179+0000 7fb2497fa700 1 --2- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fb234038320 0x7fb23403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.179+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24c10c900 con 0x7fb24c19c650 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.179+0000 7fb2497fa700 1 -- 172.21.15.70:0/1358704647 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb23c04c1f0 con 0x7fb24c19c650 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.179+0000 7fb24b7fe700 1 --2- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fb234038320 0x7fb23403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.180+0000 7fb24b7fe700 1 --2- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fb234038320 0x7fb23403a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb240006fd0 tx=0x7fb240006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.184+0000 7fb2497fa700 1 -- 172.21.15.70:0/1358704647 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb23c02d440 con 0x7fb24c19c650 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.385+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fb24c103e20 con 0x7fb234038320 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.390+0000 7fb2497fa700 1 -- 172.21.15.70:0/1358704647 <== mgr.14120 v2:172.21.15.70:6800/2939436653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fb24c103e20 con 0x7fb234038320 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fb234038320 msgr2=0x7fb23403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:40.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 --2- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fb234038320 0x7fb23403a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb240006fd0 tx=0x7fb240006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:40.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c19c650 msgr2=0x7fb24c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:40.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 --2- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c19c650 0x7fb24c19ca70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0098a0 tx=0x7fb23c011870 comp rx=0 tx=0).stop 2024-04-02T04:38:40.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 shutdown_connections 2024-04-02T04:38:40.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 --2- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fb234038320 0x7fb23403a7e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 --2- 172.21.15.70:0/1358704647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb24c19c650 0x7fb24c19ca70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 >> 172.21.15.70:0/1358704647 conn(0x7fb24c101b40 msgr2=0x7fb24c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:40.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 shutdown_connections 2024-04-02T04:38:40.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:40.392+0000 7fb252835700 1 -- 172.21.15.70:0/1358704647 wait complete. 2024-04-02T04:38:41.602 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:41 smithi070 ceph-mon[29212]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:41.602 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:41 smithi070 ceph-mon[29212]: Saving service alertmanager spec with placement count:1 2024-04-02T04:38:41.602 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:41 smithi070 ceph-mon[29212]: from='mgr.14120 172.21.15.70:0/1013569447' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:41.602 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:41 smithi070 ceph-mon[29212]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:38:42.230 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb7a8e700 1 Processor -- start 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb7a8e700 1 -- start start 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb7a8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb0104c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb7a8e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb0105230 con 0x7f5fb0104840 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb582a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb0104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb582a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb0104c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57340/0 (socket says 172.21.15.70:57340) 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb582a700 1 -- 172.21.15.70:0/3301335272 learned_addr learned my addr 172.21.15.70:0/3301335272 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb582a700 1 -- 172.21.15.70:0/3301335272 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb01077d0 con 0x7f5fb0104840 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.383+0000 7f5fb582a700 1 --2- 172.21.15.70:0/3301335272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb0104c60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00098a0 tx=0x7f5fa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c14da2049a60d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb4828700 1 -- 172.21.15.70:0/3301335272 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fa0004180 con 0x7f5fb0104840 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb4828700 1 -- 172.21.15.70:0/3301335272 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5fa00042e0 con 0x7f5fb0104840 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb4828700 1 -- 172.21.15.70:0/3301335272 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fa0011af0 con 0x7f5fb0104840 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3301335272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 msgr2=0x7f5fb0104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:42.231 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb7a8e700 1 --2- 172.21.15.70:0/3301335272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb0104c60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00098a0 tx=0x7f5fa0009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3301335272 shutdown_connections 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb7a8e700 1 --2- 172.21.15.70:0/3301335272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb0104c60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3301335272 >> 172.21.15.70:0/3301335272 conn(0x7f5fb0100190 msgr2=0x7f5fb01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3301335272 shutdown_connections 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.384+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3301335272 wait complete. 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.385+0000 7f5fb7a8e700 1 Processor -- start 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.385+0000 7f5fb7a8e700 1 -- start start 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.385+0000 7f5fb7a8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.385+0000 7f5fb7a8e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb019a7e0 con 0x7f5fb0104840 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.385+0000 7f5fb582a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.385+0000 7f5fb582a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57344/0 (socket says 172.21.15.70:57344) 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.385+0000 7f5fb582a700 1 -- 172.21.15.70:0/3401937086 learned_addr learned my addr 172.21.15.70:0/3401937086 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:42.232 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.386+0000 7f5fb582a700 1 -- 172.21.15.70:0/3401937086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa0009550 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.386+0000 7f5fb582a700 1 --2- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb019a2a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0011670 tx=0x7f5fa0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.386+0000 7f5fa6ffd700 1 -- 172.21.15.70:0/3401937086 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fa00039c0 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.386+0000 7f5fa6ffd700 1 -- 172.21.15.70:0/3401937086 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5fa0003b20 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.386+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb019a9e0 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.386+0000 7f5fa6ffd700 1 -- 172.21.15.70:0/3401937086 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fa00247f0 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.386+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb019ae80 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.387+0000 7f5fa6ffd700 1 -- 172.21.15.70:0/3401937086 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f5fa001a430 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.387+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb00623c0 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.387+0000 7f5fa6ffd700 1 --2- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5f9c038330 0x7f5f9c03a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.387+0000 7f5fa6ffd700 1 -- 172.21.15.70:0/3401937086 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5fa004c470 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.387+0000 7f5fb5029700 1 --2- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5f9c038330 0x7f5f9c03a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.387+0000 7f5fb5029700 1 --2- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5f9c038330 0x7f5f9c03a7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5fac006fd0 tx=0x7f5fac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.392+0000 7f5fa6ffd700 1 -- 172.21.15.70:0/3401937086 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5fa001a6e0 con 0x7f5fb0104840 2024-04-02T04:38:42.233 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.581+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f5fb019dcc0 con 0x7f5fb0104840 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.584+0000 7f5fa6ffd700 1 -- 172.21.15.70:0/3401937086 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f5fa0018350 con 0x7f5fb0104840 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5f9c038330 msgr2=0x7f5f9c03a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 --2- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5f9c038330 0x7f5f9c03a7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5fac006fd0 tx=0x7f5fac006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 msgr2=0x7f5fb019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 --2- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb019a2a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0011670 tx=0x7f5fa0011aa0 comp rx=0 tx=0).stop 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 shutdown_connections 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 --2- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f5f9c038330 0x7f5f9c03a7f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 --2- 172.21.15.70:0/3401937086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb0104840 0x7f5fb019a2a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 >> 172.21.15.70:0/3401937086 conn(0x7f5fb0100190 msgr2=0x7f5fb0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 shutdown_connections 2024-04-02T04:38:42.234 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:41.586+0000 7f5fb7a8e700 1 -- 172.21.15.70:0/3401937086 wait complete. 2024-04-02T04:38:42.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:42 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3401937086' entity='client.admin' 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.655+0000 7fe7f9412700 1 Processor -- start 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.655+0000 7fe7f9412700 1 -- start start 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.655+0000 7fe7f9412700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f41093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.655+0000 7fe7f9412700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4074680 con 0x7fe7f4106fe0 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.656+0000 7fe7f2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f41093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.656+0000 7fe7f2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f41093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57356/0 (socket says 172.21.15.70:57356) 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.656+0000 7fe7f2ffd700 1 -- 172.21.15.70:0/1243631986 learned_addr learned my addr 172.21.15.70:0/1243631986 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.656+0000 7fe7f2ffd700 1 -- 172.21.15.70:0/1243631986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4109910 con 0x7fe7f4106fe0 2024-04-02T04:38:43.411 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.656+0000 7fe7f2ffd700 1 --2- 172.21.15.70:0/1243631986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f41093d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009e80 tx=0x7fe7e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc92025c7977fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.656+0000 7fe7f1ffb700 1 -- 172.21.15.70:0/1243631986 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7e400e070 con 0x7fe7f4106fe0 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.656+0000 7fe7f1ffb700 1 -- 172.21.15.70:0/1243631986 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe7e400b8c0 con 0x7fe7f4106fe0 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.656+0000 7fe7f1ffb700 1 -- 172.21.15.70:0/1243631986 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7e40044d0 con 0x7fe7f4106fe0 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.657+0000 7fe7f9412700 1 -- 172.21.15.70:0/1243631986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 msgr2=0x7fe7f41093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.657+0000 7fe7f9412700 1 --2- 172.21.15.70:0/1243631986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f41093d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009e80 tx=0x7fe7e400b330 comp rx=0 tx=0).stop 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.657+0000 7fe7f9412700 1 -- 172.21.15.70:0/1243631986 shutdown_connections 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.657+0000 7fe7f9412700 1 --2- 172.21.15.70:0/1243631986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f41093d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.657+0000 7fe7f9412700 1 -- 172.21.15.70:0/1243631986 >> 172.21.15.70:0/1243631986 conn(0x7fe7f41009f0 msgr2=0x7fe7f4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:43.412 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.657+0000 7fe7f9412700 1 -- 172.21.15.70:0/1243631986 shutdown_connections 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.657+0000 7fe7f9412700 1 -- 172.21.15.70:0/1243631986 wait complete. 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.657+0000 7fe7f9412700 1 Processor -- start 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f9412700 1 -- start start 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f9412700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f9412700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f419a660 con 0x7fe7f4106fe0 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f419a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57368/0 (socket says 172.21.15.70:57368) 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f2ffd700 1 -- 172.21.15.70:0/1970371725 learned_addr learned my addr 172.21.15.70:0/1970371725 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f2ffd700 1 -- 172.21.15.70:0/1970371725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4009b30 con 0x7fe7f4106fe0 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f2ffd700 1 --2- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f419a120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4004070 tx=0x7fe7e40049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7ebfff700 1 -- 172.21.15.70:0/1970371725 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7e400e040 con 0x7fe7f4106fe0 2024-04-02T04:38:43.413 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7ebfff700 1 -- 172.21.15.70:0/1970371725 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe7e400be00 con 0x7fe7f4106fe0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f419a860 con 0x7fe7f4106fe0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7ebfff700 1 -- 172.21.15.70:0/1970371725 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7e4012770 con 0x7fe7f4106fe0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.658+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f419ad60 con 0x7fe7f4106fe0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.659+0000 7fe7ebfff700 1 -- 172.21.15.70:0/1970371725 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe7e4019040 con 0x7fe7f4106fe0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.659+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f410a700 con 0x7fe7f4106fe0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.659+0000 7fe7ebfff700 1 --2- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe7e0038320 0x7fe7e003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.659+0000 7fe7ebfff700 1 -- 172.21.15.70:0/1970371725 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe7e404bd70 con 0x7fe7f4106fe0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.660+0000 7fe7f27fc700 1 --2- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe7e0038320 0x7fe7e003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.660+0000 7fe7f27fc700 1 --2- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe7e0038320 0x7fe7e003a7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006fd0 tx=0x7fe7dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.664+0000 7fe7ebfff700 1 -- 172.21.15.70:0/1970371725 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe7e4011750 con 0x7fe7f4106fe0 2024-04-02T04:38:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.852+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe7f419dae0 con 0x7fe7f4106fe0 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.855+0000 7fe7ebfff700 1 -- 172.21.15.70:0/1970371725 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe7e4011390 con 0x7fe7f4106fe0 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe7e0038320 msgr2=0x7fe7e003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 --2- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe7e0038320 0x7fe7e003a7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006fd0 tx=0x7fe7dc006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 msgr2=0x7fe7f419a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 --2- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f419a120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4004070 tx=0x7fe7e40049f0 comp rx=0 tx=0).stop 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 shutdown_connections 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 --2- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fe7e0038320 0x7fe7e003a7e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 --2- 172.21.15.70:0/1970371725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7f4106fe0 0x7fe7f419a120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 >> 172.21.15.70:0/1970371725 conn(0x7fe7f41009f0 msgr2=0x7fe7f41016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 shutdown_connections 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:42.857+0000 7fe7f9412700 1 -- 172.21.15.70:0/1970371725 wait complete. 2024-04-02T04:38:43.415 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling the dashboard module... 2024-04-02T04:38:43.789 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:43 smithi070 ceph-mon[29212]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:38:43.789 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:43 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1970371725' entity='client.admin' 2024-04-02T04:38:44.902 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:44 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/306354066' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.893+0000 7f4fba5db700 1 Processor -- start 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.893+0000 7f4fba5db700 1 -- start start 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.893+0000 7f4fba5db700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb4104d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.893+0000 7f4fba5db700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41052e0 con 0x7f4fb41048f0 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.893+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb4104d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.893+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb4104d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57376/0 (socket says 172.21.15.70:57376) 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.893+0000 7f4fb3fff700 1 -- 172.21.15.70:0/3363313517 learned_addr learned my addr 172.21.15.70:0/3363313517 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.893+0000 7f4fb3fff700 1 -- 172.21.15.70:0/3363313517 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4107890 con 0x7f4fb41048f0 2024-04-02T04:38:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fb3fff700 1 --2- 172.21.15.70:0/3363313517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb4104d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40098a0 tx=0x7f4fa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9da5860748cc15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fb2ffd700 1 -- 172.21.15.70:0/3363313517 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa4004180 con 0x7f4fb41048f0 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fb2ffd700 1 -- 172.21.15.70:0/3363313517 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4fa40042e0 con 0x7f4fb41048f0 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fb2ffd700 1 -- 172.21.15.70:0/3363313517 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa4011af0 con 0x7f4fb41048f0 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fba5db700 1 -- 172.21.15.70:0/3363313517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 msgr2=0x7f4fb4104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fba5db700 1 --2- 172.21.15.70:0/3363313517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb4104d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40098a0 tx=0x7f4fa4009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fba5db700 1 -- 172.21.15.70:0/3363313517 shutdown_connections 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fba5db700 1 --2- 172.21.15.70:0/3363313517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb4104d10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.894+0000 7f4fba5db700 1 -- 172.21.15.70:0/3363313517 >> 172.21.15.70:0/3363313517 conn(0x7f4fb41001d0 msgr2=0x7f4fb41025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.895+0000 7f4fba5db700 1 -- 172.21.15.70:0/3363313517 shutdown_connections 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.895+0000 7f4fba5db700 1 -- 172.21.15.70:0/3363313517 wait complete. 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.895+0000 7f4fba5db700 1 Processor -- start 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.895+0000 7f4fba5db700 1 -- start start 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.895+0000 7f4fba5db700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb41071f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.895+0000 7f4fba5db700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb4107730 con 0x7f4fb41048f0 2024-04-02T04:38:45.169 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb41071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fb3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb41071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57392/0 (socket says 172.21.15.70:57392) 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fb3fff700 1 -- 172.21.15.70:0/306354066 learned_addr learned my addr 172.21.15.70:0/306354066 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fb3fff700 1 -- 172.21.15.70:0/306354066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4009550 con 0x7f4fb41048f0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fb3fff700 1 --2- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb41071f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4011670 tx=0x7f4fa4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fb17fa700 1 -- 172.21.15.70:0/306354066 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa4003a40 con 0x7f4fb41048f0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb4105840 con 0x7f4fb41048f0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fb17fa700 1 -- 172.21.15.70:0/306354066 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4fa4003ba0 con 0x7f4fb41048f0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb4105ce0 con 0x7f4fb41048f0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.896+0000 7f4fb17fa700 1 -- 172.21.15.70:0/306354066 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa40246c0 con 0x7f4fb41048f0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.897+0000 7f4fb17fa700 1 -- 172.21.15.70:0/306354066 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f4fa4024820 con 0x7f4fb41048f0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.897+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb40623c0 con 0x7f4fb41048f0 2024-04-02T04:38:45.170 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.897+0000 7f4fb17fa700 1 --2- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4f9c0382e0 0x7f4f9c03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.897+0000 7f4fb17fa700 1 -- 172.21.15.70:0/306354066 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4fa404c330 con 0x7f4fb41048f0 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.897+0000 7f4fb37fe700 1 --2- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4f9c0382e0 0x7f4f9c03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.898+0000 7f4fb37fe700 1 --2- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4f9c0382e0 0x7f4f9c03a7a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8006fd0 tx=0x7f4fa8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:43.902+0000 7f4fb17fa700 1 -- 172.21.15.70:0/306354066 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4fa4018530 con 0x7f4fb41048f0 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.148+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f4fb410ab10 con 0x7f4fb41048f0 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.590+0000 7f4fb17fa700 1 -- 172.21.15.70:0/306354066 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f4fa401ac70 con 0x7f4fb41048f0 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.591+0000 7f4fb17fa700 1 -- 172.21.15.70:0/306354066 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f4fa404c6d0 con 0x7f4fb41048f0 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4f9c0382e0 msgr2=0x7f4f9c03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 --2- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4f9c0382e0 0x7f4f9c03a7a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8006fd0 tx=0x7f4fa8006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 msgr2=0x7f4fb41071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 --2- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb41071f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4011670 tx=0x7f4fa4011aa0 comp rx=0 tx=0).stop 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 shutdown_connections 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 --2- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4f9c0382e0 0x7f4f9c03a7a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:45.171 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 --2- 172.21.15.70:0/306354066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb41048f0 0x7f4fb41071f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:45.172 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 >> 172.21.15.70:0/306354066 conn(0x7f4fb41001d0 msgr2=0x7f4fb4100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:45.172 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 shutdown_connections 2024-04-02T04:38:45.172 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:44.594+0000 7f4fba5db700 1 -- 172.21.15.70:0/306354066 wait complete. 2024-04-02T04:38:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:45 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/306354066' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-02T04:38:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:45 smithi070 ceph-mon[29212]: mgrmap e9: smithi070.igidvr(active, since 26s) 2024-04-02T04:38:46.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:38:46.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-02T04:38:46.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T04:38:46.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "smithi070.igidvr", 2024-04-02T04:38:46.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T04:38:46.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:38:46.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.588+0000 7fc7028a7700 1 Processor -- start 2024-04-02T04:38:46.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.588+0000 7fc7028a7700 1 -- start start 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.588+0000 7fc7028a7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.588+0000 7fc7028a7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc079e10 con 0x7fc6fc104e50 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.589+0000 7fc6fbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.589+0000 7fc6fbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc105270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57418/0 (socket says 172.21.15.70:57418) 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.589+0000 7fc6fbfff700 1 -- 172.21.15.70:0/4142579723 learned_addr learned my addr 172.21.15.70:0/4142579723 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.589+0000 7fc6fbfff700 1 -- 172.21.15.70:0/4142579723 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc1057b0 con 0x7fc6fc104e50 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.589+0000 7fc6fbfff700 1 --2- 172.21.15.70:0/4142579723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc105270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0098a0 tx=0x7fc6ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1294d4f988019155 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc6faffd700 1 -- 172.21.15.70:0/4142579723 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6ec004180 con 0x7fc6fc104e50 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc6faffd700 1 -- 172.21.15.70:0/4142579723 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc6ec0042e0 con 0x7fc6fc104e50 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc6faffd700 1 -- 172.21.15.70:0/4142579723 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6ec011af0 con 0x7fc6fc104e50 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc7028a7700 1 -- 172.21.15.70:0/4142579723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 msgr2=0x7fc6fc105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:46.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc7028a7700 1 --2- 172.21.15.70:0/4142579723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc105270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0098a0 tx=0x7fc6ec009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc7028a7700 1 -- 172.21.15.70:0/4142579723 shutdown_connections 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc7028a7700 1 --2- 172.21.15.70:0/4142579723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc105270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc7028a7700 1 -- 172.21.15.70:0/4142579723 >> 172.21.15.70:0/4142579723 conn(0x7fc6fc1009f0 msgr2=0x7fc6fc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc7028a7700 1 -- 172.21.15.70:0/4142579723 shutdown_connections 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.592+0000 7fc7028a7700 1 -- 172.21.15.70:0/4142579723 wait complete. 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.593+0000 7fc7028a7700 1 Processor -- start 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.593+0000 7fc7028a7700 1 -- start start 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.593+0000 7fc7028a7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.593+0000 7fc7028a7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc19c8e0 con 0x7fc6fc104e50 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.593+0000 7fc6fbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc6fbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57420/0 (socket says 172.21.15.70:57420) 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc6fbfff700 1 -- 172.21.15.70:0/105678826 learned_addr learned my addr 172.21.15.70:0/105678826 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc6fbfff700 1 -- 172.21.15.70:0/105678826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec009550 con 0x7fc6fc104e50 2024-04-02T04:38:46.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc6fbfff700 1 --2- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc19c3a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec006ba0 tx=0x7fc6ec011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc6f97fa700 1 -- 172.21.15.70:0/105678826 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6ec003b00 con 0x7fc6fc104e50 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc6f97fa700 1 -- 172.21.15.70:0/105678826 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc6ec01a8e0 con 0x7fc6fc104e50 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc19cae0 con 0x7fc6fc104e50 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc6f97fa700 1 -- 172.21.15.70:0/105678826 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6ec0199c0 con 0x7fc6fc104e50 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.594+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc19cf80 con 0x7fc6fc104e50 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.595+0000 7fc6f97fa700 1 -- 172.21.15.70:0/105678826 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fc6ec019c80 con 0x7fc6fc104e50 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.595+0000 7fc6f97fa700 1 --2- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc6e4038320 0x7fc6e403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.595+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc195c30 con 0x7fc6fc104e50 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.595+0000 7fc6f97fa700 1 -- 172.21.15.70:0/105678826 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc6ec04c600 con 0x7fc6fc104e50 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.595+0000 7fc6fb7fe700 1 -- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc6e4038320 msgr2=0x7fc6e403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2939436653 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.595+0000 7fc6fb7fe700 1 --2- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc6e4038320 0x7fc6e403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:38:46.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.600+0000 7fc6f97fa700 1 -- 172.21.15.70:0/105678826 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc6ec048080 con 0x7fc6fc104e50 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.796+0000 7fc6fb7fe700 1 -- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc6e4038320 msgr2=0x7fc6e403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2939436653 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.796+0000 7fc6fb7fe700 1 --2- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc6e4038320 0x7fc6e403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.847+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc6fc02cc70 con 0x7fc6fc104e50 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.848+0000 7fc6f97fa700 1 -- 172.21.15.70:0/105678826 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fc6ec02d440 con 0x7fc6fc104e50 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc6e4038320 msgr2=0x7fc6e403a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 --2- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc6e4038320 0x7fc6e403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 msgr2=0x7fc6fc19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 --2- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc19c3a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec006ba0 tx=0x7fc6ec011b50 comp rx=0 tx=0).stop 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 shutdown_connections 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 --2- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7fc6e4038320 0x7fc6e403a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 --2- 172.21.15.70:0/105678826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc104e50 0x7fc6fc19c3a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 >> 172.21.15.70:0/105678826 conn(0x7fc6fc1009f0 msgr2=0x7fc6fc190a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 shutdown_connections 2024-04-02T04:38:46.511 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:45.850+0000 7fc7028a7700 1 -- 172.21.15.70:0/105678826 wait complete. 2024-04-02T04:38:46.512 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2024-04-02T04:38:46.512 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 9... 2024-04-02T04:38:46.753 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:46 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/105678826' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T04:38:52.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: Active manager daemon smithi070.igidvr restarted 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: Activating manager daemon smithi070.igidvr 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: osdmap e3: 0 total, 0 up, 0 in 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: mgrmap e10: smithi070.igidvr(active, starting, since 0.00332187s) 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: Manager daemon smithi070.igidvr is now available 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:52.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:38:52.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/trash_purge_schedule"}]: dispatch 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.982+0000 7f4b27d34700 1 Processor -- start 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.982+0000 7f4b27d34700 1 -- start start 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.982+0000 7f4b27d34700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b20109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.982+0000 7f4b27d34700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20079e50 con 0x7f4b20107020 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.982+0000 7f4b25ad0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b20109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:53.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.982+0000 7f4b25ad0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b20109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39276/0 (socket says 172.21.15.70:39276) 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.982+0000 7f4b25ad0700 1 -- 172.21.15.70:0/566415225 learned_addr learned my addr 172.21.15.70:0/566415225 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.983+0000 7f4b25ad0700 1 -- 172.21.15.70:0/566415225 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b20109950 con 0x7f4b20107020 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.983+0000 7f4b25ad0700 1 --2- 172.21.15.70:0/566415225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b20109410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0098a0 tx=0x7f4b1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc80fed79fb5530d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.983+0000 7f4b24ace700 1 -- 172.21.15.70:0/566415225 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b1c004180 con 0x7f4b20107020 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.983+0000 7f4b24ace700 1 -- 172.21.15.70:0/566415225 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4b1c0042e0 con 0x7f4b20107020 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.983+0000 7f4b24ace700 1 -- 172.21.15.70:0/566415225 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b1c011af0 con 0x7f4b20107020 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.984+0000 7f4b27d34700 1 -- 172.21.15.70:0/566415225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 msgr2=0x7f4b20109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.984+0000 7f4b27d34700 1 --2- 172.21.15.70:0/566415225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b20109410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0098a0 tx=0x7f4b1c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.984+0000 7f4b27d34700 1 -- 172.21.15.70:0/566415225 shutdown_connections 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.984+0000 7f4b27d34700 1 --2- 172.21.15.70:0/566415225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b20109410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.984+0000 7f4b27d34700 1 -- 172.21.15.70:0/566415225 >> 172.21.15.70:0/566415225 conn(0x7f4b201009f0 msgr2=0x7f4b20102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.984+0000 7f4b27d34700 1 -- 172.21.15.70:0/566415225 shutdown_connections 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.984+0000 7f4b27d34700 1 -- 172.21.15.70:0/566415225 wait complete. 2024-04-02T04:38:53.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b27d34700 1 Processor -- start 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b27d34700 1 -- start start 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b27d34700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b2019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b27d34700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2019a6f0 con 0x7f4b20107020 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b25ad0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b2019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b25ad0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b2019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39292/0 (socket says 172.21.15.70:39292) 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b25ad0700 1 -- 172.21.15.70:0/1775307386 learned_addr learned my addr 172.21.15.70:0/1775307386 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b25ad0700 1 -- 172.21.15.70:0/1775307386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c009550 con 0x7f4b20107020 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b25ad0700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b2019a1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c006ba0 tx=0x7f4b1c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.985+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b1c003b00 con 0x7f4b20107020 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.986+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4b1c01a8e0 con 0x7f4b20107020 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.986+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b1c0199c0 con 0x7f4b20107020 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.986+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2019a8f0 con 0x7f4b20107020 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.986+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2019ad90 con 0x7f4b20107020 2024-04-02T04:38:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.987+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f4b1c019c80 con 0x7f4b20107020 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.987+0000 7f4b16ffd700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 0x7f4b0c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.987+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 --> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b0c03aef0 con 0x7f4b0c038320 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.987+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4b1c04c610 con 0x7f4b20107020 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.987+0000 7f4b252cf700 1 -- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 msgr2=0x7f4b0c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2939436653 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:46.987+0000 7f4b252cf700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 0x7f4b0c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:47.188+0000 7f4b252cf700 1 -- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 msgr2=0x7f4b0c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2939436653 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:47.188+0000 7f4b252cf700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 0x7f4b0c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:47.589+0000 7f4b252cf700 1 -- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 msgr2=0x7f4b0c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2939436653 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:47.589+0000 7f4b252cf700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 0x7f4b0c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:48.390+0000 7f4b252cf700 1 -- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 msgr2=0x7f4b0c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2939436653 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:48.390+0000 7f4b252cf700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 0x7f4b0c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T04:38:53.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:49.992+0000 7f4b252cf700 1 -- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 msgr2=0x7f4b0c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2939436653 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:49.992+0000 7f4b252cf700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 0x7f4b0c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:51.246+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f4b1c02c3a0 con 0x7f4b20107020 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:51.247+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 msgr2=0x7f4b0c03a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:51.247+0000 7f4b16ffd700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 0x7f4b0c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.248+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f4b1c04e410 con 0x7f4b20107020 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.248+0000 7f4b16ffd700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4b0c03bd90 0x7f4b0c03e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.248+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 --> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b0c03aef0 con 0x7f4b0c03bd90 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.249+0000 7f4b252cf700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4b0c03bd90 0x7f4b0c03e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.249+0000 7f4b252cf700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4b0c03bd90 0x7f4b0c03e180 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4b10003a10 tx=0x7f4b100092b0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.250+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mgr.14164 v2:172.21.15.70:6800/2440490727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f4b0c03aef0 con 0x7f4b0c03bd90 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.256+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 --> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4b2019b4f0 con 0x7f4b0c03bd90 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.256+0000 7f4b16ffd700 1 -- 172.21.15.70:0/1775307386 <== mgr.14164 v2:172.21.15.70:6800/2440490727 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f4b2019b4f0 con 0x7f4b0c03bd90 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4b0c03bd90 msgr2=0x7f4b0c03e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:53.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4b0c03bd90 0x7f4b0c03e180 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4b10003a10 tx=0x7f4b100092b0 comp rx=0 tx=0).stop 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 msgr2=0x7f4b2019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b2019a1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c006ba0 tx=0x7f4b1c011b50 comp rx=0 tx=0).stop 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 shutdown_connections 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4b0c03bd90 0x7f4b0c03e180 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b20107020 0x7f4b2019a1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 --2- 172.21.15.70:0/1775307386 >> [v2:172.21.15.70:6800/2939436653,v1:172.21.15.70:6801/2939436653] conn(0x7f4b0c038320 0x7f4b0c03a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 >> 172.21.15.70:0/1775307386 conn(0x7f4b201009f0 msgr2=0x7f4b20101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 shutdown_connections 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:52.257+0000 7f4b27d34700 1 -- 172.21.15.70:0/1775307386 wait complete. 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 9 is available 2024-04-02T04:38:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:Generating a dashboard self-signed certificate... 2024-04-02T04:38:53.442 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:53 smithi070 ceph-mon[29212]: [02/Apr/2024:04:38:51] ENGINE Bus STARTING 2024-04-02T04:38:53.442 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:53 smithi070 ceph-mon[29212]: [02/Apr/2024:04:38:51] ENGINE Serving on http://172.21.15.70:8765 2024-04-02T04:38:53.442 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:53 smithi070 ceph-mon[29212]: [02/Apr/2024:04:38:52] ENGINE Serving on https://172.21.15.70:7150 2024-04-02T04:38:53.442 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:53 smithi070 ceph-mon[29212]: [02/Apr/2024:04:38:52] ENGINE Bus STARTED 2024-04-02T04:38:53.442 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:53 smithi070 ceph-mon[29212]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T04:38:53.442 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:53 smithi070 ceph-mon[29212]: mgrmap e11: smithi070.igidvr(active, since 1.00551s) 2024-04-02T04:38:53.443 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:53 smithi070 ceph-mon[29212]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T04:38:54.437 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:54 smithi070 ceph-mon[29212]: mgrmap e12: smithi070.igidvr(active, since 2s) 2024-04-02T04:38:54.437 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:54 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:54.438 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:54 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:54.438 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:54 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:54.438 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:54 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:54.445 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-02T04:38:54.445 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.475+0000 7feb9fea8700 1 Processor -- start 2024-04-02T04:38:54.445 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.475+0000 7feb9fea8700 1 -- start start 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.475+0000 7feb9fea8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb98106960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.475+0000 7feb9fea8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb98106f30 con 0x7feb98106540 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.476+0000 7feb9dc44700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb98106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.476+0000 7feb9dc44700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb98106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39358/0 (socket says 172.21.15.70:39358) 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.476+0000 7feb9dc44700 1 -- 172.21.15.70:0/1269034062 learned_addr learned my addr 172.21.15.70:0/1269034062 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.476+0000 7feb9dc44700 1 -- 172.21.15.70:0/1269034062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb98107740 con 0x7feb98106540 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.476+0000 7feb9dc44700 1 --2- 172.21.15.70:0/1269034062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb98106960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb880098a0 tx=0x7feb88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b8056a13f82a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.476+0000 7feb9cc42700 1 -- 172.21.15.70:0/1269034062 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb88004180 con 0x7feb98106540 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.476+0000 7feb9cc42700 1 -- 172.21.15.70:0/1269034062 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb880042e0 con 0x7feb98106540 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.476+0000 7feb9cc42700 1 -- 172.21.15.70:0/1269034062 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb88011af0 con 0x7feb98106540 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.477+0000 7feb9fea8700 1 -- 172.21.15.70:0/1269034062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 msgr2=0x7feb98106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:54.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.477+0000 7feb9fea8700 1 --2- 172.21.15.70:0/1269034062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb98106960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb880098a0 tx=0x7feb88009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.477+0000 7feb9fea8700 1 -- 172.21.15.70:0/1269034062 shutdown_connections 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.477+0000 7feb9fea8700 1 --2- 172.21.15.70:0/1269034062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb98106960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.477+0000 7feb9fea8700 1 -- 172.21.15.70:0/1269034062 >> 172.21.15.70:0/1269034062 conn(0x7feb98101b00 msgr2=0x7feb98103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.477+0000 7feb9fea8700 1 -- 172.21.15.70:0/1269034062 shutdown_connections 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.477+0000 7feb9fea8700 1 -- 172.21.15.70:0/1269034062 wait complete. 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.478+0000 7feb9fea8700 1 Processor -- start 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.478+0000 7feb9fea8700 1 -- start start 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.478+0000 7feb9fea8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb9819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.478+0000 7feb9fea8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9819a7a0 con 0x7feb98106540 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.478+0000 7feb9dc44700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb9819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.478+0000 7feb9dc44700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb9819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39372/0 (socket says 172.21.15.70:39372) 2024-04-02T04:38:54.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.478+0000 7feb9dc44700 1 -- 172.21.15.70:0/2109523083 learned_addr learned my addr 172.21.15.70:0/2109523083 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.479+0000 7feb9dc44700 1 -- 172.21.15.70:0/2109523083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb88009550 con 0x7feb98106540 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.479+0000 7feb9dc44700 1 --2- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb9819a260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb88011670 tx=0x7feb88011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.479+0000 7feb8effd700 1 -- 172.21.15.70:0/2109523083 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb880039c0 con 0x7feb98106540 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.479+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9819a9a0 con 0x7feb98106540 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.479+0000 7feb8effd700 1 -- 172.21.15.70:0/2109523083 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb88003b20 con 0x7feb98106540 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.479+0000 7feb8effd700 1 -- 172.21.15.70:0/2109523083 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb880247f0 con 0x7feb98106540 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.479+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9819ae40 con 0x7feb98106540 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.480+0000 7feb8effd700 1 -- 172.21.15.70:0/2109523083 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7feb88024ab0 con 0x7feb98106540 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.480+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb980623c0 con 0x7feb98106540 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.480+0000 7feb8effd700 1 --2- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7feb84038330 0x7feb8403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:54.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.480+0000 7feb8effd700 1 -- 172.21.15.70:0/2109523083 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feb8804c630 con 0x7feb98106540 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.480+0000 7feb9d443700 1 --2- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7feb84038330 0x7feb8403a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.481+0000 7feb9d443700 1 --2- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7feb84038330 0x7feb8403a7f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feb94006fd0 tx=0x7feb94006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.485+0000 7feb8effd700 1 -- 172.21.15.70:0/2109523083 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feb88022310 con 0x7feb98106540 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.695+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 --> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7feb9810a940 con 0x7feb84038330 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.853+0000 7feb8effd700 1 -- 172.21.15.70:0/2109523083 <== mgr.14164 v2:172.21.15.70:6800/2440490727 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7feb9810a940 con 0x7feb84038330 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.856+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7feb84038330 msgr2=0x7feb8403a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.856+0000 7feb9fea8700 1 --2- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7feb84038330 0x7feb8403a7f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feb94006fd0 tx=0x7feb94006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.856+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 msgr2=0x7feb9819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.856+0000 7feb9fea8700 1 --2- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb9819a260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb88011670 tx=0x7feb88011aa0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.857+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 shutdown_connections 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.857+0000 7feb9fea8700 1 --2- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7feb84038330 0x7feb8403a7f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.857+0000 7feb9fea8700 1 --2- 172.21.15.70:0/2109523083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb98106540 0x7feb9819a260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.857+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 >> 172.21.15.70:0/2109523083 conn(0x7feb98101b00 msgr2=0x7feb981031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:54.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.857+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 shutdown_connections 2024-04-02T04:38:54.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:53.857+0000 7feb9fea8700 1 -- 172.21.15.70:0/2109523083 wait complete. 2024-04-02T04:38:54.450 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial admin user... 2024-04-02T04:38:55.465 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:55 smithi070 ceph-mon[29212]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:56.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$nAasrC2KDVJ8xKVQyDIv6uHNiPSzZFnuj9Ti3kMkhjdrvw2GH/xva", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712032735, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-02T04:38:56.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.876+0000 7fdcbfe46700 1 Processor -- start 2024-04-02T04:38:56.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.876+0000 7fdcbfe46700 1 -- start start 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.876+0000 7fdcbfe46700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8106e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.876+0000 7fdcbfe46700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8107410 con 0x7fdcb8106a20 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.876+0000 7fdcbdbe2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.876+0000 7fdcbdbe2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8106e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39374/0 (socket says 172.21.15.70:39374) 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.876+0000 7fdcbdbe2700 1 -- 172.21.15.70:0/2013711957 learned_addr learned my addr 172.21.15.70:0/2013711957 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.877+0000 7fdcbdbe2700 1 -- 172.21.15.70:0/2013711957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8105680 con 0x7fdcb8106a20 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.877+0000 7fdcbdbe2700 1 --2- 172.21.15.70:0/2013711957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8106e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40098a0 tx=0x7fdcb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f18faef25bc97a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.877+0000 7fdcbcbe0700 1 -- 172.21.15.70:0/2013711957 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb4004180 con 0x7fdcb8106a20 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.877+0000 7fdcbcbe0700 1 -- 172.21.15.70:0/2013711957 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdcb40042e0 con 0x7fdcb8106a20 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.877+0000 7fdcbcbe0700 1 -- 172.21.15.70:0/2013711957 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb4011af0 con 0x7fdcb8106a20 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.878+0000 7fdcbfe46700 1 -- 172.21.15.70:0/2013711957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 msgr2=0x7fdcb8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.878+0000 7fdcbfe46700 1 --2- 172.21.15.70:0/2013711957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8106e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40098a0 tx=0x7fdcb4009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.878+0000 7fdcbfe46700 1 -- 172.21.15.70:0/2013711957 shutdown_connections 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.878+0000 7fdcbfe46700 1 --2- 172.21.15.70:0/2013711957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8106e40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.878+0000 7fdcbfe46700 1 -- 172.21.15.70:0/2013711957 >> 172.21.15.70:0/2013711957 conn(0x7fdcb8100170 msgr2=0x7fdcb81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.878+0000 7fdcbfe46700 1 -- 172.21.15.70:0/2013711957 shutdown_connections 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.878+0000 7fdcbfe46700 1 -- 172.21.15.70:0/2013711957 wait complete. 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.879+0000 7fdcbfe46700 1 Processor -- start 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.879+0000 7fdcbfe46700 1 -- start start 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.879+0000 7fdcbfe46700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8190640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.879+0000 7fdcbfe46700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8190b80 con 0x7fdcb8106a20 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.879+0000 7fdcbdbe2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8190640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.879+0000 7fdcbdbe2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8190640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39388/0 (socket says 172.21.15.70:39388) 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.879+0000 7fdcbdbe2700 1 -- 172.21.15.70:0/1935568011 learned_addr learned my addr 172.21.15.70:0/1935568011 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.879+0000 7fdcbdbe2700 1 -- 172.21.15.70:0/1935568011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb4009550 con 0x7fdcb8106a20 2024-04-02T04:38:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.880+0000 7fdcbdbe2700 1 --2- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8190640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4011670 tx=0x7fdcb4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.880+0000 7fdcaeffd700 1 -- 172.21.15.70:0/1935568011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb40039f0 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.880+0000 7fdcaeffd700 1 -- 172.21.15.70:0/1935568011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdcb4003b50 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.880+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb818ec90 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.880+0000 7fdcaeffd700 1 -- 172.21.15.70:0/1935568011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb40247c0 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.880+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb818f130 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.881+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb80623c0 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.881+0000 7fdcaeffd700 1 -- 172.21.15.70:0/1935568011 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fdcb401ac00 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.882+0000 7fdcaeffd700 1 --2- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdca40383c0 0x7fdca403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.882+0000 7fdcaeffd700 1 -- 172.21.15.70:0/1935568011 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdcb404c4b0 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.882+0000 7fdcbd3e1700 1 --2- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdca40383c0 0x7fdca403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.882+0000 7fdcbd3e1700 1 --2- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdca40383c0 0x7fdca403a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000ad30 tx=0x7fdcb00093f0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:54.886+0000 7fdcaeffd700 1 -- 172.21.15.70:0/1935568011 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdcb401a430 con 0x7fdcb8106a20 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.096+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 --> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdcb80621e0 con 0x7fdca40383c0 2024-04-02T04:38:56.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.320+0000 7fdcaeffd700 1 -- 172.21.15.70:0/1935568011 <== mgr.14164 v2:172.21.15.70:6800/2440490727 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fdcb80621e0 con 0x7fdca40383c0 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.322+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdca40383c0 msgr2=0x7fdca403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.322+0000 7fdcbfe46700 1 --2- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdca40383c0 0x7fdca403a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000ad30 tx=0x7fdcb00093f0 comp rx=0 tx=0).stop 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.322+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 msgr2=0x7fdcb8190640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.322+0000 7fdcbfe46700 1 --2- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8190640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4011670 tx=0x7fdcb4011b50 comp rx=0 tx=0).stop 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.323+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 shutdown_connections 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.323+0000 7fdcbfe46700 1 --2- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdca40383c0 0x7fdca403a880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.323+0000 7fdcbfe46700 1 --2- 172.21.15.70:0/1935568011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb8106a20 0x7fdcb8190640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.323+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 >> 172.21.15.70:0/1935568011 conn(0x7fdcb8100170 msgr2=0x7fdcb8104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.323+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 shutdown_connections 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:55.323+0000 7fdcbfe46700 1 -- 172.21.15.70:0/1935568011 wait complete. 2024-04-02T04:38:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:Fetching dashboard port number... 2024-04-02T04:38:56.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:56 smithi070 ceph-mon[29212]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:38:56.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:56 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:38:57.498 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:57 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1249467223' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-02T04:38:57.506 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 8443 2024-04-02T04:38:57.506 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.442+0000 7f708a56d700 1 Processor -- start 2024-04-02T04:38:57.506 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.442+0000 7f708a56d700 1 -- start start 2024-04-02T04:38:57.506 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.443+0000 7f708a56d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f7084106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.443+0000 7f708a56d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084106f50 con 0x7f7084106560 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.443+0000 7f7083fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f7084106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.443+0000 7f7083fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f7084106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39392/0 (socket says 172.21.15.70:39392) 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.443+0000 7f7083fff700 1 -- 172.21.15.70:0/241550860 learned_addr learned my addr 172.21.15.70:0/241550860 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.443+0000 7f7083fff700 1 -- 172.21.15.70:0/241550860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7084107760 con 0x7f7084106560 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.443+0000 7f7083fff700 1 --2- 172.21.15.70:0/241550860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f7084106980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70740098a0 tx=0x7f7074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1191e50e3c62f55e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.443+0000 7f7082ffd700 1 -- 172.21.15.70:0/241550860 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7074004180 con 0x7f7084106560 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f7082ffd700 1 -- 172.21.15.70:0/241550860 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f70740042e0 con 0x7f7084106560 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f7082ffd700 1 -- 172.21.15.70:0/241550860 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7074011af0 con 0x7f7084106560 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f708a56d700 1 -- 172.21.15.70:0/241550860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 msgr2=0x7f7084106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f708a56d700 1 --2- 172.21.15.70:0/241550860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f7084106980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70740098a0 tx=0x7f7074009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f708a56d700 1 -- 172.21.15.70:0/241550860 shutdown_connections 2024-04-02T04:38:57.507 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f708a56d700 1 --2- 172.21.15.70:0/241550860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f7084106980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f708a56d700 1 -- 172.21.15.70:0/241550860 >> 172.21.15.70:0/241550860 conn(0x7f7084101b40 msgr2=0x7f7084103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f708a56d700 1 -- 172.21.15.70:0/241550860 shutdown_connections 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.444+0000 7f708a56d700 1 -- 172.21.15.70:0/241550860 wait complete. 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.445+0000 7f708a56d700 1 Processor -- start 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.445+0000 7f708a56d700 1 -- start start 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.445+0000 7f708a56d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f708407bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.445+0000 7f708a56d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708407c170 con 0x7f7084106560 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.445+0000 7f7083fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f708407bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.445+0000 7f7083fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f708407bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39408/0 (socket says 172.21.15.70:39408) 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.445+0000 7f7083fff700 1 -- 172.21.15.70:0/1249467223 learned_addr learned my addr 172.21.15.70:0/1249467223 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.446+0000 7f7083fff700 1 -- 172.21.15.70:0/1249467223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7074009550 con 0x7f7084106560 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.446+0000 7f7083fff700 1 --2- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f708407bc30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7074006ba0 tx=0x7f7074011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.446+0000 7f70817fa700 1 -- 172.21.15.70:0/1249467223 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7074003b00 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.446+0000 7f70817fa700 1 -- 172.21.15.70:0/1249467223 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f707401a8e0 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.446+0000 7f70817fa700 1 -- 172.21.15.70:0/1249467223 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7074019990 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.446+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708407a280 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.446+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708407a720 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.447+0000 7f70817fa700 1 -- 172.21.15.70:0/1249467223 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f7074019c50 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.447+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70840623c0 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.447+0000 7f70817fa700 1 --2- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f706c038320 0x7f706c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.447+0000 7f70817fa700 1 -- 172.21.15.70:0/1249467223 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f707404c6b0 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.447+0000 7f70837fe700 1 --2- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f706c038320 0x7f706c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.448+0000 7f70837fe700 1 --2- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f706c038320 0x7f706c03a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7078006fd0 tx=0x7f7078006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.452+0000 7f70817fa700 1 -- 172.21.15.70:0/1249467223 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7074018530 con 0x7f7084106560 2024-04-02T04:38:57.509 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.645+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f708410a9e0 con 0x7f7084106560 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.645+0000 7f70817fa700 1 -- 172.21.15.70:0/1249467223 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f7074018350 con 0x7f7084106560 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f706c038320 msgr2=0x7f706c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 --2- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f706c038320 0x7f706c03a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7078006fd0 tx=0x7f7078006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 msgr2=0x7f708407bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 --2- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f708407bc30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7074006ba0 tx=0x7f7074011b50 comp rx=0 tx=0).stop 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 shutdown_connections 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 --2- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f706c038320 0x7f706c03a7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 --2- 172.21.15.70:0/1249467223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7084106560 0x7f708407bc30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 >> 172.21.15.70:0/1249467223 conn(0x7f7084101b40 msgr2=0x7f7084102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 shutdown_connections 2024-04-02T04:38:57.510 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:56.648+0000 7f708a56d700 1 -- 172.21.15.70:0/1249467223 wait complete. 2024-04-02T04:38:57.513 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T04:38:57.513 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-04-02T04:38:57.520 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T04:38:57.520 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-04-02T04:38:57.520 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-04-02T04:38:57.520 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-02T04:38:57.522 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph Dashboard is now available at: 2024-04-02T04:38:57.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:38:57.522 INFO:teuthology.orchestra.run.smithi070.stdout: URL: https://smithi070.front.sepia.ceph.com:8443/ 2024-04-02T04:38:57.522 INFO:teuthology.orchestra.run.smithi070.stdout: User: admin 2024-04-02T04:38:57.522 INFO:teuthology.orchestra.run.smithi070.stdout: Password: t27zdavabv 2024-04-02T04:38:57.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:38:57.523 INFO:teuthology.orchestra.run.smithi070.stdout:Saving cluster configuration to /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config directory 2024-04-02T04:38:57.523 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling autotune for osd_memory_target 2024-04-02T04:38:58.504 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:38:58 smithi070 ceph-mon[29212]: mgrmap e13: smithi070.igidvr(active, since 6s) 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.948+0000 7f6f2637b700 1 Processor -- start 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.948+0000 7f6f2637b700 1 -- start start 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.948+0000 7f6f2637b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f20104c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.948+0000 7f6f2637b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f20105240 con 0x7f6f20104850 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.948+0000 7f6f1ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f20104c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.948+0000 7f6f1ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f20104c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55462/0 (socket says 172.21.15.70:55462) 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.948+0000 7f6f1ffff700 1 -- 172.21.15.70:0/1669549204 learned_addr learned my addr 172.21.15.70:0/1669549204 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.948+0000 7f6f1ffff700 1 -- 172.21.15.70:0/1669549204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f201077e0 con 0x7f6f20104850 2024-04-02T04:38:58.788 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f1ffff700 1 --2- 172.21.15.70:0/1669549204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f20104c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f100098a0 tx=0x7f6f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5201751274ae10 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f1effd700 1 -- 172.21.15.70:0/1669549204 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f10004180 con 0x7f6f20104850 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f1effd700 1 -- 172.21.15.70:0/1669549204 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6f100042e0 con 0x7f6f20104850 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f1effd700 1 -- 172.21.15.70:0/1669549204 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f10011af0 con 0x7f6f20104850 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f2637b700 1 -- 172.21.15.70:0/1669549204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 msgr2=0x7f6f20104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f2637b700 1 --2- 172.21.15.70:0/1669549204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f20104c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f100098a0 tx=0x7f6f10009bb0 comp rx=0 tx=0).stop 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f2637b700 1 -- 172.21.15.70:0/1669549204 shutdown_connections 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f2637b700 1 --2- 172.21.15.70:0/1669549204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f20104c70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f2637b700 1 -- 172.21.15.70:0/1669549204 >> 172.21.15.70:0/1669549204 conn(0x7f6f201001a0 msgr2=0x7f6f201025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.949+0000 7f6f2637b700 1 -- 172.21.15.70:0/1669549204 shutdown_connections 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.950+0000 7f6f2637b700 1 -- 172.21.15.70:0/1669549204 wait complete. 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.950+0000 7f6f2637b700 1 Processor -- start 2024-04-02T04:38:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.950+0000 7f6f2637b700 1 -- start start 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.950+0000 7f6f2637b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f2019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.950+0000 7f6f2637b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2019a790 con 0x7f6f20104850 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f1ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f2019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f1ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f2019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55470/0 (socket says 172.21.15.70:55470) 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f1ffff700 1 -- 172.21.15.70:0/1974920458 learned_addr learned my addr 172.21.15.70:0/1974920458 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f1ffff700 1 -- 172.21.15.70:0/1974920458 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10009550 con 0x7f6f20104850 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f1ffff700 1 --2- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f2019a250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f10011670 tx=0x7f6f10011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f1d7fa700 1 -- 172.21.15.70:0/1974920458 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f100039c0 con 0x7f6f20104850 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f1d7fa700 1 -- 172.21.15.70:0/1974920458 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6f10003b20 con 0x7f6f20104850 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2019a990 con 0x7f6f20104850 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f1d7fa700 1 -- 172.21.15.70:0/1974920458 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f100247f0 con 0x7f6f20104850 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.951+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2019ae30 con 0x7f6f20104850 2024-04-02T04:38:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.952+0000 7f6f1d7fa700 1 -- 172.21.15.70:0/1974920458 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f6f10024ab0 con 0x7f6f20104850 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.952+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f200623c0 con 0x7f6f20104850 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.952+0000 7f6f1d7fa700 1 --2- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6f080383d0 0x7f6f0803a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.952+0000 7f6f1d7fa700 1 -- 172.21.15.70:0/1974920458 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6f1004c590 con 0x7f6f20104850 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.952+0000 7f6f1f7fe700 1 --2- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6f080383d0 0x7f6f0803a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.953+0000 7f6f1f7fe700 1 --2- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6f080383d0 0x7f6f0803a890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f14006fd0 tx=0x7f6f14006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:57.957+0000 7f6f1d7fa700 1 -- 172.21.15.70:0/1974920458 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6f1001a430 con 0x7f6f20104850 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.144+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f6f2019dd30 con 0x7f6f20104850 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.144+0000 7f6f1d7fa700 1 -- 172.21.15.70:0/1974920458 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6f10022310 con 0x7f6f20104850 2024-04-02T04:38:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6f080383d0 msgr2=0x7f6f0803a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 --2- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6f080383d0 0x7f6f0803a890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f14006fd0 tx=0x7f6f14006e40 comp rx=0 tx=0).stop 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 msgr2=0x7f6f2019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 --2- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f2019a250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f10011670 tx=0x7f6f10011aa0 comp rx=0 tx=0).stop 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 shutdown_connections 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 --2- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6f080383d0 0x7f6f0803a890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 --2- 172.21.15.70:0/1974920458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f20104850 0x7f6f2019a250 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 >> 172.21.15.70:0/1974920458 conn(0x7f6f201001a0 msgr2=0x7f6f20100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 shutdown_connections 2024-04-02T04:38:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:58.146+0000 7f6f2637b700 1 -- 172.21.15.70:0/1974920458 wait complete. 2024-04-02T04:39:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.161+0000 7f3c3e9cb700 1 Processor -- start 2024-04-02T04:39:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.161+0000 7f3c3e9cb700 1 -- start start 2024-04-02T04:39:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.161+0000 7f3c3e9cb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38105870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.161+0000 7f3c3e9cb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38105e40 con 0x7f3c38105450 2024-04-02T04:39:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.161+0000 7f3c37fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38105870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.162+0000 7f3c37fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38105870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55480/0 (socket says 172.21.15.70:55480) 2024-04-02T04:39:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.162+0000 7f3c37fff700 1 -- 172.21.15.70:0/3684396532 learned_addr learned my addr 172.21.15.70:0/3684396532 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:39:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.162+0000 7f3c37fff700 1 -- 172.21.15.70:0/3684396532 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38106650 con 0x7f3c38105450 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.162+0000 7f3c37fff700 1 --2- 172.21.15.70:0/3684396532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38105870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3c280098a0 tx=0x7f3c28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0ce1dc5f4feb45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.162+0000 7f3c36ffd700 1 -- 172.21.15.70:0/3684396532 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c28004180 con 0x7f3c38105450 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.162+0000 7f3c36ffd700 1 -- 172.21.15.70:0/3684396532 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3c280042e0 con 0x7f3c38105450 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.162+0000 7f3c36ffd700 1 -- 172.21.15.70:0/3684396532 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c28011af0 con 0x7f3c38105450 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/3684396532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 msgr2=0x7f3c38105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 --2- 172.21.15.70:0/3684396532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38105870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3c280098a0 tx=0x7f3c28009bb0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/3684396532 shutdown_connections 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 --2- 172.21.15.70:0/3684396532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38105870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/3684396532 >> 172.21.15.70:0/3684396532 conn(0x7f3c381009b0 msgr2=0x7f3c38102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/3684396532 shutdown_connections 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/3684396532 wait complete. 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 Processor -- start 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.163+0000 7f3c3e9cb700 1 -- start start 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c3e9cb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38190640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c3e9cb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38190b80 con 0x7f3c38105450 2024-04-02T04:39:00.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c37fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38190640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c37fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38190640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55482/0 (socket says 172.21.15.70:55482) 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c37fff700 1 -- 172.21.15.70:0/739731899 learned_addr learned my addr 172.21.15.70:0/739731899 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c37fff700 1 -- 172.21.15.70:0/739731899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28009550 con 0x7f3c38105450 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c37fff700 1 --2- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38190640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c28011670 tx=0x7f3c28011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c357fa700 1 -- 172.21.15.70:0/739731899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c280039c0 con 0x7f3c38105450 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c357fa700 1 -- 172.21.15.70:0/739731899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3c28003b20 con 0x7f3c38105450 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3818ec90 con 0x7f3c38105450 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c357fa700 1 -- 172.21.15.70:0/739731899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c280247f0 con 0x7f3c38105450 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.164+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3818f130 con 0x7f3c38105450 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.165+0000 7f3c357fa700 1 -- 172.21.15.70:0/739731899 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f3c28024a60 con 0x7f3c38105450 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.165+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c380623c0 con 0x7f3c38105450 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.165+0000 7f3c357fa700 1 --2- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3c20038420 0x7f3c2003a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:00.070 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.165+0000 7f3c357fa700 1 -- 172.21.15.70:0/739731899 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3c2804c5d0 con 0x7f3c38105450 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.166+0000 7f3c377fe700 1 --2- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3c20038420 0x7f3c2003a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.166+0000 7f3c377fe700 1 --2- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3c20038420 0x7f3c2003a8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c006fd0 tx=0x7f3c2c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.170+0000 7f3c357fa700 1 -- 172.21.15.70:0/739731899 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3c28022720 con 0x7f3c38105450 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.471+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f3c3818f960 con 0x7f3c38105450 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.474+0000 7f3c357fa700 1 -- 172.21.15.70:0/739731899 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3c2802c350 con 0x7f3c38105450 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3c20038420 msgr2=0x7f3c2003a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 --2- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3c20038420 0x7f3c2003a8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c006fd0 tx=0x7f3c2c006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 msgr2=0x7f3c38190640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 --2- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38190640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c28011670 tx=0x7f3c28011aa0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 shutdown_connections 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 --2- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3c20038420 0x7f3c2003a8e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 --2- 172.21.15.70:0/739731899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38105450 0x7f3c38190640 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 >> 172.21.15.70:0/739731899 conn(0x7f3c381009b0 msgr2=0x7f3c380798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 shutdown_connections 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-04-02T04:38:59.476+0000 7f3c3e9cb700 1 -- 172.21.15.70:0/739731899 wait complete. 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout:Or, if you are only running a single cluster on this host: 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: ceph telemetry on 2024-04-02T04:39:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:00.073 INFO:teuthology.orchestra.run.smithi070.stdout:For more information see: 2024-04-02T04:39:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-02T04:39:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:00.073 INFO:teuthology.orchestra.run.smithi070.stdout:Bootstrap complete. 2024-04-02T04:39:00.102 INFO:tasks.cephadm:Fetching config... 2024-04-02T04:39:00.102 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:39:00.103 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-02T04:39:00.119 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-02T04:39:00.119 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:39:00.119 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-02T04:39:00.173 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-02T04:39:00.174 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:39:00.174 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/keyring of=/dev/stdout 2024-04-02T04:39:00.240 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-02T04:39:00.241 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:39:00.241 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-02T04:39:00.295 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-02T04:39:00.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL/eJTrtXsg+rXRF5Naum1qW6NUePVatDMqJngI7sYc7Rw1wvmlxQZ9SumtUkbHD80NTIqyFJH3/4e5x+b8mxpiIlWAHxDlvyGL2IBhLZKzayAtApfU5xyL50iq5egaoTuLl/VUY1sOBMKC/Ew6KuhgJpAmbjW2BZ/Hh+cHo6DlvhLciINX8+v04tVL5bFv5KdjHxd2V526wmuvTo3/YLpm2kvgEZKPAzQ3mGyYNyTxaby+WA2AGUcHL7s2512+BaepSauL3yCv+QQEwn7MZVdS0bGbRxg4Nd02bAf1d676hiPGClXfPfTtljRgSdW9klO9aRrjponaywk+uQs3+tNuBZ46T5U3E12ih3ijSB/JUvM2dXuxAYb8tWHTe3tEAgU/PMTef7qzXo0OUEhrG817FCG/jGJsYkbV+tInLI9mwPsuIYKRrQvNB1Dh5gwYHABqxwUvMJwEFYFjzEJTa7XdHYBBo45OOQGQzXFKG84TFkjKIbhPzw+bGY3bq1egX8= ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T04:39:00.373 INFO:teuthology.orchestra.run.smithi070.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL/eJTrtXsg+rXRF5Naum1qW6NUePVatDMqJngI7sYc7Rw1wvmlxQZ9SumtUkbHD80NTIqyFJH3/4e5x+b8mxpiIlWAHxDlvyGL2IBhLZKzayAtApfU5xyL50iq5egaoTuLl/VUY1sOBMKC/Ew6KuhgJpAmbjW2BZ/Hh+cHo6DlvhLciINX8+v04tVL5bFv5KdjHxd2V526wmuvTo3/YLpm2kvgEZKPAzQ3mGyYNyTxaby+WA2AGUcHL7s2512+BaepSauL3yCv+QQEwn7MZVdS0bGbRxg4Nd02bAf1d676hiPGClXfPfTtljRgSdW9klO9aRrjponaywk+uQs3+tNuBZ46T5U3E12ih3ijSB/JUvM2dXuxAYb8tWHTe3tEAgU/PMTef7qzXo0OUEhrG817FCG/jGJsYkbV+tInLI9mwPsuIYKRrQvNB1Dh5gwYHABqxwUvMJwEFYFjzEJTa7XdHYBBo45OOQGQzXFKG84TFkjKIbhPzw+bGY3bq1egX8= ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:39:00.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL/eJTrtXsg+rXRF5Naum1qW6NUePVatDMqJngI7sYc7Rw1wvmlxQZ9SumtUkbHD80NTIqyFJH3/4e5x+b8mxpiIlWAHxDlvyGL2IBhLZKzayAtApfU5xyL50iq5egaoTuLl/VUY1sOBMKC/Ew6KuhgJpAmbjW2BZ/Hh+cHo6DlvhLciINX8+v04tVL5bFv5KdjHxd2V526wmuvTo3/YLpm2kvgEZKPAzQ3mGyYNyTxaby+WA2AGUcHL7s2512+BaepSauL3yCv+QQEwn7MZVdS0bGbRxg4Nd02bAf1d676hiPGClXfPfTtljRgSdW9klO9aRrjponaywk+uQs3+tNuBZ46T5U3E12ih3ijSB/JUvM2dXuxAYb8tWHTe3tEAgU/PMTef7qzXo0OUEhrG817FCG/jGJsYkbV+tInLI9mwPsuIYKRrQvNB1Dh5gwYHABqxwUvMJwEFYFjzEJTa7XdHYBBo45OOQGQzXFKG84TFkjKIbhPzw+bGY3bq1egX8= ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T04:39:00.435 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL/eJTrtXsg+rXRF5Naum1qW6NUePVatDMqJngI7sYc7Rw1wvmlxQZ9SumtUkbHD80NTIqyFJH3/4e5x+b8mxpiIlWAHxDlvyGL2IBhLZKzayAtApfU5xyL50iq5egaoTuLl/VUY1sOBMKC/Ew6KuhgJpAmbjW2BZ/Hh+cHo6DlvhLciINX8+v04tVL5bFv5KdjHxd2V526wmuvTo3/YLpm2kvgEZKPAzQ3mGyYNyTxaby+WA2AGUcHL7s2512+BaepSauL3yCv+QQEwn7MZVdS0bGbRxg4Nd02bAf1d676hiPGClXfPfTtljRgSdW9klO9aRrjponaywk+uQs3+tNuBZ46T5U3E12ih3ijSB/JUvM2dXuxAYb8tWHTe3tEAgU/PMTef7qzXo0OUEhrG817FCG/jGJsYkbV+tInLI9mwPsuIYKRrQvNB1Dh5gwYHABqxwUvMJwEFYFjzEJTa7XdHYBBo45OOQGQzXFKG84TFkjKIbhPzw+bGY3bq1egX8= ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:39:00.450 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-02T04:39:00.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:00 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/739731899' entity='client.admin' 2024-04-02T04:39:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.213+0000 7fa40501e700 1 -- 172.21.15.70:0/1077119547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 msgr2=0x7fa4000ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.213+0000 7fa40501e700 1 --2- 172.21.15.70:0/1077119547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 0x7fa4000ffba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009910 tx=0x7fa3f0009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.213+0000 7fa40501e700 1 -- 172.21.15.70:0/1077119547 shutdown_connections 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.213+0000 7fa40501e700 1 --2- 172.21.15.70:0/1077119547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 0x7fa4000ffba0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.213+0000 7fa40501e700 1 -- 172.21.15.70:0/1077119547 >> 172.21.15.70:0/1077119547 conn(0x7fa4000fad20 msgr2=0x7fa4000fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.214+0000 7fa40501e700 1 -- 172.21.15.70:0/1077119547 shutdown_connections 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.214+0000 7fa40501e700 1 -- 172.21.15.70:0/1077119547 wait complete. 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.214+0000 7fa40501e700 1 Processor -- start 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa40501e700 1 -- start start 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa40501e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 0x7fa400068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:03.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa40501e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4001986f0 con 0x7fa4000ff780 2024-04-02T04:39:03.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa3fed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 0x7fa400068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:03.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa3fed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 0x7fa400068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55508/0 (socket says 172.21.15.70:55508) 2024-04-02T04:39:03.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa3fed9d700 1 -- 172.21.15.70:0/1712510878 learned_addr learned my addr 172.21.15.70:0/1712510878 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:39:03.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa3fed9d700 1 -- 172.21.15.70:0/1712510878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00095f0 con 0x7fa4000ff780 2024-04-02T04:39:03.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa3fed9d700 1 --2- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 0x7fa400068af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0005b40 tx=0x7fa3f0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:03.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa3fcd99700 1 -- 172.21.15.70:0/1712510878 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3f001d070 con 0x7fa4000ff780 2024-04-02T04:39:03.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.215+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa400198830 con 0x7fa4000ff780 2024-04-02T04:39:03.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.216+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa400198cd0 con 0x7fa4000ff780 2024-04-02T04:39:03.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.216+0000 7fa3fcd99700 1 -- 172.21.15.70:0/1712510878 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa3f000be10 con 0x7fa4000ff780 2024-04-02T04:39:03.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.216+0000 7fa3fcd99700 1 -- 172.21.15.70:0/1712510878 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3f000f460 con 0x7fa4000ff780 2024-04-02T04:39:03.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.216+0000 7fa3fcd99700 1 -- 172.21.15.70:0/1712510878 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fa3f000f5c0 con 0x7fa4000ff780 2024-04-02T04:39:03.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.217+0000 7fa3fcd99700 1 --2- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fa3ec038410 0x7fa3ec03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:03.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.217+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e0005320 con 0x7fa4000ff780 2024-04-02T04:39:03.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.217+0000 7fa3fcd99700 1 -- 172.21.15.70:0/1712510878 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa3f004d410 con 0x7fa4000ff780 2024-04-02T04:39:03.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.217+0000 7fa3f659c700 1 --2- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fa3ec038410 0x7fa3ec03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:03.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.217+0000 7fa3f659c700 1 --2- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fa3ec038410 0x7fa3ec03a8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8006fd0 tx=0x7fa3e8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:03.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.224+0000 7fa3fcd99700 1 -- 172.21.15.70:0/1712510878 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa3f002a360 con 0x7fa4000ff780 2024-04-02T04:39:03.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.435+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fa3e0005f70 con 0x7fa4000ff780 2024-04-02T04:39:03.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.438+0000 7fa3fcd99700 1 -- 172.21.15.70:0/1712510878 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa3f0027070 con 0x7fa4000ff780 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fa3ec038410 msgr2=0x7fa3ec03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 --2- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fa3ec038410 0x7fa3ec03a8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8006fd0 tx=0x7fa3e8006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 msgr2=0x7fa400068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 --2- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 0x7fa400068af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0005b40 tx=0x7fa3f0004740 comp rx=0 tx=0).stop 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 shutdown_connections 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 --2- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fa3ec038410 0x7fa3ec03a8d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 --2- 172.21.15.70:0/1712510878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4000ff780 0x7fa400068af0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 >> 172.21.15.70:0/1712510878 conn(0x7fa4000fad20 msgr2=0x7fa4000fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 shutdown_connections 2024-04-02T04:39:03.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:03.440+0000 7fa40501e700 1 -- 172.21.15.70:0/1712510878 wait complete. 2024-04-02T04:39:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:04 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1712510878' entity='client.admin' 2024-04-02T04:39:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:04 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:04 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:04 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:39:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:04 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:04 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:39:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:04 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:39:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:04 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:04.766 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-02T04:39:04.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-02T04:39:05.463 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:39:05.723 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:05 smithi070 ceph-mon[29212]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-04-02T04:39:07.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.888+0000 7fdffd875700 1 -- 172.21.15.70:0/231786019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 msgr2=0x7fdff80fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.888+0000 7fdffd875700 1 --2- 172.21.15.70:0/231786019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 0x7fdff80fef70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009910 tx=0x7fdfec009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.888+0000 7fdffd875700 1 -- 172.21.15.70:0/231786019 shutdown_connections 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.888+0000 7fdffd875700 1 --2- 172.21.15.70:0/231786019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 0x7fdff80fef70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.888+0000 7fdffd875700 1 -- 172.21.15.70:0/231786019 >> 172.21.15.70:0/231786019 conn(0x7fdff80fa490 msgr2=0x7fdff80fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.888+0000 7fdffd875700 1 -- 172.21.15.70:0/231786019 shutdown_connections 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.888+0000 7fdffd875700 1 -- 172.21.15.70:0/231786019 wait complete. 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.888+0000 7fdffd875700 1 Processor -- start 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdffd875700 1 -- start start 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdffd875700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 0x7fdff8101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:07.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdffd875700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff81019d0 con 0x7fdff80feb50 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdff6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 0x7fdff8101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdff6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 0x7fdff8101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45920/0 (socket says 172.21.15.70:45920) 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdff6ffd700 1 -- 172.21.15.70:0/1693876013 learned_addr learned my addr 172.21.15.70:0/1693876013 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdff6ffd700 1 -- 172.21.15.70:0/1693876013 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0095f0 con 0x7fdff80feb50 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdff6ffd700 1 --2- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 0x7fdff8101490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdfec005b40 tx=0x7fdfec004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdffc873700 1 -- 172.21.15.70:0/1693876013 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfec01d070 con 0x7fdff80feb50 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff80ffae0 con 0x7fdff80feb50 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdffc873700 1 -- 172.21.15.70:0/1693876013 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdfec00be10 con 0x7fdff80feb50 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.889+0000 7fdffc873700 1 -- 172.21.15.70:0/1693876013 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfec00f460 con 0x7fdff80feb50 2024-04-02T04:39:07.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.890+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff80fff80 con 0x7fdff80feb50 2024-04-02T04:39:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.890+0000 7fdffc873700 1 -- 172.21.15.70:0/1693876013 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fdfec00f6d0 con 0x7fdff80feb50 2024-04-02T04:39:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.890+0000 7fdffc873700 1 --2- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdfe00383c0 0x7fdfe003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.890+0000 7fdffc873700 1 -- 172.21.15.70:0/1693876013 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdfec04d260 con 0x7fdff80feb50 2024-04-02T04:39:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.891+0000 7fdff67fc700 1 --2- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdfe00383c0 0x7fdfe003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.891+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff8100990 con 0x7fdff80feb50 2024-04-02T04:39:07.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.891+0000 7fdff67fc700 1 --2- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdfe00383c0 0x7fdfe003a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8006fd0 tx=0x7fdfe8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:07.896+0000 7fdffc873700 1 -- 172.21.15.70:0/1693876013 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdfec027030 con 0x7fdff80feb50 2024-04-02T04:39:08.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.118+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 --> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fdff818f390 con 0x7fdfe00383c0 2024-04-02T04:39:08.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.121+0000 7fdffc873700 1 -- 172.21.15.70:0/1693876013 <== mgr.14164 v2:172.21.15.70:6800/2440490727 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdff818f390 con 0x7fdfe00383c0 2024-04-02T04:39:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdfe00383c0 msgr2=0x7fdfe003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 --2- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdfe00383c0 0x7fdfe003a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8006fd0 tx=0x7fdfe8006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 msgr2=0x7fdff8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 --2- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 0x7fdff8101490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdfec005b40 tx=0x7fdfec004740 comp rx=0 tx=0).stop 2024-04-02T04:39:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 shutdown_connections 2024-04-02T04:39:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 --2- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fdfe00383c0 0x7fdfe003a880 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 --2- 172.21.15.70:0/1693876013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff80feb50 0x7fdff8101490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 >> 172.21.15.70:0/1693876013 conn(0x7fdff80fa490 msgr2=0x7fdff80fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 shutdown_connections 2024-04-02T04:39:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:08.124+0000 7fdffd875700 1 -- 172.21.15.70:0/1693876013 wait complete. 2024-04-02T04:39:08.911 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: Deploying daemon crash.smithi070 on smithi070 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:08.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:08 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:08.934 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-04-02T04:39:08.934 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:39:08.935 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-04-02T04:39:08.953 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:39:08.953 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:39:09.011 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-04-02T04:39:09.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi179 2024-04-02T04:39:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:39:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.467+0000 7fc0db3eb700 1 -- 172.21.15.70:0/761395128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 msgr2=0x7fc0d4100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.467+0000 7fc0db3eb700 1 --2- 172.21.15.70:0/761395128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 0x7fc0d4100ca0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009910 tx=0x7fc0d0009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.467+0000 7fc0db3eb700 1 -- 172.21.15.70:0/761395128 shutdown_connections 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.467+0000 7fc0db3eb700 1 --2- 172.21.15.70:0/761395128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 0x7fc0d4100ca0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.467+0000 7fc0db3eb700 1 -- 172.21.15.70:0/761395128 >> 172.21.15.70:0/761395128 conn(0x7fc0d40fbe00 msgr2=0x7fc0d40fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.467+0000 7fc0db3eb700 1 -- 172.21.15.70:0/761395128 shutdown_connections 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.468+0000 7fc0db3eb700 1 -- 172.21.15.70:0/761395128 wait complete. 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.468+0000 7fc0db3eb700 1 Processor -- start 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.468+0000 7fc0db3eb700 1 -- start start 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.469+0000 7fc0db3eb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 0x7fc0d41959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.469+0000 7fc0db3eb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4195f10 con 0x7fc0d4100880 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.469+0000 7fc0d9187700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 0x7fc0d41959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.469+0000 7fc0d9187700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 0x7fc0d41959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45940/0 (socket says 172.21.15.70:45940) 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.469+0000 7fc0d9187700 1 -- 172.21.15.70:0/2694154050 learned_addr learned my addr 172.21.15.70:0/2694154050 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.469+0000 7fc0d9187700 1 -- 172.21.15.70:0/2694154050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00095f0 con 0x7fc0d4100880 2024-04-02T04:39:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.469+0000 7fc0d9187700 1 --2- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 0x7fc0d41959d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0005b40 tx=0x7fc0d0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.470+0000 7fc0ca7fc700 1 -- 172.21.15.70:0/2694154050 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0d001d070 con 0x7fc0d4100880 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.470+0000 7fc0ca7fc700 1 -- 172.21.15.70:0/2694154050 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0d000be10 con 0x7fc0d4100880 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.470+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d4196110 con 0x7fc0d4100880 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.470+0000 7fc0ca7fc700 1 -- 172.21.15.70:0/2694154050 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0d000f460 con 0x7fc0d4100880 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.470+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d4196530 con 0x7fc0d4100880 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.471+0000 7fc0ca7fc700 1 -- 172.21.15.70:0/2694154050 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fc0d000b960 con 0x7fc0d4100880 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.471+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d418f4a0 con 0x7fc0d4100880 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.471+0000 7fc0ca7fc700 1 --2- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fc0c00383c0 0x7fc0c003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.471+0000 7fc0ca7fc700 1 -- 172.21.15.70:0/2694154050 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc0d004c3c0 con 0x7fc0d4100880 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.471+0000 7fc0d8986700 1 --2- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fc0c00383c0 0x7fc0c003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.471+0000 7fc0d8986700 1 --2- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fc0c00383c0 0x7fc0c003a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4006fd0 tx=0x7fc0c4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.476+0000 7fc0ca7fc700 1 -- 172.21.15.70:0/2694154050 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc0d002a360 con 0x7fc0d4100880 2024-04-02T04:39:11.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:11.671+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 --> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0d40611d0 con 0x7fc0c00383c0 2024-04-02T04:39:12.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:12 smithi070 ceph-mon[29212]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:12.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:12 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:12.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:12 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:12.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:12 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:12.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:12 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:12.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:12 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:13.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:13 smithi070 ceph-mon[29212]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:13.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:13 smithi070 ceph-mon[29212]: Deploying daemon node-exporter.smithi070 on smithi070 2024-04-02T04:39:13.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:13 smithi070 ceph-mon[29212]: Deploying cephadm binary to smithi179 2024-04-02T04:39:14.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.385+0000 7fc0ca7fc700 1 -- 172.21.15.70:0/2694154050 <== mgr.14164 v2:172.21.15.70:6800/2440490727 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc0d40611d0 con 0x7fc0c00383c0 2024-04-02T04:39:14.391 INFO:teuthology.orchestra.run.smithi070.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-04-02T04:39:14.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fc0c00383c0 msgr2=0x7fc0c003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 --2- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fc0c00383c0 0x7fc0c003a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4006fd0 tx=0x7fc0c4006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 msgr2=0x7fc0d41959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 --2- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 0x7fc0d41959d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0005b40 tx=0x7fc0d0004740 comp rx=0 tx=0).stop 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 shutdown_connections 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 --2- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fc0c00383c0 0x7fc0c003a880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 --2- 172.21.15.70:0/2694154050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4100880 0x7fc0d41959d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 >> 172.21.15.70:0/2694154050 conn(0x7fc0d40fbe00 msgr2=0x7fc0d40fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 shutdown_connections 2024-04-02T04:39:14.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:14.388+0000 7fc0db3eb700 1 -- 172.21.15.70:0/2694154050 wait complete. 2024-04-02T04:39:14.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:14 smithi070 ceph-mon[29212]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:15.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T04:39:15.695 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:15 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:15.696 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:15 smithi070 ceph-mon[29212]: Added host smithi179 2024-04-02T04:39:15.899 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:39:16.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:16 smithi070 ceph-mon[29212]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:16.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:16 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:16.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:16 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:16.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:16 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:16.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:16 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:16.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:16 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.452+0000 7f30d9642700 1 -- 172.21.15.70:0/3744640173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 msgr2=0x7f30d40ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.452+0000 7f30d9642700 1 --2- 172.21.15.70:0/3744640173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 0x7f30d40ffba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009910 tx=0x7f30bc009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.452+0000 7f30d9642700 1 -- 172.21.15.70:0/3744640173 shutdown_connections 2024-04-02T04:39:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.452+0000 7f30d9642700 1 --2- 172.21.15.70:0/3744640173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 0x7f30d40ffba0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.452+0000 7f30d9642700 1 -- 172.21.15.70:0/3744640173 >> 172.21.15.70:0/3744640173 conn(0x7f30d40fad00 msgr2=0x7f30d40fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.452+0000 7f30d9642700 1 -- 172.21.15.70:0/3744640173 shutdown_connections 2024-04-02T04:39:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.452+0000 7f30d9642700 1 -- 172.21.15.70:0/3744640173 wait complete. 2024-04-02T04:39:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.453+0000 7f30d9642700 1 Processor -- start 2024-04-02T04:39:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.453+0000 7f30d9642700 1 -- start start 2024-04-02T04:39:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.453+0000 7f30d9642700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 0x7f30d40fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.453+0000 7f30d9642700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40fd200 con 0x7f30d40ff780 2024-04-02T04:39:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.453+0000 7f30d2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 0x7f30d40fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.453+0000 7f30d2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 0x7f30d40fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53548/0 (socket says 172.21.15.70:53548) 2024-04-02T04:39:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.453+0000 7f30d2ffd700 1 -- 172.21.15.70:0/453973351 learned_addr learned my addr 172.21.15.70:0/453973351 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:39:17.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.453+0000 7f30d2ffd700 1 -- 172.21.15.70:0/453973351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc0095f0 con 0x7f30d40ff780 2024-04-02T04:39:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.454+0000 7f30d2ffd700 1 --2- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 0x7f30d40fccc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f30bc005b40 tx=0x7f30bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.454+0000 7f30cbfff700 1 -- 172.21.15.70:0/453973351 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30bc01d070 con 0x7f30d40ff780 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.454+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d40fb310 con 0x7f30d40ff780 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.454+0000 7f30cbfff700 1 -- 172.21.15.70:0/453973351 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f30bc00be10 con 0x7f30d40ff780 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.454+0000 7f30cbfff700 1 -- 172.21.15.70:0/453973351 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30bc00f460 con 0x7f30d40ff780 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.454+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d40fb7b0 con 0x7f30d40ff780 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.455+0000 7f30cbfff700 1 -- 172.21.15.70:0/453973351 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f30bc00f6d0 con 0x7f30d40ff780 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.455+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d40623c0 con 0x7f30d40ff780 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.455+0000 7f30cbfff700 1 --2- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f30c00383c0 0x7f30c003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.455+0000 7f30cbfff700 1 -- 172.21.15.70:0/453973351 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f30bc04c4a0 con 0x7f30d40ff780 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.455+0000 7f30d27fc700 1 --2- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f30c00383c0 0x7f30c003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.456+0000 7f30d27fc700 1 --2- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f30c00383c0 0x7f30c003a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006fd0 tx=0x7f30c4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.460+0000 7f30cbfff700 1 -- 172.21.15.70:0/453973351 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f30bc020bf0 con 0x7f30d40ff780 2024-04-02T04:39:17.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.682+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 --> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30d40fb9f0 con 0x7f30c00383c0 2024-04-02T04:39:17.686 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:17 smithi070 ceph-mon[29212]: Deploying daemon alertmanager.smithi070 on smithi070 2024-04-02T04:39:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.683+0000 7f30cbfff700 1 -- 172.21.15.70:0/453973351 <== mgr.14164 v2:172.21.15.70:6800/2440490727 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f30d40fb9f0 con 0x7f30c00383c0 2024-04-02T04:39:17.687 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:39:17.687 INFO:teuthology.orchestra.run.smithi070.stdout:[{"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-04-02T04:39:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.685+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f30c00383c0 msgr2=0x7f30c003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.685+0000 7f30d9642700 1 --2- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f30c00383c0 0x7f30c003a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006fd0 tx=0x7f30c4006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.685+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 msgr2=0x7f30d40fccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.685+0000 7f30d9642700 1 --2- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 0x7f30d40fccc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f30bc005b40 tx=0x7f30bc004740 comp rx=0 tx=0).stop 2024-04-02T04:39:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.685+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 shutdown_connections 2024-04-02T04:39:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.685+0000 7f30d9642700 1 --2- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f30c00383c0 0x7f30c003a880 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.685+0000 7f30d9642700 1 --2- 172.21.15.70:0/453973351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d40ff780 0x7f30d40fccc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.685+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 >> 172.21.15.70:0/453973351 conn(0x7f30d40fad00 msgr2=0x7f30d40690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.686+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 shutdown_connections 2024-04-02T04:39:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:17.686+0000 7f30d9642700 1 -- 172.21.15.70:0/453973351 wait complete. 2024-04-02T04:39:18.550 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-02T04:39:18.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-02T04:39:18.760 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:39:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:18 smithi070 ceph-mon[29212]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:19.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:19 smithi070 ceph-mon[29212]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:39:20.668 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:20 smithi070 ceph-mon[29212]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:21.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.064+0000 7f7d57a26700 1 -- 172.21.15.70:0/2605315627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 msgr2=0x7f7d50100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:21.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.064+0000 7f7d57a26700 1 --2- 172.21.15.70:0/2605315627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 0x7f7d50100cc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009910 tx=0x7f7d4c009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:21.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.064+0000 7f7d57a26700 1 -- 172.21.15.70:0/2605315627 shutdown_connections 2024-04-02T04:39:21.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.064+0000 7f7d57a26700 1 --2- 172.21.15.70:0/2605315627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 0x7f7d50100cc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:21.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.064+0000 7f7d57a26700 1 -- 172.21.15.70:0/2605315627 >> 172.21.15.70:0/2605315627 conn(0x7f7d500fbe20 msgr2=0x7f7d500fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:21.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.064+0000 7f7d57a26700 1 -- 172.21.15.70:0/2605315627 shutdown_connections 2024-04-02T04:39:21.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.065+0000 7f7d57a26700 1 -- 172.21.15.70:0/2605315627 wait complete. 2024-04-02T04:39:21.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.065+0000 7f7d57a26700 1 Processor -- start 2024-04-02T04:39:21.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.065+0000 7f7d57a26700 1 -- start start 2024-04-02T04:39:21.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.065+0000 7f7d57a26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 0x7f7d50074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.065+0000 7f7d57a26700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d50075070 con 0x7f7d501008a0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.065+0000 7f7d557c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 0x7f7d50074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.065+0000 7f7d557c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 0x7f7d50074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53564/0 (socket says 172.21.15.70:53564) 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.065+0000 7f7d557c2700 1 -- 172.21.15.70:0/912702403 learned_addr learned my addr 172.21.15.70:0/912702403 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.066+0000 7f7d557c2700 1 -- 172.21.15.70:0/912702403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c0095f0 con 0x7f7d501008a0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.066+0000 7f7d557c2700 1 --2- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 0x7f7d50074b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c005950 tx=0x7f7d4c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.066+0000 7f7d467fc700 1 -- 172.21.15.70:0/912702403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d4c01d070 con 0x7f7d501008a0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.066+0000 7f7d467fc700 1 -- 172.21.15.70:0/912702403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7d4c00be10 con 0x7f7d501008a0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.066+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d50073180 con 0x7f7d501008a0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.066+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d50073620 con 0x7f7d501008a0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.066+0000 7f7d467fc700 1 -- 172.21.15.70:0/912702403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d4c00f460 con 0x7f7d501008a0 2024-04-02T04:39:21.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.067+0000 7f7d467fc700 1 -- 172.21.15.70:0/912702403 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f7d4c00f660 con 0x7f7d501008a0 2024-04-02T04:39:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.067+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d34005320 con 0x7f7d501008a0 2024-04-02T04:39:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.067+0000 7f7d467fc700 1 --2- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f7d3c038410 0x7f7d3c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.067+0000 7f7d467fc700 1 -- 172.21.15.70:0/912702403 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7d4c04c6e0 con 0x7f7d501008a0 2024-04-02T04:39:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.067+0000 7f7d54fc1700 1 --2- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f7d3c038410 0x7f7d3c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.067+0000 7f7d54fc1700 1 --2- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f7d3c038410 0x7f7d3c03a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d40006fd0 tx=0x7f7d40006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.072+0000 7f7d467fc700 1 -- 172.21.15.70:0/912702403 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7d4c02a360 con 0x7f7d501008a0 2024-04-02T04:39:21.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.285+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7d340059f0 con 0x7f7d501008a0 2024-04-02T04:39:21.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.713+0000 7f7d467fc700 1 -- 172.21.15.70:0/912702403 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7d4c027070 con 0x7f7d501008a0 2024-04-02T04:39:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f7d3c038410 msgr2=0x7f7d3c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 --2- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f7d3c038410 0x7f7d3c03a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d40006fd0 tx=0x7f7d40006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 msgr2=0x7f7d50074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 --2- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 0x7f7d50074b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c005950 tx=0x7f7d4c009e60 comp rx=0 tx=0).stop 2024-04-02T04:39:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 shutdown_connections 2024-04-02T04:39:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 --2- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f7d3c038410 0x7f7d3c03a8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 --2- 172.21.15.70:0/912702403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d501008a0 0x7f7d50074b30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 >> 172.21.15.70:0/912702403 conn(0x7f7d500fbe20 msgr2=0x7f7d500fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:21.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 shutdown_connections 2024-04-02T04:39:21.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:39:21.715+0000 7f7d57a26700 1 -- 172.21.15.70:0/912702403 wait complete. 2024-04-02T04:39:21.720 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2024-04-02T04:39:21.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:21 smithi070 ceph-mon[29212]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:21.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:21 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/912702403' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T04:39:22.851 INFO:tasks.cephadm:Adding mon.smithi070 on smithi070 2024-04-02T04:39:22.851 INFO:tasks.cephadm:Adding mon.smithi179 on smithi179 2024-04-02T04:39:22.851 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '2;smithi070:172.21.15.70=smithi070;smithi179:172.21.15.179=smithi179' 2024-04-02T04:39:22.939 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:22 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/912702403' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T04:39:22.939 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:22 smithi070 ceph-mon[29212]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T04:39:23.052 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:24.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:23 smithi070 ceph-mon[29212]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.742+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/3891421666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 msgr2=0x7f6cf40ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.742+0000 7f6cfa7fe700 1 --2- 172.21.15.179:0/3891421666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 0x7f6cf40ffba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009910 tx=0x7f6ce4009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.742+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/3891421666 shutdown_connections 2024-04-02T04:39:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.742+0000 7f6cfa7fe700 1 --2- 172.21.15.179:0/3891421666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 0x7f6cf40ffba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.742+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/3891421666 >> 172.21.15.179:0/3891421666 conn(0x7f6cf40fad20 msgr2=0x7f6cf40fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.742+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/3891421666 shutdown_connections 2024-04-02T04:39:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.742+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/3891421666 wait complete. 2024-04-02T04:39:24.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.743+0000 7f6cfa7fe700 1 Processor -- start 2024-04-02T04:39:24.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.743+0000 7f6cfa7fe700 1 -- start start 2024-04-02T04:39:24.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.743+0000 7f6cfa7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 0x7f6cf41948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:24.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.743+0000 7f6cfa7fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4194de0 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cf3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 0x7f6cf41948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cf3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 0x7f6cf41948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:49040/0 (socket says 172.21.15.179:49040) 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cf3fff700 1 -- 172.21.15.179:0/2152593304 learned_addr learned my addr 172.21.15.179:0/2152593304 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cf3fff700 1 -- 172.21.15.179:0/2152593304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce40095f0 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cf3fff700 1 --2- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 0x7f6cf41948a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4005b40 tx=0x7f6ce4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cf17fa700 1 -- 172.21.15.179:0/2152593304 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ce401d070 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf4194fe0 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cf17fa700 1 -- 172.21.15.179:0/2152593304 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6ce400be10 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cf17fa700 1 -- 172.21.15.179:0/2152593304 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ce400f460 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.744+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf4195480 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.745+0000 7f6cf17fa700 1 -- 172.21.15.179:0/2152593304 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f6ce40227e0 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.745+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf418e3a0 con 0x7f6cf40ff780 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.745+0000 7f6cf17fa700 1 --2- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6cdc0383c0 0x7f6cdc03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:24.749 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.745+0000 7f6cf17fa700 1 -- 172.21.15.179:0/2152593304 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6ce404c3d0 con 0x7f6cf40ff780 2024-04-02T04:39:24.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.746+0000 7f6cf37fe700 1 --2- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6cdc0383c0 0x7f6cdc03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:24.750 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.746+0000 7f6cf37fe700 1 --2- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6cdc0383c0 0x7f6cdc03a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:24.753 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.751+0000 7f6cf17fa700 1 -- 172.21.15.179:0/2152593304 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6ce4022470 con 0x7f6cf40ff780 2024-04-02T04:39:24.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.948+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 --> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi070:172.21.15.70=smithi070;smithi179:172.21.15.179=smithi179", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cf4061340 con 0x7f6cdc0383c0 2024-04-02T04:39:24.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.954+0000 7f6cf17fa700 1 -- 172.21.15.179:0/2152593304 <== mgr.14164 v2:172.21.15.70:6800/2440490727 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6cf4061340 con 0x7f6cdc0383c0 2024-04-02T04:39:24.958 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-04-02T04:39:24.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6cdc0383c0 msgr2=0x7f6cdc03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:24.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 --2- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6cdc0383c0 0x7f6cdc03a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:24.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 msgr2=0x7f6cf41948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:24.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 --2- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 0x7f6cf41948a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4005b40 tx=0x7f6ce4004740 comp rx=0 tx=0).stop 2024-04-02T04:39:24.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 shutdown_connections 2024-04-02T04:39:24.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 --2- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f6cdc0383c0 0x7f6cdc03a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:24.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 --2- 172.21.15.179:0/2152593304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cf40ff780 0x7f6cf41948a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:24.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 >> 172.21.15.179:0/2152593304 conn(0x7f6cf40fad20 msgr2=0x7f6cf40fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:24.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 shutdown_connections 2024-04-02T04:39:24.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:24.957+0000 7f6cfa7fe700 1 -- 172.21.15.179:0/2152593304 wait complete. 2024-04-02T04:39:25.512 DEBUG:teuthology.orchestra.run.smithi179:mon.smithi179> sudo journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi179.service 2024-04-02T04:39:25.514 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:25.515 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:25.741 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:25.788 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:26.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:25 smithi070 ceph-mon[29212]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi070:172.21.15.70=smithi070;smithi179:172.21.15.179=smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:26.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:25 smithi070 ceph-mon[29212]: Saving service mon spec with placement smithi070:172.21.15.70=smithi070;smithi179:172.21.15.179=smithi179;count:2 2024-04-02T04:39:26.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:25 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:26.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:25 smithi070 ceph-mon[29212]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:27.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.118+0000 7f24e892a700 1 -- 172.21.15.179:0/2320094521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 msgr2=0x7f24e4100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:27.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.118+0000 7f24e892a700 1 --2- 172.21.15.179:0/2320094521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 0x7f24e4100cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009910 tx=0x7f24d8009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:27.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.118+0000 7f24e892a700 1 -- 172.21.15.179:0/2320094521 shutdown_connections 2024-04-02T04:39:27.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.118+0000 7f24e892a700 1 --2- 172.21.15.179:0/2320094521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 0x7f24e4100cc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:27.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.118+0000 7f24e892a700 1 -- 172.21.15.179:0/2320094521 >> 172.21.15.179:0/2320094521 conn(0x7f24e40fbe20 msgr2=0x7f24e40fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.118+0000 7f24e892a700 1 -- 172.21.15.179:0/2320094521 shutdown_connections 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.118+0000 7f24e892a700 1 -- 172.21.15.179:0/2320094521 wait complete. 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.119+0000 7f24e892a700 1 Processor -- start 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.119+0000 7f24e892a700 1 -- start start 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.119+0000 7f24e892a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 0x7f24e41959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.119+0000 7f24e892a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4195ee0 con 0x7f24e41008a0 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.119+0000 7f24e259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 0x7f24e41959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.119+0000 7f24e259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 0x7f24e41959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:49054/0 (socket says 172.21.15.179:49054) 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.119+0000 7f24e259c700 1 -- 172.21.15.179:0/3495760683 learned_addr learned my addr 172.21.15.179:0/3495760683 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.120+0000 7f24e259c700 1 -- 172.21.15.179:0/3495760683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d80095f0 con 0x7f24e41008a0 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.120+0000 7f24e259c700 1 --2- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 0x7f24e41959a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f24d8005b40 tx=0x7f24d8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.120+0000 7f24d37fe700 1 -- 172.21.15.179:0/3495760683 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24d801d070 con 0x7f24e41008a0 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.120+0000 7f24d37fe700 1 -- 172.21.15.179:0/3495760683 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24d800be10 con 0x7f24e41008a0 2024-04-02T04:39:27.124 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.120+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e41960e0 con 0x7f24e41008a0 2024-04-02T04:39:27.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.120+0000 7f24d37fe700 1 -- 172.21.15.179:0/3495760683 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24d800f460 con 0x7f24e41008a0 2024-04-02T04:39:27.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.120+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e4196580 con 0x7f24e41008a0 2024-04-02T04:39:27.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.121+0000 7f24d37fe700 1 -- 172.21.15.179:0/3495760683 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f24d80227e0 con 0x7f24e41008a0 2024-04-02T04:39:27.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.121+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e418f4c0 con 0x7f24e41008a0 2024-04-02T04:39:27.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.121+0000 7f24d37fe700 1 --2- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f24cc0383c0 0x7f24cc03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:27.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.121+0000 7f24d37fe700 1 -- 172.21.15.179:0/3495760683 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f24d804c560 con 0x7f24e41008a0 2024-04-02T04:39:27.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.121+0000 7f24e1d9b700 1 --2- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f24cc0383c0 0x7f24cc03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:27.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.122+0000 7f24e1d9b700 1 --2- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f24cc0383c0 0x7f24cc03a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24d4006fd0 tx=0x7f24d4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:27.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.126+0000 7f24d37fe700 1 -- 172.21.15.179:0/3495760683 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f24d802bba0 con 0x7f24e41008a0 2024-04-02T04:39:27.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.388+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f24e402cc70 con 0x7f24e41008a0 2024-04-02T04:39:27.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.389+0000 7f24d37fe700 1 -- 172.21.15.179:0/3495760683 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f24d8027020 con 0x7f24e41008a0 2024-04-02T04:39:27.392 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:27.392 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:27.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f24cc0383c0 msgr2=0x7f24cc03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:27.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 --2- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f24cc0383c0 0x7f24cc03a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24d4006fd0 tx=0x7f24d4006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:27.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 msgr2=0x7f24e41959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:27.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 --2- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 0x7f24e41959a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f24d8005b40 tx=0x7f24d8004740 comp rx=0 tx=0).stop 2024-04-02T04:39:27.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 shutdown_connections 2024-04-02T04:39:27.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 --2- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f24cc0383c0 0x7f24cc03a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:27.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 --2- 172.21.15.179:0/3495760683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e41008a0 0x7f24e41959a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:27.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 >> 172.21.15.179:0/3495760683 conn(0x7f24e40fbe20 msgr2=0x7f24e40fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:27.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 shutdown_connections 2024-04-02T04:39:27.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:27.391+0000 7f24e892a700 1 -- 172.21.15.179:0/3495760683 wait complete. 2024-04-02T04:39:27.395 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:28.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-02T04:39:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-02T04:39:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-02T04:39:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: Deploying daemon grafana.smithi070 on smithi070 2024-04-02T04:39:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:27 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/3495760683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:28.923 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:28.924 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:29.127 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:29.173 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:30.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.655+0000 7f398343e700 1 -- 172.21.15.179:0/2632211619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 msgr2=0x7f397c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:30.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.655+0000 7f398343e700 1 --2- 172.21.15.179:0/2632211619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 0x7f397c100cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f396c009910 tx=0x7f396c009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:30.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.655+0000 7f398343e700 1 -- 172.21.15.179:0/2632211619 shutdown_connections 2024-04-02T04:39:30.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.655+0000 7f398343e700 1 --2- 172.21.15.179:0/2632211619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 0x7f397c100cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:30.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.655+0000 7f398343e700 1 -- 172.21.15.179:0/2632211619 >> 172.21.15.179:0/2632211619 conn(0x7f397c0fbe20 msgr2=0x7f397c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:30.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.655+0000 7f398343e700 1 -- 172.21.15.179:0/2632211619 shutdown_connections 2024-04-02T04:39:30.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.655+0000 7f398343e700 1 -- 172.21.15.179:0/2632211619 wait complete. 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.656+0000 7f398343e700 1 Processor -- start 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.656+0000 7f398343e700 1 -- start start 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.656+0000 7f398343e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 0x7f397c1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.656+0000 7f398343e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c195ee0 con 0x7f397c1008a0 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.656+0000 7f39811da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 0x7f397c1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.656+0000 7f39811da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 0x7f397c1959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:58694/0 (socket says 172.21.15.179:58694) 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.656+0000 7f39811da700 1 -- 172.21.15.179:0/1989371123 learned_addr learned my addr 172.21.15.179:0/1989371123 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.657+0000 7f39811da700 1 -- 172.21.15.179:0/1989371123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c0095f0 con 0x7f397c1008a0 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.657+0000 7f39811da700 1 --2- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 0x7f397c1959a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f396c005b40 tx=0x7f396c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.657+0000 7f39727fc700 1 -- 172.21.15.179:0/1989371123 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f396c01d070 con 0x7f397c1008a0 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.657+0000 7f39727fc700 1 -- 172.21.15.179:0/1989371123 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f396c00be10 con 0x7f397c1008a0 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.657+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c1960e0 con 0x7f397c1008a0 2024-04-02T04:39:30.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.657+0000 7f39727fc700 1 -- 172.21.15.179:0/1989371123 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f396c00f460 con 0x7f397c1008a0 2024-04-02T04:39:30.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.657+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c196580 con 0x7f397c1008a0 2024-04-02T04:39:30.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.658+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c18f4c0 con 0x7f397c1008a0 2024-04-02T04:39:30.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.658+0000 7f39727fc700 1 -- 172.21.15.179:0/1989371123 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f396c0227e0 con 0x7f397c1008a0 2024-04-02T04:39:30.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.658+0000 7f39727fc700 1 --2- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3968038410 0x7f396803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:30.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.658+0000 7f39727fc700 1 -- 172.21.15.179:0/1989371123 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f396c04c420 con 0x7f397c1008a0 2024-04-02T04:39:30.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.658+0000 7f39809d9700 1 --2- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3968038410 0x7f396803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:30.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.659+0000 7f39809d9700 1 --2- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3968038410 0x7f396803a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3978006fd0 tx=0x7f3978006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:30.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.663+0000 7f39727fc700 1 -- 172.21.15.179:0/1989371123 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f396c02ba00 con 0x7f397c1008a0 2024-04-02T04:39:30.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:30 smithi070 ceph-mon[29212]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:30.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.921+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f397c02cc70 con 0x7f397c1008a0 2024-04-02T04:39:30.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.922+0000 7f39727fc700 1 -- 172.21.15.179:0/1989371123 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f396c027020 con 0x7f397c1008a0 2024-04-02T04:39:30.926 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:30.926 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3968038410 msgr2=0x7f396803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 --2- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3968038410 0x7f396803a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3978006fd0 tx=0x7f3978006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 msgr2=0x7f397c1959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 --2- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 0x7f397c1959a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f396c005b40 tx=0x7f396c004740 comp rx=0 tx=0).stop 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 shutdown_connections 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 --2- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f3968038410 0x7f396803a8d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 --2- 172.21.15.179:0/1989371123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f397c1008a0 0x7f397c1959a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 >> 172.21.15.179:0/1989371123 conn(0x7f397c0fbe20 msgr2=0x7f397c0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:30.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 shutdown_connections 2024-04-02T04:39:30.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:30.924+0000 7f398343e700 1 -- 172.21.15.179:0/1989371123 wait complete. 2024-04-02T04:39:30.929 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:32.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:31 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/1989371123' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:32.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:31 smithi070 ceph-mon[29212]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:32.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:31 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:32.385 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:32.386 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:32.581 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:32.628 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:34.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.033+0000 7f5d53e49700 1 -- 172.21.15.179:0/876477990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 msgr2=0x7f5d4c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:34.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.033+0000 7f5d53e49700 1 --2- 172.21.15.179:0/876477990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 0x7f5d4c100cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400b0a0 tx=0x7f5d4400b3b0 comp rx=0 tx=0).stop 2024-04-02T04:39:34.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.033+0000 7f5d53e49700 1 -- 172.21.15.179:0/876477990 shutdown_connections 2024-04-02T04:39:34.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.033+0000 7f5d53e49700 1 --2- 172.21.15.179:0/876477990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 0x7f5d4c100cc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:34.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.033+0000 7f5d53e49700 1 -- 172.21.15.179:0/876477990 >> 172.21.15.179:0/876477990 conn(0x7f5d4c0fbe20 msgr2=0x7f5d4c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:34.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.033+0000 7f5d53e49700 1 -- 172.21.15.179:0/876477990 shutdown_connections 2024-04-02T04:39:34.038 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.033+0000 7f5d53e49700 1 -- 172.21.15.179:0/876477990 wait complete. 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.034+0000 7f5d53e49700 1 Processor -- start 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.034+0000 7f5d53e49700 1 -- start start 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.034+0000 7f5d53e49700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 0x7f5d4c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.034+0000 7f5d53e49700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c195f30 con 0x7f5d4c1008a0 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.034+0000 7f5d51be5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 0x7f5d4c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.034+0000 7f5d51be5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 0x7f5d4c1959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:58710/0 (socket says 172.21.15.179:58710) 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.034+0000 7f5d51be5700 1 -- 172.21.15.179:0/1544355498 learned_addr learned my addr 172.21.15.179:0/1544355498 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.034+0000 7f5d51be5700 1 -- 172.21.15.179:0/1544355498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44009d00 con 0x7f5d4c1008a0 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.035+0000 7f5d51be5700 1 --2- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 0x7f5d4c1959f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400b830 tx=0x7f5d4400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.035+0000 7f5d42ffd700 1 -- 172.21.15.179:0/1544355498 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d4400e040 con 0x7f5d4c1008a0 2024-04-02T04:39:34.040 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.035+0000 7f5d42ffd700 1 -- 172.21.15.179:0/1544355498 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5d44012af0 con 0x7f5d4c1008a0 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.035+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c196130 con 0x7f5d4c1008a0 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.035+0000 7f5d42ffd700 1 -- 172.21.15.179:0/1544355498 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d4401d9f0 con 0x7f5d4c1008a0 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.035+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c196550 con 0x7f5d4c1008a0 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.036+0000 7f5d42ffd700 1 -- 172.21.15.179:0/1544355498 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f5d4401dcb0 con 0x7f5d4c1008a0 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.036+0000 7f5d42ffd700 1 --2- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f5d380383c0 0x7f5d3803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.036+0000 7f5d42ffd700 1 -- 172.21.15.179:0/1544355498 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5d4404c940 con 0x7f5d4c1008a0 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.036+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c18f4c0 con 0x7f5d4c1008a0 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.036+0000 7f5d513e4700 1 --2- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f5d380383c0 0x7f5d3803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:34.041 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.036+0000 7f5d513e4700 1 --2- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f5d380383c0 0x7f5d3803a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c006fd0 tx=0x7f5d3c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:34.045 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.041+0000 7f5d42ffd700 1 -- 172.21.15.179:0/1544355498 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5d4401c3e0 con 0x7f5d4c1008a0 2024-04-02T04:39:34.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.304+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5d4c0623c0 con 0x7f5d4c1008a0 2024-04-02T04:39:34.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.304+0000 7f5d42ffd700 1 -- 172.21.15.179:0/1544355498 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5d4402c200 con 0x7f5d4c1008a0 2024-04-02T04:39:34.309 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:34.309 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:34.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f5d380383c0 msgr2=0x7f5d3803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:34.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 --2- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f5d380383c0 0x7f5d3803a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c006fd0 tx=0x7f5d3c006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 msgr2=0x7f5d4c1959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 --2- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 0x7f5d4c1959f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400b830 tx=0x7f5d4400bd40 comp rx=0 tx=0).stop 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 shutdown_connections 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 --2- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f5d380383c0 0x7f5d3803a880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 --2- 172.21.15.179:0/1544355498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c1008a0 0x7f5d4c1959f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 >> 172.21.15.179:0/1544355498 conn(0x7f5d4c0fbe20 msgr2=0x7f5d4c0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.306+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 shutdown_connections 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:34.307+0000 7f5d53e49700 1 -- 172.21.15.179:0/1544355498 wait complete. 2024-04-02T04:39:34.311 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:34.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:34 smithi070 ceph-mon[29212]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:34.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:34 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/1544355498' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:35.805 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:35.806 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:36.006 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:36.050 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:36.680 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:36 smithi070 ceph-mon[29212]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:37.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2091981871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 msgr2=0x7f4eb4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:37.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 --2- 172.21.15.179:0/2091981871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 0x7f4eb4101180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009910 tx=0x7f4ea8009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2091981871 shutdown_connections 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 --2- 172.21.15.179:0/2091981871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 0x7f4eb4101180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2091981871 >> 172.21.15.179:0/2091981871 conn(0x7f4eb40fa490 msgr2=0x7f4eb40fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2091981871 shutdown_connections 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2091981871 wait complete. 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 Processor -- start 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.490+0000 7f4ebaec5700 1 -- start start 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4ebaec5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 0x7f4eb40fc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4ebaec5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb40fc990 con 0x7f4eb4100d60 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4eb8c61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 0x7f4eb40fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4eb8c61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 0x7f4eb40fc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:58732/0 (socket says 172.21.15.179:58732) 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4eb8c61700 1 -- 172.21.15.179:0/2902248762 learned_addr learned my addr 172.21.15.179:0/2902248762 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:37.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4eb8c61700 1 -- 172.21.15.179:0/2902248762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea80095f0 con 0x7f4eb4100d60 2024-04-02T04:39:37.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4eb8c61700 1 --2- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 0x7f4eb40fc450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005b40 tx=0x7f4ea8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:37.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4eb1ffb700 1 -- 172.21.15.179:0/2902248762 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ea801d070 con 0x7f4eb4100d60 2024-04-02T04:39:37.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4eb1ffb700 1 -- 172.21.15.179:0/2902248762 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ea800be10 con 0x7f4eb4100d60 2024-04-02T04:39:37.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb40faaa0 con 0x7f4eb4100d60 2024-04-02T04:39:37.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.491+0000 7f4eb1ffb700 1 -- 172.21.15.179:0/2902248762 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ea800f460 con 0x7f4eb4100d60 2024-04-02T04:39:37.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.492+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb40faf40 con 0x7f4eb4100d60 2024-04-02T04:39:37.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.492+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb40623c0 con 0x7f4eb4100d60 2024-04-02T04:39:37.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.493+0000 7f4eb1ffb700 1 -- 172.21.15.179:0/2902248762 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f4ea80227e0 con 0x7f4eb4100d60 2024-04-02T04:39:37.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.493+0000 7f4eb1ffb700 1 --2- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4ea4038410 0x7f4ea403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:37.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.493+0000 7f4eb1ffb700 1 -- 172.21.15.179:0/2902248762 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4ea804c420 con 0x7f4eb4100d60 2024-04-02T04:39:37.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.493+0000 7f4eb3fff700 1 --2- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4ea4038410 0x7f4ea403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:37.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.493+0000 7f4eb3fff700 1 --2- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4ea4038410 0x7f4ea403a8d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0006fd0 tx=0x7f4ea0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:37.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.498+0000 7f4eb1ffb700 1 -- 172.21.15.179:0/2902248762 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4ea802ba00 con 0x7f4eb4100d60 2024-04-02T04:39:37.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.763+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4eb418edb0 con 0x7f4eb4100d60 2024-04-02T04:39:37.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.764+0000 7f4eb1ffb700 1 -- 172.21.15.179:0/2902248762 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4ea8027030 con 0x7f4eb4100d60 2024-04-02T04:39:37.768 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:37.768 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4ea4038410 msgr2=0x7f4ea403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 --2- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4ea4038410 0x7f4ea403a8d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0006fd0 tx=0x7f4ea0006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 msgr2=0x7f4eb40fc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 --2- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 0x7f4eb40fc450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005b40 tx=0x7f4ea8004740 comp rx=0 tx=0).stop 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 shutdown_connections 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 --2- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4ea4038410 0x7f4ea403a8d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 --2- 172.21.15.179:0/2902248762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eb4100d60 0x7f4eb40fc450 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 >> 172.21.15.179:0/2902248762 conn(0x7f4eb40fa490 msgr2=0x7f4eb40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 shutdown_connections 2024-04-02T04:39:37.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:37.766+0000 7f4ebaec5700 1 -- 172.21.15.179:0/2902248762 wait complete. 2024-04-02T04:39:37.771 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:38.633 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:38 smithi070 ceph-mon[29212]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:38.633 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:38 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/2902248762' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:39.334 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:39.334 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:39.537 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:39.583 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:40.427 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:40 smithi070 ceph-mon[29212]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.934+0000 7f9f6e353700 1 -- 172.21.15.179:0/80661894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 msgr2=0x7f9f680ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.934+0000 7f9f6e353700 1 --2- 172.21.15.179:0/80661894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 0x7f9f680ffbb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009910 tx=0x7f9f58009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.934+0000 7f9f6e353700 1 -- 172.21.15.179:0/80661894 shutdown_connections 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.934+0000 7f9f6e353700 1 --2- 172.21.15.179:0/80661894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 0x7f9f680ffbb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.934+0000 7f9f6e353700 1 -- 172.21.15.179:0/80661894 >> 172.21.15.179:0/80661894 conn(0x7f9f680fad30 msgr2=0x7f9f680fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.934+0000 7f9f6e353700 1 -- 172.21.15.179:0/80661894 shutdown_connections 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.934+0000 7f9f6e353700 1 -- 172.21.15.179:0/80661894 wait complete. 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.935+0000 7f9f6e353700 1 Processor -- start 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.935+0000 7f9f6e353700 1 -- start start 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.935+0000 7f9f6e353700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 0x7f9f681948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.935+0000 7f9f6e353700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68194de0 con 0x7f9f680ff790 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.935+0000 7f9f67fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 0x7f9f681948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.935+0000 7f9f67fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 0x7f9f681948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:36986/0 (socket says 172.21.15.179:36986) 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.935+0000 7f9f67fff700 1 -- 172.21.15.179:0/3921857278 learned_addr learned my addr 172.21.15.179:0/3921857278 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.935+0000 7f9f67fff700 1 -- 172.21.15.179:0/3921857278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580095f0 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.936+0000 7f9f67fff700 1 --2- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 0x7f9f681948a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f58005b40 tx=0x7f9f58004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.936+0000 7f9f657fa700 1 -- 172.21.15.179:0/3921857278 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f5801d070 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.936+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f68194fe0 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.936+0000 7f9f657fa700 1 -- 172.21.15.179:0/3921857278 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f5800be10 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.936+0000 7f9f657fa700 1 -- 172.21.15.179:0/3921857278 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f5800f460 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.936+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f68195480 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.937+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6818e3b0 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.937+0000 7f9f657fa700 1 -- 172.21.15.179:0/3921857278 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f9f580227e0 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.937+0000 7f9f657fa700 1 --2- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f9f500383c0 0x7f9f5003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.937+0000 7f9f657fa700 1 -- 172.21.15.179:0/3921857278 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9f5804c4a0 con 0x7f9f680ff790 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.937+0000 7f9f677fe700 1 --2- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f9f500383c0 0x7f9f5003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.938+0000 7f9f677fe700 1 --2- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f9f500383c0 0x7f9f5003a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c006fd0 tx=0x7f9f5c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:40.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:40.942+0000 7f9f657fa700 1 -- 172.21.15.179:0/3921857278 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9f5802bba0 con 0x7f9f680ff790 2024-04-02T04:39:41.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.207+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9f6802cc70 con 0x7f9f680ff790 2024-04-02T04:39:41.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.208+0000 7f9f657fa700 1 -- 172.21.15.179:0/3921857278 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f9f58027020 con 0x7f9f680ff790 2024-04-02T04:39:41.213 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:41.213 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:41.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f9f500383c0 msgr2=0x7f9f5003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:41.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 --2- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f9f500383c0 0x7f9f5003a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c006fd0 tx=0x7f9f5c006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:41.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 msgr2=0x7f9f681948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:41.215 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 --2- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 0x7f9f681948a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f58005b40 tx=0x7f9f58004740 comp rx=0 tx=0).stop 2024-04-02T04:39:41.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 shutdown_connections 2024-04-02T04:39:41.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 --2- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f9f500383c0 0x7f9f5003a880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:41.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 --2- 172.21.15.179:0/3921857278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f680ff790 0x7f9f681948a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:41.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 >> 172.21.15.179:0/3921857278 conn(0x7f9f680fad30 msgr2=0x7f9f680fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:41.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 shutdown_connections 2024-04-02T04:39:41.216 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:41.210+0000 7f9f6e353700 1 -- 172.21.15.179:0/3921857278 wait complete. 2024-04-02T04:39:41.217 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:41.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:41.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:41.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:41.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:41.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:41.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:41.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:41.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:41.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/3921857278' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:41.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:41 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:42 smithi070 ceph-mon[29212]: Deploying daemon prometheus.smithi070 on smithi070 2024-04-02T04:39:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:42 smithi070 ceph-mon[29212]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:42.806 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:42.806 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:43.001 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:43.047 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:44.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.387+0000 7fb566c9c700 1 -- 172.21.15.179:0/2636674800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 msgr2=0x7fb560100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:44.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.387+0000 7fb566c9c700 1 --2- 172.21.15.179:0/2636674800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 0x7fb560100ca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb554009910 tx=0x7fb554009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:44.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.387+0000 7fb566c9c700 1 -- 172.21.15.179:0/2636674800 shutdown_connections 2024-04-02T04:39:44.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.387+0000 7fb566c9c700 1 --2- 172.21.15.179:0/2636674800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 0x7fb560100ca0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:44.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.387+0000 7fb566c9c700 1 -- 172.21.15.179:0/2636674800 >> 172.21.15.179:0/2636674800 conn(0x7fb5600fbe20 msgr2=0x7fb5600fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:44.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.387+0000 7fb566c9c700 1 -- 172.21.15.179:0/2636674800 shutdown_connections 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.387+0000 7fb566c9c700 1 -- 172.21.15.179:0/2636674800 wait complete. 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.388+0000 7fb566c9c700 1 Processor -- start 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.388+0000 7fb566c9c700 1 -- start start 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.388+0000 7fb566c9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 0x7fb560195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.388+0000 7fb566c9c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560195ed0 con 0x7fb560100880 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.388+0000 7fb564a38700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 0x7fb560195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.388+0000 7fb564a38700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 0x7fb560195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:37004/0 (socket says 172.21.15.179:37004) 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.388+0000 7fb564a38700 1 -- 172.21.15.179:0/3244222520 learned_addr learned my addr 172.21.15.179:0/3244222520 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.389+0000 7fb564a38700 1 -- 172.21.15.179:0/3244222520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540095f0 con 0x7fb560100880 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.389+0000 7fb564a38700 1 --2- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 0x7fb560195990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb554005b40 tx=0x7fb554004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.389+0000 7fb55dffb700 1 -- 172.21.15.179:0/3244222520 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55401d070 con 0x7fb560100880 2024-04-02T04:39:44.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.389+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5601960d0 con 0x7fb560100880 2024-04-02T04:39:44.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.389+0000 7fb55dffb700 1 -- 172.21.15.179:0/3244222520 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb55400be10 con 0x7fb560100880 2024-04-02T04:39:44.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.389+0000 7fb55dffb700 1 -- 172.21.15.179:0/3244222520 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55400f460 con 0x7fb560100880 2024-04-02T04:39:44.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.389+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb560196570 con 0x7fb560100880 2024-04-02T04:39:44.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.390+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56018f4a0 con 0x7fb560100880 2024-04-02T04:39:44.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.390+0000 7fb55dffb700 1 -- 172.21.15.179:0/3244222520 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fb5540227e0 con 0x7fb560100880 2024-04-02T04:39:44.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.390+0000 7fb55dffb700 1 --2- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fb5500383c0 0x7fb55003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:44.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.390+0000 7fb55dffb700 1 -- 172.21.15.179:0/3244222520 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb55404c470 con 0x7fb560100880 2024-04-02T04:39:44.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.390+0000 7fb55ffff700 1 --2- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fb5500383c0 0x7fb55003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:44.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.391+0000 7fb55ffff700 1 --2- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fb5500383c0 0x7fb55003a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006fd0 tx=0x7fb54c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:44.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.399+0000 7fb55dffb700 1 -- 172.21.15.179:0/3244222520 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb55402bba0 con 0x7fb560100880 2024-04-02T04:39:44.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.668+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb56002cc70 con 0x7fb560100880 2024-04-02T04:39:44.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.668+0000 7fb55dffb700 1 -- 172.21.15.179:0/3244222520 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb554027020 con 0x7fb560100880 2024-04-02T04:39:44.672 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:44.673 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.670+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fb5500383c0 msgr2=0x7fb55003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.670+0000 7fb566c9c700 1 --2- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fb5500383c0 0x7fb55003a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006fd0 tx=0x7fb54c006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.671+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 msgr2=0x7fb560195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.671+0000 7fb566c9c700 1 --2- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 0x7fb560195990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb554005b40 tx=0x7fb554004740 comp rx=0 tx=0).stop 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.671+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 shutdown_connections 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.671+0000 7fb566c9c700 1 --2- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7fb5500383c0 0x7fb55003a880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.671+0000 7fb566c9c700 1 --2- 172.21.15.179:0/3244222520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560100880 0x7fb560195990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.671+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 >> 172.21.15.179:0/3244222520 conn(0x7fb5600fbe20 msgr2=0x7fb5600fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.671+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 shutdown_connections 2024-04-02T04:39:44.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:44.671+0000 7fb566c9c700 1 -- 172.21.15.179:0/3244222520 wait complete. 2024-04-02T04:39:44.676 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:44 smithi070 ceph-mon[29212]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:45 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/3244222520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:46.217 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:46.218 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:46.417 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:46.463 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:46 smithi070 ceph-mon[29212]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:47.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.942+0000 7f1a294f8700 1 -- 172.21.15.179:0/979709556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 msgr2=0x7f1a24100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:47.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.942+0000 7f1a294f8700 1 --2- 172.21.15.179:0/979709556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 0x7f1a24100ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009910 tx=0x7f1a18009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.942+0000 7f1a294f8700 1 -- 172.21.15.179:0/979709556 shutdown_connections 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.942+0000 7f1a294f8700 1 --2- 172.21.15.179:0/979709556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 0x7f1a24100ca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.942+0000 7f1a294f8700 1 -- 172.21.15.179:0/979709556 >> 172.21.15.179:0/979709556 conn(0x7f1a240fbe00 msgr2=0x7f1a240fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.942+0000 7f1a294f8700 1 -- 172.21.15.179:0/979709556 shutdown_connections 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.942+0000 7f1a294f8700 1 -- 172.21.15.179:0/979709556 wait complete. 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.943+0000 7f1a294f8700 1 Processor -- start 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.943+0000 7f1a294f8700 1 -- start start 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.943+0000 7f1a294f8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 0x7f1a241959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.943+0000 7f1a294f8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24195f10 con 0x7f1a24100880 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a22ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 0x7f1a241959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a22ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 0x7f1a241959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:58122/0 (socket says 172.21.15.179:58122) 2024-04-02T04:39:47.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a22ffd700 1 -- 172.21.15.179:0/620087026 learned_addr learned my addr 172.21.15.179:0/620087026 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:47.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a22ffd700 1 -- 172.21.15.179:0/620087026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180095f0 con 0x7f1a24100880 2024-04-02T04:39:47.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a22ffd700 1 --2- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 0x7f1a241959d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1a18005b40 tx=0x7f1a18004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:47.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a0bfff700 1 -- 172.21.15.179:0/620087026 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a1801d070 con 0x7f1a24100880 2024-04-02T04:39:47.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a0bfff700 1 -- 172.21.15.179:0/620087026 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a1800be10 con 0x7f1a24100880 2024-04-02T04:39:47.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a24196110 con 0x7f1a24100880 2024-04-02T04:39:47.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a0bfff700 1 -- 172.21.15.179:0/620087026 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a1800f460 con 0x7f1a24100880 2024-04-02T04:39:47.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.944+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a241965b0 con 0x7f1a24100880 2024-04-02T04:39:47.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.945+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2418f4a0 con 0x7f1a24100880 2024-04-02T04:39:47.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.945+0000 7f1a0bfff700 1 -- 172.21.15.179:0/620087026 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f1a180227e0 con 0x7f1a24100880 2024-04-02T04:39:47.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.945+0000 7f1a0bfff700 1 --2- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f1a0c0383c0 0x7f1a0c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:47.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.945+0000 7f1a0bfff700 1 -- 172.21.15.179:0/620087026 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1a1804c560 con 0x7f1a24100880 2024-04-02T04:39:47.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.946+0000 7f1a227fc700 1 --2- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f1a0c0383c0 0x7f1a0c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:47.951 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.946+0000 7f1a227fc700 1 --2- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f1a0c0383c0 0x7f1a0c03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1a14006fd0 tx=0x7f1a14006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:47.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:47.951+0000 7f1a0bfff700 1 -- 172.21.15.179:0/620087026 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1a1802bba0 con 0x7f1a24100880 2024-04-02T04:39:48.217 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.213+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1a2402cc70 con 0x7f1a24100880 2024-04-02T04:39:48.218 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.214+0000 7f1a0bfff700 1 -- 172.21.15.179:0/620087026 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1a18027020 con 0x7f1a24100880 2024-04-02T04:39:48.218 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:48.218 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:48.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.216+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f1a0c0383c0 msgr2=0x7f1a0c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.216+0000 7f1a294f8700 1 --2- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f1a0c0383c0 0x7f1a0c03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1a14006fd0 tx=0x7f1a14006e40 comp rx=0 tx=0).stop 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.216+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 msgr2=0x7f1a241959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.216+0000 7f1a294f8700 1 --2- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 0x7f1a241959d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1a18005b40 tx=0x7f1a18004740 comp rx=0 tx=0).stop 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.216+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 shutdown_connections 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.216+0000 7f1a294f8700 1 --2- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f1a0c0383c0 0x7f1a0c03a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.216+0000 7f1a294f8700 1 --2- 172.21.15.179:0/620087026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a24100880 0x7f1a241959d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.216+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 >> 172.21.15.179:0/620087026 conn(0x7f1a240fbe00 msgr2=0x7f1a240fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.217+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 shutdown_connections 2024-04-02T04:39:48.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:48.217+0000 7f1a294f8700 1 -- 172.21.15.179:0/620087026 wait complete. 2024-04-02T04:39:48.222 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:48.673 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:48 smithi070 ceph-mon[29212]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:48.673 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:48 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/620087026' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:49.775 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:49.776 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:49.978 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:50.024 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:50.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:50 smithi070 ceph-mon[29212]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:39:50.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:50 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:50.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:50 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:51.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 -- 172.21.15.179:0/3441654216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 msgr2=0x7f452c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:51.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 --2- 172.21.15.179:0/3441654216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 0x7f452c103700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f451c009910 tx=0x7f451c009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:51.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 -- 172.21.15.179:0/3441654216 shutdown_connections 2024-04-02T04:39:51.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 --2- 172.21.15.179:0/3441654216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 0x7f452c103700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:51.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 -- 172.21.15.179:0/3441654216 >> 172.21.15.179:0/3441654216 conn(0x7f452c0fad00 msgr2=0x7f452c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:51.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 -- 172.21.15.179:0/3441654216 shutdown_connections 2024-04-02T04:39:51.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 -- 172.21.15.179:0/3441654216 wait complete. 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 Processor -- start 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.578+0000 7f453187c700 1 -- start start 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f453187c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 0x7f452c100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f453187c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c1011b0 con 0x7f452c101310 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f452affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 0x7f452c100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f452affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 0x7f452c100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:58152/0 (socket says 172.21.15.179:58152) 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f452affd700 1 -- 172.21.15.179:0/3168262572 learned_addr learned my addr 172.21.15.179:0/3168262572 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f452affd700 1 -- 172.21.15.179:0/3168262572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c0095f0 con 0x7f452c101310 2024-04-02T04:39:51.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f452affd700 1 --2- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 0x7f452c100c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f451c005b40 tx=0x7f451c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:51.584 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f453087a700 1 -- 172.21.15.179:0/3168262572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f451c01d070 con 0x7f452c101310 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f453087a700 1 -- 172.21.15.179:0/3168262572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f451c00be10 con 0x7f452c101310 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c0ff2c0 con 0x7f452c101310 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.579+0000 7f453087a700 1 -- 172.21.15.179:0/3168262572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f451c00f460 con 0x7f452c101310 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.580+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c0ff760 con 0x7f452c101310 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.580+0000 7f453087a700 1 -- 172.21.15.179:0/3168262572 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7f451c00f6d0 con 0x7f452c101310 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.581+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c0623c0 con 0x7f452c101310 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.581+0000 7f453087a700 1 --2- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4514038410 0x7f451403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.581+0000 7f453087a700 1 -- 172.21.15.179:0/3168262572 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f451c04c540 con 0x7f452c101310 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.581+0000 7f452a7fc700 1 -- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4514038410 msgr2=0x7f451403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2440490727 2024-04-02T04:39:51.585 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.581+0000 7f452a7fc700 1 --2- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4514038410 0x7f451403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:39:51.588 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.586+0000 7f453087a700 1 -- 172.21.15.179:0/3168262572 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f451c020a30 con 0x7f452c101310 2024-04-02T04:39:51.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:51.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-02T04:39:51.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:51 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:51.784 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.782+0000 7f452a7fc700 1 -- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4514038410 msgr2=0x7f451403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2440490727 2024-04-02T04:39:51.785 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.782+0000 7f452a7fc700 1 --2- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4514038410 0x7f451403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:39:51.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.850+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f452c18ee80 con 0x7f452c101310 2024-04-02T04:39:51.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.851+0000 7f453087a700 1 -- 172.21.15.179:0/3168262572 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f451c027030 con 0x7f452c101310 2024-04-02T04:39:51.854 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:51.854 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4514038410 msgr2=0x7f451403a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 --2- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4514038410 0x7f451403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 msgr2=0x7f452c100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 --2- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 0x7f452c100c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f451c005b40 tx=0x7f451c004740 comp rx=0 tx=0).stop 2024-04-02T04:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 shutdown_connections 2024-04-02T04:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 --2- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7f4514038410 0x7f451403a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 --2- 172.21.15.179:0/3168262572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f452c101310 0x7f452c100c70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:51.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 >> 172.21.15.179:0/3168262572 conn(0x7f452c0fad00 msgr2=0x7f452c0fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 shutdown_connections 2024-04-02T04:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:51.853+0000 7f453187c700 1 -- 172.21.15.179:0/3168262572 wait complete. 2024-04-02T04:39:51.857 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:52.773 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:52 smithi070 ceph-mon[29212]: from='mgr.14164 172.21.15.70:0/3574356308' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-02T04:39:52.773 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:52 smithi070 ceph-mon[29212]: mgrmap e14: smithi070.igidvr(active, since 60s) 2024-04-02T04:39:52.774 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:52 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/3168262572' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:53.411 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:53.411 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:53.613 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:53.658 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:54.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.952+0000 7ff593093700 1 -- 172.21.15.179:0/479853885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 msgr2=0x7ff58c0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:54.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.952+0000 7ff593093700 1 --2- 172.21.15.179:0/479853885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 0x7ff58c0ffbb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff588009910 tx=0x7ff588009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:54.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.952+0000 7ff593093700 1 -- 172.21.15.179:0/479853885 shutdown_connections 2024-04-02T04:39:54.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.952+0000 7ff593093700 1 --2- 172.21.15.179:0/479853885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 0x7ff58c0ffbb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:54.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.952+0000 7ff593093700 1 -- 172.21.15.179:0/479853885 >> 172.21.15.179:0/479853885 conn(0x7ff58c0fad30 msgr2=0x7ff58c0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:54.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.952+0000 7ff593093700 1 -- 172.21.15.179:0/479853885 shutdown_connections 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.952+0000 7ff593093700 1 -- 172.21.15.179:0/479853885 wait complete. 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff593093700 1 Processor -- start 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff593093700 1 -- start start 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff593093700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 0x7ff58c1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff593093700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c195ef0 con 0x7ff58c0ff790 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff590e2f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 0x7ff58c1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff590e2f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 0x7ff58c1959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:58172/0 (socket says 172.21.15.179:58172) 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff590e2f700 1 -- 172.21.15.179:0/3444428084 learned_addr learned my addr 172.21.15.179:0/3444428084 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff590e2f700 1 -- 172.21.15.179:0/3444428084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880095f0 con 0x7ff58c0ff790 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.953+0000 7ff590e2f700 1 --2- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 0x7ff58c1959b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff588005b40 tx=0x7ff588004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.954+0000 7ff581ffb700 1 -- 172.21.15.179:0/3444428084 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff58801d070 con 0x7ff58c0ff790 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.954+0000 7ff581ffb700 1 -- 172.21.15.179:0/3444428084 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff58800be10 con 0x7ff58c0ff790 2024-04-02T04:39:54.958 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.954+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c1960f0 con 0x7ff58c0ff790 2024-04-02T04:39:54.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.954+0000 7ff581ffb700 1 -- 172.21.15.179:0/3444428084 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff58800f460 con 0x7ff58c0ff790 2024-04-02T04:39:54.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.954+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c196590 con 0x7ff58c0ff790 2024-04-02T04:39:54.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.955+0000 7ff581ffb700 1 -- 172.21.15.179:0/3444428084 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7ff58800f6d0 con 0x7ff58c0ff790 2024-04-02T04:39:54.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.955+0000 7ff581ffb700 1 --2- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7ff574038410 0x7ff57403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:54.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.955+0000 7ff581ffb700 1 -- 172.21.15.179:0/3444428084 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff58804c670 con 0x7ff58c0ff790 2024-04-02T04:39:54.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.955+0000 7ff583fff700 1 -- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7ff574038410 msgr2=0x7ff57403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2440490727 2024-04-02T04:39:54.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.955+0000 7ff583fff700 1 --2- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7ff574038410 0x7ff57403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:39:54.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.955+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c0623c0 con 0x7ff58c0ff790 2024-04-02T04:39:54.962 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:54.960+0000 7ff581ffb700 1 -- 172.21.15.179:0/3444428084 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff588027070 con 0x7ff58c0ff790 2024-04-02T04:39:55.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.156+0000 7ff583fff700 1 -- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7ff574038410 msgr2=0x7ff57403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2440490727 2024-04-02T04:39:55.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.156+0000 7ff583fff700 1 --2- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7ff574038410 0x7ff57403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:39:55.230 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.227+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff58c198fb0 con 0x7ff58c0ff790 2024-04-02T04:39:55.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.228+0000 7ff581ffb700 1 -- 172.21.15.179:0/3444428084 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff58802c440 con 0x7ff58c0ff790 2024-04-02T04:39:55.231 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:55.231 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:55.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7ff574038410 msgr2=0x7ff57403a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 --2- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7ff574038410 0x7ff57403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 msgr2=0x7ff58c1959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 --2- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 0x7ff58c1959b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff588005b40 tx=0x7ff588004740 comp rx=0 tx=0).stop 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 shutdown_connections 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 --2- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:6800/2440490727,v1:172.21.15.70:6801/2440490727] conn(0x7ff574038410 0x7ff57403a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 --2- 172.21.15.179:0/3444428084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff58c0ff790 0x7ff58c1959b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 >> 172.21.15.179:0/3444428084 conn(0x7ff58c0fad30 msgr2=0x7ff58c0fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 shutdown_connections 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:55.230+0000 7ff593093700 1 -- 172.21.15.179:0/3444428084 wait complete. 2024-04-02T04:39:55.234 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:55.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:55 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/3444428084' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:56.921 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:39:56.921 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:39:57.124 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:57.170 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:39:58.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.442+0000 7f3b10015700 1 -- 172.21.15.179:0/235960971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 msgr2=0x7f3b080ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.442+0000 7f3b10015700 1 --2- 172.21.15.179:0/235960971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 0x7f3b080ffba0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009910 tx=0x7f3b04009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:58.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 -- 172.21.15.179:0/235960971 shutdown_connections 2024-04-02T04:39:58.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 --2- 172.21.15.179:0/235960971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 0x7f3b080ffba0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 -- 172.21.15.179:0/235960971 >> 172.21.15.179:0/235960971 conn(0x7f3b080fad20 msgr2=0x7f3b080fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 -- 172.21.15.179:0/235960971 shutdown_connections 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 -- 172.21.15.179:0/235960971 wait complete. 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 Processor -- start 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 -- start start 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 0x7f3b081948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.443+0000 7f3b10015700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b08194de0 con 0x7f3b080ff780 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3b0ddb1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 0x7f3b081948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3b0ddb1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 0x7f3b081948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:54066/0 (socket says 172.21.15.179:54066) 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3b0ddb1700 1 -- 172.21.15.179:0/1637032304 learned_addr learned my addr 172.21.15.179:0/1637032304 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3b0ddb1700 1 -- 172.21.15.179:0/1637032304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b040095f0 con 0x7f3b080ff780 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3b0ddb1700 1 --2- 172.21.15.179:0/1637032304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 0x7f3b081948a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005950 tx=0x7f3b04009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3afeffd700 1 -- 172.21.15.179:0/1637032304 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b0401d070 con 0x7f3b080ff780 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b08194fe0 con 0x7f3b080ff780 2024-04-02T04:39:58.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3afeffd700 1 -- 172.21.15.179:0/1637032304 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3b0400be10 con 0x7f3b080ff780 2024-04-02T04:39:58.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3afeffd700 1 -- 172.21.15.179:0/1637032304 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b0400f460 con 0x7f3b080ff780 2024-04-02T04:39:58.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.444+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b08195480 con 0x7f3b080ff780 2024-04-02T04:39:58.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.445+0000 7f3afeffd700 1 -- 172.21.15.179:0/1637032304 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 15) v1 ==== 45077+0+0 (secure 0 0 0) 0x7f3b0400b960 con 0x7f3b080ff780 2024-04-02T04:39:58.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.445+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0818e3a0 con 0x7f3b080ff780 2024-04-02T04:39:58.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.445+0000 7f3afeffd700 1 -- 172.21.15.179:0/1637032304 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3b0404c160 con 0x7f3b080ff780 2024-04-02T04:39:58.453 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.450+0000 7f3afeffd700 1 -- 172.21.15.179:0/1637032304 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3b0402bba0 con 0x7f3b080ff780 2024-04-02T04:39:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: Active manager daemon smithi070.igidvr restarted 2024-04-02T04:39:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: Activating manager daemon smithi070.igidvr 2024-04-02T04:39:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: osdmap e5: 0 total, 0 up, 0 in 2024-04-02T04:39:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: mgrmap e15: smithi070.igidvr(active, starting, since 0.00306283s) 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: Manager daemon smithi070.igidvr is now available 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:58.730 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.727+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3b0802cc70 con 0x7f3b080ff780 2024-04-02T04:39:58.731 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.728+0000 7f3afeffd700 1 -- 172.21.15.179:0/1637032304 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3b04027030 con 0x7f3b080ff780 2024-04-02T04:39:58.731 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:39:58.731 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.730+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 msgr2=0x7f3b081948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.730+0000 7f3b10015700 1 --2- 172.21.15.179:0/1637032304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 0x7f3b081948a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005950 tx=0x7f3b04009e60 comp rx=0 tx=0).stop 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.730+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 shutdown_connections 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.730+0000 7f3b10015700 1 --2- 172.21.15.179:0/1637032304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b080ff780 0x7f3b081948a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.730+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 >> 172.21.15.179:0/1637032304 conn(0x7f3b080fad20 msgr2=0x7f3b080fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.730+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 shutdown_connections 2024-04-02T04:39:58.734 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:39:58.730+0000 7f3b10015700 1 -- 172.21.15.179:0/1637032304 wait complete. 2024-04-02T04:39:58.734 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:39:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:59 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:39:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:59 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/trash_purge_schedule"}]: dispatch 2024-04-02T04:39:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:59 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/1637032304' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:39:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:59 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:39:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:39:59 smithi070 ceph-mon[29212]: mgrmap e16: smithi070.igidvr(active, since 1.00541s) 2024-04-02T04:40:00.355 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:40:00.355 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:40:00.565 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:40:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:00 smithi070 ceph-mon[29212]: [02/Apr/2024:04:39:58] ENGINE Bus STARTING 2024-04-02T04:40:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:00 smithi070 ceph-mon[29212]: [02/Apr/2024:04:39:59] ENGINE Serving on https://172.21.15.70:7150 2024-04-02T04:40:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:00 smithi070 ceph-mon[29212]: [02/Apr/2024:04:39:59] ENGINE Serving on http://172.21.15.70:8765 2024-04-02T04:40:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:00 smithi070 ceph-mon[29212]: [02/Apr/2024:04:39:59] ENGINE Bus STARTED 2024-04-02T04:40:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:01.447 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:40:01.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:01 smithi070 ceph-mon[29212]: mgrmap e17: smithi070.igidvr(active, since 2s) 2024-04-02T04:40:03.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 -- 172.21.15.179:0/1559097795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 msgr2=0x7f8d3c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:03.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 --2- 172.21.15.179:0/1559097795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 0x7f8d3c0ffba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009910 tx=0x7f8d38009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:03.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 -- 172.21.15.179:0/1559097795 shutdown_connections 2024-04-02T04:40:03.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 --2- 172.21.15.179:0/1559097795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 0x7f8d3c0ffba0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:03.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 -- 172.21.15.179:0/1559097795 >> 172.21.15.179:0/1559097795 conn(0x7f8d3c0fad00 msgr2=0x7f8d3c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:03.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 -- 172.21.15.179:0/1559097795 shutdown_connections 2024-04-02T04:40:03.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 -- 172.21.15.179:0/1559097795 wait complete. 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 Processor -- start 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.915+0000 7f8d434e7700 1 -- start start 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d434e7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 0x7f8d3c18c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d434e7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c18cca0 con 0x7f8d3c0ff780 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d41283700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 0x7f8d3c18c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d41283700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 0x7f8d3c18c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:54082/0 (socket says 172.21.15.179:54082) 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d41283700 1 -- 172.21.15.179:0/4228256895 learned_addr learned my addr 172.21.15.179:0/4228256895 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d41283700 1 -- 172.21.15.179:0/4228256895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380095f0 con 0x7f8d3c0ff780 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d41283700 1 --2- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 0x7f8d3c18c760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d38005b40 tx=0x7f8d38004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d327fc700 1 -- 172.21.15.179:0/4228256895 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d3801d070 con 0x7f8d3c0ff780 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.916+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c18adb0 con 0x7f8d3c0ff780 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.917+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c18b250 con 0x7f8d3c0ff780 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.917+0000 7f8d327fc700 1 -- 172.21.15.179:0/4228256895 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8d3800be10 con 0x7f8d3c0ff780 2024-04-02T04:40:03.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.917+0000 7f8d327fc700 1 -- 172.21.15.179:0/4228256895 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d3800f460 con 0x7f8d3c0ff780 2024-04-02T04:40:03.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.917+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20005320 con 0x7f8d3c0ff780 2024-04-02T04:40:03.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.917+0000 7f8d327fc700 1 -- 172.21.15.179:0/4228256895 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f8d3800b960 con 0x7f8d3c0ff780 2024-04-02T04:40:03.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.918+0000 7f8d327fc700 1 --2- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f8d280384b0 0x7f8d2803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:03.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.918+0000 7f8d327fc700 1 -- 172.21.15.179:0/4228256895 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f8d3804c530 con 0x7f8d3c0ff780 2024-04-02T04:40:03.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.918+0000 7f8d40a82700 1 --2- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f8d280384b0 0x7f8d2803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:03.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.918+0000 7f8d40a82700 1 --2- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f8d280384b0 0x7f8d2803a970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c006fd0 tx=0x7f8d2c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:03.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:03.923+0000 7f8d327fc700 1 -- 172.21.15.179:0/4228256895 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8d3802bba0 con 0x7f8d3c0ff780 2024-04-02T04:40:04.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.202+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8d200059f0 con 0x7f8d3c0ff780 2024-04-02T04:40:04.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.202+0000 7f8d327fc700 1 -- 172.21.15.179:0/4228256895 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8d38027030 con 0x7f8d3c0ff780 2024-04-02T04:40:04.206 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:40:04.206 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:40:04.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f8d280384b0 msgr2=0x7f8d2803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:04.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 --2- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f8d280384b0 0x7f8d2803a970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c006fd0 tx=0x7f8d2c006e40 comp rx=0 tx=0).stop 2024-04-02T04:40:04.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 msgr2=0x7f8d3c18c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:04.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 --2- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 0x7f8d3c18c760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d38005b40 tx=0x7f8d38004740 comp rx=0 tx=0).stop 2024-04-02T04:40:04.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 shutdown_connections 2024-04-02T04:40:04.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 --2- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f8d280384b0 0x7f8d2803a970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:04.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 --2- 172.21.15.179:0/4228256895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d3c0ff780 0x7f8d3c18c760 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:04.208 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 >> 172.21.15.179:0/4228256895 conn(0x7f8d3c0fad00 msgr2=0x7f8d3c0fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:04.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 shutdown_connections 2024-04-02T04:40:04.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:04.205+0000 7f8d434e7700 1 -- 172.21.15.179:0/4228256895 wait complete. 2024-04-02T04:40:04.209 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:40:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:40:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:03 smithi070 ceph-mon[29212]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:40:05.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:05.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:05.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/4228256895' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:40:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:05.807 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:40:05.807 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:40:06.003 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:06.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:05 smithi070 ceph-mon[29212]: Deploying daemon ceph-exporter.smithi179 on smithi179 2024-04-02T04:40:07.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.948+0000 7fc27c685700 1 -- 172.21.15.179:0/117947966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 msgr2=0x7fc274106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:07.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.948+0000 7fc27c685700 1 --2- 172.21.15.179:0/117947966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 0x7fc274106c00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00b0a0 tx=0x7fc26c00b3b0 comp rx=0 tx=0).stop 2024-04-02T04:40:07.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.948+0000 7fc27c685700 1 -- 172.21.15.179:0/117947966 shutdown_connections 2024-04-02T04:40:07.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.948+0000 7fc27c685700 1 --2- 172.21.15.179:0/117947966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 0x7fc274106c00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:07.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.948+0000 7fc27c685700 1 -- 172.21.15.179:0/117947966 >> 172.21.15.179:0/117947966 conn(0x7fc2740fc350 msgr2=0x7fc2740fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:07.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.948+0000 7fc27c685700 1 -- 172.21.15.179:0/117947966 shutdown_connections 2024-04-02T04:40:07.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.948+0000 7fc27c685700 1 -- 172.21.15.179:0/117947966 wait complete. 2024-04-02T04:40:07.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.949+0000 7fc27c685700 1 Processor -- start 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.949+0000 7fc27c685700 1 -- start start 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.949+0000 7fc27c685700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 0x7fc274073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.949+0000 7fc27c685700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc274075060 con 0x7fc274106820 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.949+0000 7fc27a421700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 0x7fc274073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.949+0000 7fc27a421700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 0x7fc274073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:54070/0 (socket says 172.21.15.179:54070) 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.949+0000 7fc27a421700 1 -- 172.21.15.179:0/61868235 learned_addr learned my addr 172.21.15.179:0/61868235 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.950+0000 7fc27a421700 1 -- 172.21.15.179:0/61868235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c009d00 con 0x7fc274106820 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.950+0000 7fc27a421700 1 --2- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 0x7fc274073090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00d010 tx=0x7fc26c003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.950+0000 7fc26b7fe700 1 -- 172.21.15.179:0/61868235 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc26c00e040 con 0x7fc274106820 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.950+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc274073690 con 0x7fc274106820 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.950+0000 7fc26b7fe700 1 -- 172.21.15.179:0/61868235 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc26c0044f0 con 0x7fc274106820 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.950+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc274073b10 con 0x7fc274106820 2024-04-02T04:40:07.954 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.950+0000 7fc26b7fe700 1 -- 172.21.15.179:0/61868235 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc26c01d920 con 0x7fc274106820 2024-04-02T04:40:07.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.951+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc274104ad0 con 0x7fc274106820 2024-04-02T04:40:07.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.951+0000 7fc26b7fe700 1 -- 172.21.15.179:0/61868235 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fc26c004040 con 0x7fc274106820 2024-04-02T04:40:07.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.951+0000 7fc26b7fe700 1 --2- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc2600384b0 0x7fc26003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:07.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.951+0000 7fc26b7fe700 1 -- 172.21.15.179:0/61868235 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc26c04c360 con 0x7fc274106820 2024-04-02T04:40:07.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.951+0000 7fc279c20700 1 --2- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc2600384b0 0x7fc26003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:07.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.952+0000 7fc279c20700 1 --2- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc2600384b0 0x7fc26003a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc264006fd0 tx=0x7fc264006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:07.959 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:07.956+0000 7fc26b7fe700 1 -- 172.21.15.179:0/61868235 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc26c011570 con 0x7fc274106820 2024-04-02T04:40:08.235 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.232+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc2740623c0 con 0x7fc274106820 2024-04-02T04:40:08.236 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.233+0000 7fc26b7fe700 1 -- 172.21.15.179:0/61868235 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc26c011390 con 0x7fc274106820 2024-04-02T04:40:08.236 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:40:08.236 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc2600384b0 msgr2=0x7fc26003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 --2- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc2600384b0 0x7fc26003a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc264006fd0 tx=0x7fc264006e40 comp rx=0 tx=0).stop 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 msgr2=0x7fc274073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 --2- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 0x7fc274073090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00d010 tx=0x7fc26c003ad0 comp rx=0 tx=0).stop 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 shutdown_connections 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 --2- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc2600384b0 0x7fc26003a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 --2- 172.21.15.179:0/61868235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc274106820 0x7fc274073090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 >> 172.21.15.179:0/61868235 conn(0x7fc2740fc350 msgr2=0x7fc274108c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:08.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 shutdown_connections 2024-04-02T04:40:08.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:08.235+0000 7fc27c685700 1 -- 172.21.15.179:0/61868235 wait complete. 2024-04-02T04:40:08.239 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:40:09.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:09.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:09.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/61868235' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:40:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi179", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:40:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi179", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-02T04:40:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:09 smithi070 ceph-mon[29212]: Deploying daemon crash.smithi179 on smithi179 2024-04-02T04:40:09.817 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:40:09.818 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:40:10.018 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:11.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.641+0000 7fd4ed3af700 1 -- 172.21.15.179:0/1747717293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 msgr2=0x7fd4e8106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:11.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.641+0000 7fd4ed3af700 1 --2- 172.21.15.179:0/1747717293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 0x7fd4e8106c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009910 tx=0x7fd4d0009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:11.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.641+0000 7fd4ed3af700 1 -- 172.21.15.179:0/1747717293 shutdown_connections 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.641+0000 7fd4ed3af700 1 --2- 172.21.15.179:0/1747717293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 0x7fd4e8106c00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.641+0000 7fd4ed3af700 1 -- 172.21.15.179:0/1747717293 >> 172.21.15.179:0/1747717293 conn(0x7fd4e80fc350 msgr2=0x7fd4e80fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.641+0000 7fd4ed3af700 1 -- 172.21.15.179:0/1747717293 shutdown_connections 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.641+0000 7fd4ed3af700 1 -- 172.21.15.179:0/1747717293 wait complete. 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.641+0000 7fd4ed3af700 1 Processor -- start 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.642+0000 7fd4ed3af700 1 -- start start 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.642+0000 7fd4ed3af700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 0x7fd4e8195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.642+0000 7fd4ed3af700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8196260 con 0x7fd4e8106820 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.642+0000 7fd4e6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 0x7fd4e8195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.642+0000 7fd4e6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 0x7fd4e8195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:54104/0 (socket says 172.21.15.179:54104) 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.642+0000 7fd4e6ffd700 1 -- 172.21.15.179:0/34961730 learned_addr learned my addr 172.21.15.179:0/34961730 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.642+0000 7fd4e6ffd700 1 -- 172.21.15.179:0/34961730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00095f0 con 0x7fd4e8106820 2024-04-02T04:40:11.646 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.643+0000 7fd4e6ffd700 1 --2- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 0x7fd4e8195d20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005b40 tx=0x7fd4d0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.643+0000 7fd4dffff700 1 -- 172.21.15.179:0/34961730 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4d001d070 con 0x7fd4e8106820 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.643+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e8196460 con 0x7fd4e8106820 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.643+0000 7fd4dffff700 1 -- 172.21.15.179:0/34961730 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4d00044d0 con 0x7fd4e8106820 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.643+0000 7fd4dffff700 1 -- 172.21.15.179:0/34961730 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4d0003910 con 0x7fd4e8106820 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.643+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e81968e0 con 0x7fd4e8106820 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.644+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e80623c0 con 0x7fd4e8106820 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.644+0000 7fd4dffff700 1 -- 172.21.15.179:0/34961730 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fd4d0003b90 con 0x7fd4e8106820 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.644+0000 7fd4dffff700 1 --2- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd4d40384b0 0x7fd4d403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.644+0000 7fd4dffff700 1 -- 172.21.15.179:0/34961730 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd4d004c460 con 0x7fd4e8106820 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.644+0000 7fd4e67fc700 1 --2- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd4d40384b0 0x7fd4d403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:11.648 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.645+0000 7fd4e67fc700 1 --2- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd4d40384b0 0x7fd4d403a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006fd0 tx=0x7fd4d8006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:11.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.649+0000 7fd4dffff700 1 -- 172.21.15.179:0/34961730 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd4d002a540 con 0x7fd4e8106820 2024-04-02T04:40:11.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.947+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd4e8104630 con 0x7fd4e8106820 2024-04-02T04:40:11.950 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.947+0000 7fd4dffff700 1 -- 172.21.15.179:0/34961730 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fd4d002a360 con 0x7fd4e8106820 2024-04-02T04:40:11.951 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:40:11.951 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:40:11.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd4d40384b0 msgr2=0x7fd4d403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:11.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 --2- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd4d40384b0 0x7fd4d403a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006fd0 tx=0x7fd4d8006e40 comp rx=0 tx=0).stop 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 msgr2=0x7fd4e8195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 --2- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 0x7fd4e8195d20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005b40 tx=0x7fd4d0004c80 comp rx=0 tx=0).stop 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 shutdown_connections 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 --2- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd4d40384b0 0x7fd4d403a970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 --2- 172.21.15.179:0/34961730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4e8106820 0x7fd4e8195d20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 >> 172.21.15.179:0/34961730 conn(0x7fd4e80fc350 msgr2=0x7fd4e8108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.949+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 shutdown_connections 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:11.950+0000 7fd4ed3af700 1 -- 172.21.15.179:0/34961730 wait complete. 2024-04-02T04:40:11.953 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:40:12.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:11 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/34961730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:40:13.505 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:40:13.506 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:40:13.727 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:13 smithi070 ceph-mon[29212]: Deploying daemon node-exporter.smithi179 on smithi179 2024-04-02T04:40:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:14.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.174+0000 7f0db0efd700 1 -- 172.21.15.179:0/4119237948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 msgr2=0x7f0dac106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.174+0000 7f0db0efd700 1 --2- 172.21.15.179:0/4119237948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 0x7f0dac106be0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009910 tx=0x7f0d9c009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.174+0000 7f0db0efd700 1 -- 172.21.15.179:0/4119237948 shutdown_connections 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.174+0000 7f0db0efd700 1 --2- 172.21.15.179:0/4119237948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 0x7f0dac106be0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.174+0000 7f0db0efd700 1 -- 172.21.15.179:0/4119237948 >> 172.21.15.179:0/4119237948 conn(0x7f0dac0fc2f0 msgr2=0x7f0dac0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.174+0000 7f0db0efd700 1 -- 172.21.15.179:0/4119237948 shutdown_connections 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.174+0000 7f0db0efd700 1 -- 172.21.15.179:0/4119237948 wait complete. 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.175+0000 7f0db0efd700 1 Processor -- start 2024-04-02T04:40:16.220 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.175+0000 7f0db0efd700 1 -- start start 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.175+0000 7f0db0efd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 0x7f0dac073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.175+0000 7f0db0efd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac199d40 con 0x7f0dac106800 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.175+0000 7f0daa59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 0x7f0dac073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.175+0000 7f0daa59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 0x7f0dac073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:54136/0 (socket says 172.21.15.179:54136) 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.175+0000 7f0daa59c700 1 -- 172.21.15.179:0/3976506100 learned_addr learned my addr 172.21.15.179:0/3976506100 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.176+0000 7f0daa59c700 1 -- 172.21.15.179:0/3976506100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c0095f0 con 0x7f0dac106800 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.176+0000 7f0daa59c700 1 --2- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 0x7f0dac073090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c005b40 tx=0x7f0d9c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.176+0000 7f0d9b7fe700 1 -- 172.21.15.179:0/3976506100 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d9c01d070 con 0x7f0dac106800 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.176+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac074e70 con 0x7f0dac106800 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.176+0000 7f0d9b7fe700 1 -- 172.21.15.179:0/3976506100 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0d9c0044d0 con 0x7f0dac106800 2024-04-02T04:40:16.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.176+0000 7f0d9b7fe700 1 -- 172.21.15.179:0/3976506100 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d9c003910 con 0x7f0dac106800 2024-04-02T04:40:16.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.176+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac0737a0 con 0x7f0dac106800 2024-04-02T04:40:16.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.177+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac104ab0 con 0x7f0dac106800 2024-04-02T04:40:16.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.177+0000 7f0d9b7fe700 1 -- 172.21.15.179:0/3976506100 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f0d9c00b960 con 0x7f0dac106800 2024-04-02T04:40:16.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.177+0000 7f0d9b7fe700 1 --2- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0d940384b0 0x7f0d9403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:16.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.177+0000 7f0d9b7fe700 1 -- 172.21.15.179:0/3976506100 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f0d9c04c5b0 con 0x7f0dac106800 2024-04-02T04:40:16.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.177+0000 7f0da9d9b700 1 --2- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0d940384b0 0x7f0d9403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:16.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.178+0000 7f0da9d9b700 1 --2- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0d940384b0 0x7f0d9403a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006fd0 tx=0x7f0da0006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:16.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.182+0000 7f0d9b7fe700 1 -- 172.21.15.179:0/3976506100 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0d9c02a540 con 0x7f0dac106800 2024-04-02T04:40:16.451 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.447+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0dac104490 con 0x7f0dac106800 2024-04-02T04:40:16.452 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.448+0000 7f0d9b7fe700 1 -- 172.21.15.179:0/3976506100 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0d9c02a360 con 0x7f0dac106800 2024-04-02T04:40:16.452 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:40:16.453 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:40:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0d940384b0 msgr2=0x7f0d9403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 --2- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0d940384b0 0x7f0d9403a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006fd0 tx=0x7f0da0006e40 comp rx=0 tx=0).stop 2024-04-02T04:40:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 msgr2=0x7f0dac073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 --2- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 0x7f0dac073090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c005b40 tx=0x7f0d9c004c80 comp rx=0 tx=0).stop 2024-04-02T04:40:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 shutdown_connections 2024-04-02T04:40:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 --2- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0d940384b0 0x7f0d9403a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 --2- 172.21.15.179:0/3976506100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dac106800 0x7f0dac073090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:16.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 >> 172.21.15.179:0/3976506100 conn(0x7f0dac0fc2f0 msgr2=0x7f0dac108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:16.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 shutdown_connections 2024-04-02T04:40:16.456 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:16.450+0000 7f0db0efd700 1 -- 172.21.15.179:0/3976506100 wait complete. 2024-04-02T04:40:16.456 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:40:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:16 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/3976506100' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:40:18.511 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:40:18.511 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:40:18.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:18.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:18.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:18.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:18.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:40:18.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T04:40:18.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:40:18.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:18.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:18 smithi070 ceph-mon[29212]: Deploying daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:40:18.713 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:19.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:19 smithi070 ceph-mon[29212]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:19.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:20.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.385+0000 7fa242ee9700 1 -- 172.21.15.179:0/1559040636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 msgr2=0x7fa23c0ff500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:20.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.385+0000 7fa242ee9700 1 --2- 172.21.15.179:0/1559040636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 0x7fa23c0ff500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa230009910 tx=0x7fa230009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:20.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.385+0000 7fa242ee9700 1 -- 172.21.15.179:0/1559040636 shutdown_connections 2024-04-02T04:40:20.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.385+0000 7fa242ee9700 1 --2- 172.21.15.179:0/1559040636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 0x7fa23c0ff500 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:20.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.385+0000 7fa242ee9700 1 -- 172.21.15.179:0/1559040636 >> 172.21.15.179:0/1559040636 conn(0x7fa23c075e80 msgr2=0x7fa23c076290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:20.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.385+0000 7fa242ee9700 1 -- 172.21.15.179:0/1559040636 shutdown_connections 2024-04-02T04:40:20.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.385+0000 7fa242ee9700 1 -- 172.21.15.179:0/1559040636 wait complete. 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.386+0000 7fa242ee9700 1 Processor -- start 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.386+0000 7fa242ee9700 1 -- start start 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.386+0000 7fa242ee9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 0x7fa23c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.386+0000 7fa242ee9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c19afd0 con 0x7fa23c0ff120 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.386+0000 7fa240c85700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 0x7fa23c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.386+0000 7fa240c85700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 0x7fa23c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:59600/0 (socket says 172.21.15.179:59600) 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.386+0000 7fa240c85700 1 -- 172.21.15.179:0/3368798140 learned_addr learned my addr 172.21.15.179:0/3368798140 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.386+0000 7fa240c85700 1 -- 172.21.15.179:0/3368798140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2300095f0 con 0x7fa23c0ff120 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.387+0000 7fa240c85700 1 --2- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 0x7fa23c19aa90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2300092e0 tx=0x7fa230004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.387+0000 7fa239ffb700 1 -- 172.21.15.179:0/3368798140 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa23001d070 con 0x7fa23c0ff120 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.387+0000 7fa239ffb700 1 -- 172.21.15.179:0/3368798140 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa23000be10 con 0x7fa23c0ff120 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.387+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23c19b1d0 con 0x7fa23c0ff120 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.387+0000 7fa239ffb700 1 -- 172.21.15.179:0/3368798140 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa230022e10 con 0x7fa23c0ff120 2024-04-02T04:40:20.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.387+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c194a80 con 0x7fa23c0ff120 2024-04-02T04:40:20.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.388+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23c19b5f0 con 0x7fa23c0ff120 2024-04-02T04:40:20.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.388+0000 7fa239ffb700 1 -- 172.21.15.179:0/3368798140 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fa230022470 con 0x7fa23c0ff120 2024-04-02T04:40:20.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.388+0000 7fa239ffb700 1 --2- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fa22c038460 0x7fa22c03a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:20.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.388+0000 7fa239ffb700 1 -- 172.21.15.179:0/3368798140 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa23004c620 con 0x7fa23c0ff120 2024-04-02T04:40:20.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.388+0000 7fa23bfff700 1 --2- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fa22c038460 0x7fa22c03a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:20.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.389+0000 7fa23bfff700 1 --2- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fa22c038460 0x7fa22c03a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa228006fd0 tx=0x7fa228006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:20.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.393+0000 7fa239ffb700 1 -- 172.21.15.179:0/3368798140 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa23002a540 con 0x7fa23c0ff120 2024-04-02T04:40:20.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.665+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa23c194d10 con 0x7fa23c0ff120 2024-04-02T04:40:20.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.666+0000 7fa239ffb700 1 -- 172.21.15.179:0/3368798140 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fa230027030 con 0x7fa23c0ff120 2024-04-02T04:40:20.670 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:40:20.671 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.668+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fa22c038460 msgr2=0x7fa22c03a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.668+0000 7fa242ee9700 1 --2- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fa22c038460 0x7fa22c03a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa228006fd0 tx=0x7fa228006e40 comp rx=0 tx=0).stop 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.669+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 msgr2=0x7fa23c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.669+0000 7fa242ee9700 1 --2- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 0x7fa23c19aa90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2300092e0 tx=0x7fa230004c80 comp rx=0 tx=0).stop 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.669+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 shutdown_connections 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.669+0000 7fa242ee9700 1 --2- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fa22c038460 0x7fa22c03a920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.669+0000 7fa242ee9700 1 --2- 172.21.15.179:0/3368798140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa23c0ff120 0x7fa23c19aa90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.669+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 >> 172.21.15.179:0/3368798140 conn(0x7fa23c075e80 msgr2=0x7fa23c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.669+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 shutdown_connections 2024-04-02T04:40:20.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:20.669+0000 7fa242ee9700 1 -- 172.21.15.179:0/3368798140 wait complete. 2024-04-02T04:40:20.674 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:40:21.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:21 smithi070 ceph-mon[29212]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:21.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:21 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/3368798140' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:40:22.436 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:40:22.437 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:40:22.721 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:22.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:22.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:22.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:22.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:40:22.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:22.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:22 smithi070 ceph-mon[29212]: Deploying daemon mon.smithi179 on smithi179 2024-04-02T04:40:22.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:22 smithi070 ceph-mon[29212]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:24 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:25.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:25 smithi070 ceph-mon[29212]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:25.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.762+0000 7fe8173b5700 1 -- 172.21.15.179:0/132404005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 msgr2=0x7fe810105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.762+0000 7fe8173b5700 1 --2- 172.21.15.179:0/132404005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 0x7fe810105b60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009910 tx=0x7fe80c009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:25.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.762+0000 7fe8173b5700 1 -- 172.21.15.179:0/132404005 shutdown_connections 2024-04-02T04:40:25.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.762+0000 7fe8173b5700 1 --2- 172.21.15.179:0/132404005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 0x7fe810105b60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.762+0000 7fe8173b5700 1 -- 172.21.15.179:0/132404005 >> 172.21.15.179:0/132404005 conn(0x7fe810075290 msgr2=0x7fe8100756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:25.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.762+0000 7fe8173b5700 1 -- 172.21.15.179:0/132404005 shutdown_connections 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.762+0000 7fe8173b5700 1 -- 172.21.15.179:0/132404005 wait complete. 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.762+0000 7fe8173b5700 1 Processor -- start 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.763+0000 7fe8173b5700 1 -- start start 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.763+0000 7fe8173b5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 0x7fe81019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.763+0000 7fe8173b5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81019afb0 con 0x7fe810105780 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.763+0000 7fe815151700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 0x7fe81019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.763+0000 7fe815151700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 0x7fe81019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.179:59634/0 (socket says 172.21.15.179:59634) 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.763+0000 7fe815151700 1 -- 172.21.15.179:0/2270942258 learned_addr learned my addr 172.21.15.179:0/2270942258 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.763+0000 7fe815151700 1 -- 172.21.15.179:0/2270942258 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0095f0 con 0x7fe810105780 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.764+0000 7fe815151700 1 --2- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 0x7fe81019aa70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe80c005b40 tx=0x7fe80c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.764+0000 7fe8067fc700 1 -- 172.21.15.179:0/2270942258 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe80c01d070 con 0x7fe810105780 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.764+0000 7fe8067fc700 1 -- 172.21.15.179:0/2270942258 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe80c0044d0 con 0x7fe810105780 2024-04-02T04:40:25.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.764+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81019b1b0 con 0x7fe810105780 2024-04-02T04:40:25.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.764+0000 7fe8067fc700 1 -- 172.21.15.179:0/2270942258 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe80c003930 con 0x7fe810105780 2024-04-02T04:40:25.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.764+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81019b570 con 0x7fe810105780 2024-04-02T04:40:25.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.765+0000 7fe8067fc700 1 -- 172.21.15.179:0/2270942258 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fe80c003c70 con 0x7fe810105780 2024-04-02T04:40:25.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.765+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8100623c0 con 0x7fe810105780 2024-04-02T04:40:25.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.765+0000 7fe8067fc700 1 --2- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe7fc0384b0 0x7fe7fc03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:25.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.765+0000 7fe8067fc700 1 -- 172.21.15.179:0/2270942258 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fe80c04c5a0 con 0x7fe810105780 2024-04-02T04:40:25.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.765+0000 7fe814950700 1 --2- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe7fc0384b0 0x7fe7fc03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:25.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.766+0000 7fe814950700 1 --2- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe7fc0384b0 0x7fe7fc03a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe800006fd0 tx=0x7fe800006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:25.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:25.770+0000 7fe8067fc700 1 -- 172.21.15.179:0/2270942258 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe80c02a950 con 0x7fe810105780 2024-04-02T04:40:26.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.057+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe8101035f0 con 0x7fe810105780 2024-04-02T04:40:26.061 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.057+0000 7fe8067fc700 1 -- 172.21.15.179:0/2270942258 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fe80c02a360 con 0x7fe810105780 2024-04-02T04:40:26.062 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:40:26.062 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:37:39.761198Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe7fc0384b0 msgr2=0x7fe7fc03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 --2- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe7fc0384b0 0x7fe7fc03a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe800006fd0 tx=0x7fe800006e40 comp rx=0 tx=0).stop 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 msgr2=0x7fe81019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 --2- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 0x7fe81019aa70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe80c005b40 tx=0x7fe80c004c80 comp rx=0 tx=0).stop 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 shutdown_connections 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 --2- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe7fc0384b0 0x7fe7fc03a970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 --2- 172.21.15.179:0/2270942258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe810105780 0x7fe81019aa70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 >> 172.21.15.179:0/2270942258 conn(0x7fe810075290 msgr2=0x7fe8100fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:26.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 shutdown_connections 2024-04-02T04:40:26.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:26.060+0000 7fe8173b5700 1 -- 172.21.15.179:0/2270942258 wait complete. 2024-04-02T04:40:26.065 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-02T04:40:27.951 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:40:27.954 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:40:28.159 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi179/config 2024-04-02T04:40:31.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:31.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:31.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:40:31.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: mon.smithi070 calling monitor election 2024-04-02T04:40:31.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:40:31.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:40:31.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/469321594' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:40:31.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:31.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:31.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:40:31.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: mon.smithi179 calling monitor election 2024-04-02T04:40:31.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:40:31.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:31.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:40:31.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:40:31.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: mon.smithi070 is new leader, mons smithi070,smithi179 in quorum (ranks 0,1) 2024-04-02T04:40:31.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: monmap e2: 2 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],smithi179=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-02T04:40:31.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: fsmap 2024-04-02T04:40:31.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: osdmap e5: 0 total, 0 up, 0 in 2024-04-02T04:40:31.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: mgrmap e17: smithi070.igidvr(active, since 33s) 2024-04-02T04:40:31.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:40:31.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/469321594' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:40:31.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: overall HEALTH_OK 2024-04-02T04:40:31.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/469321594' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:40:31.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/469321594' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:40:31.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:31.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:31.784 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:31.784 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:31.784 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:32 smithi179 ceph-mon[36603]: mgrmap e18: smithi070.igidvr(active, since 33s), standbys: smithi179.labmvs 2024-04-02T04:40:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:32 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi179.labmvs", "id": "smithi179.labmvs"}]: dispatch 2024-04-02T04:40:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:32 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:40:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:32 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:40:32.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:32 smithi070 ceph-mon[29212]: mgrmap e18: smithi070.igidvr(active, since 33s), standbys: smithi179.labmvs 2024-04-02T04:40:32.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:32 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi179.labmvs", "id": "smithi179.labmvs"}]: dispatch 2024-04-02T04:40:32.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:32 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:40:32.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:32 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:40:32.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.653+0000 7f517acb3700 1 -- 172.21.15.179:0/3792350985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51640035c0 msgr2=0x7f5164005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:32.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.653+0000 7f517acb3700 1 --2- 172.21.15.179:0/3792350985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51640035c0 0x7f5164005a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5174100870 tx=0x7f516800feb0 comp rx=0 tx=0).stop 2024-04-02T04:40:32.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.653+0000 7f517acb3700 1 -- 172.21.15.179:0/3792350985 shutdown_connections 2024-04-02T04:40:32.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.653+0000 7f517acb3700 1 --2- 172.21.15.179:0/3792350985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51640035c0 0x7f5164005a50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.653+0000 7f517acb3700 1 -- 172.21.15.179:0/3792350985 >> 172.21.15.179:0/3792350985 conn(0x7f51740fc430 msgr2=0x7f51740fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.653+0000 7f517acb3700 1 -- 172.21.15.179:0/3792350985 shutdown_connections 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.653+0000 7f517acb3700 1 -- 172.21.15.179:0/3792350985 wait complete. 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.654+0000 7f517acb3700 1 Processor -- start 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.654+0000 7f517acb3700 1 -- start start 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.654+0000 7f517acb3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 0x7f5174195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.654+0000 7f517acb3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5174196480 0x7f517419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.654+0000 7f517acb3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5174196aa0 con 0x7f5174196480 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.654+0000 7f517acb3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5174196c10 con 0x7f51640035c0 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5178a4f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 0x7f5174195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5178a4f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 0x7f5174195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38286/0 (socket says 172.21.15.179:38286) 2024-04-02T04:40:32.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5178a4f700 1 -- 172.21.15.179:0/3965109804 learned_addr learned my addr 172.21.15.179:0/3965109804 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5173fff700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5174196480 0x7f517419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5178a4f700 1 -- 172.21.15.179:0/3965109804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 msgr2=0x7f5174195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5178a4f700 1 -- 172.21.15.179:0/3965109804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 msgr2=0x7f5174195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5178a4f700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 0x7f5174195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5178a4f700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 0x7f5174195f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5173fff700 1 -- 172.21.15.179:0/3965109804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 msgr2=0x7f5174195f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5173fff700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 0x7f5174195f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5173fff700 1 -- 172.21.15.179:0/3965109804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516800f900 con 0x7f5174196480 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5173fff700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5174196480 0x7f517419a8f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f516000b4d0 tx=0x7f516000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5171ffb700 1 -- 172.21.15.179:0/3965109804 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5160011800 con 0x7f5174196480 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5171ffb700 1 -- 172.21.15.179:0/3965109804 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5160011e10 con 0x7f5174196480 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517419aef0 con 0x7f5174196480 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.655+0000 7f5171ffb700 1 -- 172.21.15.179:0/3965109804 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f516000f2e0 con 0x7f5174196480 2024-04-02T04:40:32.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.656+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517419b440 con 0x7f5174196480 2024-04-02T04:40:32.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.657+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517419b710 con 0x7f5174196480 2024-04-02T04:40:32.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.657+0000 7f5171ffb700 1 -- 172.21.15.179:0/3965109804 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f5160010380 con 0x7f5174196480 2024-04-02T04:40:32.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.657+0000 7f5171ffb700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f515806c560 0x7f515806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:32.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.657+0000 7f5171ffb700 1 -- 172.21.15.179:0/3965109804 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f516008a2d0 con 0x7f5174196480 2024-04-02T04:40:32.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.657+0000 7f5178a4f700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f515806c560 0x7f515806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:32.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.658+0000 7f5178a4f700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f515806c560 0x7f515806ea20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5168005ed0 tx=0x7f5168005e60 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:32.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.662+0000 7f5171ffb700 1 -- 172.21.15.179:0/3965109804 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5160055b90 con 0x7f5174196480 2024-04-02T04:40:32.956 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.952+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f51741a0b90 con 0x7f5174196480 2024-04-02T04:40:32.957 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.953+0000 7f5171ffb700 1 -- 172.21.15.179:0/3965109804 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1022 (secure 0 0 0) 0x7f5160015410 con 0x7f5174196480 2024-04-02T04:40:32.957 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-02T04:40:32.957 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:40:26.229830Z","created":"2024-04-02T04:37:39.761198Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi179","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f515806c560 msgr2=0x7f515806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f515806c560 0x7f515806ea20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5168005ed0 tx=0x7f5168005e60 comp rx=0 tx=0).stop 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5174196480 msgr2=0x7f517419a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5174196480 0x7f517419a8f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f516000b4d0 tx=0x7f516000b890 comp rx=0 tx=0).stop 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 shutdown_connections 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f515806c560 0x7f515806ea20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51640035c0 0x7f5174195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 --2- 172.21.15.179:0/3965109804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5174196480 0x7f517419a8f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 >> 172.21.15.179:0/3965109804 conn(0x7f51740fc430 msgr2=0x7f517418e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:32.960 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 shutdown_connections 2024-04-02T04:40:32.961 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:40:32.956+0000 7f517acb3700 1 -- 172.21.15.179:0/3965109804 wait complete. 2024-04-02T04:40:32.961 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-04-02T04:40:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:33 smithi179 ceph-mon[36603]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:33 smithi179 ceph-mon[36603]: from='client.? 172.21.15.179:0/3965109804' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:40:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:33 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:33.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:33 smithi070 ceph-mon[29212]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:33.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:33 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/3965109804' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:40:33.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:33 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:33.932 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-02T04:40:33.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-02T04:40:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:34.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:40:34.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:40:34.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:40:35.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:40:35.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:35.627 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:35 smithi070 ceph-mon[29212]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:35.627 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:35 smithi070 ceph-mon[29212]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:35.627 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:35 smithi070 ceph-mon[29212]: Reconfiguring mon.smithi070 (unknown last config time)... 2024-04-02T04:40:35.627 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:35 smithi070 ceph-mon[29212]: Reconfiguring daemon mon.smithi070 on smithi070 2024-04-02T04:40:35.627 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:35 smithi070 ceph-mon[29212]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:35.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:35 smithi179 ceph-mon[36603]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:35 smithi179 ceph-mon[36603]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:40:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:35 smithi179 ceph-mon[36603]: Reconfiguring mon.smithi070 (unknown last config time)... 2024-04-02T04:40:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:35 smithi179 ceph-mon[36603]: Reconfiguring daemon mon.smithi070 on smithi070 2024-04-02T04:40:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:35 smithi179 ceph-mon[36603]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.920+0000 7f4c3c483700 1 -- 172.21.15.70:0/2305077663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c34105a20 msgr2=0x7f4c34105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.920+0000 7f4c3c483700 1 --2- 172.21.15.70:0/2305077663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c34105a20 0x7f4c34105e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009910 tx=0x7f4c30009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.920+0000 7f4c3c483700 1 -- 172.21.15.70:0/2305077663 shutdown_connections 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.920+0000 7f4c3c483700 1 --2- 172.21.15.70:0/2305077663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c340ffb60 0x7f4c340fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.920+0000 7f4c3c483700 1 --2- 172.21.15.70:0/2305077663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c34105a20 0x7f4c34105e00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.920+0000 7f4c3c483700 1 -- 172.21.15.70:0/2305077663 >> 172.21.15.70:0/2305077663 conn(0x7f4c340754a0 msgr2=0x7f4c340758b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.920+0000 7f4c3c483700 1 -- 172.21.15.70:0/2305077663 shutdown_connections 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.920+0000 7f4c3c483700 1 -- 172.21.15.70:0/2305077663 wait complete. 2024-04-02T04:40:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3c483700 1 Processor -- start 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3c483700 1 -- start start 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3c483700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c340ffb60 0x7f4c341955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3c483700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c34105a20 0x7f4c34195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3c483700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c341961e0 con 0x7f4c340ffb60 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3c483700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34199f70 con 0x7f4c34105a20 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3a21f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c340ffb60 0x7f4c341955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3a21f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c340ffb60 0x7f4c341955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56640/0 (socket says 172.21.15.70:56640) 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c3a21f700 1 -- 172.21.15.70:0/3655140209 learned_addr learned my addr 172.21.15.70:0/3655140209 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.921+0000 7f4c39a1e700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c34105a20 0x7f4c34195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c3a21f700 1 -- 172.21.15.70:0/3655140209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c34105a20 msgr2=0x7f4c34195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c3a21f700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c34105a20 0x7f4c34195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c3a21f700 1 -- 172.21.15.70:0/3655140209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300095f0 con 0x7f4c340ffb60 2024-04-02T04:40:36.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c3a21f700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c340ffb60 0x7f4c341955c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4c30006010 tx=0x7f4c3000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c2b7fe700 1 -- 172.21.15.70:0/3655140209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c3001d070 con 0x7f4c340ffb60 2024-04-02T04:40:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3419a1f0 con 0x7f4c340ffb60 2024-04-02T04:40:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c2b7fe700 1 -- 172.21.15.70:0/3655140209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c3000f460 con 0x7f4c340ffb60 2024-04-02T04:40:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c2b7fe700 1 -- 172.21.15.70:0/3655140209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c30017630 con 0x7f4c340ffb60 2024-04-02T04:40:36.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.922+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3419a6e0 con 0x7f4c340ffb60 2024-04-02T04:40:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.923+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3404f2e0 con 0x7f4c340ffb60 2024-04-02T04:40:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.923+0000 7f4c2b7fe700 1 -- 172.21.15.70:0/3655140209 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f4c3000f5d0 con 0x7f4c340ffb60 2024-04-02T04:40:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.924+0000 7f4c2b7fe700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4c2006c460 0x7f4c2006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.924+0000 7f4c2b7fe700 1 -- 172.21.15.70:0/3655140209 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f4c3008bb10 con 0x7f4c340ffb60 2024-04-02T04:40:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.924+0000 7f4c39a1e700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4c2006c460 0x7f4c2006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.925+0000 7f4c39a1e700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4c2006c460 0x7f4c2006e920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c34196be0 tx=0x7f4c2400b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:36.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:36.929+0000 7f4c2b7fe700 1 -- 172.21.15.70:0/3655140209 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4c300573d0 con 0x7f4c340ffb60 2024-04-02T04:40:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.122+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4c3419aaa0 con 0x7f4c340ffb60 2024-04-02T04:40:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.122+0000 7f4c2b7fe700 1 -- 172.21.15.70:0/3655140209 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f4c3008b2d0 con 0x7f4c340ffb60 2024-04-02T04:40:37.126 INFO:teuthology.orchestra.run.smithi070.stdout:# minimal ceph.conf for a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:40:37.126 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-04-02T04:40:37.126 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:40:37.126 INFO:teuthology.orchestra.run.smithi070.stdout: mon_host = [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-02T04:40:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4c2006c460 msgr2=0x7f4c2006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4c2006c460 0x7f4c2006e920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c34196be0 tx=0x7f4c2400b410 comp rx=0 tx=0).stop 2024-04-02T04:40:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c340ffb60 msgr2=0x7f4c341955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c340ffb60 0x7f4c341955c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4c30006010 tx=0x7f4c3000bd70 comp rx=0 tx=0).stop 2024-04-02T04:40:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 shutdown_connections 2024-04-02T04:40:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4c2006c460 0x7f4c2006e920 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c340ffb60 0x7f4c341955c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 --2- 172.21.15.70:0/3655140209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c34105a20 0x7f4c34195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 >> 172.21.15.70:0/3655140209 conn(0x7f4c340754a0 msgr2=0x7f4c340fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 shutdown_connections 2024-04-02T04:40:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:37.124+0000 7f4c3c483700 1 -- 172.21.15.70:0/3655140209 wait complete. 2024-04-02T04:40:37.699 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:37.699 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:37.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:37.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: Reconfiguring mgr.smithi070.igidvr (unknown last config time)... 2024-04-02T04:40:37.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.igidvr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:40:37.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:40:37.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:37.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: Reconfiguring daemon mgr.smithi070.igidvr on smithi070 2024-04-02T04:40:37.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:37 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3655140209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:37.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:37.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: Reconfiguring mgr.smithi070.igidvr (unknown last config time)... 2024-04-02T04:40:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.igidvr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:40:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:40:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: Reconfiguring daemon mgr.smithi070.igidvr on smithi070 2024-04-02T04:40:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:37 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3655140209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:38.106 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-02T04:40:38.106 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:40:38.107 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T04:40:38.135 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:40:38.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:40:38.206 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:40:38.206 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T04:40:38.242 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:40:38.243 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:40:38.313 INFO:tasks.cephadm:Deploying OSDs... 2024-04-02T04:40:38.313 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:40:38.313 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T04:40:38.327 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T04:40:38.328 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-04-02T04:40:38.382 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T04:40:38.382 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:40:38.383 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-02T04:40:38.383 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:38.383 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:40:38.383 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-02 04:39:49.836457847 +0000 2024-04-02T04:40:38.383 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-02 04:34:16.371281591 +0000 2024-04-02T04:40:38.383 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-02 04:34:16.371281591 +0000 2024-04-02T04:40:38.383 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-02 04:34:16.371281591 +0000 2024-04-02T04:40:38.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T04:40:38.448 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-02T04:40:38.449 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-02T04:40:38.449 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.458e-05 s, 6.1 MB/s 2024-04-02T04:40:38.450 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T04:40:38.506 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-04-02T04:40:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T04:40:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:40:38.561 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-02T04:40:38.561 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:38.561 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:40:38.561 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-02 04:39:49.836457847 +0000 2024-04-02T04:40:38.562 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-02 04:34:16.694272344 +0000 2024-04-02T04:40:38.562 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-02 04:34:16.694272344 +0000 2024-04-02T04:40:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-02 04:34:16.694272344 +0000 2024-04-02T04:40:38.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T04:40:38.627 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-02T04:40:38.627 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-02T04:40:38.627 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.6091e-05 s, 5.9 MB/s 2024-04-02T04:40:38.629 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T04:40:38.684 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-04-02T04:40:38.739 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T04:40:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:40:38.740 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-02T04:40:38.740 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:38.740 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:40:38.740 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-02 04:39:49.836457847 +0000 2024-04-02T04:40:38.740 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-02 04:34:17.035262580 +0000 2024-04-02T04:40:38.740 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-02 04:34:17.035262580 +0000 2024-04-02T04:40:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-02 04:34:17.035262580 +0000 2024-04-02T04:40:38.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T04:40:38.805 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-02T04:40:38.805 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-02T04:40:38.805 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 9.2373e-05 s, 5.5 MB/s 2024-04-02T04:40:38.807 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T04:40:38.862 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-04-02T04:40:38.917 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T04:40:38.917 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:40:38.918 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-02T04:40:38.918 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:38.918 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:40:38.918 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-02 04:39:49.837457819 +0000 2024-04-02T04:40:38.918 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-02 04:34:17.364253160 +0000 2024-04-02T04:40:38.918 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-02 04:34:17.364253160 +0000 2024-04-02T04:40:38.918 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-02 04:34:17.364253160 +0000 2024-04-02T04:40:38.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T04:40:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-02T04:40:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-02T04:40:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 9.0698e-05 s, 5.6 MB/s 2024-04-02T04:40:38.988 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T04:40:39.044 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:40:39.044 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T04:40:39.062 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T04:40:39.063 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-04-02T04:40:39.120 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T04:40:39.120 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:40:39.120 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-02T04:40:39.120 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:39.121 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:40:39.121 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-02 04:40:25.489788138 +0000 2024-04-02T04:40:39.121 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-02 04:34:16.419953680 +0000 2024-04-02T04:40:39.121 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-02 04:34:16.419953680 +0000 2024-04-02T04:40:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-04-02 04:34:16.419953680 +0000 2024-04-02T04:40:39.121 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T04:40:39.190 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-02T04:40:39.191 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-02T04:40:39.191 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000286962 s, 1.8 MB/s 2024-04-02T04:40:39.192 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T04:40:39.250 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-04-02T04:40:39.307 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T04:40:39.308 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:40:39.308 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-04-02T04:40:39.308 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:39.308 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:40:39.308 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-02 04:40:25.489788138 +0000 2024-04-02T04:40:39.308 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-02 04:34:16.765945938 +0000 2024-04-02T04:40:39.308 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-02 04:34:16.765945938 +0000 2024-04-02T04:40:39.308 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-04-02 04:34:16.765945938 +0000 2024-04-02T04:40:39.308 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T04:40:39.376 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-02T04:40:39.376 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-02T04:40:39.376 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000207504 s, 2.5 MB/s 2024-04-02T04:40:39.377 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T04:40:39.434 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-04-02T04:40:39.491 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T04:40:39.491 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:40:39.491 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-02T04:40:39.491 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:39.491 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:40:39.491 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-02 04:40:25.490788116 +0000 2024-04-02T04:40:39.491 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-02 04:34:17.090938665 +0000 2024-04-02T04:40:39.491 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-02 04:34:17.090938665 +0000 2024-04-02T04:40:39.492 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-04-02 04:34:17.090938665 +0000 2024-04-02T04:40:39.492 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T04:40:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:39 smithi070 ceph-mon[29212]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:39 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:39 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:39 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:40:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:39 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:39.557 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-02T04:40:39.557 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-02T04:40:39.557 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000218166 s, 2.3 MB/s 2024-04-02T04:40:39.558 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T04:40:39.615 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-02 04:40:25.490788116 +0000 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-02 04:34:17.413931437 +0000 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-02 04:34:17.413931437 +0000 2024-04-02T04:40:39.672 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-04-02 04:34:17.413931437 +0000 2024-04-02T04:40:39.673 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T04:40:39.736 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:39 smithi179 ceph-mon[36603]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:39.736 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:39 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:39.736 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:39 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:39.736 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:39 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:40:39.736 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:39 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:39.738 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-02T04:40:39.739 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-02T04:40:39.739 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000172836 s, 3.0 MB/s 2024-04-02T04:40:39.740 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T04:40:39.796 INFO:tasks.cephadm:Deploying osd.0 on smithi070 with /dev/vg_nvme/lv_4... 2024-04-02T04:40:39.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T04:40:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:40:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:40 smithi070 ceph-mon[29212]: Reconfiguring ceph-exporter.smithi070 (monmap changed)... 2024-04-02T04:40:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:40 smithi070 ceph-mon[29212]: Reconfiguring daemon ceph-exporter.smithi070 on smithi070 2024-04-02T04:40:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:40:40.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:40.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:40 smithi179 ceph-mon[36603]: Reconfiguring ceph-exporter.smithi070 (monmap changed)... 2024-04-02T04:40:40.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:40 smithi179 ceph-mon[36603]: Reconfiguring daemon ceph-exporter.smithi070 on smithi070 2024-04-02T04:40:40.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:40.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:40.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:40:40.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:41.664 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:41 smithi070 ceph-mon[29212]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:41.664 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:41 smithi070 ceph-mon[29212]: Reconfiguring crash.smithi070 (monmap changed)... 2024-04-02T04:40:41.664 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:41 smithi070 ceph-mon[29212]: Reconfiguring daemon crash.smithi070 on smithi070 2024-04-02T04:40:41.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:41 smithi179 ceph-mon[36603]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:41.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:41 smithi179 ceph-mon[36603]: Reconfiguring crash.smithi070 (monmap changed)... 2024-04-02T04:40:41.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:41 smithi179 ceph-mon[36603]: Reconfiguring daemon crash.smithi070 on smithi070 2024-04-02T04:40:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T04:40:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T04:40:42.484 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-04-02T04:40:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-04-02T04:40:42.484 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00864835 s, 1.2 GB/s 2024-04-02T04:40:42.484 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-04-02T04:40:42.991 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi070:vg_nvme/lv_4 2024-04-02T04:40:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:40:43.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:43 smithi070 ceph-mon[29212]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:43.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:43.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:43 smithi179 ceph-mon[36603]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:43.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:44 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:44 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:44 smithi070 ceph-mon[29212]: Reconfiguring alertmanager.smithi070 (dependencies changed)... 2024-04-02T04:40:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:44 smithi070 ceph-mon[29212]: Reconfiguring daemon alertmanager.smithi070 on smithi070 2024-04-02T04:40:44.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:44 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:44.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:44 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:44.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:44 smithi179 ceph-mon[36603]: Reconfiguring alertmanager.smithi070 (dependencies changed)... 2024-04-02T04:40:44.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:44 smithi179 ceph-mon[36603]: Reconfiguring daemon alertmanager.smithi070 on smithi070 2024-04-02T04:40:45.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.570+0000 7f1664bb9700 1 -- 172.21.15.70:0/3033002551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 msgr2=0x7f16600fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:45.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.570+0000 7f1664bb9700 1 --2- 172.21.15.70:0/3033002551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 0x7f16600fffc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f165800b0a0 tx=0x7f165800b3b0 comp rx=0 tx=0).stop 2024-04-02T04:40:45.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.570+0000 7f1664bb9700 1 -- 172.21.15.70:0/3033002551 shutdown_connections 2024-04-02T04:40:45.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.570+0000 7f1664bb9700 1 --2- 172.21.15.70:0/3033002551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1660100da0 0x7f1660101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:45.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.570+0000 7f1664bb9700 1 --2- 172.21.15.70:0/3033002551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 0x7f16600fffc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:45.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.570+0000 7f1664bb9700 1 -- 172.21.15.70:0/3033002551 >> 172.21.15.70:0/3033002551 conn(0x7f16600fb120 msgr2=0x7f16600fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:45.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.570+0000 7f1664bb9700 1 -- 172.21.15.70:0/3033002551 shutdown_connections 2024-04-02T04:40:45.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.570+0000 7f1664bb9700 1 -- 172.21.15.70:0/3033002551 wait complete. 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f1664bb9700 1 Processor -- start 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f1664bb9700 1 -- start start 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f1664bb9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 0x7f1660103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f1664bb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1660100da0 0x7f1660102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f1664bb9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166019a1f0 con 0x7f16600ffba0 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f1664bb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166019a330 con 0x7f1660100da0 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f165e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 0x7f1660103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f165e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 0x7f1660103b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56644/0 (socket says 172.21.15.70:56644) 2024-04-02T04:40:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.571+0000 7f165e59c700 1 -- 172.21.15.70:0/1025370706 learned_addr learned my addr 172.21.15.70:0/1025370706 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f165e59c700 1 -- 172.21.15.70:0/1025370706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1660100da0 msgr2=0x7f1660102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f165e59c700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1660100da0 0x7f1660102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f165e59c700 1 -- 172.21.15.70:0/1025370706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1658009d00 con 0x7f16600ffba0 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f165e59c700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 0x7f1660103b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f16580077b0 tx=0x7f1658003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f16577fe700 1 -- 172.21.15.70:0/1025370706 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165800e040 con 0x7f16600ffba0 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16601026c0 con 0x7f16600ffba0 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f16577fe700 1 -- 172.21.15.70:0/1025370706 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f165800ce60 con 0x7f16600ffba0 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f16577fe700 1 -- 172.21.15.70:0/1025370706 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165801db10 con 0x7f16600ffba0 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.572+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1660102b80 con 0x7f16600ffba0 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.573+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1660102e60 con 0x7f16600ffba0 2024-04-02T04:40:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.573+0000 7f16577fe700 1 -- 172.21.15.70:0/1025370706 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f165801dc90 con 0x7f16600ffba0 2024-04-02T04:40:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.574+0000 7f16577fe700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f164c06c460 0x7f164c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.574+0000 7f165dd9b700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f164c06c460 0x7f164c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.574+0000 7f16577fe700 1 -- 172.21.15.70:0/1025370706 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f165808c1a0 con 0x7f16600ffba0 2024-04-02T04:40:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.574+0000 7f165dd9b700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f164c06c460 0x7f164c06e920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16480097b0 tx=0x7f1648006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:45.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.578+0000 7f16577fe700 1 -- 172.21.15.70:0/1025370706 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f165805b4f0 con 0x7f16600ffba0 2024-04-02T04:40:45.768 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:45 smithi070 ceph-mon[29212]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:45.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:40:45.781+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f16601a2df0 con 0x7f164c06c460 2024-04-02T04:40:45.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:45 smithi179 ceph-mon[36603]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:47.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:46 smithi070 ceph-mon[29212]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:47.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:46 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:40:47.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:46 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:40:47.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:46 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:47.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:46 smithi070 ceph-mon[29212]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:47.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:46 smithi179 ceph-mon[36603]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:47.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:46 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:40:47.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:46 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:40:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:46 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:46 smithi179 ceph-mon[36603]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:49.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:49 smithi070 ceph-mon[29212]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:49.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:49 smithi179 ceph-mon[36603]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:51.269 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:51 smithi070 ceph-mon[29212]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:51.353 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:51 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:51.353 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:51 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:51.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:51 smithi179 ceph-mon[36603]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:51.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:51 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:51.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:51 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:40:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:52 smithi070 ceph-mon[29212]: Reconfiguring grafana.smithi070 (dependencies changed)... 2024-04-02T04:40:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:52 smithi070 ceph-mon[29212]: Reconfiguring daemon grafana.smithi070 on smithi070 2024-04-02T04:40:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:52 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/60313158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b5e5510-c4db-4073-b470-c777b557d77e"}]: dispatch 2024-04-02T04:40:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:52 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/60313158' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b5e5510-c4db-4073-b470-c777b557d77e"}]': finished 2024-04-02T04:40:52.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:52 smithi070 ceph-mon[29212]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T04:40:52.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:40:52.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:52 smithi179 ceph-mon[36603]: Reconfiguring grafana.smithi070 (dependencies changed)... 2024-04-02T04:40:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:52 smithi179 ceph-mon[36603]: Reconfiguring daemon grafana.smithi070 on smithi070 2024-04-02T04:40:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:52 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/60313158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b5e5510-c4db-4073-b470-c777b557d77e"}]: dispatch 2024-04-02T04:40:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:52 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/60313158' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b5e5510-c4db-4073-b470-c777b557d77e"}]': finished 2024-04-02T04:40:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:52 smithi179 ceph-mon[36603]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T04:40:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:40:53.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:53 smithi070 ceph-mon[29212]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:53.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:53 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/38913887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:40:53.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:53 smithi179 ceph-mon[36603]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:53.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:53 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/38913887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:40:55.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:55 smithi070 ceph-mon[29212]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:55.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:55 smithi179 ceph-mon[36603]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:56.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:56 smithi070 ceph-mon[29212]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:57.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:56 smithi179 ceph-mon[36603]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:58.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:40:59 smithi070 ceph-mon[29212]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:40:59.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:40:59 smithi179 ceph-mon[36603]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:01.354 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:01 smithi070 ceph-mon[29212]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:01.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:01 smithi179 ceph-mon[36603]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:02.490 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:02 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:02.491 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:02 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:02.491 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:02 smithi070 ceph-mon[29212]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-04-02T04:41:02.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:02 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:02.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:02 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:02.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:02 smithi179 ceph-mon[36603]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-04-02T04:41:03.441 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:03 smithi070 ceph-mon[29212]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-04-02T04:41:03.441 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:03 smithi070 ceph-mon[29212]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:03.441 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:41:03.441 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:03.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:03 smithi179 ceph-mon[36603]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-04-02T04:41:03.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:03 smithi179 ceph-mon[36603]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:03.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:03 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:41:03.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:03 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:04.487 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:04 smithi070 ceph-mon[29212]: Deploying daemon osd.0 on smithi070 2024-04-02T04:41:04.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:04 smithi179 ceph-mon[36603]: Deploying daemon osd.0 on smithi070 2024-04-02T04:41:05.475 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:05 smithi070 ceph-mon[29212]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:05.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:05 smithi179 ceph-mon[36603]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:07.017 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:06 smithi070 ceph-mon[29212]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:07.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:06 smithi179 ceph-mon[36603]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:08.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:08 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:08.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:08 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:08.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:08 smithi179 ceph-mon[36603]: Reconfiguring ceph-exporter.smithi179 (monmap changed)... 2024-04-02T04:41:08.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:08 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:41:08.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:08 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:08.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:08 smithi179 ceph-mon[36603]: Reconfiguring daemon ceph-exporter.smithi179 on smithi179 2024-04-02T04:41:08.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:08 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:08.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:08 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:08.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:08 smithi070 ceph-mon[29212]: Reconfiguring ceph-exporter.smithi179 (monmap changed)... 2024-04-02T04:41:08.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:08 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:41:08.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:08 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:08.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:08 smithi070 ceph-mon[29212]: Reconfiguring daemon ceph-exporter.smithi179 on smithi179 2024-04-02T04:41:09.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:09 smithi179 ceph-mon[36603]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi179", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:41:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:09.753 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:09 smithi070 ceph-mon[29212]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:09.753 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:09.753 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:09.753 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi179", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:41:09.753 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:10.721 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:10 smithi179 ceph-mon[36603]: Reconfiguring crash.smithi179 (monmap changed)... 2024-04-02T04:41:10.721 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:10 smithi179 ceph-mon[36603]: Reconfiguring daemon crash.smithi179 on smithi179 2024-04-02T04:41:10.722 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:10.722 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:10.722 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:41:10.722 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:41:10.722 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:10 smithi070 ceph-mon[29212]: Reconfiguring crash.smithi179 (monmap changed)... 2024-04-02T04:41:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:10 smithi070 ceph-mon[29212]: Reconfiguring daemon crash.smithi179 on smithi179 2024-04-02T04:41:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:41:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:41:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:11.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:11 smithi070 ceph-mon[29212]: Reconfiguring mgr.smithi179.labmvs (monmap changed)... 2024-04-02T04:41:11.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:11 smithi070 ceph-mon[29212]: Reconfiguring daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:41:11.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:11 smithi070 ceph-mon[29212]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:11.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:11 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:11.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:11 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:11.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:11 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:41:11.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:11 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:41:11.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:11 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:11 smithi179 ceph-mon[36603]: Reconfiguring mgr.smithi179.labmvs (monmap changed)... 2024-04-02T04:41:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:11 smithi179 ceph-mon[36603]: Reconfiguring daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:41:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:11 smithi179 ceph-mon[36603]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:11 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:11 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:11 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:41:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:11 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:41:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:11 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:12.792 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:12 smithi179 ceph-mon[36603]: Reconfiguring mon.smithi179 (monmap changed)... 2024-04-02T04:41:12.792 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:12 smithi179 ceph-mon[36603]: Reconfiguring daemon mon.smithi179 on smithi179 2024-04-02T04:41:12.792 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:12 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:12.792 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:12 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:12.797 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:12 smithi070 ceph-mon[29212]: Reconfiguring mon.smithi179 (monmap changed)... 2024-04-02T04:41:12.797 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:12 smithi070 ceph-mon[29212]: Reconfiguring daemon mon.smithi179 on smithi179 2024-04-02T04:41:12.797 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:12 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:12.797 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:12 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi070.front.sepia.ceph.com:3000"}]: dispatch 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:41:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi070.front.sepia.ceph.com:9095"}]: dispatch 2024-04-02T04:41:13.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:13.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:13.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:13.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi070.front.sepia.ceph.com:3000"}]: dispatch 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi070.front.sepia.ceph.com:9095"}]: dispatch 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:14.693 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:14 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:41:14.693 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:14 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-04-02T04:41:14.693 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:14 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:41:14.693 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:14 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi070.front.sepia.ceph.com:3000"}]: dispatch 2024-04-02T04:41:14.693 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:14 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:41:14.693 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:14 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi070.front.sepia.ceph.com:9095"}]: dispatch 2024-04-02T04:41:14.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:14 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:41:14.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:14 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-04-02T04:41:14.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:14 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:41:14.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:14 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi070.front.sepia.ceph.com:3000"}]: dispatch 2024-04-02T04:41:14.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:14 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:41:14.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:14 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi070.front.sepia.ceph.com:9095"}]: dispatch 2024-04-02T04:41:15.317 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 0 on host 'smithi070' 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.310+0000 7f16577fe700 1 -- 172.21.15.70:0/1025370706 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f16601a2df0 con 0x7f164c06c460 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.313+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f164c06c460 msgr2=0x7f164c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.313+0000 7f1664bb9700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f164c06c460 0x7f164c06e920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16480097b0 tx=0x7f1648006d20 comp rx=0 tx=0).stop 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.313+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 msgr2=0x7f1660103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.313+0000 7f1664bb9700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 0x7f1660103b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f16580077b0 tx=0x7f1658003ce0 comp rx=0 tx=0).stop 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.314+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 shutdown_connections 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.314+0000 7f1664bb9700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f164c06c460 0x7f164c06e920 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16480097b0 tx=0x7f1648006d20 comp rx=0 tx=0).stop 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.314+0000 7f1664bb9700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16600ffba0 0x7f1660103b30 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f16580077b0 tx=0x7f1658003ce0 comp rx=0 tx=0).stop 2024-04-02T04:41:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.314+0000 7f1664bb9700 1 --2- 172.21.15.70:0/1025370706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1660100da0 0x7f1660102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:15.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.314+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 >> 172.21.15.70:0/1025370706 conn(0x7f16600fb120 msgr2=0x7f1660068e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:15.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.314+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 shutdown_connections 2024-04-02T04:41:15.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:15.314+0000 7f1664bb9700 1 -- 172.21.15.70:0/1025370706 wait complete. 2024-04-02T04:41:15.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:15 smithi070 ceph-mon[29212]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:15.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:15.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:15.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:15.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:15.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:15 smithi179 ceph-mon[36603]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:15.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:15.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:15.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:15.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:15.964 DEBUG:teuthology.orchestra.run.smithi070:osd.0> sudo journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.0.service 2024-04-02T04:41:15.967 INFO:tasks.cephadm:Deploying osd.1 on smithi070 with /dev/vg_nvme/lv_3... 2024-04-02T04:41:15.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T04:41:16.231 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:41:16 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[47987]: 2024-04-02T04:41:16.096+0000 7fb5027f5640 -1 osd.0 0 log_to_monitors true 2024-04-02T04:41:16.731 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:41:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:16 smithi070 ceph-mon[29212]: from='osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T04:41:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:16 smithi070 ceph-mon[29212]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:17.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:16 smithi179 ceph-mon[36603]: from='osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T04:41:17.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:16 smithi179 ceph-mon[36603]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:17 smithi070 ceph-mon[29212]: from='osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T04:41:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:17 smithi070 ceph-mon[29212]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T04:41:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:17 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:41:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:17 smithi070 ceph-mon[29212]: from='osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:41:18.030 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:41:17 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[47987]: 2024-04-02T04:41:17.742+0000 7fb4f8e5b700 -1 osd.0 0 waiting for initial osdmap 2024-04-02T04:41:18.031 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:41:17 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[47987]: 2024-04-02T04:41:17.745+0000 7fb4f1c4a700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:41:18.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:17 smithi179 ceph-mon[36603]: from='osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T04:41:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:17 smithi179 ceph-mon[36603]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T04:41:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:17 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:41:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:17 smithi179 ceph-mon[36603]: from='osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:41:19.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:18 smithi070 ceph-mon[29212]: from='osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-04-02T04:41:19.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:18 smithi070 ceph-mon[29212]: osdmap e8: 1 total, 0 up, 1 in 2024-04-02T04:41:19.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:41:19.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:41:19.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:18 smithi070 ceph-mon[29212]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:19.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:18 smithi179 ceph-mon[36603]: from='osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-04-02T04:41:19.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:18 smithi179 ceph-mon[36603]: osdmap e8: 1 total, 0 up, 1 in 2024-04-02T04:41:19.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:18 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:41:19.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:18 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:41:19.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:18 smithi179 ceph-mon[36603]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:41:19.750 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T04:41:19.750 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T04:41:19.750 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-04-02T04:41:19.750 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-04-02T04:41:19.751 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0106275 s, 987 MB/s 2024-04-02T04:41:19.751 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-04-02T04:41:20.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:19 smithi070 ceph-mon[29212]: purged_snaps scrub starts 2024-04-02T04:41:20.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:19 smithi070 ceph-mon[29212]: purged_snaps scrub ok 2024-04-02T04:41:20.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:19 smithi070 ceph-mon[29212]: osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] boot 2024-04-02T04:41:20.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:19 smithi070 ceph-mon[29212]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T04:41:20.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:41:20.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:19 smithi179 ceph-mon[36603]: purged_snaps scrub starts 2024-04-02T04:41:20.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:19 smithi179 ceph-mon[36603]: purged_snaps scrub ok 2024-04-02T04:41:20.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:19 smithi179 ceph-mon[36603]: osd.0 [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] boot 2024-04-02T04:41:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:19 smithi179 ceph-mon[36603]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T04:41:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:41:20.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi070:vg_nvme/lv_3 2024-04-02T04:41:20.822 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:41:21.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:20 smithi070 ceph-mon[29212]: osdmap e10: 1 total, 1 up, 1 in 2024-04-02T04:41:21.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:20 smithi070 ceph-mon[29212]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:21.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:20 smithi179 ceph-mon[36603]: osdmap e10: 1 total, 1 up, 1 in 2024-04-02T04:41:21.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:20 smithi179 ceph-mon[36603]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:21.934 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:21 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:21.935 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:21 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:22.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:21 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:22.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:21 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.447+0000 7f8432b36700 1 -- 172.21.15.70:0/2651748820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 msgr2=0x7f842c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.447+0000 7f8432b36700 1 --2- 172.21.15.70:0/2651748820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 0x7f842c100030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8420009910 tx=0x7f8420009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.447+0000 7f8432b36700 1 -- 172.21.15.70:0/2651748820 shutdown_connections 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.447+0000 7f8432b36700 1 --2- 172.21.15.70:0/2651748820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f842c100e10 0x7f842c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 --2- 172.21.15.70:0/2651748820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 0x7f842c100030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 -- 172.21.15.70:0/2651748820 >> 172.21.15.70:0/2651748820 conn(0x7f842c0fb190 msgr2=0x7f842c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 -- 172.21.15.70:0/2651748820 shutdown_connections 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 -- 172.21.15.70:0/2651748820 wait complete. 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 Processor -- start 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 -- start start 2024-04-02T04:41:22.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 0x7f842c0690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f842c100e10 0x7f842c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c1970a0 con 0x7f842c0ffc10 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f8432b36700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c19db70 con 0x7f842c100e10 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.448+0000 7f84308d2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 0x7f842c0690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f84308d2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 0x7f842c0690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42736/0 (socket says 172.21.15.70:42736) 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f84308d2700 1 -- 172.21.15.70:0/2062327226 learned_addr learned my addr 172.21.15.70:0/2062327226 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f842bfff700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f842c100e10 0x7f842c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f84308d2700 1 -- 172.21.15.70:0/2062327226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f842c100e10 msgr2=0x7f842c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f84308d2700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f842c100e10 0x7f842c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f84308d2700 1 -- 172.21.15.70:0/2062327226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200095f0 con 0x7f842c0ffc10 2024-04-02T04:41:22.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f84308d2700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 0x7f842c0690e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f842000bdd0 tx=0x7f842000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:22.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f8429ffb700 1 -- 172.21.15.70:0/2062327226 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842001d070 con 0x7f842c0ffc10 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f8429ffb700 1 -- 172.21.15.70:0/2062327226 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f842000f460 con 0x7f842c0ffc10 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f8429ffb700 1 -- 172.21.15.70:0/2062327226 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8420021620 con 0x7f842c0ffc10 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.449+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c19ddc0 con 0x7f842c0ffc10 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.450+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c19e2b0 con 0x7f842c0ffc10 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.451+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c108700 con 0x7f842c0ffc10 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.451+0000 7f8429ffb700 1 -- 172.21.15.70:0/2062327226 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f8420004af0 con 0x7f842c0ffc10 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.451+0000 7f8429ffb700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f841c06c460 0x7f841c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.451+0000 7f8429ffb700 1 -- 172.21.15.70:0/2062327226 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f842008bcb0 con 0x7f842c0ffc10 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.452+0000 7f842bfff700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f841c06c460 0x7f841c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:22.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.452+0000 7f842bfff700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f841c06c460 0x7f841c06e920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f842c069660 tx=0x7f841800a300 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:22.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.456+0000 7f8429ffb700 1 -- 172.21.15.70:0/2062327226 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f842005aec0 con 0x7f842c0ffc10 2024-04-02T04:41:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:22.667+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f842c105760 con 0x7f841c06c460 2024-04-02T04:41:22.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:22 smithi070 ceph-mon[29212]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:22.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:41:22.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:41:22.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:23.151 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:22 smithi179 ceph-mon[36603]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:23.151 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:41:23.151 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:41:23.152 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:23.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:23 smithi070 ceph-mon[29212]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:24.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:23 smithi179 ceph-mon[36603]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:25.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:24 smithi070 ceph-mon[29212]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:25.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:24 smithi179 ceph-mon[36603]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: Detected new or changed devices on smithi070 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: Adjusting osd_memory_target on smithi070 to 11840M 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:27.459 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:41:27.460 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:27.460 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:27 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:27.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:27.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: Detected new or changed devices on smithi070 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: Adjusting osd_memory_target on smithi070 to 11840M 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:27 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:28.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:28 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3369086600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b751ceaf-b734-4280-ba22-82683b9776ae"}]: dispatch 2024-04-02T04:41:28.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:28 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3369086600' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b751ceaf-b734-4280-ba22-82683b9776ae"}]': finished 2024-04-02T04:41:28.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:28 smithi070 ceph-mon[29212]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T04:41:28.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:28.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:28 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/4117307821' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:41:28.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:28.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:28.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:28 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3369086600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b751ceaf-b734-4280-ba22-82683b9776ae"}]: dispatch 2024-04-02T04:41:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:28 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3369086600' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b751ceaf-b734-4280-ba22-82683b9776ae"}]': finished 2024-04-02T04:41:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:28 smithi179 ceph-mon[36603]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T04:41:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:28 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/4117307821' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:41:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:29.668 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:29 smithi070 ceph-mon[29212]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:29.669 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:29 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:29.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:29 smithi179 ceph-mon[36603]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:29.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:29 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:31.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:31 smithi070 ceph-mon[29212]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:31.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:31.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:31.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:31.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:41:31.494 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:31.624 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:31 smithi179 ceph-mon[36603]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:31.624 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:31.624 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:31.624 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:31.624 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:41:31.624 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:33.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:33 smithi070 ceph-mon[29212]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:33 smithi179 ceph-mon[36603]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:35.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:35 smithi070 ceph-mon[29212]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:35.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:41:35.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:35.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:35 smithi179 ceph-mon[36603]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:41:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:36.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:36 smithi070 ceph-mon[29212]: Deploying daemon osd.1 on smithi070 2024-04-02T04:41:36.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:36 smithi179 ceph-mon[36603]: Deploying daemon osd.1 on smithi070 2024-04-02T04:41:37.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:37 smithi070 ceph-mon[29212]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:37.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:37 smithi179 ceph-mon[36603]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:39.464 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:39 smithi070 ceph-mon[29212]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:39.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:39 smithi179 ceph-mon[36603]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:40.977 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:40.977 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:40.977 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:40.977 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:40 smithi179 ceph-mon[36603]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:41.026 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:41.026 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:41.026 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:41.026 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:40 smithi070 ceph-mon[29212]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:42 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:42 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:42.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:42 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:42.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:42 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:43.386 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 1 on host 'smithi070' 2024-04-02T04:41:43.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.383+0000 7f8429ffb700 1 -- 172.21.15.70:0/2062327226 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f842c105760 con 0x7f841c06c460 2024-04-02T04:41:43.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.385+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f841c06c460 msgr2=0x7f841c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.385+0000 7f8432b36700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f841c06c460 0x7f841c06e920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f842c069660 tx=0x7f841800a300 comp rx=0 tx=0).stop 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 msgr2=0x7f842c0690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 0x7f842c0690e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f842000bdd0 tx=0x7f842000be00 comp rx=0 tx=0).stop 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 shutdown_connections 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f841c06c460 0x7f841c06e920 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f842c0ffc10 0x7f842c0690e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 --2- 172.21.15.70:0/2062327226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f842c100e10 0x7f842c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 >> 172.21.15.70:0/2062327226 conn(0x7f842c0fb190 msgr2=0x7f842c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 shutdown_connections 2024-04-02T04:41:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:43.386+0000 7f8432b36700 1 -- 172.21.15.70:0/2062327226 wait complete. 2024-04-02T04:41:43.579 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:43 smithi070 ceph-mon[29212]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:43.579 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:43.579 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:43.579 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:43.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:43 smithi179 ceph-mon[36603]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:44.308 DEBUG:teuthology.orchestra.run.smithi070:osd.1> sudo journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.1.service 2024-04-02T04:41:44.311 INFO:tasks.cephadm:Deploying osd.2 on smithi070 with /dev/vg_nvme/lv_2... 2024-04-02T04:41:44.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T04:41:44.581 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:44 smithi070 ceph-mon[29212]: from='osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T04:41:44.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:44 smithi179 ceph-mon[36603]: from='osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T04:41:45.034 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:41:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:45 smithi070 ceph-mon[29212]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:45 smithi070 ceph-mon[29212]: from='osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T04:41:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:45 smithi070 ceph-mon[29212]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T04:41:45.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:45 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:45.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:45 smithi070 ceph-mon[29212]: from='osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:41:45.780 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:41:45 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[55485]: 2024-04-02T04:41:45.470+0000 7f0eeb82f700 -1 osd.1 0 waiting for initial osdmap 2024-04-02T04:41:45.780 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:41:45 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[55485]: 2024-04-02T04:41:45.475+0000 7f0ee5e21700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:41:45.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:45 smithi179 ceph-mon[36603]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:45.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:45 smithi179 ceph-mon[36603]: from='osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T04:41:45.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:45 smithi179 ceph-mon[36603]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T04:41:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:45 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:45 smithi179 ceph-mon[36603]: from='osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:41:46.690 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:46 smithi070 ceph-mon[29212]: from='osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-04-02T04:41:46.690 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:46 smithi070 ceph-mon[29212]: osdmap e13: 2 total, 1 up, 2 in 2024-04-02T04:41:46.691 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:46 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:46.691 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:46 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:46.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:46 smithi179 ceph-mon[36603]: from='osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-04-02T04:41:46.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:46 smithi179 ceph-mon[36603]: osdmap e13: 2 total, 1 up, 2 in 2024-04-02T04:41:46.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:46 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:46.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:46 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:47.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:47 smithi070 ceph-mon[29212]: purged_snaps scrub starts 2024-04-02T04:41:47.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:47 smithi070 ceph-mon[29212]: purged_snaps scrub ok 2024-04-02T04:41:47.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:47 smithi070 ceph-mon[29212]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:47.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:47 smithi070 ceph-mon[29212]: osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] boot 2024-04-02T04:41:47.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:47 smithi070 ceph-mon[29212]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T04:41:47.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:47 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:47.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:47 smithi179 ceph-mon[36603]: purged_snaps scrub starts 2024-04-02T04:41:47.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:47 smithi179 ceph-mon[36603]: purged_snaps scrub ok 2024-04-02T04:41:47.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:47 smithi179 ceph-mon[36603]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:41:47.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:47 smithi179 ceph-mon[36603]: osd.1 [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] boot 2024-04-02T04:41:47.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:47 smithi179 ceph-mon[36603]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T04:41:47.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:47 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:41:48.060 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T04:41:48.060 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T04:41:48.060 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-04-02T04:41:48.061 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-04-02T04:41:48.061 INFO:teuthology.orchestra.run.smithi070.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00870187 s, 1.2 GB/s 2024-04-02T04:41:48.061 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-04-02T04:41:48.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:48 smithi070 ceph-mon[29212]: osdmap e15: 2 total, 2 up, 2 in 2024-04-02T04:41:48.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:48 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:48.668 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:48 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:48.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi070:vg_nvme/lv_2 2024-04-02T04:41:48.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:48 smithi179 ceph-mon[36603]: osdmap e15: 2 total, 2 up, 2 in 2024-04-02T04:41:48.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:48 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:48.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:48 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:49.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:49 smithi070 ceph-mon[29212]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:49.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:49 smithi179 ceph-mon[36603]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:49.968 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:41:51.629 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:51 smithi070 ceph-mon[29212]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:51.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:51 smithi179 ceph-mon[36603]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:52.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.547+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/464164254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac101ea0 msgr2=0x7f7aac102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.547+0000 7f7ab2f8e700 1 --2- 172.21.15.70:0/464164254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac101ea0 0x7f7aac102300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009960 tx=0x7f7a98009c70 comp rx=0 tx=0).stop 2024-04-02T04:41:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.547+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/464164254 shutdown_connections 2024-04-02T04:41:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.547+0000 7f7ab2f8e700 1 --2- 172.21.15.70:0/464164254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac101ea0 0x7f7aac102300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.547+0000 7f7ab2f8e700 1 --2- 172.21.15.70:0/464164254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac100ca0 0x7f7aac1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.547+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/464164254 >> 172.21.15.70:0/464164254 conn(0x7f7aac0fc260 msgr2=0x7f7aac0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.547+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/464164254 shutdown_connections 2024-04-02T04:41:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.547+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/464164254 wait complete. 2024-04-02T04:41:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab2f8e700 1 Processor -- start 2024-04-02T04:41:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab2f8e700 1 -- start start 2024-04-02T04:41:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab2f8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac100ca0 0x7f7aac1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab2f8e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac101ea0 0x7f7aac1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab2f8e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac197000 con 0x7f7aac100ca0 2024-04-02T04:41:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab2f8e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac197140 con 0x7f7aac101ea0 2024-04-02T04:41:52.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab0d2a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac100ca0 0x7f7aac1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:52.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab0d2a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac100ca0 0x7f7aac1964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39116/0 (socket says 172.21.15.70:39116) 2024-04-02T04:41:52.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab0d2a700 1 -- 172.21.15.70:0/2388983209 learned_addr learned my addr 172.21.15.70:0/2388983209 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:41:52.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7aabfff700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac101ea0 0x7f7aac1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:52.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab0d2a700 1 -- 172.21.15.70:0/2388983209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac101ea0 msgr2=0x7f7aac1969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:52.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.548+0000 7f7ab0d2a700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac101ea0 0x7f7aac1969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:52.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.549+0000 7f7ab0d2a700 1 -- 172.21.15.70:0/2388983209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980095f0 con 0x7f7aac100ca0 2024-04-02T04:41:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.549+0000 7f7ab0d2a700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac100ca0 0x7f7aac1964a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000d900 tx=0x7f7aa000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.549+0000 7f7aa9ffb700 1 -- 172.21.15.70:0/2388983209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aa00041f0 con 0x7f7aac100ca0 2024-04-02T04:41:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.549+0000 7f7aa9ffb700 1 -- 172.21.15.70:0/2388983209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7aa0004350 con 0x7f7aac100ca0 2024-04-02T04:41:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.549+0000 7f7aa9ffb700 1 -- 172.21.15.70:0/2388983209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aa0003da0 con 0x7f7aac100ca0 2024-04-02T04:41:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.549+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac19bbf0 con 0x7f7aac100ca0 2024-04-02T04:41:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.549+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac19c140 con 0x7f7aac100ca0 2024-04-02T04:41:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.550+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac109800 con 0x7f7aac100ca0 2024-04-02T04:41:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.551+0000 7f7aa9ffb700 1 -- 172.21.15.70:0/2388983209 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f7aa0009730 con 0x7f7aac100ca0 2024-04-02T04:41:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.551+0000 7f7aa9ffb700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f7a9c06c460 0x7f7a9c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.551+0000 7f7aa9ffb700 1 -- 172.21.15.70:0/2388983209 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f7aa008abb0 con 0x7f7aac100ca0 2024-04-02T04:41:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.551+0000 7f7aabfff700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f7a9c06c460 0x7f7a9c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.552+0000 7f7aabfff700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f7a9c06c460 0x7f7a9c06e920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b5c0 tx=0x7f7a98005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.556+0000 7f7aa9ffb700 1 -- 172.21.15.70:0/2388983209 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7aa0059b70 con 0x7f7aac100ca0 2024-04-02T04:41:52.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:41:52.778+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7aac1067f0 con 0x7f7a9c06c460 2024-04-02T04:41:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: Detected new or changed devices on smithi070 2024-04-02T04:41:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: Adjusting osd_memory_target on smithi070 to 5920M 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:52 smithi179 ceph-mon[36603]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:52.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: Detected new or changed devices on smithi070 2024-04-02T04:41:52.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:52.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:52.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:41:52.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:41:52.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: Adjusting osd_memory_target on smithi070 to 5920M 2024-04-02T04:41:52.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:52.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:52.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:41:52.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:52.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:52.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:52 smithi070 ceph-mon[29212]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:53.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:53 smithi070 ceph-mon[29212]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:53.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:41:53.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:41:53.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:53.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:53.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:54.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:53 smithi179 ceph-mon[36603]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:54.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:41:54.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:41:54.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:54.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:54.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:54.981 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:54 smithi070 ceph-mon[29212]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:55.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:54 smithi179 ceph-mon[36603]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:57.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:56 smithi070 ceph-mon[29212]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:57.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:56 smithi179 ceph-mon[36603]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:57.748 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:57 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/92526427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32080ab6-7809-4523-aab7-ca7a6d263ba0"}]: dispatch 2024-04-02T04:41:57.748 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:57 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/92526427' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32080ab6-7809-4523-aab7-ca7a6d263ba0"}]': finished 2024-04-02T04:41:57.749 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:57 smithi070 ceph-mon[29212]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T04:41:57.749 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:57 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:41:57.749 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:57 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/388745447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:41:58.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:57 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/92526427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32080ab6-7809-4523-aab7-ca7a6d263ba0"}]: dispatch 2024-04-02T04:41:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:57 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/92526427' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32080ab6-7809-4523-aab7-ca7a6d263ba0"}]': finished 2024-04-02T04:41:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:57 smithi179 ceph-mon[36603]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T04:41:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:57 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:41:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:57 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/388745447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:41:59.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:59.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:59.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:59.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:41:59.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:59.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:58 smithi070 ceph-mon[29212]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:59.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:41:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:59.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:59.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:59.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:59.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:41:59.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:41:59.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:58 smithi179 ceph-mon[36603]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:41:59.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:41:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:01.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:01 smithi070 ceph-mon[29212]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:01.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:01 smithi179 ceph-mon[36603]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:03.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:03 smithi070 ceph-mon[29212]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:03.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:03 smithi179 ceph-mon[36603]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:05.338 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:05 smithi070 ceph-mon[29212]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:05.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:42:05.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:05.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:05 smithi070 ceph-mon[29212]: Deploying daemon osd.2 on smithi070 2024-04-02T04:42:05.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:05 smithi179 ceph-mon[36603]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:42:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:05 smithi179 ceph-mon[36603]: Deploying daemon osd.2 on smithi070 2024-04-02T04:42:06.911 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:06 smithi070 ceph-mon[29212]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:07.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:06 smithi179 ceph-mon[36603]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:09.504 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:09 smithi070 ceph-mon[29212]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:09.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:09 smithi179 ceph-mon[36603]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:10 smithi070 ceph-mon[29212]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:11.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:11.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:11.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:11.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:10 smithi179 ceph-mon[36603]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:13.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:13 smithi070 ceph-mon[29212]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:13.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:13.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:13 smithi179 ceph-mon[36603]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:13.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.686+0000 7f7aa9ffb700 1 -- 172.21.15.70:0/2388983209 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7aac1067f0 con 0x7f7a9c06c460 2024-04-02T04:42:13.690 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 2 on host 'smithi070' 2024-04-02T04:42:13.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f7a9c06c460 msgr2=0x7f7a9c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:13.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f7a9c06c460 0x7f7a9c06e920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b5c0 tx=0x7f7a98005a90 comp rx=0 tx=0).stop 2024-04-02T04:42:13.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac100ca0 msgr2=0x7f7aac1964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:13.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac100ca0 0x7f7aac1964a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000d900 tx=0x7f7aa000dcc0 comp rx=0 tx=0).stop 2024-04-02T04:42:13.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 shutdown_connections 2024-04-02T04:42:13.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f7a9c06c460 0x7f7a9c06e920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:13.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aac100ca0 0x7f7aac1964a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:13.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 --2- 172.21.15.70:0/2388983209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7aac101ea0 0x7f7aac1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:13.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.688+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 >> 172.21.15.70:0/2388983209 conn(0x7f7aac0fc260 msgr2=0x7f7aac1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:13.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.689+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 shutdown_connections 2024-04-02T04:42:13.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:42:13.689+0000 7f7ab2f8e700 1 -- 172.21.15.70:0/2388983209 wait complete. 2024-04-02T04:42:14.455 DEBUG:teuthology.orchestra.run.smithi070:osd.2> sudo journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.2.service 2024-04-02T04:42:14.458 INFO:tasks.cephadm:Deploying osd.3 on smithi179 with /dev/vg_nvme/lv_4... 2024-04-02T04:42:14.458 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T04:42:14.671 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi179/config 2024-04-02T04:42:14.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:14 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:14.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:14 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:14.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:14 smithi070 ceph-mon[29212]: from='osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T04:42:14.716 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:14 smithi070 ceph-mon[29212]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:14.787 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:14 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:14.787 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:14 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:14.787 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:14 smithi179 ceph-mon[36603]: from='osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T04:42:14.787 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:14 smithi179 ceph-mon[36603]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:15.694 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:15 smithi070 ceph-mon[29212]: from='osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T04:42:15.694 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:15 smithi070 ceph-mon[29212]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T04:42:15.694 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:15.694 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:15 smithi070 ceph-mon[29212]: from='osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:42:15.695 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:15 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[63325]: 2024-04-02T04:42:15.685+0000 7f370242d700 -1 osd.2 0 waiting for initial osdmap 2024-04-02T04:42:15.695 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:15 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[63325]: 2024-04-02T04:42:15.690+0000 7f36faa1b700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:42:15.956 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:15 smithi179 ceph-mon[36603]: from='osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T04:42:15.956 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:15 smithi179 ceph-mon[36603]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T04:42:15.956 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:15.956 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:15 smithi179 ceph-mon[36603]: from='osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:42:16.454 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T04:42:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T04:42:16.455 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-04-02T04:42:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-04-02T04:42:16.455 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114689 s, 914 MB/s 2024-04-02T04:42:16.455 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-04-02T04:42:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:16 smithi070 ceph-mon[29212]: from='osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-04-02T04:42:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:16 smithi070 ceph-mon[29212]: osdmap e18: 3 total, 2 up, 3 in 2024-04-02T04:42:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:16 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:16 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:16 smithi070 ceph-mon[29212]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:17.052 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:16 smithi179 ceph-mon[36603]: from='osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-04-02T04:42:17.052 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:16 smithi179 ceph-mon[36603]: osdmap e18: 3 total, 2 up, 3 in 2024-04-02T04:42:17.052 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:16 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:17.052 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:16 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:17.052 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:16 smithi179 ceph-mon[36603]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:42:17.073 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-04-02T04:42:17.279 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi179/config 2024-04-02T04:42:17.959 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:17 smithi179 ceph-mon[36603]: purged_snaps scrub starts 2024-04-02T04:42:17.959 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:17 smithi179 ceph-mon[36603]: purged_snaps scrub ok 2024-04-02T04:42:17.959 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:17 smithi179 ceph-mon[36603]: osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] boot 2024-04-02T04:42:17.960 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:17 smithi179 ceph-mon[36603]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T04:42:17.960 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:17 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:17.960 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:17 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:17.960 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:17 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:17 smithi070 ceph-mon[29212]: purged_snaps scrub starts 2024-04-02T04:42:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:17 smithi070 ceph-mon[29212]: purged_snaps scrub ok 2024-04-02T04:42:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:17 smithi070 ceph-mon[29212]: osd.2 [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] boot 2024-04-02T04:42:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:17 smithi070 ceph-mon[29212]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T04:42:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:17 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:17 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:17 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:18.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.687+0000 7f2892e80700 1 -- 172.21.15.179:0/2359410687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c100ca0 msgr2=0x7f288c1010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:18.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.687+0000 7f2892e80700 1 --2- 172.21.15.179:0/2359410687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c100ca0 0x7f288c1010c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2880009910 tx=0x7f2880009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:18.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.687+0000 7f2892e80700 1 -- 172.21.15.179:0/2359410687 shutdown_connections 2024-04-02T04:42:18.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.687+0000 7f2892e80700 1 --2- 172.21.15.179:0/2359410687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f288c101ea0 0x7f288c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:18.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.687+0000 7f2892e80700 1 --2- 172.21.15.179:0/2359410687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c100ca0 0x7f288c1010c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.687+0000 7f2892e80700 1 -- 172.21.15.179:0/2359410687 >> 172.21.15.179:0/2359410687 conn(0x7f288c0fc260 msgr2=0x7f288c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.687+0000 7f2892e80700 1 -- 172.21.15.179:0/2359410687 shutdown_connections 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.687+0000 7f2892e80700 1 -- 172.21.15.179:0/2359410687 wait complete. 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f2892e80700 1 Processor -- start 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f2892e80700 1 -- start start 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f2892e80700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f288c100ca0 0x7f288c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f2892e80700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c101ea0 0x7f288c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f2892e80700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0736c0 con 0x7f288c100ca0 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f2892e80700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c073800 con 0x7f288c101ea0 2024-04-02T04:42:18.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f288bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c101ea0 0x7f288c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f288bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c101ea0 0x7f288c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41116/0 (socket says 172.21.15.179:41116) 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f2890c1c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f288c100ca0 0x7f288c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f288bfff700 1 -- 172.21.15.179:0/2210579193 learned_addr learned my addr 172.21.15.179:0/2210579193 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f288bfff700 1 -- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f288c100ca0 msgr2=0x7f288c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f288bfff700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f288c100ca0 0x7f288c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f288bfff700 1 -- 172.21.15.179:0/2210579193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28800095f0 con 0x7f288c101ea0 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.688+0000 7f288bfff700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c101ea0 0x7f288c073180 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f287800e920 tx=0x7f287800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.689+0000 7f2889ffb700 1 -- 172.21.15.179:0/2210579193 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f287800c9d0 con 0x7f288c101ea0 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.689+0000 7f2889ffb700 1 -- 172.21.15.179:0/2210579193 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2878010430 con 0x7f288c101ea0 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.689+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c073ae0 con 0x7f288c101ea0 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.689+0000 7f2889ffb700 1 -- 172.21.15.179:0/2210579193 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2878018650 con 0x7f288c101ea0 2024-04-02T04:42:18.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.689+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c19e530 con 0x7f288c101ea0 2024-04-02T04:42:18.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.690+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288c109800 con 0x7f288c101ea0 2024-04-02T04:42:18.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.690+0000 7f2889ffb700 1 -- 172.21.15.179:0/2210579193 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f2878018910 con 0x7f288c101ea0 2024-04-02T04:42:18.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.690+0000 7f2889ffb700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f287c06c460 0x7f287c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:18.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.691+0000 7f2889ffb700 1 -- 172.21.15.179:0/2210579193 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2846+0+0 (secure 0 0 0) 0x7f2878014070 con 0x7f288c101ea0 2024-04-02T04:42:18.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.691+0000 7f2890c1c700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f287c06c460 0x7f287c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:18.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.691+0000 7f2890c1c700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f287c06c460 0x7f287c06e920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f288000b6f0 tx=0x7f2880005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:18.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.695+0000 7f2889ffb700 1 -- 172.21.15.179:0/2210579193 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f28780520e0 con 0x7f288c101ea0 2024-04-02T04:42:18.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:18 smithi179 ceph-mon[36603]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T04:42:18.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:18 smithi179 ceph-mon[36603]: pgmap v78: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:18.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:18 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T04:42:18.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:18.891+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f288c1067f0 con 0x7f287c06c460 2024-04-02T04:42:19.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:18 smithi070 ceph-mon[29212]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T04:42:19.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:18 smithi070 ceph-mon[29212]: pgmap v78: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:19.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T04:42:19.929 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T04:42:19.929 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:19 smithi070 ceph-mon[29212]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T04:42:19.929 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T04:42:19.929 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:19 smithi070 ceph-mon[29212]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:19.929 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:42:19.929 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:42:19.929 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:19.978 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T04:42:19.979 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:19 smithi179 ceph-mon[36603]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T04:42:19.979 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T04:42:19.979 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:19 smithi179 ceph-mon[36603]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:19.979 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:42:19.979 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:42:19.979 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:20.764 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:42:20 smithi070 sudo[68080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:42:20.764 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:42:20 smithi070 sudo[68080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:20.764 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:42:20 smithi070 sudo[68080]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:20.764 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:42:20 smithi070 sudo[68080]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:20.764 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: Detected new or changed devices on smithi070 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: Adjusting osd_memory_target on smithi070 to 3946M 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:20.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:20.766 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:20.766 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:20.766 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:20 smithi070 ceph-mon[29212]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:20.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: Detected new or changed devices on smithi070 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: Adjusting osd_memory_target on smithi070 to 3946M 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:20.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:20.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:20.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:20 smithi179 ceph-mon[36603]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:21.029 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:42:20 smithi070 sudo[68133]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:42:21.029 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:42:20 smithi070 sudo[68133]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:21.029 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:42:20 smithi070 sudo[68133]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:21.029 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:42:20 smithi070 sudo[68133]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:21.302 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68154]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:42:21.302 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68154]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:21.302 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68154]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:21.302 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68154]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:21.563 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68200]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:42:21.563 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68200]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:21.564 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68200]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:21.564 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68200]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:21.825 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68261]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:42:21.825 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68261]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:21.825 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68261]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:21.825 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68261]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:21.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:21 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:21.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:21 smithi070 ceph-mon[29212]: osdmap e23: 3 total, 3 up, 3 in 2024-04-02T04:42:21.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:21 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/1763845191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d71a0bb-5e7a-4339-afe7-06174dea4986"}]: dispatch 2024-04-02T04:42:21.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:21 smithi070 ceph-mon[29212]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d71a0bb-5e7a-4339-afe7-06174dea4986"}]: dispatch 2024-04-02T04:42:21.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:21 smithi070 ceph-mon[29212]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d71a0bb-5e7a-4339-afe7-06174dea4986"}]': finished 2024-04-02T04:42:21.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:21 smithi070 ceph-mon[29212]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T04:42:21.826 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:21 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:22.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:21 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:21 smithi179 ceph-mon[36603]: osdmap e23: 3 total, 3 up, 3 in 2024-04-02T04:42:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:21 smithi179 ceph-mon[36603]: from='client.? 172.21.15.179:0/1763845191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d71a0bb-5e7a-4339-afe7-06174dea4986"}]: dispatch 2024-04-02T04:42:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:21 smithi179 ceph-mon[36603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d71a0bb-5e7a-4339-afe7-06174dea4986"}]: dispatch 2024-04-02T04:42:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:21 smithi179 ceph-mon[36603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d71a0bb-5e7a-4339-afe7-06174dea4986"}]': finished 2024-04-02T04:42:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:21 smithi179 ceph-mon[36603]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T04:42:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:21 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:22.084 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68286]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:42:22.084 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:22.084 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68286]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:22.084 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:42:21 smithi070 sudo[68286]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:22.342 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 sudo[68321]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:42:22.342 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 sudo[68321]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:22.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 sudo[68321]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:22.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 sudo[68321]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 sudo[68364]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:42:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 sudo[68364]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 sudo[68364]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 sudo[68364]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:22.855 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 sudo[40665]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:42:22.855 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 sudo[40665]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:22.855 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 sudo[40665]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:22.855 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 sudo[40665]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:22.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:22.858+0000 7f2889ffb700 1 -- 172.21.15.179:0/2210579193 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f28780566a0 con 0x7f288c101ea0 2024-04-02T04:42:23.230 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 sudo[40856]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:42:23.230 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 sudo[40856]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:42:23.230 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 sudo[40856]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:42:23.230 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 sudo[40856]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:42:23.231 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 ceph-mon[36603]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T04:42:23.231 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 ceph-mon[36603]: from='client.? 172.21.15.179:0/358339452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:42:23.231 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 ceph-mon[36603]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:23.231 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 ceph-mon[36603]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T04:42:23.231 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:42:23.231 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:42:23.231 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:42:23.231 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:42:23.249 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 ceph-mon[29212]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T04:42:23.249 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/358339452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:42:23.249 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 ceph-mon[29212]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:23.250 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 ceph-mon[29212]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T04:42:23.250 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:42:23.250 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:42:23.250 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:42:23.250 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:42:24.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:23 smithi070 ceph-mon[29212]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T04:42:24.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:23 smithi070 ceph-mon[29212]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T04:42:24.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:23 smithi070 ceph-mon[29212]: mgrmap e19: smithi070.igidvr(active, since 2m), standbys: smithi179.labmvs 2024-04-02T04:42:24.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:23 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:24.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:23 smithi179 ceph-mon[36603]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T04:42:24.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:23 smithi179 ceph-mon[36603]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T04:42:24.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:23 smithi179 ceph-mon[36603]: mgrmap e19: smithi070.igidvr(active, since 2m), standbys: smithi179.labmvs 2024-04-02T04:42:24.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:23 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:25.523 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:25 smithi179 ceph-mon[36603]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:25.523 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:25 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:25.523 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:25 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:25.523 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:25 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:25.523 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:25 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:25.523 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:25 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:25.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:25 smithi070 ceph-mon[29212]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:25.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:25 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:25.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:25 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:25.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:25 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:25.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:25 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:25.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:25 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:26.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:26 smithi179 ceph-mon[36603]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:27.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:26 smithi070 ceph-mon[29212]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:28.389 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:28.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:29.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:29 smithi070 ceph-mon[29212]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-04-02T04:42:29.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:29 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:42:29.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:29 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:29.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:29 smithi070 ceph-mon[29212]: Deploying daemon osd.3 on smithi179 2024-04-02T04:42:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:29 smithi179 ceph-mon[36603]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-04-02T04:42:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:29 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:42:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:29 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:29 smithi179 ceph-mon[36603]: Deploying daemon osd.3 on smithi179 2024-04-02T04:42:31.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:31 smithi070 ceph-mon[29212]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-02T04:42:31.531 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:31 smithi179 ceph-mon[36603]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-02T04:42:33.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:33 smithi179 ceph-mon[36603]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-02T04:42:33.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:33 smithi070 ceph-mon[29212]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-02T04:42:35.009 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:35.010 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:35.010 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:35.010 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:34 smithi179 ceph-mon[36603]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-02T04:42:35.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:35.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:35.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:35.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:34 smithi070 ceph-mon[29212]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-02T04:42:37.007 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:36 smithi179 ceph-mon[36603]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-02T04:42:37.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:36 smithi070 ceph-mon[29212]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-02T04:42:37.062 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 3 on host 'smithi179' 2024-04-02T04:42:37.063 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.058+0000 7f2889ffb700 1 -- 172.21.15.179:0/2210579193 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f288c1067f0 con 0x7f287c06c460 2024-04-02T04:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f287c06c460 msgr2=0x7f287c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.064 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f287c06c460 0x7f287c06e920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f288000b6f0 tx=0x7f2880005940 comp rx=0 tx=0).stop 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c101ea0 msgr2=0x7f288c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c101ea0 0x7f288c073180 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f287800e920 tx=0x7f287800ec30 comp rx=0 tx=0).stop 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 shutdown_connections 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f287c06c460 0x7f287c06e920 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f288c100ca0 0x7f288c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 --2- 172.21.15.179:0/2210579193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f288c101ea0 0x7f288c073180 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 >> 172.21.15.179:0/2210579193 conn(0x7f288c0fc260 msgr2=0x7f288c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 shutdown_connections 2024-04-02T04:42:37.065 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:37.060+0000 7f2892e80700 1 -- 172.21.15.179:0/2210579193 wait complete. 2024-04-02T04:42:37.935 DEBUG:teuthology.orchestra.run.smithi179:osd.3> sudo journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.3.service 2024-04-02T04:42:37.937 INFO:tasks.cephadm:Deploying osd.4 on smithi179 with /dev/vg_nvme/lv_3... 2024-04-02T04:42:37.937 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T04:42:38.181 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi179/config 2024-04-02T04:42:38.253 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:38 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:38.254 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:38 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:38.254 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:38 smithi179 ceph-mon[36603]: from='osd.3 [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:42:38.254 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:38 smithi179 ceph-mon[36603]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:42:38.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:38 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:38.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:38 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:38.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:38 smithi070 ceph-mon[29212]: from='osd.3 [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:42:38.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:38 smithi070 ceph-mon[29212]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:42:39.264 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-mon[36603]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T04:42:39.264 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-mon[36603]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T04:42:39.264 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:39.264 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-mon[36603]: from='osd.3 [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:42:39.265 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:39.265 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-mon[36603]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:42:39.265 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:39.265 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-mon[36603]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:39.265 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[42695]: 2024-04-02T04:42:39.050+0000 7f49c75cb700 -1 osd.3 0 waiting for initial osdmap 2024-04-02T04:42:39.265 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:42:39 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[42695]: 2024-04-02T04:42:39.056+0000 7f49c2bbf700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:42:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:39 smithi070 ceph-mon[29212]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T04:42:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:39 smithi070 ceph-mon[29212]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T04:42:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:39 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:39 smithi070 ceph-mon[29212]: from='osd.3 [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:42:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:39 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:39 smithi070 ceph-mon[29212]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:42:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:39 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:39 smithi070 ceph-mon[29212]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:42:40.011 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T04:42:40.011 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T04:42:40.011 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-04-02T04:42:40.012 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-04-02T04:42:40.012 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130902 s, 801 MB/s 2024-04-02T04:42:40.012 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-04-02T04:42:40.222 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:40 smithi179 ceph-mon[36603]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-02T04:42:40.222 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:40 smithi179 ceph-mon[36603]: osdmap e26: 4 total, 3 up, 4 in 2024-04-02T04:42:40.222 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:40.222 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:40 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:40.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:40 smithi070 ceph-mon[29212]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-02T04:42:40.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:40 smithi070 ceph-mon[29212]: osdmap e26: 4 total, 3 up, 4 in 2024-04-02T04:42:40.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:40.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:40 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:41 smithi179 ceph-mon[36603]: purged_snaps scrub starts 2024-04-02T04:42:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:41 smithi179 ceph-mon[36603]: purged_snaps scrub ok 2024-04-02T04:42:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:41 smithi179 ceph-mon[36603]: osd.3 [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] boot 2024-04-02T04:42:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:41 smithi179 ceph-mon[36603]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T04:42:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:41 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:41 smithi179 ceph-mon[36603]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:41 smithi179 ceph-mon[36603]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T04:42:41.359 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-04-02T04:42:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:41 smithi070 ceph-mon[29212]: purged_snaps scrub starts 2024-04-02T04:42:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:41 smithi070 ceph-mon[29212]: purged_snaps scrub ok 2024-04-02T04:42:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:41 smithi070 ceph-mon[29212]: osd.3 [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] boot 2024-04-02T04:42:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:41 smithi070 ceph-mon[29212]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T04:42:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:41 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:41 smithi070 ceph-mon[29212]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:41 smithi070 ceph-mon[29212]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T04:42:41.901 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi179/config 2024-04-02T04:42:43.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:42 smithi070 ceph-mon[29212]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T04:42:43.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:42 smithi070 ceph-mon[29212]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:43.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:42 smithi179 ceph-mon[36603]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T04:42:43.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:42 smithi179 ceph-mon[36603]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:43.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: Detected new or changed devices on smithi179 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: Adjusting osd_memory_target on smithi179 to 14912M 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:43.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:44.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T04:42:44.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: Detected new or changed devices on smithi179 2024-04-02T04:42:44.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:44.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:44.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: Adjusting osd_memory_target on smithi179 to 14912M 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:44.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:42:44.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 -- 172.21.15.179:0/537128138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130100ca0 msgr2=0x7ff1301010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:44.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 --2- 172.21.15.179:0/537128138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130100ca0 0x7ff1301010c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff120009910 tx=0x7ff120009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:44.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 -- 172.21.15.179:0/537128138 shutdown_connections 2024-04-02T04:42:44.047 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 --2- 172.21.15.179:0/537128138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130101ea0 0x7ff130102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:44.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 --2- 172.21.15.179:0/537128138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130100ca0 0x7ff1301010c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:44.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 -- 172.21.15.179:0/537128138 >> 172.21.15.179:0/537128138 conn(0x7ff1300fc260 msgr2=0x7ff1300fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:44.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 -- 172.21.15.179:0/537128138 shutdown_connections 2024-04-02T04:42:44.048 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 -- 172.21.15.179:0/537128138 wait complete. 2024-04-02T04:42:44.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.043+0000 7ff134d9f700 1 Processor -- start 2024-04-02T04:42:44.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff134d9f700 1 -- start start 2024-04-02T04:42:44.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff134d9f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130100ca0 0x7ff130196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:44.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff134d9f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130101ea0 0x7ff130196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:44.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff134d9f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1301970b0 con 0x7ff130100ca0 2024-04-02T04:42:44.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff134d9f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1301971f0 con 0x7ff130101ea0 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff12dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130101ea0 0x7ff130196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff12dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130101ea0 0x7ff130196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48626/0 (socket says 172.21.15.179:48626) 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff12dd9b700 1 -- 172.21.15.179:0/2089631247 learned_addr learned my addr 172.21.15.179:0/2089631247 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff12e59c700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130100ca0 0x7ff130196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff12dd9b700 1 -- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130100ca0 msgr2=0x7ff130196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff12dd9b700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130100ca0 0x7ff130196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff12dd9b700 1 -- 172.21.15.179:0/2089631247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200095f0 con 0x7ff130101ea0 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff12dd9b700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130101ea0 0x7ff130196a90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff12400b4b0 tx=0x7ff12400b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff11f7fe700 1 -- 172.21.15.179:0/2089631247 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff124011800 con 0x7ff130101ea0 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13019bca0 con 0x7ff130101ea0 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff11f7fe700 1 -- 172.21.15.179:0/2089631247 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff124011e10 con 0x7ff130101ea0 2024-04-02T04:42:44.052 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.044+0000 7ff11f7fe700 1 -- 172.21.15.179:0/2089631247 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff12400f2e0 con 0x7ff130101ea0 2024-04-02T04:42:44.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.045+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13019c1f0 con 0x7ff130101ea0 2024-04-02T04:42:44.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.045+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff130109880 con 0x7ff130101ea0 2024-04-02T04:42:44.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.046+0000 7ff11f7fe700 1 -- 172.21.15.179:0/2089631247 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff124010380 con 0x7ff130101ea0 2024-04-02T04:42:44.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.046+0000 7ff11f7fe700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff11806c4e0 0x7ff11806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:44.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.046+0000 7ff11f7fe700 1 -- 172.21.15.179:0/2089631247 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7ff12408ad20 con 0x7ff130101ea0 2024-04-02T04:42:44.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.046+0000 7ff12e59c700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff11806c4e0 0x7ff11806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:44.053 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.047+0000 7ff12e59c700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff11806c4e0 0x7ff11806e9a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff120006010 tx=0x7ff12000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:44.055 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.051+0000 7ff11f7fe700 1 -- 172.21.15.179:0/2089631247 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff124059630 con 0x7ff130101ea0 2024-04-02T04:42:44.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:42:44.245+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1301067f0 con 0x7ff11806c4e0 2024-04-02T04:42:45.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:44 smithi070 ceph-mon[29212]: osdmap e31: 4 total, 4 up, 4 in 2024-04-02T04:42:45.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:44 smithi070 ceph-mon[29212]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T04:42:45.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:44 smithi070 ceph-mon[29212]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:45.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:44 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:42:45.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:44 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:42:45.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:44 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:45.031 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:44 smithi179 ceph-mon[36603]: osdmap e31: 4 total, 4 up, 4 in 2024-04-02T04:42:45.031 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:44 smithi179 ceph-mon[36603]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T04:42:45.031 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:44 smithi179 ceph-mon[36603]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:45.031 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:44 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:42:45.031 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:44 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:42:45.032 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:44 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:47.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:46 smithi070 ceph-mon[29212]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-02T04:42:47.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:46 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/876507607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "754e6f09-2a8e-42e2-a998-5f5a1addcb29"}]: dispatch 2024-04-02T04:42:47.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:46 smithi070 ceph-mon[29212]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "754e6f09-2a8e-42e2-a998-5f5a1addcb29"}]: dispatch 2024-04-02T04:42:47.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:46 smithi070 ceph-mon[29212]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "754e6f09-2a8e-42e2-a998-5f5a1addcb29"}]': finished 2024-04-02T04:42:47.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:46 smithi070 ceph-mon[29212]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T04:42:47.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:46 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:42:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:46 smithi179 ceph-mon[36603]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-02T04:42:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:46 smithi179 ceph-mon[36603]: from='client.? 172.21.15.179:0/876507607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "754e6f09-2a8e-42e2-a998-5f5a1addcb29"}]: dispatch 2024-04-02T04:42:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:46 smithi179 ceph-mon[36603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "754e6f09-2a8e-42e2-a998-5f5a1addcb29"}]: dispatch 2024-04-02T04:42:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:46 smithi179 ceph-mon[36603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "754e6f09-2a8e-42e2-a998-5f5a1addcb29"}]': finished 2024-04-02T04:42:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:46 smithi179 ceph-mon[36603]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T04:42:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:46 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:42:47.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:47 smithi179 ceph-mon[36603]: from='client.? 172.21.15.179:0/2362329765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:42:48.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:47 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/2362329765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:42:49.076 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:48 smithi179 ceph-mon[36603]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T04:42:49.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:48 smithi070 ceph-mon[29212]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T04:42:51.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:51 smithi179 ceph-mon[36603]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:51.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:51 smithi070 ceph-mon[29212]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:53.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:53 smithi070 ceph-mon[29212]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:53.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:53 smithi179 ceph-mon[36603]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:54.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:54 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:42:54.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:54 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:54.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:54 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:42:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:54 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:55.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:55 smithi070 ceph-mon[29212]: Deploying daemon osd.4 on smithi179 2024-04-02T04:42:55.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:55 smithi070 ceph-mon[29212]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:55.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:55 smithi179 ceph-mon[36603]: Deploying daemon osd.4 on smithi179 2024-04-02T04:42:55.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:55 smithi179 ceph-mon[36603]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:56.934 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:56 smithi179 ceph-mon[36603]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:57.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:56 smithi070 ceph-mon[29212]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:59.414 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:42:59 smithi179 ceph-mon[36603]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:42:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:42:59 smithi070 ceph-mon[29212]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:43:00.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:00.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:00.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:00.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:00.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:01.738 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:01 smithi179 ceph-mon[36603]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:43:01.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:01 smithi070 ceph-mon[29212]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:43:02.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.507+0000 7ff11f7fe700 1 -- 172.21.15.179:0/2089631247 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff1301067f0 con 0x7ff11806c4e0 2024-04-02T04:43:02.514 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 4 on host 'smithi179' 2024-04-02T04:43:02.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff11806c4e0 msgr2=0x7ff11806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:02.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff11806c4e0 0x7ff11806e9a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff120006010 tx=0x7ff12000b540 comp rx=0 tx=0).stop 2024-04-02T04:43:02.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130101ea0 msgr2=0x7ff130196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:02.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130101ea0 0x7ff130196a90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff12400b4b0 tx=0x7ff12400b7c0 comp rx=0 tx=0).stop 2024-04-02T04:43:02.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 shutdown_connections 2024-04-02T04:43:02.515 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff11806c4e0 0x7ff11806e9a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:02.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff130100ca0 0x7ff130196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:02.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 --2- 172.21.15.179:0/2089631247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff130101ea0 0x7ff130196a90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:02.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 >> 172.21.15.179:0/2089631247 conn(0x7ff1300fc260 msgr2=0x7ff1301050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:02.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 shutdown_connections 2024-04-02T04:43:02.516 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:02.510+0000 7ff134d9f700 1 -- 172.21.15.179:0/2089631247 wait complete. 2024-04-02T04:43:03.186 DEBUG:teuthology.orchestra.run.smithi179:osd.4> sudo journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.4.service 2024-04-02T04:43:03.188 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_2... 2024-04-02T04:43:03.189 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T04:43:03.468 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:43:03 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[49174]: 2024-04-02T04:43:03.330+0000 7f0a82782640 -1 osd.4 0 log_to_monitors true 2024-04-02T04:43:03.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:03 smithi179 ceph-mon[36603]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:43:03.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:03 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:03.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:03 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:03.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:03 smithi179 ceph-mon[36603]: from='osd.4 [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:43:03.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:03 smithi179 ceph-mon[36603]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:43:03.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:03 smithi070 ceph-mon[29212]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:43:03.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:03.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:03.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:03 smithi070 ceph-mon[29212]: from='osd.4 [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:43:03.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:03 smithi070 ceph-mon[29212]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:43:03.997 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi179/config 2024-04-02T04:43:04.760 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:43:04 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[49174]: 2024-04-02T04:43:04.507+0000 7f0a775e5700 -1 osd.4 0 waiting for initial osdmap 2024-04-02T04:43:04.760 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:43:04 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[49174]: 2024-04-02T04:43:04.513+0000 7f0a723d8700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:43:04.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:04 smithi179 ceph-mon[36603]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T04:43:04.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:04 smithi179 ceph-mon[36603]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T04:43:04.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:04 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:04.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:04 smithi179 ceph-mon[36603]: from='osd.4 [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:43:04.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:04 smithi179 ceph-mon[36603]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:43:04.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:04 smithi070 ceph-mon[29212]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T04:43:04.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:04 smithi070 ceph-mon[29212]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T04:43:04.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:04 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:04.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:04 smithi070 ceph-mon[29212]: from='osd.4 [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:43:04.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:04 smithi070 ceph-mon[29212]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:43:05.713 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:05 smithi179 ceph-mon[36603]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:43:05.714 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:05 smithi179 ceph-mon[36603]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-02T04:43:05.714 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:05 smithi179 ceph-mon[36603]: osdmap e34: 5 total, 4 up, 5 in 2024-04-02T04:43:05.714 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:05.714 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:05.714 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:05.714 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:05.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:05 smithi070 ceph-mon[29212]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-02T04:43:05.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:05 smithi070 ceph-mon[29212]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-02T04:43:05.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:05 smithi070 ceph-mon[29212]: osdmap e34: 5 total, 4 up, 5 in 2024-04-02T04:43:05.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:05.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:05.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:05.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:05.839 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T04:43:05.840 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T04:43:05.840 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-04-02T04:43:05.840 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-04-02T04:43:05.840 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0138756 s, 756 MB/s 2024-04-02T04:43:05.840 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-04-02T04:43:06.436 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-04-02T04:43:06.671 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:06 smithi179 ceph-mon[36603]: purged_snaps scrub starts 2024-04-02T04:43:06.672 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:06 smithi179 ceph-mon[36603]: purged_snaps scrub ok 2024-04-02T04:43:06.672 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:06 smithi179 ceph-mon[36603]: osd.4 [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] boot 2024-04-02T04:43:06.672 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:06 smithi179 ceph-mon[36603]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T04:43:06.672 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:06 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:06.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:06 smithi070 ceph-mon[29212]: purged_snaps scrub starts 2024-04-02T04:43:06.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:06 smithi070 ceph-mon[29212]: purged_snaps scrub ok 2024-04-02T04:43:06.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:06 smithi070 ceph-mon[29212]: osd.4 [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] boot 2024-04-02T04:43:06.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:06 smithi070 ceph-mon[29212]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T04:43:06.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:06 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:07.322 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi179/config 2024-04-02T04:43:07.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:07 smithi179 ceph-mon[36603]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:07.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:07 smithi179 ceph-mon[36603]: osdmap e36: 5 total, 5 up, 5 in 2024-04-02T04:43:07.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:07 smithi070 ceph-mon[29212]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:07.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:07 smithi070 ceph-mon[29212]: osdmap e36: 5 total, 5 up, 5 in 2024-04-02T04:43:09.639 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:09.640 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:09.640 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:09.640 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:09.640 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:09.640 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:09.640 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:09.640 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:09.640 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:09.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:09.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:09.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:09.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:09.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:09.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:09.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:09.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:09.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:09.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 -- 172.21.15.179:0/266351436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec101eb0 msgr2=0x7fd1ec102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:09.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 --2- 172.21.15.179:0/266351436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec101eb0 0x7fd1ec102310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009910 tx=0x7fd1e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:09.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 -- 172.21.15.179:0/266351436 shutdown_connections 2024-04-02T04:43:09.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 --2- 172.21.15.179:0/266351436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec101eb0 0x7fd1ec102310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:09.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 --2- 172.21.15.179:0/266351436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec100cb0 0x7fd1ec1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:09.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 -- 172.21.15.179:0/266351436 >> 172.21.15.179:0/266351436 conn(0x7fd1ec0fc230 msgr2=0x7fd1ec0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:09.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 -- 172.21.15.179:0/266351436 shutdown_connections 2024-04-02T04:43:09.860 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 -- 172.21.15.179:0/266351436 wait complete. 2024-04-02T04:43:09.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 Processor -- start 2024-04-02T04:43:09.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 -- start start 2024-04-02T04:43:09.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.857+0000 7fd1f2472700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec100cb0 0x7fd1ec1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:09.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1f2472700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec101eb0 0x7fd1ec196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:09.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1f2472700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec196fb0 con 0x7fd1ec101eb0 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1f2472700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1970f0 con 0x7fd1ec100cb0 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec100cb0 0x7fd1ec1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec100cb0 0x7fd1ec1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49818/0 (socket says 172.21.15.179:49818) 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1ebfff700 1 -- 172.21.15.179:0/2748226901 learned_addr learned my addr 172.21.15.179:0/2748226901 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1eb7fe700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec101eb0 0x7fd1ec196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1ebfff700 1 -- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec101eb0 msgr2=0x7fd1ec196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1ebfff700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec101eb0 0x7fd1ec196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1ebfff700 1 -- 172.21.15.179:0/2748226901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e00095f0 con 0x7fd1ec100cb0 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1eb7fe700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec101eb0 0x7fd1ec196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1ebfff700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec100cb0 0x7fd1ec1964e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c910 tx=0x7fd1dc00ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1e97fa700 1 -- 172.21.15.179:0/2748226901 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1dc0041f0 con 0x7fd1ec100cb0 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec19bbb0 con 0x7fd1ec100cb0 2024-04-02T04:43:09.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1e97fa700 1 -- 172.21.15.179:0/2748226901 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1dc007db0 con 0x7fd1ec100cb0 2024-04-02T04:43:09.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1e97fa700 1 -- 172.21.15.179:0/2748226901 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1dc003d60 con 0x7fd1ec100cb0 2024-04-02T04:43:09.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.858+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec19c0d0 con 0x7fd1ec100cb0 2024-04-02T04:43:09.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.859+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec109810 con 0x7fd1ec100cb0 2024-04-02T04:43:09.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.860+0000 7fd1e97fa700 1 -- 172.21.15.179:0/2748226901 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd1dc007900 con 0x7fd1ec100cb0 2024-04-02T04:43:09.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.860+0000 7fd1e97fa700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd1d406c4e0 0x7fd1d406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:09.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.860+0000 7fd1e97fa700 1 -- 172.21.15.179:0/2748226901 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fd1dc08b050 con 0x7fd1ec100cb0 2024-04-02T04:43:09.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.860+0000 7fd1eb7fe700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd1d406c4e0 0x7fd1d406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:09.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.861+0000 7fd1eb7fe700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd1d406c4e0 0x7fd1d406e9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b5c0 tx=0x7fd1e0005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:09.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:09.865+0000 7fd1e97fa700 1 -- 172.21.15.179:0/2748226901 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd1dc0597b0 con 0x7fd1ec100cb0 2024-04-02T04:43:10.059 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:10.056+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1ec106800 con 0x7fd1d406c4e0 2024-04-02T04:43:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: Detected new or changed devices on smithi179 2024-04-02T04:43:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: Adjusting osd_memory_target on smithi179 to 7456M 2024-04-02T04:43:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:43:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:43:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:10.809 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: Detected new or changed devices on smithi179 2024-04-02T04:43:10.809 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: Adjusting osd_memory_target on smithi179 to 7456M 2024-04-02T04:43:10.809 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:10.809 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:10.810 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:10.810 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:10.810 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:43:10.810 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:43:10.810 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:11.708 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:11 smithi179 ceph-mon[36603]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:11.708 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:11 smithi179 ceph-mon[36603]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:12.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:11 smithi070 ceph-mon[29212]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:12.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:11 smithi070 ceph-mon[29212]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:13.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:12 smithi070 ceph-mon[29212]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:13.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:12 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/677141829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a73805-a770-4ed1-96f4-e64cf88810eb"}]: dispatch 2024-04-02T04:43:13.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:12 smithi070 ceph-mon[29212]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a73805-a770-4ed1-96f4-e64cf88810eb"}]: dispatch 2024-04-02T04:43:13.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:12 smithi070 ceph-mon[29212]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a73805-a770-4ed1-96f4-e64cf88810eb"}]': finished 2024-04-02T04:43:13.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:12 smithi070 ceph-mon[29212]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T04:43:13.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:12 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:13.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:12 smithi179 ceph-mon[36603]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:13.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:12 smithi179 ceph-mon[36603]: from='client.? 172.21.15.179:0/677141829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a73805-a770-4ed1-96f4-e64cf88810eb"}]: dispatch 2024-04-02T04:43:13.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:12 smithi179 ceph-mon[36603]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a73805-a770-4ed1-96f4-e64cf88810eb"}]: dispatch 2024-04-02T04:43:13.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:12 smithi179 ceph-mon[36603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a73805-a770-4ed1-96f4-e64cf88810eb"}]': finished 2024-04-02T04:43:13.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:12 smithi179 ceph-mon[36603]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T04:43:13.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:12 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:13 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/2804435537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:43:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:14.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:13 smithi179 ceph-mon[36603]: from='client.? 172.21.15.179:0/2804435537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:43:14.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:15.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:14 smithi070 ceph-mon[29212]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:15.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:14 smithi179 ceph-mon[36603]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:16 smithi070 ceph-mon[29212]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:17.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:16 smithi179 ceph-mon[36603]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:19.469 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:19 smithi179 ceph-mon[36603]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:19.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:19 smithi070 ceph-mon[29212]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:20.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:43:20.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:20.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:43:20.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:21.514 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:21 smithi179 ceph-mon[36603]: Deploying daemon osd.5 on smithi179 2024-04-02T04:43:21.514 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:21 smithi179 ceph-mon[36603]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:21.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:21 smithi070 ceph-mon[29212]: Deploying daemon osd.5 on smithi179 2024-04-02T04:43:21.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:21 smithi070 ceph-mon[29212]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:23.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:23 smithi179 ceph-mon[36603]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:23.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:23 smithi070 ceph-mon[29212]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:25.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:25 smithi070 ceph-mon[29212]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:25.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:25 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:25.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:25 smithi179 ceph-mon[36603]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:25.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:25 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:26.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:26 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:26.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:26 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:26.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:26 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:26 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:27.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:27 smithi070 ceph-mon[29212]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:27.533 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:27 smithi179 ceph-mon[36603]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:28.171 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-04-02T04:43:28.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.166+0000 7fd1e97fa700 1 -- 172.21.15.179:0/2748226901 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd1ec106800 con 0x7fd1d406c4e0 2024-04-02T04:43:28.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd1d406c4e0 msgr2=0x7fd1d406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:28.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd1d406c4e0 0x7fd1d406e9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b5c0 tx=0x7fd1e0005960 comp rx=0 tx=0).stop 2024-04-02T04:43:28.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec100cb0 msgr2=0x7fd1ec1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:28.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec100cb0 0x7fd1ec1964e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c910 tx=0x7fd1dc00ccd0 comp rx=0 tx=0).stop 2024-04-02T04:43:28.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 shutdown_connections 2024-04-02T04:43:28.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd1d406c4e0 0x7fd1d406e9a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:28.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ec100cb0 0x7fd1ec1964e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:28.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 --2- 172.21.15.179:0/2748226901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec101eb0 0x7fd1ec196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:28.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 >> 172.21.15.179:0/2748226901 conn(0x7fd1ec0fc230 msgr2=0x7fd1ec1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:28.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 shutdown_connections 2024-04-02T04:43:28.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:28.169+0000 7fd1f2472700 1 -- 172.21.15.179:0/2748226901 wait complete. 2024-04-02T04:43:29.106 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.5.service 2024-04-02T04:43:29.109 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-02T04:43:29.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T04:43:29.330 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:29.438 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:29 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:29.438 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:29 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:29.438 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:29 smithi179 ceph-mon[36603]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:29.438 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:29 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:29.439 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:43:29 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[55862]: 2024-04-02T04:43:29.178+0000 7fc08048e640 -1 osd.5 0 log_to_monitors true 2024-04-02T04:43:29.450 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:29 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:29.450 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:29 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:29.450 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:29 smithi070 ceph-mon[29212]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:29.451 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:29 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:30.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:30 smithi179 ceph-mon[36603]: from='osd.5 [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:43:30.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:30 smithi179 ceph-mon[36603]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:43:30.468 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:30 smithi070 ceph-mon[29212]: from='osd.5 [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:43:30.468 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:30 smithi070 ceph-mon[29212]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:43:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.137+0000 7f1ed95cf700 1 -- 172.21.15.70:0/357584570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed4101790 msgr2=0x7f1ed4103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.137+0000 7f1ed95cf700 1 --2- 172.21.15.70:0/357584570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed4101790 0x7f1ed4103b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009910 tx=0x7f1ec4009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.137+0000 7f1ed95cf700 1 -- 172.21.15.70:0/357584570 shutdown_connections 2024-04-02T04:43:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.137+0000 7f1ed95cf700 1 --2- 172.21.15.70:0/357584570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ed41040c0 0x7f1ed41064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.137+0000 7f1ed95cf700 1 --2- 172.21.15.70:0/357584570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed4101790 0x7f1ed4103b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.137+0000 7f1ed95cf700 1 -- 172.21.15.70:0/357584570 >> 172.21.15.70:0/357584570 conn(0x7f1ed40fb160 msgr2=0x7f1ed40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.137+0000 7f1ed95cf700 1 -- 172.21.15.70:0/357584570 shutdown_connections 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.137+0000 7f1ed95cf700 1 -- 172.21.15.70:0/357584570 wait complete. 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed95cf700 1 Processor -- start 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed95cf700 1 -- start start 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed95cf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ed4101790 0x7f1ed41967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed95cf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed41040c0 0x7f1ed4196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed95cf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed4197320 con 0x7f1ed41040c0 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed95cf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed4197460 con 0x7f1ed4101790 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed41040c0 0x7f1ed4196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed27fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed41040c0 0x7f1ed4196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50222/0 (socket says 172.21.15.70:50222) 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ed4101790 0x7f1ed41967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.138+0000 7f1ed27fc700 1 -- 172.21.15.70:0/3661917702 learned_addr learned my addr 172.21.15.70:0/3661917702 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:43:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ed27fc700 1 -- 172.21.15.70:0/3661917702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ed4101790 msgr2=0x7f1ed41967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ed27fc700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ed4101790 0x7f1ed41967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ed27fc700 1 -- 172.21.15.70:0/3661917702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec40095f0 con 0x7f1ed41040c0 2024-04-02T04:43:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ed27fc700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed41040c0 0x7f1ed4196d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800d8d0 tx=0x7f1ec800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ebbfff700 1 -- 172.21.15.70:0/3661917702 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec80098a0 con 0x7f1ed41040c0 2024-04-02T04:43:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ebbfff700 1 -- 172.21.15.70:0/3661917702 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1ec8010460 con 0x7f1ed41040c0 2024-04-02T04:43:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed419bf10 con 0x7f1ed41040c0 2024-04-02T04:43:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ebbfff700 1 -- 172.21.15.70:0/3661917702 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec800f5f0 con 0x7f1ed41040c0 2024-04-02T04:43:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.139+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed419c430 con 0x7f1ed41040c0 2024-04-02T04:43:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.140+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed4190630 con 0x7f1ed41040c0 2024-04-02T04:43:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.141+0000 7f1ebbfff700 1 -- 172.21.15.70:0/3661917702 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1ec8009a00 con 0x7f1ed41040c0 2024-04-02T04:43:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.141+0000 7f1ebbfff700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1ebc06c4e0 0x7f1ebc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.141+0000 7f1ebbfff700 1 -- 172.21.15.70:0/3661917702 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4342+0+0 (secure 0 0 0) 0x7f1ec808b270 con 0x7f1ed41040c0 2024-04-02T04:43:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.141+0000 7f1ed2ffd700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1ebc06c4e0 0x7f1ebc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.141+0000 7f1ed2ffd700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1ebc06c4e0 0x7f1ebc06e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006010 tx=0x7f1ec400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:31.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.146+0000 7f1ebbfff700 1 -- 172.21.15.70:0/3661917702 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1ec8059940 con 0x7f1ed41040c0 2024-04-02T04:43:31.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.335+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1ed4068af0 con 0x7f1ed41040c0 2024-04-02T04:43:31.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:31 smithi070 ceph-mon[29212]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T04:43:31.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:31 smithi070 ceph-mon[29212]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T04:43:31.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:31.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:31 smithi070 ceph-mon[29212]: from='osd.5 [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:43:31.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:31.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:31 smithi070 ceph-mon[29212]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:43:31.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:31 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:31.339 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:31 smithi070 ceph-mon[29212]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:31.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.336+0000 7f1ebbfff700 1 -- 172.21.15.70:0/3661917702 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f1ec80594d0 con 0x7f1ed41040c0 2024-04-02T04:43:31.340 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:43:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1ebc06c4e0 msgr2=0x7f1ebc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1ebc06c4e0 0x7f1ebc06e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006010 tx=0x7f1ec400b540 comp rx=0 tx=0).stop 2024-04-02T04:43:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed41040c0 msgr2=0x7f1ed4196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed41040c0 0x7f1ed4196d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800d8d0 tx=0x7f1ec800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:43:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 shutdown_connections 2024-04-02T04:43:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1ebc06c4e0 0x7f1ebc06e9a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ed4101790 0x7f1ed41967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 --2- 172.21.15.70:0/3661917702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed41040c0 0x7f1ed4196d00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:31.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 >> 172.21.15.70:0/3661917702 conn(0x7f1ed40fb160 msgr2=0x7f1ed40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:31.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 shutdown_connections 2024-04-02T04:43:31.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:31.339+0000 7f1ed95cf700 1 -- 172.21.15.70:0/3661917702 wait complete. 2024-04-02T04:43:31.410 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[55862]: 2024-04-02T04:43:31.166+0000 7fc0752f1700 -1 osd.5 0 waiting for initial osdmap 2024-04-02T04:43:31.410 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[55862]: 2024-04-02T04:43:31.174+0000 7fc0710e6700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:43:31.411 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-mon[36603]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T04:43:31.411 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-mon[36603]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T04:43:31.411 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:31.411 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-mon[36603]: from='osd.5 [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:43:31.411 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:31.411 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-mon[36603]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:43:31.411 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:31.411 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:31 smithi179 ceph-mon[36603]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:32.124 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1712032985,"num_in_osds":6,"osd_in_since":1712032992,"num_remapped_pgs":0} 2024-04-02T04:43:32.309 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:32 smithi179 ceph-mon[36603]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-02T04:43:32.309 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:32 smithi179 ceph-mon[36603]: osdmap e39: 6 total, 5 up, 6 in 2024-04-02T04:43:32.309 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:32 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:32.309 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:32 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:32.310 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:32 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3661917702' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T04:43:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:32 smithi070 ceph-mon[29212]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-02T04:43:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:32 smithi070 ceph-mon[29212]: osdmap e39: 6 total, 5 up, 6 in 2024-04-02T04:43:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:32 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:32 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:32 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3661917702' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T04:43:33.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T04:43:33.343 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:33.464 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:33 smithi179 ceph-mon[36603]: purged_snaps scrub starts 2024-04-02T04:43:33.464 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:33 smithi179 ceph-mon[36603]: purged_snaps scrub ok 2024-04-02T04:43:33.464 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:33 smithi179 ceph-mon[36603]: osd.5 [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] boot 2024-04-02T04:43:33.464 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:33 smithi179 ceph-mon[36603]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T04:43:33.464 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:33 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:33.464 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:33 smithi179 ceph-mon[36603]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:33.464 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:33 smithi070 ceph-mon[29212]: purged_snaps scrub starts 2024-04-02T04:43:33.465 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:33 smithi070 ceph-mon[29212]: purged_snaps scrub ok 2024-04-02T04:43:33.465 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:33 smithi070 ceph-mon[29212]: osd.5 [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] boot 2024-04-02T04:43:33.465 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:33 smithi070 ceph-mon[29212]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T04:43:33.465 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:33 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:33.465 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:33 smithi070 ceph-mon[29212]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:43:34.494 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T04:43:34.494 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.494 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.494 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:34.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:34.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:34.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:34.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:43:35.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.198+0000 7f1a3253f700 1 -- 172.21.15.70:0/1643743581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 msgr2=0x7f1a2c0ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.198+0000 7f1a3253f700 1 --2- 172.21.15.70:0/1643743581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 0x7f1a2c0ff380 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009910 tx=0x7f1a1c009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.198+0000 7f1a3253f700 1 -- 172.21.15.70:0/1643743581 shutdown_connections 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.198+0000 7f1a3253f700 1 --2- 172.21.15.70:0/1643743581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a2c101f40 0x7f1a2c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.198+0000 7f1a3253f700 1 --2- 172.21.15.70:0/1643743581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 0x7f1a2c0ff380 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.198+0000 7f1a3253f700 1 -- 172.21.15.70:0/1643743581 >> 172.21.15.70:0/1643743581 conn(0x7f1a2c0fa8c0 msgr2=0x7f1a2c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.198+0000 7f1a3253f700 1 -- 172.21.15.70:0/1643743581 shutdown_connections 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.198+0000 7f1a3253f700 1 -- 172.21.15.70:0/1643743581 wait complete. 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a3253f700 1 Processor -- start 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a3253f700 1 -- start start 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a3253f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 0x7f1a2c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a3253f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a2c101f40 0x7f1a2c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:35.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a3253f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c197010 con 0x7f1a2c0fef60 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a3253f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c197150 con 0x7f1a2c101f40 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a2bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 0x7f1a2c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a2bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 0x7f1a2c1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50244/0 (socket says 172.21.15.70:50244) 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a2bfff700 1 -- 172.21.15.70:0/3851389230 learned_addr learned my addr 172.21.15.70:0/3851389230 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a2b7fe700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a2c101f40 0x7f1a2c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a2bfff700 1 -- 172.21.15.70:0/3851389230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a2c101f40 msgr2=0x7f1a2c1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a2bfff700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a2c101f40 0x7f1a2c1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a2bfff700 1 -- 172.21.15.70:0/3851389230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c0095f0 con 0x7f1a2c0fef60 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.199+0000 7f1a2b7fe700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a2c101f40 0x7f1a2c1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.200+0000 7f1a2bfff700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 0x7f1a2c1964b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00bda0 tx=0x7f1a1c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:35.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.200+0000 7f1a297fa700 1 -- 172.21.15.70:0/3851389230 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1c01d070 con 0x7f1a2c0fef60 2024-04-02T04:43:35.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.200+0000 7f1a297fa700 1 -- 172.21.15.70:0/3851389230 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a1c004c50 con 0x7f1a2c0fef60 2024-04-02T04:43:35.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.200+0000 7f1a297fa700 1 -- 172.21.15.70:0/3851389230 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1c00f690 con 0x7f1a2c0fef60 2024-04-02T04:43:35.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.200+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c19bba0 con 0x7f1a2c0fef60 2024-04-02T04:43:35.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.200+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c19c040 con 0x7f1a2c0fef60 2024-04-02T04:43:35.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.201+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c109800 con 0x7f1a2c0fef60 2024-04-02T04:43:35.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.201+0000 7f1a297fa700 1 -- 172.21.15.70:0/3851389230 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1a1c0229b0 con 0x7f1a2c0fef60 2024-04-02T04:43:35.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.202+0000 7f1a297fa700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a1406c4e0 0x7f1a1406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:35.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.202+0000 7f1a297fa700 1 -- 172.21.15.70:0/3851389230 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1a1c08cb10 con 0x7f1a2c0fef60 2024-04-02T04:43:35.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.202+0000 7f1a2b7fe700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a1406c4e0 0x7f1a1406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:35.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.202+0000 7f1a2b7fe700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a1406c4e0 0x7f1a1406e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1a20005fd0 tx=0x7f1a20005dc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.206+0000 7f1a297fa700 1 -- 172.21.15.70:0/3851389230 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1a1c05b170 con 0x7f1a2c0fef60 2024-04-02T04:43:35.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.395+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1a2c19c710 con 0x7f1a2c0fef60 2024-04-02T04:43:35.398 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:35 smithi070 ceph-mon[29212]: Detected new or changed devices on smithi179 2024-04-02T04:43:35.399 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:35 smithi070 ceph-mon[29212]: Adjusting osd_memory_target on smithi179 to 4970M 2024-04-02T04:43:35.399 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:35 smithi070 ceph-mon[29212]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:35.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.395+0000 7f1a297fa700 1 -- 172.21.15.70:0/3851389230 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f1a1c05fe60 con 0x7f1a2c0fef60 2024-04-02T04:43:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:43:35.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.397+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a1406c4e0 msgr2=0x7f1a1406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:35.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.397+0000 7f1a3253f700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a1406c4e0 0x7f1a1406e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1a20005fd0 tx=0x7f1a20005dc0 comp rx=0 tx=0).stop 2024-04-02T04:43:35.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.397+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 msgr2=0x7f1a2c1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:35.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.397+0000 7f1a3253f700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 0x7f1a2c1964b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00bda0 tx=0x7f1a1c00be80 comp rx=0 tx=0).stop 2024-04-02T04:43:35.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.397+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 shutdown_connections 2024-04-02T04:43:35.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.397+0000 7f1a3253f700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a1406c4e0 0x7f1a1406e9a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:35.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.397+0000 7f1a3253f700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2c0fef60 0x7f1a2c1964b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:35.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.397+0000 7f1a3253f700 1 --2- 172.21.15.70:0/3851389230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a2c101f40 0x7f1a2c1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:35.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.398+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 >> 172.21.15.70:0/3851389230 conn(0x7f1a2c0fa8c0 msgr2=0x7f1a2c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:35.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.398+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 shutdown_connections 2024-04-02T04:43:35.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:35.398+0000 7f1a3253f700 1 -- 172.21.15.70:0/3851389230 wait complete. 2024-04-02T04:43:35.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:35 smithi179 ceph-mon[36603]: Detected new or changed devices on smithi179 2024-04-02T04:43:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:35 smithi179 ceph-mon[36603]: Adjusting osd_memory_target on smithi179 to 4970M 2024-04-02T04:43:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:35 smithi179 ceph-mon[36603]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:36.313 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1712033012,"num_in_osds":6,"osd_in_since":1712032992,"num_remapped_pgs":0} 2024-04-02T04:43:36.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T04:43:36.525 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:36.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:36 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3851389230' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T04:43:36.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:36 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3851389230' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T04:43:37.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:37 smithi070 ceph-mon[29212]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:37.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:37 smithi179 ceph-mon[36603]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:38.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.211+0000 7ff1d0c89700 1 -- 172.21.15.70:0/917663502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0ff5a0 msgr2=0x7ff1cc0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.211+0000 7ff1d0c89700 1 --2- 172.21.15.70:0/917663502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0ff5a0 0x7ff1cc0ff980 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009910 tx=0x7ff1bc009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.211+0000 7ff1d0c89700 1 -- 172.21.15.70:0/917663502 shutdown_connections 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.211+0000 7ff1d0c89700 1 --2- 172.21.15.70:0/917663502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1cc0fff50 0x7ff1cc105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.211+0000 7ff1d0c89700 1 --2- 172.21.15.70:0/917663502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0ff5a0 0x7ff1cc0ff980 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.211+0000 7ff1d0c89700 1 -- 172.21.15.70:0/917663502 >> 172.21.15.70:0/917663502 conn(0x7ff1cc074e30 msgr2=0x7ff1cc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.211+0000 7ff1d0c89700 1 -- 172.21.15.70:0/917663502 shutdown_connections 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.211+0000 7ff1d0c89700 1 -- 172.21.15.70:0/917663502 wait complete. 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1d0c89700 1 Processor -- start 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1d0c89700 1 -- start start 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1d0c89700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1cc0ff5a0 0x7ff1cc19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1d0c89700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0fff50 0x7ff1cc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1d0c89700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc19d2a0 con 0x7ff1cc0fff50 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1d0c89700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc196750 con 0x7ff1cc0ff5a0 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1c9d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0fff50 0x7ff1cc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1c9d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0fff50 0x7ff1cc19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54362/0 (socket says 172.21.15.70:54362) 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1c9d9b700 1 -- 172.21.15.70:0/3798860244 learned_addr learned my addr 172.21.15.70:0/3798860244 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1ca59c700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1cc0ff5a0 0x7ff1cc19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1c9d9b700 1 -- 172.21.15.70:0/3798860244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1cc0ff5a0 msgr2=0x7ff1cc19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1c9d9b700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1cc0ff5a0 0x7ff1cc19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1c9d9b700 1 -- 172.21.15.70:0/3798860244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0095f0 con 0x7ff1cc0fff50 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1ca59c700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1cc0ff5a0 0x7ff1cc19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1c9d9b700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0fff50 0x7ff1cc19cc10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b480 tx=0x7ff1c000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.212+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/3798860244 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1c00118c0 con 0x7ff1cc0fff50 2024-04-02T04:43:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.213+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/3798860244 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff1c0010380 con 0x7ff1cc0fff50 2024-04-02T04:43:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.213+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc196a30 con 0x7ff1cc0fff50 2024-04-02T04:43:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.213+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/3798860244 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1c000f4f0 con 0x7ff1cc0fff50 2024-04-02T04:43:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.213+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc196f80 con 0x7ff1cc0fff50 2024-04-02T04:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.214+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc109270 con 0x7ff1cc0fff50 2024-04-02T04:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.214+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/3798860244 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff1c0011a20 con 0x7ff1cc0fff50 2024-04-02T04:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.214+0000 7ff1bb7fe700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff1b406c4e0 0x7ff1b406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.214+0000 7ff1ca59c700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff1b406c4e0 0x7ff1b406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.214+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/3798860244 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff1c008b1a0 con 0x7ff1cc0fff50 2024-04-02T04:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.215+0000 7ff1ca59c700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff1b406c4e0 0x7ff1b406e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00b5c0 tx=0x7ff1bc005480 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.219+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/3798860244 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff1c0059750 con 0x7ff1cc0fff50 2024-04-02T04:43:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.409+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1cc04f2e0 con 0x7ff1cc0fff50 2024-04-02T04:43:38.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.410+0000 7ff1bb7fe700 1 -- 172.21.15.70:0/3798860244 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11337 (secure 0 0 0) 0x7ff1c00592e0 con 0x7ff1cc0fff50 2024-04-02T04:43:38.414 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:43:38.414 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:37:49.993595+0000","modified":"2024-04-02T04:43:33.164788+0000","last_up_change":"2024-04-02T04:43:32.163505+0000","last_in_change":"2024-04-02T04:43:12.453452+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:42:18.484513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b5e5510-c4db-4073-b470-c777b557d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6803","nonce":1163584079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6805","nonce":1163584079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6809","nonce":1163584079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6807","nonce":1163584079}]},"public_addr":"172.21.15.70:6803/1163584079","cluster_addr":"172.21.15.70:6805/1163584079","heartbeat_back_addr":"172.21.15.70:6809/1163584079","heartbeat_front_addr":"172.21.15.70:6807/1163584079","state":["exists","up"]},{"osd":1,"uuid":"b751ceaf-b734-4280-ba22-82683b9776ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6811","nonce":3081297537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6813","nonce":3081297537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3081297537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6815","nonce":3081297537}]},"public_addr":"172.21.15.70:6811/3081297537","cluster_addr":"172.21.15.70:6813/3081297537","heartbeat_back_addr":"172.21.15.70:6817/3081297537","heartbeat_front_addr":"172.21.15.70:6815/3081297537","state":["exists","up"]},{"osd":2,"uuid":"32080ab6-7809-4523-aab7-ca7a6d263ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6819","nonce":428679321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6821","nonce":428679321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6825","nonce":428679321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6823","nonce":428679321}]},"public_addr":"172.21.15.70:6819/428679321","cluster_addr":"172.21.15.70:6821/428679321","heartbeat_back_addr":"172.21.15.70:6825/428679321","heartbeat_front_addr":"172.21.15.70:6823/428679321","state":["exists","up"]},{"osd":3,"uuid":"0d71a0bb-5e7a-4339-afe7-06174dea4986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6801","nonce":1047738021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6803","nonce":1047738021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6807","nonce":1047738021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6805","nonce":1047738021}]},"public_addr":"172.21.15.179:6801/1047738021","cluster_addr":"172.21.15.179:6803/1047738021","heartbeat_back_addr":"172.21.15.179:6807/1047738021","heartbeat_front_addr":"172.21.15.179:6805/1047738021","state":["exists","up"]},{"osd":4,"uuid":"754e6f09-2a8e-42e2-a998-5f5a1addcb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6809","nonce":2251755550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6811","nonce":2251755550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6815","nonce":2251755550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6813","nonce":2251755550}]},"public_addr":"172.21.15.179:6809/2251755550","cluster_addr":"172.21.15.179:6811/2251755550","heartbeat_back_addr":"172.21.15.179:6815/2251755550","heartbeat_front_addr":"172.21.15.179:6813/2251755550","state":["exists","up"]},{"osd":5,"uuid":"46a73805-a770-4ed1-96f4-e64cf88810eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6817","nonce":2668632021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6819","nonce":2668632021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6823","nonce":2668632021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6821","nonce":2668632021}]},"public_addr":"172.21.15.179:6817/2668632021","cluster_addr":"172.21.15.179:6819/2668632021","heartbeat_back_addr":"172.21.15.179:6823/2668632021","heartbeat_front_addr":"172.21.15.179:6821/2668632021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:17.145441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:44.622049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:14.717560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:38.493463+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:04.319117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:30.171864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/110321404":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/2937797857":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6801/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1721127786":"2024-04-03T04:38:51.244832+0000","172.21.15.70:6800/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/4005781191":"2024-04-03T04:38:18.397033+0000","172.21.15.70:6800/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/4149690148":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/2446527261":"2024-04-03T04:38:18.397033+0000","172.21.15.70:6801/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:6801/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/3512109322":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/2463094374":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/940716355":"2024-04-03T04:38:18.397033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff1b406c4e0 msgr2=0x7ff1b406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff1b406c4e0 0x7ff1b406e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00b5c0 tx=0x7ff1bc005480 comp rx=0 tx=0).stop 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0fff50 msgr2=0x7ff1cc19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0fff50 0x7ff1cc19cc10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b480 tx=0x7ff1c000b840 comp rx=0 tx=0).stop 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 shutdown_connections 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff1b406c4e0 0x7ff1b406e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1cc0ff5a0 0x7ff1cc19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 --2- 172.21.15.70:0/3798860244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1cc0fff50 0x7ff1cc19cc10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 >> 172.21.15.70:0/3798860244 conn(0x7ff1cc074e30 msgr2=0x7ff1cc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 shutdown_connections 2024-04-02T04:43:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:38.412+0000 7ff1d0c89700 1 -- 172.21.15.70:0/3798860244 wait complete. 2024-04-02T04:43:39.032 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:42:18.484513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T04:43:39.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-02T04:43:39.238 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:39.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:39 smithi070 ceph-mon[29212]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:39.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:39 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3798860244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:43:39.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:39 smithi179 ceph-mon[36603]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:39.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:39 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3798860244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:43:40.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.946+0000 7fafb2642700 1 -- 172.21.15.70:0/3006268526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 msgr2=0x7fafac105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:40.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.946+0000 7fafb2642700 1 --2- 172.21.15.70:0/3006268526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 0x7fafac105fb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009910 tx=0x7faf9c009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:40.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.946+0000 7fafb2642700 1 -- 172.21.15.70:0/3006268526 shutdown_connections 2024-04-02T04:43:40.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.946+0000 7fafb2642700 1 --2- 172.21.15.70:0/3006268526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafac0ffc20 0x7fafac100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:40.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.946+0000 7fafb2642700 1 --2- 172.21.15.70:0/3006268526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 0x7fafac105fb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:40.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.946+0000 7fafb2642700 1 -- 172.21.15.70:0/3006268526 >> 172.21.15.70:0/3006268526 conn(0x7fafac075670 msgr2=0x7fafac075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.946+0000 7fafb2642700 1 -- 172.21.15.70:0/3006268526 shutdown_connections 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.946+0000 7fafb2642700 1 -- 172.21.15.70:0/3006268526 wait complete. 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafb2642700 1 Processor -- start 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafb2642700 1 -- start start 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafb2642700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafac0ffc20 0x7fafac101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafb2642700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 0x7fafac101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafb2642700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac1022c0 con 0x7fafac105bd0 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafb2642700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac102400 con 0x7fafac0ffc20 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafab7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 0x7fafac101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafab7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 0x7fafac101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54372/0 (socket says 172.21.15.70:54372) 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafab7fe700 1 -- 172.21.15.70:0/4040187874 learned_addr learned my addr 172.21.15.70:0/4040187874 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.947+0000 7fafabfff700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafac0ffc20 0x7fafac101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:40.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafab7fe700 1 -- 172.21.15.70:0/4040187874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafac0ffc20 msgr2=0x7fafac101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:40.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafab7fe700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafac0ffc20 0x7fafac101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:40.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafab7fe700 1 -- 172.21.15.70:0/4040187874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0095f0 con 0x7fafac105bd0 2024-04-02T04:43:40.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafab7fe700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 0x7fafac101d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fafa000c8a0 tx=0x7fafa000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:40.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafa97fa700 1 -- 172.21.15.70:0/4040187874 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa000cea0 con 0x7fafac105bd0 2024-04-02T04:43:40.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafa97fa700 1 -- 172.21.15.70:0/4040187874 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fafa0004830 con 0x7fafac105bd0 2024-04-02T04:43:40.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac1a2ac0 con 0x7fafac105bd0 2024-04-02T04:43:40.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafa97fa700 1 -- 172.21.15.70:0/4040187874 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa00055b0 con 0x7fafac105bd0 2024-04-02T04:43:40.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.948+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac1a2ed0 con 0x7fafac105bd0 2024-04-02T04:43:40.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.949+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac04f2e0 con 0x7fafac105bd0 2024-04-02T04:43:40.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.949+0000 7fafa97fa700 1 -- 172.21.15.70:0/4040187874 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fafa0007ad0 con 0x7fafac105bd0 2024-04-02T04:43:40.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.950+0000 7fafa97fa700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faf9406c4e0 0x7faf9406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:40.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.950+0000 7fafa97fa700 1 -- 172.21.15.70:0/4040187874 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fafa008b240 con 0x7fafac105bd0 2024-04-02T04:43:40.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.950+0000 7fafabfff700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faf9406c4e0 0x7faf9406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:40.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.950+0000 7fafabfff700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faf9406c4e0 0x7faf9406e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b5c0 tx=0x7faf9c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:40.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:40.954+0000 7fafa97fa700 1 -- 172.21.15.70:0/4040187874 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fafa0055d60 con 0x7fafac105bd0 2024-04-02T04:43:41.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.140+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fafac102d70 con 0x7fafac105bd0 2024-04-02T04:43:41.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.141+0000 7fafa97fa700 1 -- 172.21.15.70:0/4040187874 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fafac102d70 con 0x7fafac105bd0 2024-04-02T04:43:41.145 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faf9406c4e0 msgr2=0x7faf9406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faf9406c4e0 0x7faf9406e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b5c0 tx=0x7faf9c01a040 comp rx=0 tx=0).stop 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 msgr2=0x7fafac101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 0x7fafac101d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fafa000c8a0 tx=0x7fafa000cc60 comp rx=0 tx=0).stop 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 shutdown_connections 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faf9406c4e0 0x7faf9406e9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafac0ffc20 0x7fafac101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 --2- 172.21.15.70:0/4040187874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafac105bd0 0x7fafac101d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 >> 172.21.15.70:0/4040187874 conn(0x7fafac075670 msgr2=0x7fafac068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:41.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 shutdown_connections 2024-04-02T04:43:41.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:41.143+0000 7fafb2642700 1 -- 172.21.15.70:0/4040187874 wait complete. 2024-04-02T04:43:41.517 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:41 smithi070 ceph-mon[29212]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:41.517 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:41 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/4040187874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:43:41.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:41 smithi179 ceph-mon[36603]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:41 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/4040187874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:43:41.949 INFO:tasks.cephadm:Setting up client nodes... 2024-04-02T04:43:41.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T04:43:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:43.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:43 smithi070 ceph-mon[29212]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:43.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:43.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:43 smithi179 ceph-mon[36603]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.943+0000 7fb65bf38700 1 -- 172.21.15.70:0/2508460041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 msgr2=0x7fb654101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.943+0000 7fb65bf38700 1 --2- 172.21.15.70:0/2508460041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 0x7fb654101220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb650009910 tx=0x7fb650009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.943+0000 7fb65bf38700 1 -- 172.21.15.70:0/2508460041 shutdown_connections 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.943+0000 7fb65bf38700 1 --2- 172.21.15.70:0/2508460041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 0x7fb654101220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.943+0000 7fb65bf38700 1 --2- 172.21.15.70:0/2508460041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6540ffbc0 0x7fb6540fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.943+0000 7fb65bf38700 1 -- 172.21.15.70:0/2508460041 >> 172.21.15.70:0/2508460041 conn(0x7fb6540fb160 msgr2=0x7fb6540fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.943+0000 7fb65bf38700 1 -- 172.21.15.70:0/2508460041 shutdown_connections 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.943+0000 7fb65bf38700 1 -- 172.21.15.70:0/2508460041 wait complete. 2024-04-02T04:43:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb65bf38700 1 Processor -- start 2024-04-02T04:43:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb65bf38700 1 -- start start 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb65bf38700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6540ffbc0 0x7fb654195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb65bf38700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 0x7fb654195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb65bf38700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb654196260 con 0x7fb654100dc0 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb65bf38700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541963a0 con 0x7fb6540ffbc0 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb6594d3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 0x7fb654195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb6594d3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 0x7fb654195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54400/0 (socket says 172.21.15.70:54400) 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb6594d3700 1 -- 172.21.15.70:0/12419578 learned_addr learned my addr 172.21.15.70:0/12419578 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.944+0000 7fb659cd4700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6540ffbc0 0x7fb654195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb6594d3700 1 -- 172.21.15.70:0/12419578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6540ffbc0 msgr2=0x7fb654195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb6594d3700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6540ffbc0 0x7fb654195700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb6594d3700 1 -- 172.21.15.70:0/12419578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500095f0 con 0x7fb654100dc0 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb6594d3700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 0x7fb654195c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb650006010 tx=0x7fb65000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb64affd700 1 -- 172.21.15.70:0/12419578 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb65001d070 con 0x7fb654100dc0 2024-04-02T04:43:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb64affd700 1 -- 172.21.15.70:0/12419578 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb65000f460 con 0x7fb654100dc0 2024-04-02T04:43:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65419adf0 con 0x7fb654100dc0 2024-04-02T04:43:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb64affd700 1 -- 172.21.15.70:0/12419578 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb650017630 con 0x7fb654100dc0 2024-04-02T04:43:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.945+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb654069170 con 0x7fb654100dc0 2024-04-02T04:43:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.946+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb654069bb0 con 0x7fb654100dc0 2024-04-02T04:43:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.946+0000 7fb64affd700 1 -- 172.21.15.70:0/12419578 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb650003a60 con 0x7fb654100dc0 2024-04-02T04:43:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.947+0000 7fb64affd700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb64006c4e0 0x7fb64006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.947+0000 7fb659cd4700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb64006c4e0 0x7fb64006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.947+0000 7fb64affd700 1 -- 172.21.15.70:0/12419578 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb65008c920 con 0x7fb654100dc0 2024-04-02T04:43:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.947+0000 7fb659cd4700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb64006c4e0 0x7fb64006e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb644005950 tx=0x7fb6440058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:43.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:43.951+0000 7fb64affd700 1 -- 172.21.15.70:0/12419578 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb65005af50 con 0x7fb654100dc0 2024-04-02T04:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.215+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb65418f0f0 con 0x7fb654100dc0 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.219+0000 7fb64affd700 1 -- 172.21.15.70:0/12419578 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb65005aae0 con 0x7fb654100dc0 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQAAjQtm28j4DBAAq4Z2zpG+ep6WcqcLfRFx8Q== 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb64006c4e0 msgr2=0x7fb64006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb64006c4e0 0x7fb64006e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb644005950 tx=0x7fb6440058e0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 msgr2=0x7fb654195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 0x7fb654195c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb650006010 tx=0x7fb65000be20 comp rx=0 tx=0).stop 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 shutdown_connections 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb64006c4e0 0x7fb64006e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6540ffbc0 0x7fb654195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 --2- 172.21.15.70:0/12419578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb654100dc0 0x7fb654195c40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 >> 172.21.15.70:0/12419578 conn(0x7fb6540fb160 msgr2=0x7fb654104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:44.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 shutdown_connections 2024-04-02T04:43:44.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:44.222+0000 7fb65bf38700 1 -- 172.21.15.70:0/12419578 wait complete. 2024-04-02T04:43:44.524 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:44 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/12419578' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:43:44.524 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:44 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/12419578' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T04:43:44.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:44 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/12419578' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:43:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:44 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/12419578' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T04:43:44.956 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:43:44.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-02T04:43:44.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T04:43:45.001 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T04:43:45.217 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi179/config 2024-04-02T04:43:45.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:45 smithi179 ceph-mon[36603]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:45.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:45 smithi070 ceph-mon[29212]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:46.820 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.815+0000 7ff5e589c700 1 -- 172.21.15.179:0/3121887328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 msgr2=0x7ff5e0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.815+0000 7ff5e589c700 1 --2- 172.21.15.179:0/3121887328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 0x7ff5e0102310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009910 tx=0x7ff5d4009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.815+0000 7ff5e589c700 1 -- 172.21.15.179:0/3121887328 shutdown_connections 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.815+0000 7ff5e589c700 1 --2- 172.21.15.179:0/3121887328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 0x7ff5e0102310 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.815+0000 7ff5e589c700 1 --2- 172.21.15.179:0/3121887328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5e0100cb0 0x7ff5e01010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.815+0000 7ff5e589c700 1 -- 172.21.15.179:0/3121887328 >> 172.21.15.179:0/3121887328 conn(0x7ff5e00fc230 msgr2=0x7ff5e00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.815+0000 7ff5e589c700 1 -- 172.21.15.179:0/3121887328 shutdown_connections 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.816+0000 7ff5e589c700 1 -- 172.21.15.179:0/3121887328 wait complete. 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.816+0000 7ff5e589c700 1 Processor -- start 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.816+0000 7ff5e589c700 1 -- start start 2024-04-02T04:43:46.821 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.816+0000 7ff5e589c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5e0100cb0 0x7ff5e0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.816+0000 7ff5e589c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 0x7ff5e0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.816+0000 7ff5e589c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00736c0 con 0x7ff5e0100cb0 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.816+0000 7ff5e589c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0073800 con 0x7ff5e0101eb0 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.816+0000 7ff5de7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 0x7ff5e0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5de7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 0x7ff5e0073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:57256/0 (socket says 172.21.15.179:57256) 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5de7fc700 1 -- 172.21.15.179:0/2759921936 learned_addr learned my addr 172.21.15.179:0/2759921936 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5deffd700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5e0100cb0 0x7ff5e0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5de7fc700 1 -- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5e0100cb0 msgr2=0x7ff5e0074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5de7fc700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5e0100cb0 0x7ff5e0074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5de7fc700 1 -- 172.21.15.179:0/2759921936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d40095f0 con 0x7ff5e0101eb0 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5de7fc700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 0x7ff5e0073180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400bd70 tx=0x7ff5d400be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5e489a700 1 -- 172.21.15.179:0/2759921936 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5d401d070 con 0x7ff5e0101eb0 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5e489a700 1 -- 172.21.15.179:0/2759921936 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff5d4022470 con 0x7ff5e0101eb0 2024-04-02T04:43:46.822 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e0073a80 con 0x7ff5e0101eb0 2024-04-02T04:43:46.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5e489a700 1 -- 172.21.15.179:0/2759921936 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5d400f670 con 0x7ff5e0101eb0 2024-04-02T04:43:46.823 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.817+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e0073f70 con 0x7ff5e0101eb0 2024-04-02T04:43:46.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.818+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e0109810 con 0x7ff5e0101eb0 2024-04-02T04:43:46.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.819+0000 7ff5e489a700 1 -- 172.21.15.179:0/2759921936 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff5d400f8e0 con 0x7ff5e0101eb0 2024-04-02T04:43:46.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.819+0000 7ff5e489a700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff5c806c4e0 0x7ff5c806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:46.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.819+0000 7ff5e489a700 1 -- 172.21.15.179:0/2759921936 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff5d408ce50 con 0x7ff5e0101eb0 2024-04-02T04:43:46.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.819+0000 7ff5deffd700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff5c806c4e0 0x7ff5c806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:46.824 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.820+0000 7ff5deffd700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff5c806c4e0 0x7ff5c806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0005fd0 tx=0x7ff5d0005e50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:46.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:46.823+0000 7ff5e489a700 1 -- 172.21.15.179:0/2759921936 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff5d405b5d0 con 0x7ff5e0101eb0 2024-04-02T04:43:47.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:46 smithi070 ceph-mon[29212]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:47.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:46 smithi179 ceph-mon[36603]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:47.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.089+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff5e019ec10 con 0x7ff5e0101eb0 2024-04-02T04:43:47.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.094+0000 7ff5e489a700 1 -- 172.21.15.179:0/2759921936 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff5d4027020 con 0x7ff5e0101eb0 2024-04-02T04:43:47.098 INFO:teuthology.orchestra.run.smithi179.stdout:[client.1] 2024-04-02T04:43:47.098 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQADjQtmkHFtBRAAuU4I8KnsaSo/uwS6NbbY3g== 2024-04-02T04:43:47.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff5c806c4e0 msgr2=0x7ff5c806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:47.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff5c806c4e0 0x7ff5c806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0005fd0 tx=0x7ff5d0005e50 comp rx=0 tx=0).stop 2024-04-02T04:43:47.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 msgr2=0x7ff5e0073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:47.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 0x7ff5e0073180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400bd70 tx=0x7ff5d400be50 comp rx=0 tx=0).stop 2024-04-02T04:43:47.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 shutdown_connections 2024-04-02T04:43:47.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff5c806c4e0 0x7ff5c806e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:47.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5e0100cb0 0x7ff5e0074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:47.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 --2- 172.21.15.179:0/2759921936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5e0101eb0 0x7ff5e0073180 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:47.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 >> 172.21.15.179:0/2759921936 conn(0x7ff5e00fc230 msgr2=0x7ff5e01050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:47.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 shutdown_connections 2024-04-02T04:43:47.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:43:47.097+0000 7ff5e589c700 1 -- 172.21.15.179:0/2759921936 wait complete. 2024-04-02T04:43:47.704 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:43:47.704 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-02T04:43:47.704 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-02T04:43:47.746 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T04:43:47.746 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-02T04:43:47.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-02T04:43:47.963 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:48.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:47 smithi070 ceph-mon[29212]: from='client.? 172.21.15.179:0/2759921936' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:43:48.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:47 smithi070 ceph-mon[29212]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:43:48.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:47 smithi070 ceph-mon[29212]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T04:43:48.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:47 smithi179 ceph-mon[36603]: from='client.? 172.21.15.179:0/2759921936' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:43:48.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:47 smithi179 ceph-mon[36603]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:43:48.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:47 smithi179 ceph-mon[36603]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T04:43:49.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:48 smithi070 ceph-mon[29212]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:49.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:48 smithi179 ceph-mon[36603]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.622+0000 7f8858664700 1 -- 172.21.15.70:0/1668486673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 msgr2=0x7f8850100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.622+0000 7f8858664700 1 --2- 172.21.15.70:0/1668486673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 0x7f8850100080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8840009910 tx=0x7f8840009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.622+0000 7f8858664700 1 -- 172.21.15.70:0/1668486673 shutdown_connections 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.622+0000 7f8858664700 1 --2- 172.21.15.70:0/1668486673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 0x7f8850100080 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 --2- 172.21.15.70:0/1668486673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8850105bd0 0x7f8850105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 -- 172.21.15.70:0/1668486673 >> 172.21.15.70:0/1668486673 conn(0x7f8850075670 msgr2=0x7f8850075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 -- 172.21.15.70:0/1668486673 shutdown_connections 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 -- 172.21.15.70:0/1668486673 wait complete. 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 Processor -- start 2024-04-02T04:43:49.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 -- start start 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 0x7f885019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8850105bd0 0x7f885019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885019d2c0 con 0x7f88500ffc20 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.623+0000 7f8858664700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8850196740 con 0x7f8850105bd0 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8856400700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 0x7f885019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8856400700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 0x7f885019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35598/0 (socket says 172.21.15.70:35598) 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8856400700 1 -- 172.21.15.70:0/1233151314 learned_addr learned my addr 172.21.15.70:0/1233151314 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8855bff700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8850105bd0 0x7f885019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8856400700 1 -- 172.21.15.70:0/1233151314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8850105bd0 msgr2=0x7f885019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8856400700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8850105bd0 0x7f885019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8856400700 1 -- 172.21.15.70:0/1233151314 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400095f0 con 0x7f88500ffc20 2024-04-02T04:43:49.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8855bff700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8850105bd0 0x7f885019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:43:49.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8856400700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 0x7f885019c6a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f884c00d8d0 tx=0x7f884c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:49.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f88477fe700 1 -- 172.21.15.70:0/1233151314 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c009940 con 0x7f88500ffc20 2024-04-02T04:43:49.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8850196a20 con 0x7f88500ffc20 2024-04-02T04:43:49.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f88477fe700 1 -- 172.21.15.70:0/1233151314 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f884c010460 con 0x7f88500ffc20 2024-04-02T04:43:49.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f88477fe700 1 -- 172.21.15.70:0/1233151314 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c00f5f0 con 0x7f88500ffc20 2024-04-02T04:43:49.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.624+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8850196f70 con 0x7f88500ffc20 2024-04-02T04:43:49.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.625+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885004f2e0 con 0x7f88500ffc20 2024-04-02T04:43:49.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.626+0000 7f88477fe700 1 -- 172.21.15.70:0/1233151314 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f884c009aa0 con 0x7f88500ffc20 2024-04-02T04:43:49.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.626+0000 7f88477fe700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f883c06c4e0 0x7f883c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:49.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.626+0000 7f88477fe700 1 -- 172.21.15.70:0/1233151314 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f884c08b520 con 0x7f88500ffc20 2024-04-02T04:43:49.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.626+0000 7f8855bff700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f883c06c4e0 0x7f883c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:49.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.626+0000 7f8855bff700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f883c06c4e0 0x7f883c06e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8840005b40 tx=0x7f884001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:49.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.630+0000 7f88477fe700 1 -- 172.21.15.70:0/1233151314 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f884c059ad0 con 0x7f88500ffc20 2024-04-02T04:43:49.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.878+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8850197280 con 0x7f88500ffc20 2024-04-02T04:43:49.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.881+0000 7f88477fe700 1 -- 172.21.15.70:0/1233151314 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173037 (secure 0 0 0) 0x7f884c059660 con 0x7f88500ffc20 2024-04-02T04:43:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:43:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f883c06c4e0 msgr2=0x7f883c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f883c06c4e0 0x7f883c06e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8840005b40 tx=0x7f884001a040 comp rx=0 tx=0).stop 2024-04-02T04:43:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 msgr2=0x7f885019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 0x7f885019c6a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f884c00d8d0 tx=0x7f884c00dc90 comp rx=0 tx=0).stop 2024-04-02T04:43:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 shutdown_connections 2024-04-02T04:43:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f883c06c4e0 0x7f883c06e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ffc20 0x7f885019c6a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 --2- 172.21.15.70:0/1233151314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8850105bd0 0x7f885019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 >> 172.21.15.70:0/1233151314 conn(0x7f8850075670 msgr2=0x7f8850103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 shutdown_connections 2024-04-02T04:43:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:49.884+0000 7f8858664700 1 -- 172.21.15.70:0/1233151314 wait complete. 2024-04-02T04:43:50.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:49 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1233151314' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T04:43:50.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:49 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1233151314' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T04:43:50.613 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":19,"active_gid":14213,"active_name":"smithi070.igidvr","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":2814127860},{"type":"v1","addr":"172.21.15.70:6801","nonce":2814127860}]},"active_addr":"172.21.15.70:6801/2814127860","active_change":"2024-04-02T04:39:58.163282+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14238,"name":"smithi179.labmvs","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.70:8443/","prometheus":"http://172.21.15.70:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1353385001}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1700662291}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":39315361}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2716917447}]}]} 2024-04-02T04:43:50.622 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-02T04:43:50.622 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-02T04:43:50.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T04:43:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:51.195 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:50 smithi070 ceph-mon[29212]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:51.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:50 smithi179 ceph-mon[36603]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.371+0000 7f4925605700 1 -- 172.21.15.70:0/2344188448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 msgr2=0x7f4920100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.371+0000 7f4925605700 1 --2- 172.21.15.70:0/2344188448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 0x7f4920100080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4914009910 tx=0x7f4914009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.371+0000 7f4925605700 1 -- 172.21.15.70:0/2344188448 shutdown_connections 2024-04-02T04:43:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.371+0000 7f4925605700 1 --2- 172.21.15.70:0/2344188448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 0x7f4920100080 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.371+0000 7f4925605700 1 --2- 172.21.15.70:0/2344188448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4920105bd0 0x7f4920105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.371+0000 7f4925605700 1 -- 172.21.15.70:0/2344188448 >> 172.21.15.70:0/2344188448 conn(0x7f4920075670 msgr2=0x7f4920075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.371+0000 7f4925605700 1 -- 172.21.15.70:0/2344188448 shutdown_connections 2024-04-02T04:43:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.371+0000 7f4925605700 1 -- 172.21.15.70:0/2344188448 wait complete. 2024-04-02T04:43:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f4925605700 1 Processor -- start 2024-04-02T04:43:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f4925605700 1 -- start start 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f4925605700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 0x7f4920197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f4925605700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4920105bd0 0x7f4920197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f4925605700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4920198280 con 0x7f49200ffc20 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f4925605700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49201983c0 con 0x7f4920105bd0 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f491effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 0x7f4920197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f491effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 0x7f4920197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35614/0 (socket says 172.21.15.70:35614) 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.372+0000 7f491effd700 1 -- 172.21.15.70:0/3618745826 learned_addr learned my addr 172.21.15.70:0/3618745826 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f491effd700 1 -- 172.21.15.70:0/3618745826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4920105bd0 msgr2=0x7f4920197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f491e7fc700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4920105bd0 0x7f4920197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f491effd700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4920105bd0 0x7f4920197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f491effd700 1 -- 172.21.15.70:0/3618745826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140095f0 con 0x7f49200ffc20 2024-04-02T04:43:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f491effd700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 0x7f4920197720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f491000b4b0 tx=0x7f491000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f4907fff700 1 -- 172.21.15.70:0/3618745826 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4910011800 con 0x7f49200ffc20 2024-04-02T04:43:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49201a0790 con 0x7f49200ffc20 2024-04-02T04:43:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f4907fff700 1 -- 172.21.15.70:0/3618745826 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4910011e10 con 0x7f49200ffc20 2024-04-02T04:43:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f4907fff700 1 -- 172.21.15.70:0/3618745826 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491000f2e0 con 0x7f49200ffc20 2024-04-02T04:43:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.373+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49201a0c50 con 0x7f49200ffc20 2024-04-02T04:43:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.374+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49201080d0 con 0x7f49200ffc20 2024-04-02T04:43:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.375+0000 7f4907fff700 1 -- 172.21.15.70:0/3618745826 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4910010380 con 0x7f49200ffc20 2024-04-02T04:43:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.375+0000 7f4907fff700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f490806c4e0 0x7f490806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.375+0000 7f4907fff700 1 -- 172.21.15.70:0/3618745826 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f491008b130 con 0x7f49200ffc20 2024-04-02T04:43:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.375+0000 7f491e7fc700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f490806c4e0 0x7f490806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.375+0000 7f491e7fc700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f490806c4e0 0x7f490806e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f491400b5c0 tx=0x7f4914005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:52.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.380+0000 7f4907fff700 1 -- 172.21.15.70:0/3618745826 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4910055c50 con 0x7f49200ffc20 2024-04-02T04:43:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.571+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f492004f2e0 con 0x7f49200ffc20 2024-04-02T04:43:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.571+0000 7f4907fff700 1 -- 172.21.15.70:0/3618745826 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11337 (secure 0 0 0) 0x7f4910059270 con 0x7f49200ffc20 2024-04-02T04:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:37:49.993595+0000","modified":"2024-04-02T04:43:33.164788+0000","last_up_change":"2024-04-02T04:43:32.163505+0000","last_in_change":"2024-04-02T04:43:12.453452+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:42:18.484513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b5e5510-c4db-4073-b470-c777b557d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6803","nonce":1163584079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6805","nonce":1163584079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6809","nonce":1163584079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6807","nonce":1163584079}]},"public_addr":"172.21.15.70:6803/1163584079","cluster_addr":"172.21.15.70:6805/1163584079","heartbeat_back_addr":"172.21.15.70:6809/1163584079","heartbeat_front_addr":"172.21.15.70:6807/1163584079","state":["exists","up"]},{"osd":1,"uuid":"b751ceaf-b734-4280-ba22-82683b9776ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6811","nonce":3081297537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6813","nonce":3081297537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3081297537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6815","nonce":3081297537}]},"public_addr":"172.21.15.70:6811/3081297537","cluster_addr":"172.21.15.70:6813/3081297537","heartbeat_back_addr":"172.21.15.70:6817/3081297537","heartbeat_front_addr":"172.21.15.70:6815/3081297537","state":["exists","up"]},{"osd":2,"uuid":"32080ab6-7809-4523-aab7-ca7a6d263ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6819","nonce":428679321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6821","nonce":428679321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6825","nonce":428679321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6823","nonce":428679321}]},"public_addr":"172.21.15.70:6819/428679321","cluster_addr":"172.21.15.70:6821/428679321","heartbeat_back_addr":"172.21.15.70:6825/428679321","heartbeat_front_addr":"172.21.15.70:6823/428679321","state":["exists","up"]},{"osd":3,"uuid":"0d71a0bb-5e7a-4339-afe7-06174dea4986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6801","nonce":1047738021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6803","nonce":1047738021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6807","nonce":1047738021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6805","nonce":1047738021}]},"public_addr":"172.21.15.179:6801/1047738021","cluster_addr":"172.21.15.179:6803/1047738021","heartbeat_back_addr":"172.21.15.179:6807/1047738021","heartbeat_front_addr":"172.21.15.179:6805/1047738021","state":["exists","up"]},{"osd":4,"uuid":"754e6f09-2a8e-42e2-a998-5f5a1addcb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6809","nonce":2251755550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6811","nonce":2251755550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6815","nonce":2251755550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6813","nonce":2251755550}]},"public_addr":"172.21.15.179:6809/2251755550","cluster_addr":"172.21.15.179:6811/2251755550","heartbeat_back_addr":"172.21.15.179:6815/2251755550","heartbeat_front_addr":"172.21.15.179:6813/2251755550","state":["exists","up"]},{"osd":5,"uuid":"46a73805-a770-4ed1-96f4-e64cf88810eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6817","nonce":2668632021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6819","nonce":2668632021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6823","nonce":2668632021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6821","nonce":2668632021}]},"public_addr":"172.21.15.179:6817/2668632021","cluster_addr":"172.21.15.179:6819/2668632021","heartbeat_back_addr":"172.21.15.179:6823/2668632021","heartbeat_front_addr":"172.21.15.179:6821/2668632021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:17.145441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:44.622049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:14.717560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:38.493463+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:04.319117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:30.171864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/110321404":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/2937797857":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6801/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1721127786":"2024-04-03T04:38:51.244832+0000","172.21.15.70:6800/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/4005781191":"2024-04-03T04:38:18.397033+0000","172.21.15.70:6800/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/4149690148":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/2446527261":"2024-04-03T04:38:18.397033+0000","172.21.15.70:6801/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:6801/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/3512109322":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/2463094374":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/940716355":"2024-04-03T04:38:18.397033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:43:52.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.573+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f490806c4e0 msgr2=0x7f490806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:52.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.573+0000 7f4925605700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f490806c4e0 0x7f490806e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f491400b5c0 tx=0x7f4914005960 comp rx=0 tx=0).stop 2024-04-02T04:43:52.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.573+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 msgr2=0x7f4920197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:52.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.573+0000 7f4925605700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 0x7f4920197720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f491000b4b0 tx=0x7f491000b870 comp rx=0 tx=0).stop 2024-04-02T04:43:52.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.573+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 shutdown_connections 2024-04-02T04:43:52.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.573+0000 7f4925605700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f490806c4e0 0x7f490806e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:52.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.573+0000 7f4925605700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49200ffc20 0x7f4920197720 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:52.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.573+0000 7f4925605700 1 --2- 172.21.15.70:0/3618745826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4920105bd0 0x7f4920197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:52.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.574+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 >> 172.21.15.70:0/3618745826 conn(0x7f4920075670 msgr2=0x7f49200fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:52.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.574+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 shutdown_connections 2024-04-02T04:43:52.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:52.574+0000 7f4925605700 1 -- 172.21.15.70:0/3618745826 wait complete. 2024-04-02T04:43:53.250 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-02T04:43:53.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T04:43:53.457 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:53.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:53 smithi070 ceph-mon[29212]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:53.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:53 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3618745826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:43:53.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:53 smithi179 ceph-mon[36603]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:53 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3618745826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:43:55.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.157+0000 7fd130445700 1 -- 172.21.15.70:0/798734053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd128105bd0 msgr2=0x7fd128105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:55.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.157+0000 7fd130445700 1 --2- 172.21.15.70:0/798734053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd128105bd0 0x7fd128105fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b0a0 tx=0x7fd12000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:43:55.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.157+0000 7fd130445700 1 -- 172.21.15.70:0/798734053 shutdown_connections 2024-04-02T04:43:55.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.157+0000 7fd130445700 1 --2- 172.21.15.70:0/798734053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1280ffc20 0x7fd128100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.157+0000 7fd130445700 1 --2- 172.21.15.70:0/798734053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd128105bd0 0x7fd128105fb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.157+0000 7fd130445700 1 -- 172.21.15.70:0/798734053 >> 172.21.15.70:0/798734053 conn(0x7fd128075670 msgr2=0x7fd128075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.157+0000 7fd130445700 1 -- 172.21.15.70:0/798734053 shutdown_connections 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.157+0000 7fd130445700 1 -- 172.21.15.70:0/798734053 wait complete. 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd130445700 1 Processor -- start 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd130445700 1 -- start start 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd130445700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1280ffc20 0x7fd12810bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd130445700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd128105bd0 0x7fd12810c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd130445700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12810c790 con 0x7fd1280ffc20 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd130445700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12810c8d0 con 0x7fd128105bd0 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd12e1e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1280ffc20 0x7fd12810bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd12e1e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1280ffc20 0x7fd12810bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35628/0 (socket says 172.21.15.70:35628) 2024-04-02T04:43:55.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd12e1e1700 1 -- 172.21.15.70:0/2429083005 learned_addr learned my addr 172.21.15.70:0/2429083005 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd12e1e1700 1 -- 172.21.15.70:0/2429083005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd128105bd0 msgr2=0x7fd12810c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd12e1e1700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd128105bd0 0x7fd12810c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.158+0000 7fd12e1e1700 1 -- 172.21.15.70:0/2429083005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120009d00 con 0x7fd1280ffc20 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.159+0000 7fd12e1e1700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1280ffc20 0x7fd12810bd10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd120000f80 tx=0x7fd12000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.159+0000 7fd11f7fe700 1 -- 172.21.15.70:0/2429083005 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12000e040 con 0x7fd1280ffc20 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.159+0000 7fd11f7fe700 1 -- 172.21.15.70:0/2429083005 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1200044f0 con 0x7fd1280ffc20 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.159+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12810cb50 con 0x7fd1280ffc20 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.159+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1281a2e40 con 0x7fd1280ffc20 2024-04-02T04:43:55.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.159+0000 7fd11f7fe700 1 -- 172.21.15.70:0/2429083005 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12001d9f0 con 0x7fd1280ffc20 2024-04-02T04:43:55.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.160+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1281080f0 con 0x7fd1280ffc20 2024-04-02T04:43:55.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.160+0000 7fd11f7fe700 1 -- 172.21.15.70:0/2429083005 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd12001db50 con 0x7fd1280ffc20 2024-04-02T04:43:55.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.160+0000 7fd11f7fe700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd11406c4e0 0x7fd11406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:55.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.161+0000 7fd11f7fe700 1 -- 172.21.15.70:0/2429083005 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd12008cf40 con 0x7fd1280ffc20 2024-04-02T04:43:55.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.161+0000 7fd12d9e0700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd11406c4e0 0x7fd11406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:55.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.161+0000 7fd12d9e0700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd11406c4e0 0x7fd11406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd12810d4b0 tx=0x7fd118006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:55.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.165+0000 7fd11f7fe700 1 -- 172.21.15.70:0/2429083005 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd1200c00e0 con 0x7fd1280ffc20 2024-04-02T04:43:55.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.357+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd12804f2e0 con 0x7fd1280ffc20 2024-04-02T04:43:55.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.358+0000 7fd11f7fe700 1 -- 172.21.15.70:0/2429083005 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11337 (secure 0 0 0) 0x7fd120057ad0 con 0x7fd1280ffc20 2024-04-02T04:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:37:49.993595+0000","modified":"2024-04-02T04:43:33.164788+0000","last_up_change":"2024-04-02T04:43:32.163505+0000","last_in_change":"2024-04-02T04:43:12.453452+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:42:18.484513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b5e5510-c4db-4073-b470-c777b557d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6803","nonce":1163584079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6805","nonce":1163584079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6809","nonce":1163584079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6807","nonce":1163584079}]},"public_addr":"172.21.15.70:6803/1163584079","cluster_addr":"172.21.15.70:6805/1163584079","heartbeat_back_addr":"172.21.15.70:6809/1163584079","heartbeat_front_addr":"172.21.15.70:6807/1163584079","state":["exists","up"]},{"osd":1,"uuid":"b751ceaf-b734-4280-ba22-82683b9776ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6811","nonce":3081297537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6813","nonce":3081297537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3081297537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6815","nonce":3081297537}]},"public_addr":"172.21.15.70:6811/3081297537","cluster_addr":"172.21.15.70:6813/3081297537","heartbeat_back_addr":"172.21.15.70:6817/3081297537","heartbeat_front_addr":"172.21.15.70:6815/3081297537","state":["exists","up"]},{"osd":2,"uuid":"32080ab6-7809-4523-aab7-ca7a6d263ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6819","nonce":428679321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6821","nonce":428679321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6825","nonce":428679321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6823","nonce":428679321}]},"public_addr":"172.21.15.70:6819/428679321","cluster_addr":"172.21.15.70:6821/428679321","heartbeat_back_addr":"172.21.15.70:6825/428679321","heartbeat_front_addr":"172.21.15.70:6823/428679321","state":["exists","up"]},{"osd":3,"uuid":"0d71a0bb-5e7a-4339-afe7-06174dea4986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6801","nonce":1047738021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6803","nonce":1047738021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6807","nonce":1047738021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6805","nonce":1047738021}]},"public_addr":"172.21.15.179:6801/1047738021","cluster_addr":"172.21.15.179:6803/1047738021","heartbeat_back_addr":"172.21.15.179:6807/1047738021","heartbeat_front_addr":"172.21.15.179:6805/1047738021","state":["exists","up"]},{"osd":4,"uuid":"754e6f09-2a8e-42e2-a998-5f5a1addcb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6809","nonce":2251755550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6811","nonce":2251755550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6815","nonce":2251755550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6813","nonce":2251755550}]},"public_addr":"172.21.15.179:6809/2251755550","cluster_addr":"172.21.15.179:6811/2251755550","heartbeat_back_addr":"172.21.15.179:6815/2251755550","heartbeat_front_addr":"172.21.15.179:6813/2251755550","state":["exists","up"]},{"osd":5,"uuid":"46a73805-a770-4ed1-96f4-e64cf88810eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6817","nonce":2668632021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6819","nonce":2668632021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6823","nonce":2668632021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6821","nonce":2668632021}]},"public_addr":"172.21.15.179:6817/2668632021","cluster_addr":"172.21.15.179:6819/2668632021","heartbeat_back_addr":"172.21.15.179:6823/2668632021","heartbeat_front_addr":"172.21.15.179:6821/2668632021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:17.145441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:44.622049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:14.717560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:38.493463+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:04.319117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:30.171864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/110321404":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/2937797857":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6801/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1721127786":"2024-04-03T04:38:51.244832+0000","172.21.15.70:6800/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/4005781191":"2024-04-03T04:38:18.397033+0000","172.21.15.70:6800/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/4149690148":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/2446527261":"2024-04-03T04:38:18.397033+0000","172.21.15.70:6801/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:6801/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/3512109322":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/2463094374":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/940716355":"2024-04-03T04:38:18.397033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:43:55.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd11406c4e0 msgr2=0x7fd11406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd11406c4e0 0x7fd11406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd12810d4b0 tx=0x7fd118006d20 comp rx=0 tx=0).stop 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1280ffc20 msgr2=0x7fd12810bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1280ffc20 0x7fd12810bd10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd120000f80 tx=0x7fd12000cdb0 comp rx=0 tx=0).stop 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 shutdown_connections 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd11406c4e0 0x7fd11406e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1280ffc20 0x7fd12810bd10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 --2- 172.21.15.70:0/2429083005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd128105bd0 0x7fd12810c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 >> 172.21.15.70:0/2429083005 conn(0x7fd128075670 msgr2=0x7fd1280fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 shutdown_connections 2024-04-02T04:43:55.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:43:55.360+0000 7fd130445700 1 -- 172.21.15.70:0/2429083005 wait complete. 2024-04-02T04:43:55.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:55 smithi179 ceph-mon[36603]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:55.698 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:55 smithi070 ceph-mon[29212]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:56.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-02T04:43:56.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-02T04:43:56.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-02T04:43:56.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-02T04:43:56.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-02T04:43:56.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-02T04:43:56.414 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:56.415 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:56.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:56 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2429083005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:43:56.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:56 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2429083005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:43:56.701 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:56.701 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:56.702 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:57.092 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:43:57.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:57 smithi070 ceph-mon[29212]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:57.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:57 smithi179 ceph-mon[36603]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:58.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:58.698 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:59.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:43:59 smithi179 ceph-mon[36603]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:43:59.628 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:43:59 smithi070 ceph-mon[29212]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:01.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:01 smithi179 ceph-mon[36603]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:01.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:01 smithi070 ceph-mon[29212]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.698+0000 7f5c295c0700 1 -- 172.21.15.70:0/1872606335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c24105bd0 msgr2=0x7f5c24105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.698+0000 7f5c295c0700 1 --2- 172.21.15.70:0/1872606335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c24105bd0 0x7f5c24105fb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009910 tx=0x7f5c18009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.698+0000 7f5c295c0700 1 -- 172.21.15.70:0/1872606335 shutdown_connections 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.698+0000 7f5c295c0700 1 --2- 172.21.15.70:0/1872606335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c240ffc20 0x7f5c24100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.698+0000 7f5c295c0700 1 --2- 172.21.15.70:0/1872606335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c24105bd0 0x7f5c24105fb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.698+0000 7f5c295c0700 1 -- 172.21.15.70:0/1872606335 >> 172.21.15.70:0/1872606335 conn(0x7f5c24075670 msgr2=0x7f5c24075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.698+0000 7f5c295c0700 1 -- 172.21.15.70:0/1872606335 shutdown_connections 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.698+0000 7f5c295c0700 1 -- 172.21.15.70:0/1872606335 wait complete. 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c295c0700 1 Processor -- start 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c295c0700 1 -- start start 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c295c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c240ffc20 0x7f5c24101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:02.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c295c0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c24105bd0 0x7f5c24101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c295c0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c241022c0 con 0x7f5c240ffc20 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c295c0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c24102400 con 0x7f5c24105bd0 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c22ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c240ffc20 0x7f5c24101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c22ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c240ffc20 0x7f5c24101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36862/0 (socket says 172.21.15.70:36862) 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c22ffd700 1 -- 172.21.15.70:0/3320186047 learned_addr learned my addr 172.21.15.70:0/3320186047 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c227fc700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c24105bd0 0x7f5c24101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c22ffd700 1 -- 172.21.15.70:0/3320186047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c24105bd0 msgr2=0x7f5c24101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c22ffd700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c24105bd0 0x7f5c24101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c22ffd700 1 -- 172.21.15.70:0/3320186047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c180095f0 con 0x7f5c240ffc20 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.699+0000 7f5c22ffd700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c240ffc20 0x7f5c24101840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5c18006010 tx=0x7f5c1800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.700+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c1801d070 con 0x7f5c240ffc20 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.700+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c24102680 con 0x7f5c240ffc20 2024-04-02T04:44:02.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.700+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5c1800f460 con 0x7f5c240ffc20 2024-04-02T04:44:02.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.700+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c18017630 con 0x7f5c240ffc20 2024-04-02T04:44:02.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.700+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c241a2d50 con 0x7f5c240ffc20 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.700+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5c2410cfa0 con 0x7f5c240ffc20 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.701+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5c1800f5d0 con 0x7f5c240ffc20 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.701+0000 7f5c0bfff700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5c0c06c4e0 0x7f5c0c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.702+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5c1808c940 con 0x7f5c240ffc20 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.702+0000 7f5c227fc700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5c0c06c4e0 0x7f5c0c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.702+0000 7f5c0bfff700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] conn(0x7f5c0c0721c0 0x7f5c0c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.702+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 --> [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c0c074c90 con 0x7f5c0c0721c0 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.702+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c180912a0 con 0x7f5c240ffc20 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.702+0000 7f5c237fe700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] conn(0x7f5c0c0721c0 0x7f5c0c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:02.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.702+0000 7f5c227fc700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5c0c06c4e0 0x7f5c0c06e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c24103030 tx=0x7f5c1400b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:02.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.702+0000 7f5c237fe700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] conn(0x7f5c0c0721c0 0x7f5c0c0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:02.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.703+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 <== osd.0 v2:172.21.15.70:6802/1163584079 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5c0c074c90 con 0x7f5c0c0721c0 2024-04-02T04:44:02.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.718+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 --> [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5c2404f2e0 con 0x7f5c0c0721c0 2024-04-02T04:44:02.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.718+0000 7f5c0bfff700 1 -- 172.21.15.70:0/3320186047 <== osd.0 v2:172.21.15.70:6802/1163584079 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5c2404f2e0 con 0x7f5c0c0721c0 2024-04-02T04:44:02.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.718+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] conn(0x7f5c0c0721c0 msgr2=0x7f5c0c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:02.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.718+0000 7f5c295c0700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] conn(0x7f5c0c0721c0 0x7f5c0c0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:02.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5c0c06c4e0 msgr2=0x7f5c0c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:02.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5c0c06c4e0 0x7f5c0c06e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c24103030 tx=0x7f5c1400b410 comp rx=0 tx=0).stop 2024-04-02T04:44:02.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c240ffc20 msgr2=0x7f5c24101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:02.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c240ffc20 0x7f5c24101840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5c18006010 tx=0x7f5c1800bd70 comp rx=0 tx=0).stop 2024-04-02T04:44:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 shutdown_connections 2024-04-02T04:44:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6802/1163584079,v1:172.21.15.70:6803/1163584079] conn(0x7f5c0c0721c0 0x7f5c0c0745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5c0c06c4e0 0x7f5c0c06e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c240ffc20 0x7f5c24101840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 --2- 172.21.15.70:0/3320186047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c24105bd0 0x7f5c24101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 >> 172.21.15.70:0/3320186047 conn(0x7f5c24075670 msgr2=0x7f5c24068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 shutdown_connections 2024-04-02T04:44:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:02.719+0000 7f5c295c0700 1 -- 172.21.15.70:0/3320186047 wait complete. 2024-04-02T04:44:03.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.194+0000 7f11dc61a700 1 -- 172.21.15.70:0/413034471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 msgr2=0x7f11d4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:03.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.194+0000 7f11dc61a700 1 --2- 172.21.15.70:0/413034471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 0x7f11d4101280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009910 tx=0x7f11cc009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:03.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.194+0000 7f11dc61a700 1 -- 172.21.15.70:0/413034471 shutdown_connections 2024-04-02T04:44:03.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.194+0000 7f11dc61a700 1 --2- 172.21.15.70:0/413034471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d4101850 0x7f11d41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.194+0000 7f11dc61a700 1 --2- 172.21.15.70:0/413034471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 0x7f11d4101280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.194+0000 7f11dc61a700 1 -- 172.21.15.70:0/413034471 >> 172.21.15.70:0/413034471 conn(0x7f11d40fc750 msgr2=0x7f11d40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:03.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.194+0000 7f11dc61a700 1 -- 172.21.15.70:0/413034471 shutdown_connections 2024-04-02T04:44:03.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.194+0000 7f11dc61a700 1 -- 172.21.15.70:0/413034471 wait complete. 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11dc61a700 1 Processor -- start 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11dc61a700 1 -- start start 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11dc61a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 0x7f11d419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11dc61a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d4101850 0x7f11d419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11dc61a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d419d2c0 con 0x7f11d4100ea0 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11dc61a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d4196770 con 0x7f11d4101850 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11da3b6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 0x7f11d419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11da3b6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 0x7f11d419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36836/0 (socket says 172.21.15.70:36836) 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11da3b6700 1 -- 172.21.15.70:0/4207211669 learned_addr learned my addr 172.21.15.70:0/4207211669 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11d9bb5700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d4101850 0x7f11d419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11da3b6700 1 -- 172.21.15.70:0/4207211669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d4101850 msgr2=0x7f11d419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11da3b6700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d4101850 0x7f11d419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.195+0000 7f11da3b6700 1 -- 172.21.15.70:0/4207211669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc0095f0 con 0x7f11d4100ea0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.196+0000 7f11da3b6700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 0x7f11d419c6f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0092e0 tx=0x7f11cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.196+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc01d070 con 0x7f11d4100ea0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.196+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f11cc004fa0 con 0x7f11d4100ea0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.196+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc00f460 con 0x7f11d4100ea0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.196+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d41969f0 con 0x7f11d4100ea0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.196+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d4196ee0 con 0x7f11d4100ea0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.197+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f11d410e0b0 con 0x7f11d4100ea0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.197+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f11cc022810 con 0x7f11d4100ea0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11d37fe700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f11bc06c4e0 0x7f11bc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11d9bb5700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f11bc06c4e0 0x7f11bc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f11cc08c8d0 con 0x7f11d4100ea0 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11d37fe700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] conn(0x7f11bc0721c0 0x7f11bc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 --> [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11bc074c90 con 0x7f11bc0721c0 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11cc0912a0 con 0x7f11d4100ea0 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11dabb7700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] conn(0x7f11bc0721c0 0x7f11bc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11d9bb5700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f11bc06c4e0 0x7f11bc06e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11c8005950 tx=0x7f11c80058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.198+0000 7f11dabb7700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] conn(0x7f11bc0721c0 0x7f11bc0745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.199+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 <== osd.1 v2:172.21.15.70:6810/3081297537 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f11bc074c90 con 0x7f11bc0721c0 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 --> [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f11d404f2e0 con 0x7f11bc0721c0 2024-04-02T04:44:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11d37fe700 1 -- 172.21.15.70:0/4207211669 <== osd.1 v2:172.21.15.70:6810/3081297537 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f11d404f2e0 con 0x7f11bc0721c0 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] conn(0x7f11bc0721c0 msgr2=0x7f11bc0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] conn(0x7f11bc0721c0 0x7f11bc0745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f11bc06c4e0 msgr2=0x7f11bc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f11bc06c4e0 0x7f11bc06e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11c8005950 tx=0x7f11c80058e0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 msgr2=0x7f11d419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 0x7f11d419c6f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0092e0 tx=0x7f11cc00be20 comp rx=0 tx=0).stop 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 shutdown_connections 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6810/3081297537,v1:172.21.15.70:6811/3081297537] conn(0x7f11bc0721c0 0x7f11bc0745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f11bc06c4e0 0x7f11bc06e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4100ea0 0x7f11d419c6f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 --2- 172.21.15.70:0/4207211669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d4101850 0x7f11d419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:03.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 >> 172.21.15.70:0/4207211669 conn(0x7f11d40fc750 msgr2=0x7f11d40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:03.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.215+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 shutdown_connections 2024-04-02T04:44:03.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:01.216+0000 7f11dc61a700 1 -- 172.21.15.70:0/4207211669 wait complete. 2024-04-02T04:44:03.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:03 smithi070 ceph-mon[29212]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:03.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:03 smithi179 ceph-mon[36603]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:03.934 INFO:teuthology.orchestra.run.smithi070.stdout:60129542173 2024-04-02T04:44:03.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 -- 172.21.15.70:0/317133204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8101850 msgr2=0x7fd8e81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/317133204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8101850 0x7fd8e81058a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009910 tx=0x7fd8d8009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 -- 172.21.15.70:0/317133204 shutdown_connections 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/317133204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8101850 0x7fd8e81058a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/317133204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e8100ea0 0x7fd8e8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 -- 172.21.15.70:0/317133204 >> 172.21.15.70:0/317133204 conn(0x7fd8e80fc750 msgr2=0x7fd8e80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 -- 172.21.15.70:0/317133204 shutdown_connections 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 -- 172.21.15.70:0/317133204 wait complete. 2024-04-02T04:44:04.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 Processor -- start 2024-04-02T04:44:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.768+0000 7fd8edc5d700 1 -- start start 2024-04-02T04:44:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8edc5d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8100ea0 0x7fd8e819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8edc5d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e8101850 0x7fd8e819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8edc5d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e819d2d0 con 0x7fd8e8100ea0 2024-04-02T04:44:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8edc5d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e8196780 con 0x7fd8e8101850 2024-04-02T04:44:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8100ea0 0x7fd8e819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8100ea0 0x7fd8e819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36866/0 (socket says 172.21.15.70:36866) 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e77fe700 1 -- 172.21.15.70:0/1696826635 learned_addr learned my addr 172.21.15.70:0/1696826635 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e6ffd700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e8101850 0x7fd8e819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e77fe700 1 -- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e8101850 msgr2=0x7fd8e819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e77fe700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e8101850 0x7fd8e819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e77fe700 1 -- 172.21.15.70:0/1696826635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d80095f0 con 0x7fd8e8100ea0 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e6ffd700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e8101850 0x7fd8e819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e77fe700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8100ea0 0x7fd8e819c700 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00d8d0 tx=0x7fd8dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8dc009940 con 0x7fd8e8100ea0 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd8dc010460 con 0x7fd8e8100ea0 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8dc00f5f0 con 0x7fd8e8100ea0 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e8196a60 con 0x7fd8e8100ea0 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.769+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e8196fb0 con 0x7fd8e8100ea0 2024-04-02T04:44:04.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.770+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd8e810e0c0 con 0x7fd8e8100ea0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.771+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd8dc0105d0 con 0x7fd8e8100ea0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.771+0000 7fd8e4ff9700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd8d006c4e0 0x7fd8d006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.771+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd8dc08b3f0 con 0x7fd8e8100ea0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.771+0000 7fd8e6ffd700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd8d006c4e0 0x7fd8d006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.771+0000 7fd8e4ff9700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] conn(0x7fd8d00721c0 0x7fd8d00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.771+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 --> [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8d0074c90 con 0x7fd8d00721c0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.771+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8dc08dac0 con 0x7fd8e8100ea0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.772+0000 7fd8e6ffd700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd8d006c4e0 0x7fd8d006e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8006010 tx=0x7fd8d800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.772+0000 7fd8e7fff700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] conn(0x7fd8d00721c0 0x7fd8d00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.772+0000 7fd8e7fff700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] conn(0x7fd8d00721c0 0x7fd8d00745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:04.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.773+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 <== osd.4 v2:172.21.15.179:6808/2251755550 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd8d0074c90 con 0x7fd8d00721c0 2024-04-02T04:44:04.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.788+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 --> [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd8e804f2e0 con 0x7fd8d00721c0 2024-04-02T04:44:04.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.788+0000 7fd8e4ff9700 1 -- 172.21.15.70:0/1696826635 <== osd.4 v2:172.21.15.179:6808/2251755550 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd8e804f2e0 con 0x7fd8d00721c0 2024-04-02T04:44:04.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] conn(0x7fd8d00721c0 msgr2=0x7fd8d00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:04.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] conn(0x7fd8d00721c0 0x7fd8d00745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:04.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd8d006c4e0 msgr2=0x7fd8d006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:04.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd8d006c4e0 0x7fd8d006e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8006010 tx=0x7fd8d800b540 comp rx=0 tx=0).stop 2024-04-02T04:44:04.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8100ea0 msgr2=0x7fd8e819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:04.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8100ea0 0x7fd8e819c700 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00d8d0 tx=0x7fd8dc00dc90 comp rx=0 tx=0).stop 2024-04-02T04:44:04.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 shutdown_connections 2024-04-02T04:44:04.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd8d006c4e0 0x7fd8d006e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:04.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:6808/2251755550,v1:172.21.15.179:6809/2251755550] conn(0x7fd8d00721c0 0x7fd8d00745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:04.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e8100ea0 0x7fd8e819c700 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:04.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 --2- 172.21.15.70:0/1696826635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8e8101850 0x7fd8e819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:04.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 >> 172.21.15.70:0/1696826635 conn(0x7fd8e80fc750 msgr2=0x7fd8e80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:04.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 shutdown_connections 2024-04-02T04:44:04.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:04.789+0000 7fd8edc5d700 1 -- 172.21.15.70:0/1696826635 wait complete. 2024-04-02T04:44:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.011+0000 7f48b83f9700 1 -- 172.21.15.70:0/2813744388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 msgr2=0x7f48b0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.011+0000 7f48b83f9700 1 --2- 172.21.15.70:0/2813744388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 0x7f48b0101280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009910 tx=0x7f48ac009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.011+0000 7f48b83f9700 1 -- 172.21.15.70:0/2813744388 shutdown_connections 2024-04-02T04:44:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.011+0000 7f48b83f9700 1 --2- 172.21.15.70:0/2813744388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48b0101850 0x7f48b01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.011+0000 7f48b83f9700 1 --2- 172.21.15.70:0/2813744388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 0x7f48b0101280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.011+0000 7f48b83f9700 1 -- 172.21.15.70:0/2813744388 >> 172.21.15.70:0/2813744388 conn(0x7f48b00fc750 msgr2=0x7f48b00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.011+0000 7f48b83f9700 1 -- 172.21.15.70:0/2813744388 shutdown_connections 2024-04-02T04:44:05.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.011+0000 7f48b83f9700 1 -- 172.21.15.70:0/2813744388 wait complete. 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b83f9700 1 Processor -- start 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b83f9700 1 -- start start 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b83f9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 0x7f48b0196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b83f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48b0101850 0x7f48b0196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b83f9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0197560 con 0x7f48b0100ea0 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b83f9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b019b370 con 0x7f48b0101850 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b6195700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 0x7f48b0196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b6195700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 0x7f48b0196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36886/0 (socket says 172.21.15.70:36886) 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b6195700 1 -- 172.21.15.70:0/1293624007 learned_addr learned my addr 172.21.15.70:0/1293624007 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b5994700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48b0101850 0x7f48b0196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b6195700 1 -- 172.21.15.70:0/1293624007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48b0101850 msgr2=0x7f48b0196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b6195700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48b0101850 0x7f48b0196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.012+0000 7f48b6195700 1 -- 172.21.15.70:0/1293624007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac0095f0 con 0x7f48b0100ea0 2024-04-02T04:44:05.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.013+0000 7f48b6195700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 0x7f48b0196990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48ac006010 tx=0x7f48ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:05.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.013+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48ac01d070 con 0x7f48b0100ea0 2024-04-02T04:44:05.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.013+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b019b5f0 con 0x7f48b0100ea0 2024-04-02T04:44:05.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.013+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f48ac00f460 con 0x7f48b0100ea0 2024-04-02T04:44:05.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.013+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48ac021640 con 0x7f48b0100ea0 2024-04-02T04:44:05.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.013+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b019bae0 con 0x7f48b0100ea0 2024-04-02T04:44:05.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.014+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48b010e160 con 0x7f48b0100ea0 2024-04-02T04:44:05.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.014+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f48ac00faf0 con 0x7f48b0100ea0 2024-04-02T04:44:05.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48a77fe700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f489c06c4e0 0x7f489c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:05.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48b5994700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f489c06c4e0 0x7f489c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:05.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f48ac08c870 con 0x7f48b0100ea0 2024-04-02T04:44:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48a77fe700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] conn(0x7f489c0721c0 0x7f489c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 --> [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f489c074c90 con 0x7f489c0721c0 2024-04-02T04:44:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ac0912a0 con 0x7f48b0100ea0 2024-04-02T04:44:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48b6996700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] conn(0x7f489c0721c0 0x7f489c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48b5994700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f489c06c4e0 0x7f489c06e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006fd0 tx=0x7f48a0008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.015+0000 7f48b6996700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] conn(0x7f489c0721c0 0x7f489c0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:05.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.016+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 <== osd.2 v2:172.21.15.70:6818/428679321 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f489c074c90 con 0x7f489c0721c0 2024-04-02T04:44:05.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.031+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 --> [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f48b004f2e0 con 0x7f489c0721c0 2024-04-02T04:44:05.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.031+0000 7f48a77fe700 1 -- 172.21.15.70:0/1293624007 <== osd.2 v2:172.21.15.70:6818/428679321 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f48b004f2e0 con 0x7f489c0721c0 2024-04-02T04:44:05.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] conn(0x7f489c0721c0 msgr2=0x7f489c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:05.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] conn(0x7f489c0721c0 0x7f489c0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:05.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f489c06c4e0 msgr2=0x7f489c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:05.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f489c06c4e0 0x7f489c06e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006fd0 tx=0x7f48a0008040 comp rx=0 tx=0).stop 2024-04-02T04:44:05.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 msgr2=0x7f48b0196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 0x7f48b0196990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48ac006010 tx=0x7f48ac00be20 comp rx=0 tx=0).stop 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 shutdown_connections 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6818/428679321,v1:172.21.15.70:6819/428679321] conn(0x7f489c0721c0 0x7f489c0745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f489c06c4e0 0x7f489c06e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48b0100ea0 0x7f48b0196990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 --2- 172.21.15.70:0/1293624007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48b0101850 0x7f48b0196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 >> 172.21.15.70:0/1293624007 conn(0x7f48b00fc750 msgr2=0x7f48b00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 shutdown_connections 2024-04-02T04:44:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:05.032+0000 7f48b83f9700 1 -- 172.21.15.70:0/1293624007 wait complete. 2024-04-02T04:44:05.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:05 smithi179 ceph-mon[36603]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:05.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:05 smithi070 ceph-mon[29212]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:05.932 INFO:teuthology.orchestra.run.smithi070.stdout:38654705699 2024-04-02T04:44:05.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-02T04:44:06.859 INFO:teuthology.orchestra.run.smithi070.stdout:81604378648 2024-04-02T04:44:06.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-02T04:44:07.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:06 smithi070 ceph-mon[29212]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:07.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:06 smithi179 ceph-mon[36603]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.713+0000 7f54649bd700 1 -- 172.21.15.70:0/3316743830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460101850 msgr2=0x7f54601058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.713+0000 7f54649bd700 1 --2- 172.21.15.70:0/3316743830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460101850 0x7f54601058a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5450009910 tx=0x7f5450009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.713+0000 7f54649bd700 1 -- 172.21.15.70:0/3316743830 shutdown_connections 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.713+0000 7f54649bd700 1 --2- 172.21.15.70:0/3316743830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460101850 0x7f54601058a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.713+0000 7f54649bd700 1 --2- 172.21.15.70:0/3316743830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5460100ea0 0x7f5460101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.713+0000 7f54649bd700 1 -- 172.21.15.70:0/3316743830 >> 172.21.15.70:0/3316743830 conn(0x7f54600fc750 msgr2=0x7f54600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.713+0000 7f54649bd700 1 -- 172.21.15.70:0/3316743830 shutdown_connections 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.713+0000 7f54649bd700 1 -- 172.21.15.70:0/3316743830 wait complete. 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f54649bd700 1 Processor -- start 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f54649bd700 1 -- start start 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f54649bd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100ea0 0x7f5460196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f54649bd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5460101850 0x7f5460196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f54649bd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5460197520 con 0x7f5460100ea0 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f54649bd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546019b330 con 0x7f5460101850 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f545e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100ea0 0x7f5460196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f545e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100ea0 0x7f5460196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55948/0 (socket says 172.21.15.70:55948) 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f545e59c700 1 -- 172.21.15.70:0/1742470294 learned_addr learned my addr 172.21.15.70:0/1742470294 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f545e59c700 1 -- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5460101850 msgr2=0x7f5460196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f545e59c700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5460101850 0x7f5460196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f545e59c700 1 -- 172.21.15.70:0/1742470294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500095f0 con 0x7f5460100ea0 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.714+0000 7f545e59c700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100ea0 0x7f5460196950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f545400b4b0 tx=0x7f545400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.715+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5454011800 con 0x7f5460100ea0 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.715+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546019b610 con 0x7f5460100ea0 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.715+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5454011e10 con 0x7f5460100ea0 2024-04-02T04:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.715+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f545400f2e0 con 0x7f5460100ea0 2024-04-02T04:44:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.715+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546019bb60 con 0x7f5460100ea0 2024-04-02T04:44:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.715+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f546010e140 con 0x7f5460100ea0 2024-04-02T04:44:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.716+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5454010380 con 0x7f5460100ea0 2024-04-02T04:44:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.716+0000 7f544f7fe700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f544806c4e0 0x7f544806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.717+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f545408b750 con 0x7f5460100ea0 2024-04-02T04:44:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.717+0000 7f545dd9b700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f544806c4e0 0x7f544806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.717+0000 7f544f7fe700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] conn(0x7f54480721c0 0x7f54480745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.717+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 --> [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5448074c90 con 0x7f54480721c0 2024-04-02T04:44:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.717+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f545408bbb0 con 0x7f5460100ea0 2024-04-02T04:44:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.717+0000 7f545dd9b700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f544806c4e0 0x7f544806e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54500059d0 tx=0x7f5450005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.717+0000 7f545ed9d700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] conn(0x7f54480721c0 0x7f54480745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.717+0000 7f545ed9d700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] conn(0x7f54480721c0 0x7f54480745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.718+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 <== osd.5 v2:172.21.15.179:6816/2668632021 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5448074c90 con 0x7f54480721c0 2024-04-02T04:44:07.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.738+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 --> [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f546004f2e0 con 0x7f54480721c0 2024-04-02T04:44:07.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.739+0000 7f544f7fe700 1 -- 172.21.15.70:0/1742470294 <== osd.5 v2:172.21.15.179:6816/2668632021 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f546004f2e0 con 0x7f54480721c0 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] conn(0x7f54480721c0 msgr2=0x7f54480745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] conn(0x7f54480721c0 0x7f54480745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f544806c4e0 msgr2=0x7f544806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f544806c4e0 0x7f544806e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54500059d0 tx=0x7f5450005960 comp rx=0 tx=0).stop 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100ea0 msgr2=0x7f5460196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100ea0 0x7f5460196950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f545400b4b0 tx=0x7f545400b870 comp rx=0 tx=0).stop 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 shutdown_connections 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:6816/2668632021,v1:172.21.15.179:6817/2668632021] conn(0x7f54480721c0 0x7f54480745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f544806c4e0 0x7f544806e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:07.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5460100ea0 0x7f5460196950 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:07.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 --2- 172.21.15.70:0/1742470294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5460101850 0x7f5460196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:07.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 >> 172.21.15.70:0/1742470294 conn(0x7f54600fc750 msgr2=0x7f54600fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:07.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 shutdown_connections 2024-04-02T04:44:07.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:07.740+0000 7f54649bd700 1 -- 172.21.15.70:0/1742470294 wait complete. 2024-04-02T04:44:08.838 INFO:teuthology.orchestra.run.smithi070.stdout:171798691849 2024-04-02T04:44:08.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-02T04:44:08.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.390+0000 7f16fe37e700 1 -- 172.21.15.70:0/126594031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 msgr2=0x7f16f80ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:08.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.390+0000 7f16fe37e700 1 --2- 172.21.15.70:0/126594031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 0x7f16f80ffa50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009910 tx=0x7f16ec009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:08.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.390+0000 7f16fe37e700 1 -- 172.21.15.70:0/126594031 shutdown_connections 2024-04-02T04:44:08.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.390+0000 7f16fe37e700 1 --2- 172.21.15.70:0/126594031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16f8100020 0x7f16f8103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:08.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.390+0000 7f16fe37e700 1 --2- 172.21.15.70:0/126594031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 0x7f16f80ffa50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.390+0000 7f16fe37e700 1 -- 172.21.15.70:0/126594031 >> 172.21.15.70:0/126594031 conn(0x7f16f8074e30 msgr2=0x7f16f8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.390+0000 7f16fe37e700 1 -- 172.21.15.70:0/126594031 shutdown_connections 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.390+0000 7f16fe37e700 1 -- 172.21.15.70:0/126594031 wait complete. 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.391+0000 7f16fe37e700 1 Processor -- start 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.391+0000 7f16fe37e700 1 -- start start 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.391+0000 7f16fe37e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 0x7f16f819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.391+0000 7f16fe37e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16f8100020 0x7f16f819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16fe37e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f819d230 con 0x7f16f80ff670 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16fe37e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f81966b0 con 0x7f16f8100020 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 0x7f16f819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 0x7f16f819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55962/0 (socket says 172.21.15.70:55962) 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f7fff700 1 -- 172.21.15.70:0/3921557907 learned_addr learned my addr 172.21.15.70:0/3921557907 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:08.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f77fe700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16f8100020 0x7f16f819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f7fff700 1 -- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16f8100020 msgr2=0x7f16f819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f7fff700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16f8100020 0x7f16f819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f7fff700 1 -- 172.21.15.70:0/3921557907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec0095f0 con 0x7f16f80ff670 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f7fff700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 0x7f16f819c610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16ec006010 tx=0x7f16ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16ec01d070 con 0x7f16f80ff670 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f8196930 con 0x7f16f80ff670 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f16ec00f460 con 0x7f16f80ff670 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16ec021640 con 0x7f16f80ff670 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.392+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f8196e20 con 0x7f16f80ff670 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.393+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16f810cfd0 con 0x7f16f80ff670 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.394+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f16ec00fab0 con 0x7f16f80ff670 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.394+0000 7f16f57fa700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f16e00708f0 0x7f16e0072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:08.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.394+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f16ec08c7d0 con 0x7f16f80ff670 2024-04-02T04:44:08.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.394+0000 7f16f77fe700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f16e00708f0 0x7f16e0072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:08.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.394+0000 7f16f57fa700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] conn(0x7f16e00765d0 0x7f16e00789f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:08.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.394+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 --> [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16e00790a0 con 0x7f16e00765d0 2024-04-02T04:44:08.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.394+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16ec0912a0 con 0x7f16f80ff670 2024-04-02T04:44:08.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.395+0000 7f16f77fe700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f16e00708f0 0x7f16e0072db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16e8005fd0 tx=0x7f16e800a560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:08.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.395+0000 7f16fc91b700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] conn(0x7f16e00765d0 0x7f16e00789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:08.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.395+0000 7f16fc91b700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] conn(0x7f16e00765d0 0x7f16e00789f0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:08.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.396+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 <== osd.3 v2:172.21.15.179:6800/1047738021 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f16e00790a0 con 0x7f16e00765d0 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.411+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 --> [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f16f804f2e0 con 0x7f16e00765d0 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16f57fa700 1 -- 172.21.15.70:0/3921557907 <== osd.3 v2:172.21.15.179:6800/1047738021 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f16f804f2e0 con 0x7f16e00765d0 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] conn(0x7f16e00765d0 msgr2=0x7f16e00789f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] conn(0x7f16e00765d0 0x7f16e00789f0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f16e00708f0 msgr2=0x7f16e0072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f16e00708f0 0x7f16e0072db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16e8005fd0 tx=0x7f16e800a560 comp rx=0 tx=0).stop 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 msgr2=0x7f16f819c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 0x7f16f819c610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16ec006010 tx=0x7f16ec00be20 comp rx=0 tx=0).stop 2024-04-02T04:44:08.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 shutdown_connections 2024-04-02T04:44:08.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f16e00708f0 0x7f16e0072db0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:08.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16f80ff670 0x7f16f819c610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:08.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:6800/1047738021,v1:172.21.15.179:6801/1047738021] conn(0x7f16e00765d0 0x7f16e00789f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:08.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 --2- 172.21.15.70:0/3921557907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16f8100020 0x7f16f819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:08.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 >> 172.21.15.70:0/3921557907 conn(0x7f16f8074e30 msgr2=0x7f16f80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:08.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 shutdown_connections 2024-04-02T04:44:08.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:08.412+0000 7f16fe37e700 1 -- 172.21.15.70:0/3921557907 wait complete. 2024-04-02T04:44:09.153 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:09.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:09 smithi070 ceph-mon[29212]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:09.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:09 smithi179 ceph-mon[36603]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:09.680 INFO:teuthology.orchestra.run.smithi070.stdout:150323855374 2024-04-02T04:44:09.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-02T04:44:10.234 INFO:teuthology.orchestra.run.smithi070.stdout:115964117011 2024-04-02T04:44:10.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-02T04:44:10.258 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:10.258 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:11.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:11 smithi070 ceph-mon[29212]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:11.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:11 smithi179 ceph-mon[36603]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:13.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.189+0000 7fe96f66d700 1 -- 172.21.15.70:0/1722806540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 msgr2=0x7fe9681070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:13.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.189+0000 7fe96f66d700 1 --2- 172.21.15.70:0/1722806540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 0x7fe9681070b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe964009910 tx=0x7fe964009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:13.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.189+0000 7fe96f66d700 1 -- 172.21.15.70:0/1722806540 shutdown_connections 2024-04-02T04:44:13.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.189+0000 7fe96f66d700 1 --2- 172.21.15.70:0/1722806540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe968068580 0x7fe9680689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:13.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.189+0000 7fe96f66d700 1 --2- 172.21.15.70:0/1722806540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 0x7fe9681070b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:13.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.189+0000 7fe96f66d700 1 -- 172.21.15.70:0/1722806540 >> 172.21.15.70:0/1722806540 conn(0x7fe968075670 msgr2=0x7fe968075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:13.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.189+0000 7fe96f66d700 1 -- 172.21.15.70:0/1722806540 shutdown_connections 2024-04-02T04:44:13.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.189+0000 7fe96f66d700 1 -- 172.21.15.70:0/1722806540 wait complete. 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.190+0000 7fe96f66d700 1 Processor -- start 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.190+0000 7fe96f66d700 1 -- start start 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96f66d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe968068580 0x7fe96819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96f66d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 0x7fe96819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96f66d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96819d230 con 0x7fe968106cd0 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96f66d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681966e0 con 0x7fe968068580 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96cc08700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 0x7fe96819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96cc08700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 0x7fe96819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55976/0 (socket says 172.21.15.70:55976) 2024-04-02T04:44:13.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96cc08700 1 -- 172.21.15.70:0/2058113515 learned_addr learned my addr 172.21.15.70:0/2058113515 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96d409700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe968068580 0x7fe96819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96cc08700 1 -- 172.21.15.70:0/2058113515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe968068580 msgr2=0x7fe96819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96cc08700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe968068580 0x7fe96819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96cc08700 1 -- 172.21.15.70:0/2058113515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640095f0 con 0x7fe968106cd0 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe96cc08700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 0x7fe96819cba0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe95800d8d0 tx=0x7fe95800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe95e7fc700 1 -- 172.21.15.70:0/2058113515 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9580098a0 con 0x7fe968106cd0 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.191+0000 7fe95e7fc700 1 -- 172.21.15.70:0/2058113515 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe958010460 con 0x7fe968106cd0 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.192+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9681969c0 con 0x7fe968106cd0 2024-04-02T04:44:13.197 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:13.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.192+0000 7fe95e7fc700 1 -- 172.21.15.70:0/2058113515 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe95800f5f0 con 0x7fe968106cd0 2024-04-02T04:44:13.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.192+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe968196f10 con 0x7fe968106cd0 2024-04-02T04:44:13.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.193+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9681091d0 con 0x7fe968106cd0 2024-04-02T04:44:13.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.194+0000 7fe95e7fc700 1 -- 172.21.15.70:0/2058113515 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe958009a00 con 0x7fe968106cd0 2024-04-02T04:44:13.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.194+0000 7fe95e7fc700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe95406c4e0 0x7fe95406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:13.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.194+0000 7fe95e7fc700 1 -- 172.21.15.70:0/2058113515 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe95808b380 con 0x7fe968106cd0 2024-04-02T04:44:13.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.194+0000 7fe96d409700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe95406c4e0 0x7fe95406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:13.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.195+0000 7fe96d409700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe95406c4e0 0x7fe95406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe96400b5c0 tx=0x7fe96401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:13.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.199+0000 7fe95e7fc700 1 -- 172.21.15.70:0/2058113515 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe958059930 con 0x7fe968106cd0 2024-04-02T04:44:13.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.397+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe968197c90 con 0x7fe968106cd0 2024-04-02T04:44:13.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.397+0000 7fe95e7fc700 1 -- 172.21.15.70:0/2058113515 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe9580594c0 con 0x7fe968106cd0 2024-04-02T04:44:13.402 INFO:teuthology.orchestra.run.smithi070.stdout:60129542175 2024-04-02T04:44:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe95406c4e0 msgr2=0x7fe95406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe95406c4e0 0x7fe95406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe96400b5c0 tx=0x7fe96401a040 comp rx=0 tx=0).stop 2024-04-02T04:44:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 msgr2=0x7fe96819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 0x7fe96819cba0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe95800d8d0 tx=0x7fe95800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:44:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 shutdown_connections 2024-04-02T04:44:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe95406c4e0 0x7fe95406e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe968068580 0x7fe96819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 --2- 172.21.15.70:0/2058113515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe968106cd0 0x7fe96819cba0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.399+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 >> 172.21.15.70:0/2058113515 conn(0x7fe968075670 msgr2=0x7fe9680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.400+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 shutdown_connections 2024-04-02T04:44:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:13.400+0000 7fe96f66d700 1 -- 172.21.15.70:0/2058113515 wait complete. 2024-04-02T04:44:13.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:13 smithi070 ceph-mon[29212]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:13.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:13.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:13 smithi179 ceph-mon[36603]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:13.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:13.828 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:14.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:14 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2058113515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T04:44:14.597 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:14 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2058113515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T04:44:14.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542175 for osd.1 2024-04-02T04:44:14.621 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:15.428 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:15 smithi070 ceph-mon[29212]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:15.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:15 smithi179 ceph-mon[36603]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.011+0000 7f3963c69700 1 -- 172.21.15.70:0/2672792483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0fff50 msgr2=0x7f395c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.011+0000 7f3963c69700 1 --2- 172.21.15.70:0/2672792483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0fff50 0x7f395c105840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f394c009910 tx=0x7f394c009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.011+0000 7f3963c69700 1 -- 172.21.15.70:0/2672792483 shutdown_connections 2024-04-02T04:44:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.011+0000 7f3963c69700 1 --2- 172.21.15.70:0/2672792483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0fff50 0x7f395c105840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.011+0000 7f3963c69700 1 --2- 172.21.15.70:0/2672792483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c0ff5a0 0x7f395c0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.011+0000 7f3963c69700 1 -- 172.21.15.70:0/2672792483 >> 172.21.15.70:0/2672792483 conn(0x7f395c074e30 msgr2=0x7f395c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.011+0000 7f3963c69700 1 -- 172.21.15.70:0/2672792483 shutdown_connections 2024-04-02T04:44:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.011+0000 7f3963c69700 1 -- 172.21.15.70:0/2672792483 wait complete. 2024-04-02T04:44:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3963c69700 1 Processor -- start 2024-04-02T04:44:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3963c69700 1 -- start start 2024-04-02T04:44:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3963c69700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0ff5a0 0x7f395c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3963c69700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c0fff50 0x7f395c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3963c69700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19d260 con 0x7f395c0ff5a0 2024-04-02T04:44:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3963c69700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c196710 con 0x7f395c0fff50 2024-04-02T04:44:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3961a05700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0ff5a0 0x7f395c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3961a05700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0ff5a0 0x7f395c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55980/0 (socket says 172.21.15.70:55980) 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3961a05700 1 -- 172.21.15.70:0/23325760 learned_addr learned my addr 172.21.15.70:0/23325760 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3961a05700 1 -- 172.21.15.70:0/23325760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c0fff50 msgr2=0x7f395c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3961204700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c0fff50 0x7f395c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3961a05700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c0fff50 0x7f395c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3961a05700 1 -- 172.21.15.70:0/23325760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c0095f0 con 0x7f395c0ff5a0 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.012+0000 7f3961a05700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0ff5a0 0x7f395c19c690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f395800d8d0 tx=0x7f395800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.013+0000 7f3952ffd700 1 -- 172.21.15.70:0/23325760 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3958009940 con 0x7f395c0ff5a0 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.013+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c1969f0 con 0x7f395c0ff5a0 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.013+0000 7f3952ffd700 1 -- 172.21.15.70:0/23325760 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3958010460 con 0x7f395c0ff5a0 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.013+0000 7f3952ffd700 1 -- 172.21.15.70:0/23325760 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f395800f5f0 con 0x7f395c0ff5a0 2024-04-02T04:44:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.013+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c196f40 con 0x7f395c0ff5a0 2024-04-02T04:44:16.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.014+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c109200 con 0x7f395c0ff5a0 2024-04-02T04:44:16.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.014+0000 7f3952ffd700 1 -- 172.21.15.70:0/23325760 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f39580105d0 con 0x7f395c0ff5a0 2024-04-02T04:44:16.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.014+0000 7f3952ffd700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f394806c4e0 0x7f394806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:16.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.015+0000 7f3952ffd700 1 -- 172.21.15.70:0/23325760 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f395808b4f0 con 0x7f395c0ff5a0 2024-04-02T04:44:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.015+0000 7f3961204700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f394806c4e0 0x7f394806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.015+0000 7f3961204700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f394806c4e0 0x7f394806e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f394c006010 tx=0x7f394c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:16.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.019+0000 7f3952ffd700 1 -- 172.21.15.70:0/23325760 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3958059aa0 con 0x7f395c0ff5a0 2024-04-02T04:44:16.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.215+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f395c04f2e0 con 0x7f395c0ff5a0 2024-04-02T04:44:16.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.215+0000 7f3952ffd700 1 -- 172.21.15.70:0/23325760 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3958059630 con 0x7f395c0ff5a0 2024-04-02T04:44:16.219 INFO:teuthology.orchestra.run.smithi070.stdout:38654705701 2024-04-02T04:44:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f394806c4e0 msgr2=0x7f394806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f394806c4e0 0x7f394806e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f394c006010 tx=0x7f394c00b540 comp rx=0 tx=0).stop 2024-04-02T04:44:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0ff5a0 msgr2=0x7f395c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0ff5a0 0x7f395c19c690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f395800d8d0 tx=0x7f395800dc90 comp rx=0 tx=0).stop 2024-04-02T04:44:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 shutdown_connections 2024-04-02T04:44:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f394806c4e0 0x7f394806e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f395c0ff5a0 0x7f395c19c690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 --2- 172.21.15.70:0/23325760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c0fff50 0x7f395c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 >> 172.21.15.70:0/23325760 conn(0x7f395c074e30 msgr2=0x7f395c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 shutdown_connections 2024-04-02T04:44:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:16.217+0000 7f3963c69700 1 -- 172.21.15.70:0/23325760 wait complete. 2024-04-02T04:44:16.970 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:16 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/23325760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T04:44:16.971 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:16 smithi070 ceph-mon[29212]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:17.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:16 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/23325760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T04:44:17.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:16 smithi179 ceph-mon[36603]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:18.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.012+0000 7f64fc1e4700 1 -- 172.21.15.70:0/304250394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40ff5a0 msgr2=0x7f64f40ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.012+0000 7f64fc1e4700 1 --2- 172.21.15.70:0/304250394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40ff5a0 0x7f64f40ff980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009910 tx=0x7f64f0009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:18.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.012+0000 7f64fc1e4700 1 -- 172.21.15.70:0/304250394 shutdown_connections 2024-04-02T04:44:18.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.012+0000 7f64fc1e4700 1 --2- 172.21.15.70:0/304250394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64f40fff50 0x7f64f4105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.012+0000 7f64fc1e4700 1 --2- 172.21.15.70:0/304250394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40ff5a0 0x7f64f40ff980 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.012+0000 7f64fc1e4700 1 -- 172.21.15.70:0/304250394 >> 172.21.15.70:0/304250394 conn(0x7f64f4074e30 msgr2=0x7f64f4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:18.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.012+0000 7f64fc1e4700 1 -- 172.21.15.70:0/304250394 shutdown_connections 2024-04-02T04:44:18.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.012+0000 7f64fc1e4700 1 -- 172.21.15.70:0/304250394 wait complete. 2024-04-02T04:44:18.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64fc1e4700 1 Processor -- start 2024-04-02T04:44:18.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64fc1e4700 1 -- start start 2024-04-02T04:44:18.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64fc1e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64f40ff5a0 0x7f64f419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64fc1e4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40fff50 0x7f64f419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64fc1e4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f419d1c0 con 0x7f64f40fff50 2024-04-02T04:44:18.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64fc1e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4196700 con 0x7f64f40ff5a0 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64f977f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40fff50 0x7f64f419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64f977f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40fff50 0x7f64f419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34340/0 (socket says 172.21.15.70:34340) 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64f977f700 1 -- 172.21.15.70:0/3167807174 learned_addr learned my addr 172.21.15.70:0/3167807174 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64f9f80700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64f40ff5a0 0x7f64f419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64f977f700 1 -- 172.21.15.70:0/3167807174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64f40ff5a0 msgr2=0x7f64f419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64f977f700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64f40ff5a0 0x7f64f419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.013+0000 7f64f977f700 1 -- 172.21.15.70:0/3167807174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00095f0 con 0x7f64f40fff50 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.014+0000 7f64f9f80700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64f40ff5a0 0x7f64f419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.014+0000 7f64f977f700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40fff50 0x7f64f419cbc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c960 tx=0x7f64e400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.014+0000 7f64eaffd700 1 -- 172.21.15.70:0/3167807174 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e4007a70 con 0x7f64f40fff50 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.014+0000 7f64eaffd700 1 -- 172.21.15.70:0/3167807174 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f64e4007bd0 con 0x7f64f40fff50 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.014+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f41969e0 con 0x7f64f40fff50 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.014+0000 7f64eaffd700 1 -- 172.21.15.70:0/3167807174 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e4018630 con 0x7f64f40fff50 2024-04-02T04:44:18.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.014+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4196f30 con 0x7f64f40fff50 2024-04-02T04:44:18.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.015+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f4109270 con 0x7f64f40fff50 2024-04-02T04:44:18.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.016+0000 7f64eaffd700 1 -- 172.21.15.70:0/3167807174 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f64e40188a0 con 0x7f64f40fff50 2024-04-02T04:44:18.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.016+0000 7f64eaffd700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f64e006c4e0 0x7f64e006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.016+0000 7f64eaffd700 1 -- 172.21.15.70:0/3167807174 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f64e408b510 con 0x7f64f40fff50 2024-04-02T04:44:18.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.016+0000 7f64f9f80700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f64e006c4e0 0x7f64e006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.017+0000 7f64f9f80700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f64e006c4e0 0x7f64e006e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005b40 tx=0x7f64f001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.021+0000 7f64eaffd700 1 -- 172.21.15.70:0/3167807174 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f64e4059b30 con 0x7f64f40fff50 2024-04-02T04:44:18.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.215+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f64f404f2e0 con 0x7f64f40fff50 2024-04-02T04:44:18.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.215+0000 7f64eaffd700 1 -- 172.21.15.70:0/3167807174 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64e40596c0 con 0x7f64f40fff50 2024-04-02T04:44:18.220 INFO:teuthology.orchestra.run.smithi070.stdout:81604378650 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f64e006c4e0 msgr2=0x7f64e006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f64e006c4e0 0x7f64e006e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005b40 tx=0x7f64f001a040 comp rx=0 tx=0).stop 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40fff50 msgr2=0x7f64f419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40fff50 0x7f64f419cbc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c960 tx=0x7f64e400cd20 comp rx=0 tx=0).stop 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 shutdown_connections 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f64e006c4e0 0x7f64e006e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64f40ff5a0 0x7f64f419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 --2- 172.21.15.70:0/3167807174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f40fff50 0x7f64f419cbc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 >> 172.21.15.70:0/3167807174 conn(0x7f64f4074e30 msgr2=0x7f64f40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 shutdown_connections 2024-04-02T04:44:18.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:18.218+0000 7f64fc1e4700 1 -- 172.21.15.70:0/3167807174 wait complete. 2024-04-02T04:44:18.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:18 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3167807174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T04:44:18.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:18 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3167807174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T04:44:19.396 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:19 smithi070 ceph-mon[29212]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:19.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:19 smithi179 ceph-mon[36603]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:19.809 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-04-02T04:44:19.809 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:20.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.397+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2140357821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 msgr2=0x7fcbe8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.397+0000 7fcbecbb4700 1 --2- 172.21.15.70:0/2140357821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 0x7fcbe8100080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009910 tx=0x7fcbdc009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:20.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.397+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2140357821 shutdown_connections 2024-04-02T04:44:20.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.397+0000 7fcbecbb4700 1 --2- 172.21.15.70:0/2140357821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 0x7fcbe8100080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.397+0000 7fcbecbb4700 1 --2- 172.21.15.70:0/2140357821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe8105bd0 0x7fcbe8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.397+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2140357821 >> 172.21.15.70:0/2140357821 conn(0x7fcbe8075670 msgr2=0x7fcbe8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:20.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.397+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2140357821 shutdown_connections 2024-04-02T04:44:20.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.397+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2140357821 wait complete. 2024-04-02T04:44:20.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbecbb4700 1 Processor -- start 2024-04-02T04:44:20.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbecbb4700 1 -- start start 2024-04-02T04:44:20.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbecbb4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 0x7fcbe8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:20.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbecbb4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe8105bd0 0x7fcbe8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:20.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbecbb4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe81022c0 con 0x7fcbe80ffc20 2024-04-02T04:44:20.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbecbb4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe8102400 con 0x7fcbe8105bd0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbe659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 0x7fcbe8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbe659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 0x7fcbe8101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34366/0 (socket says 172.21.15.70:34366) 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbe659c700 1 -- 172.21.15.70:0/2943963319 learned_addr learned my addr 172.21.15.70:0/2943963319 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.398+0000 7fcbe5d9b700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe8105bd0 0x7fcbe8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbe5d9b700 1 -- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 msgr2=0x7fcbe8101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbe5d9b700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 0x7fcbe8101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbe5d9b700 1 -- 172.21.15.70:0/2943963319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0095f0 con 0x7fcbe8105bd0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbe5d9b700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe8105bd0 0x7fcbe8101d80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009910 tx=0x7fcbdc00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbd77fe700 1 -- 172.21.15.70:0/2943963319 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbdc01d070 con 0x7fcbe8105bd0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbd77fe700 1 -- 172.21.15.70:0/2943963319 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcbdc00f460 con 0x7fcbe8105bd0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe8102680 con 0x7fcbe8105bd0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbd77fe700 1 -- 172.21.15.70:0/2943963319 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbdc017630 con 0x7fcbe8105bd0 2024-04-02T04:44:20.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.399+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe81a2d60 con 0x7fcbe8105bd0 2024-04-02T04:44:20.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.400+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe804f2e0 con 0x7fcbe8105bd0 2024-04-02T04:44:20.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.401+0000 7fcbd77fe700 1 -- 172.21.15.70:0/2943963319 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fcbdc017790 con 0x7fcbe8105bd0 2024-04-02T04:44:20.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.401+0000 7fcbd77fe700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcbd006c4e0 0x7fcbd006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.401+0000 7fcbd77fe700 1 -- 172.21.15.70:0/2943963319 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcbdc08c860 con 0x7fcbe8105bd0 2024-04-02T04:44:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.401+0000 7fcbe659c700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcbd006c4e0 0x7fcbd006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.401+0000 7fcbe659c700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcbd006c4e0 0x7fcbd006e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8005950 tx=0x7fcbd800a400 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.406+0000 7fcbd77fe700 1 -- 172.21.15.70:0/2943963319 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcbdc057550 con 0x7fcbe8105bd0 2024-04-02T04:44:20.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.603+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcbe8102d70 con 0x7fcbe8105bd0 2024-04-02T04:44:20.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.603+0000 7fcbd77fe700 1 -- 172.21.15.70:0/2943963319 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcbdc05ab70 con 0x7fcbe8105bd0 2024-04-02T04:44:20.607 INFO:teuthology.orchestra.run.smithi070.stdout:171798691852 2024-04-02T04:44:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcbd006c4e0 msgr2=0x7fcbd006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcbd006c4e0 0x7fcbd006e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8005950 tx=0x7fcbd800a400 comp rx=0 tx=0).stop 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe8105bd0 msgr2=0x7fcbe8101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe8105bd0 0x7fcbe8101d80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009910 tx=0x7fcbdc00bd70 comp rx=0 tx=0).stop 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 shutdown_connections 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcbd006c4e0 0x7fcbd006e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe80ffc20 0x7fcbe8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 --2- 172.21.15.70:0/2943963319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe8105bd0 0x7fcbe8101d80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 >> 172.21.15.70:0/2943963319 conn(0x7fcbe8075670 msgr2=0x7fcbe8068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.605+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 shutdown_connections 2024-04-02T04:44:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:20.606+0000 7fcbecbb4700 1 -- 172.21.15.70:0/2943963319 wait complete. 2024-04-02T04:44:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.001+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3958115120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288100ea0 msgr2=0x7fc288101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.001+0000 7fc28f8bd700 1 --2- 172.21.15.70:0/3958115120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288100ea0 0x7fc288101280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc284009910 tx=0x7fc284009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.001+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3958115120 shutdown_connections 2024-04-02T04:44:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.001+0000 7fc28f8bd700 1 --2- 172.21.15.70:0/3958115120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc288101850 0x7fc2881058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.001+0000 7fc28f8bd700 1 --2- 172.21.15.70:0/3958115120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288100ea0 0x7fc288101280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.001+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3958115120 >> 172.21.15.70:0/3958115120 conn(0x7fc2880fc750 msgr2=0x7fc2880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.001+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3958115120 shutdown_connections 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.001+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3958115120 wait complete. 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28f8bd700 1 Processor -- start 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28f8bd700 1 -- start start 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28f8bd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc288100ea0 0x7fc288196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28f8bd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288101850 0x7fc288196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28f8bd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288197560 con 0x7fc288101850 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28f8bd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28819b370 con 0x7fc288100ea0 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28ce58700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288101850 0x7fc288196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28ce58700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288101850 0x7fc288196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34376/0 (socket says 172.21.15.70:34376) 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.002+0000 7fc28ce58700 1 -- 172.21.15.70:0/3195004762 learned_addr learned my addr 172.21.15.70:0/3195004762 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc28ce58700 1 -- 172.21.15.70:0/3195004762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc288100ea0 msgr2=0x7fc288196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc28ce58700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc288100ea0 0x7fc288196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc28ce58700 1 -- 172.21.15.70:0/3195004762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2840095f0 con 0x7fc288101850 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc28ce58700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288101850 0x7fc288196ed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc27800b480 tx=0x7fc27800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc27e7fc700 1 -- 172.21.15.70:0/3195004762 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2780118c0 con 0x7fc288101850 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28819b650 con 0x7fc288101850 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc27e7fc700 1 -- 172.21.15.70:0/3195004762 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc278010380 con 0x7fc288101850 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc27e7fc700 1 -- 172.21.15.70:0/3195004762 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc27800f4f0 con 0x7fc288101850 2024-04-02T04:44:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.003+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28819bba0 con 0x7fc288101850 2024-04-02T04:44:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.004+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288109240 con 0x7fc288101850 2024-04-02T04:44:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.004+0000 7fc27e7fc700 1 -- 172.21.15.70:0/3195004762 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc2780104f0 con 0x7fc288101850 2024-04-02T04:44:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.005+0000 7fc27e7fc700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc27406c4e0 0x7fc27406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.005+0000 7fc28d659700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc27406c4e0 0x7fc27406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.005+0000 7fc27e7fc700 1 -- 172.21.15.70:0/3195004762 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc27808b150 con 0x7fc288101850 2024-04-02T04:44:21.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.005+0000 7fc28d659700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc27406c4e0 0x7fc27406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc28400b5c0 tx=0x7fc284005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.009+0000 7fc27e7fc700 1 -- 172.21.15.70:0/3195004762 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc278059700 con 0x7fc288101850 2024-04-02T04:44:21.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.205+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc288197cf0 con 0x7fc288101850 2024-04-02T04:44:21.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.205+0000 7fc27e7fc700 1 -- 172.21.15.70:0/3195004762 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc278059290 con 0x7fc288101850 2024-04-02T04:44:21.209 INFO:teuthology.orchestra.run.smithi070.stdout:150323855377 2024-04-02T04:44:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc27406c4e0 msgr2=0x7fc27406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc27406c4e0 0x7fc27406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc28400b5c0 tx=0x7fc284005960 comp rx=0 tx=0).stop 2024-04-02T04:44:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288101850 msgr2=0x7fc288196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288101850 0x7fc288196ed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc27800b480 tx=0x7fc27800b840 comp rx=0 tx=0).stop 2024-04-02T04:44:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 shutdown_connections 2024-04-02T04:44:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc27406c4e0 0x7fc27406e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc288100ea0 0x7fc288196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 --2- 172.21.15.70:0/3195004762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc288101850 0x7fc288196ed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 >> 172.21.15.70:0/3195004762 conn(0x7fc2880fc750 msgr2=0x7fc2880fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 shutdown_connections 2024-04-02T04:44:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:21.207+0000 7fc28f8bd700 1 -- 172.21.15.70:0/3195004762 wait complete. 2024-04-02T04:44:21.518 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:21 smithi070 ceph-mon[29212]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:21.518 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:21 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2943963319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T04:44:21.518 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:21 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3195004762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T04:44:21.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:21 smithi179 ceph-mon[36603]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:21 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2943963319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T04:44:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:21 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3195004762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T04:44:21.598 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-04-02T04:44:21.598 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:22.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.413+0000 7f24a2880700 1 -- 172.21.15.70:0/493060462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 msgr2=0x7f249c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.413+0000 7f24a2880700 1 --2- 172.21.15.70:0/493060462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 0x7f249c1047a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2490009910 tx=0x7f2490009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.413+0000 7f24a2880700 1 -- 172.21.15.70:0/493060462 shutdown_connections 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.413+0000 7f24a2880700 1 --2- 172.21.15.70:0/493060462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 0x7f249c1047a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.413+0000 7f24a2880700 1 --2- 172.21.15.70:0/493060462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f249c0ffda0 0x7f249c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.413+0000 7f24a2880700 1 -- 172.21.15.70:0/493060462 >> 172.21.15.70:0/493060462 conn(0x7f249c0fb650 msgr2=0x7f249c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.413+0000 7f24a2880700 1 -- 172.21.15.70:0/493060462 shutdown_connections 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.413+0000 7f24a2880700 1 -- 172.21.15.70:0/493060462 wait complete. 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f24a2880700 1 Processor -- start 2024-04-02T04:44:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f24a2880700 1 -- start start 2024-04-02T04:44:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f24a2880700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f249c0ffda0 0x7f249c195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f24a2880700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 0x7f249c195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f24a2880700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c195fd0 con 0x7f249c100750 2024-04-02T04:44:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f24a2880700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c196140 con 0x7f249c0ffda0 2024-04-02T04:44:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f249b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 0x7f249c195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f249b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 0x7f249c195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34388/0 (socket says 172.21.15.70:34388) 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f249b7fe700 1 -- 172.21.15.70:0/624775488 learned_addr learned my addr 172.21.15.70:0/624775488 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f249bfff700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f249c0ffda0 0x7f249c195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f249b7fe700 1 -- 172.21.15.70:0/624775488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f249c0ffda0 msgr2=0x7f249c195550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f249b7fe700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f249c0ffda0 0x7f249c195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.414+0000 7f249b7fe700 1 -- 172.21.15.70:0/624775488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24900095f0 con 0x7f249c100750 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.415+0000 7f249b7fe700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 0x7f249c195a90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24900092e0 tx=0x7f249000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.415+0000 7f24997fa700 1 -- 172.21.15.70:0/624775488 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f249001d070 con 0x7f249c100750 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.415+0000 7f24997fa700 1 -- 172.21.15.70:0/624775488 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f249000f460 con 0x7f249c100750 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.415+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c196360 con 0x7f249c100750 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.415+0000 7f24997fa700 1 -- 172.21.15.70:0/624775488 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2490017630 con 0x7f249c100750 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.415+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c1a2ea0 con 0x7f249c100750 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.416+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c1081a0 con 0x7f249c100750 2024-04-02T04:44:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.416+0000 7f24997fa700 1 -- 172.21.15.70:0/624775488 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f2490003a60 con 0x7f249c100750 2024-04-02T04:44:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.417+0000 7f24997fa700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f248406c4e0 0x7f248406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.417+0000 7f24997fa700 1 -- 172.21.15.70:0/624775488 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f249008c920 con 0x7f249c100750 2024-04-02T04:44:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.417+0000 7f249bfff700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f248406c4e0 0x7f248406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.417+0000 7f249bfff700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f248406c4e0 0x7f248406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f248c005950 tx=0x7f248c0058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.425+0000 7f24997fa700 1 -- 172.21.15.70:0/624775488 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f249005aed0 con 0x7f249c100750 2024-04-02T04:44:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.624+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f249c069520 con 0x7f249c100750 2024-04-02T04:44:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.624+0000 7f24997fa700 1 -- 172.21.15.70:0/624775488 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f249005aa60 con 0x7f249c100750 2024-04-02T04:44:22.628 INFO:teuthology.orchestra.run.smithi070.stdout:115964117014 2024-04-02T04:44:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f248406c4e0 msgr2=0x7f248406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f248406c4e0 0x7f248406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f248c005950 tx=0x7f248c0058e0 comp rx=0 tx=0).stop 2024-04-02T04:44:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 msgr2=0x7f249c195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 0x7f249c195a90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24900092e0 tx=0x7f249000be20 comp rx=0 tx=0).stop 2024-04-02T04:44:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 shutdown_connections 2024-04-02T04:44:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f248406c4e0 0x7f248406e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f249c0ffda0 0x7f249c195550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 --2- 172.21.15.70:0/624775488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f249c100750 0x7f249c195a90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.626+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 >> 172.21.15.70:0/624775488 conn(0x7f249c0fb650 msgr2=0x7f249c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.627+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 shutdown_connections 2024-04-02T04:44:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:22.627+0000 7f24a2880700 1 -- 172.21.15.70:0/624775488 wait complete. 2024-04-02T04:44:22.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691852 for osd.5 2024-04-02T04:44:22.829 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:23.412 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:23 smithi070 ceph-mon[29212]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:23.412 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:23 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/624775488' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T04:44:23.531 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855374 got 150323855377 for osd.4 2024-04-02T04:44:23.531 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:23 smithi179 ceph-mon[36603]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:23 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/624775488' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T04:44:24.001 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117011 got 115964117014 for osd.3 2024-04-02T04:44:24.002 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:24.002 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-02T04:44:24.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T04:44:24.217 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:25.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:25 smithi179 ceph-mon[36603]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:25.596 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:25 smithi070 ceph-mon[29212]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.002+0000 7fd47dba0700 1 -- 172.21.15.70:0/1841690971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 msgr2=0x7fd478100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.002+0000 7fd47dba0700 1 --2- 172.21.15.70:0/1841690971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 0x7fd478100180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd468009910 tx=0x7fd468009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.002+0000 7fd47dba0700 1 -- 172.21.15.70:0/1841690971 shutdown_connections 2024-04-02T04:44:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.002+0000 7fd47dba0700 1 --2- 172.21.15.70:0/1841690971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd478100750 0x7fd4781047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.002+0000 7fd47dba0700 1 --2- 172.21.15.70:0/1841690971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 0x7fd478100180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.002+0000 7fd47dba0700 1 -- 172.21.15.70:0/1841690971 >> 172.21.15.70:0/1841690971 conn(0x7fd4780fb650 msgr2=0x7fd4780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.002+0000 7fd47dba0700 1 -- 172.21.15.70:0/1841690971 shutdown_connections 2024-04-02T04:44:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.002+0000 7fd47dba0700 1 -- 172.21.15.70:0/1841690971 wait complete. 2024-04-02T04:44:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd47dba0700 1 Processor -- start 2024-04-02T04:44:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd47dba0700 1 -- start start 2024-04-02T04:44:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd47dba0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 0x7fd478196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd47dba0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd478100750 0x7fd478196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd47dba0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd478197550 con 0x7fd4780ffda0 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd47dba0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47819b360 con 0x7fd478100750 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd4777fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 0x7fd478196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd4777fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 0x7fd478196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34400/0 (socket says 172.21.15.70:34400) 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd4777fe700 1 -- 172.21.15.70:0/32057545 learned_addr learned my addr 172.21.15.70:0/32057545 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd476ffd700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd478100750 0x7fd478196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd4777fe700 1 -- 172.21.15.70:0/32057545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd478100750 msgr2=0x7fd478196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd4777fe700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd478100750 0x7fd478196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd4777fe700 1 -- 172.21.15.70:0/32057545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680095f0 con 0x7fd4780ffda0 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.003+0000 7fd4777fe700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 0x7fd478196980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd468006010 tx=0x7fd46800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.004+0000 7fd474ff9700 1 -- 172.21.15.70:0/32057545 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd46801d070 con 0x7fd4780ffda0 2024-04-02T04:44:26.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.004+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47819b5e0 con 0x7fd4780ffda0 2024-04-02T04:44:26.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.004+0000 7fd474ff9700 1 -- 172.21.15.70:0/32057545 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd46800f460 con 0x7fd4780ffda0 2024-04-02T04:44:26.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.004+0000 7fd474ff9700 1 -- 172.21.15.70:0/32057545 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd468021640 con 0x7fd4780ffda0 2024-04-02T04:44:26.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.004+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47819bad0 con 0x7fd4780ffda0 2024-04-02T04:44:26.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.005+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47804f2e0 con 0x7fd4780ffda0 2024-04-02T04:44:26.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.005+0000 7fd474ff9700 1 -- 172.21.15.70:0/32057545 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd46800faf0 con 0x7fd4780ffda0 2024-04-02T04:44:26.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.005+0000 7fd474ff9700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd46006c4e0 0x7fd46006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:26.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.005+0000 7fd474ff9700 1 -- 172.21.15.70:0/32057545 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd46808c870 con 0x7fd4780ffda0 2024-04-02T04:44:26.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.006+0000 7fd476ffd700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd46006c4e0 0x7fd46006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:26.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.006+0000 7fd476ffd700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd46006c4e0 0x7fd46006e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd46c006fd0 tx=0x7fd46c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:26.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.010+0000 7fd474ff9700 1 -- 172.21.15.70:0/32057545 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd46805ae20 con 0x7fd4780ffda0 2024-04-02T04:44:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.192+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd47819bd40 con 0x7fd46006c4e0 2024-04-02T04:44:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.194+0000 7fd474ff9700 1 -- 172.21.15.70:0/32057545 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27889 (secure 0 0 0) 0x7fd47819bd40 con 0x7fd46006c4e0 2024-04-02T04:44:26.198 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:44:26.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd46006c4e0 msgr2=0x7fd46006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:26.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd46006c4e0 0x7fd46006e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd46c006fd0 tx=0x7fd46c008040 comp rx=0 tx=0).stop 2024-04-02T04:44:26.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 msgr2=0x7fd478196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:26.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 0x7fd478196980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd468006010 tx=0x7fd46800be20 comp rx=0 tx=0).stop 2024-04-02T04:44:26.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 shutdown_connections 2024-04-02T04:44:26.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd46006c4e0 0x7fd46006e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:26.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4780ffda0 0x7fd478196980 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:26.201 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 --2- 172.21.15.70:0/32057545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd478100750 0x7fd478196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:26.201 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 >> 172.21.15.70:0/32057545 conn(0x7fd4780fb650 msgr2=0x7fd478104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:26.201 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 shutdown_connections 2024-04-02T04:44:26.201 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:26.196+0000 7fd47dba0700 1 -- 172.21.15.70:0/32057545 wait complete. 2024-04-02T04:44:26.201 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-04-02T04:44:26.815 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-04-02T04:44:24.235958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002030"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":184,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-02T04:43:32.166314+0000","last_change":"2024-04-02T04:43:05.509572+0000","last_active":"2024-04-02T04:43:32.166314+0000","last_peered":"2024-04-02T04:43:32.166314+0000","last_clean":"2024-04-02T04:43:32.166314+0000","last_became_active":"2024-04-02T04:43:05.509143+0000","last_became_peered":"2024-04-02T04:43:05.509143+0000","last_unstale":"2024-04-02T04:43:32.166314+0000","last_undegraded":"2024-04-02T04:43:32.166314+0000","last_fullsized":"2024-04-02T04:43:32.166314+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T04:42:18.696201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T04:42:18.696201+0000","last_clean_scrub_stamp":"2024-04-02T04:42:18.696201+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:18:01.223562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.377},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Tue Apr 2 04:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Apr 2 04:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.437}]},{"osd":3,"last update":"Tue Apr 2 04:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.57699999999999996}]},{"osd":4,"last update":"Tue Apr 2 04:44:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.22800000000000001}]},{"osd":1,"last update":"Tue Apr 2 04:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.245}]},{"osd":3,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.66200000000000003}]},{"osd":4,"last update":"Tue Apr 2 04:44:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Apr 2 04:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.315,"15min":0.307},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.255},{"interface":"front","average":{"1min":0.376,"5min":0.34699999999999998,"15min":0.34200000000000003},"min":{"1min":0.221,"5min":0.218,"15min":0.218},"max":{"1min":0.51200000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.38800000000000001}]},{"osd":2,"last update":"Tue Apr 2 04:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":6.1269999999999998,"5min":6.1269999999999998,"15min":6.1269999999999998},"last":0.50800000000000001}]},{"osd":4,"last update":"Tue Apr 2 04:44:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:43:50 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.30099999999999999,"15min":0.29899999999999999},"min":{"1min":0.20799999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.313},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.34100000000000003,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45400000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.247}]},{"osd":2,"last update":"Tue Apr 2 04:43:19 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.504}]},{"osd":4,"last update":"Tue Apr 2 04:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T04:44:26.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T04:44:27.020 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:27.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:26 smithi070 ceph-mon[29212]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:44:27.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:26 smithi070 ceph-mon[29212]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:27.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:26 smithi179 ceph-mon[36603]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:44:27.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:26 smithi179 ceph-mon[36603]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:28.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:28.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:28.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 -- 172.21.15.70:0/3840098057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 msgr2=0x7f1a641070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:28.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 --2- 172.21.15.70:0/3840098057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 0x7f1a641070b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009910 tx=0x7f1a60009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:28.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 -- 172.21.15.70:0/3840098057 shutdown_connections 2024-04-02T04:44:28.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 --2- 172.21.15.70:0/3840098057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64068580 0x7f1a640689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 --2- 172.21.15.70:0/3840098057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 0x7f1a641070b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 -- 172.21.15.70:0/3840098057 >> 172.21.15.70:0/3840098057 conn(0x7f1a64075670 msgr2=0x7f1a64075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 -- 172.21.15.70:0/3840098057 shutdown_connections 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 -- 172.21.15.70:0/3840098057 wait complete. 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 Processor -- start 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.624+0000 7f1a6b634700 1 -- start start 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a6b634700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64068580 0x7f1a6419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a6b634700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 0x7f1a6419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a6b634700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6419d2b0 con 0x7f1a64106cd0 2024-04-02T04:44:28.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a6b634700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64196760 con 0x7f1a64068580 2024-04-02T04:44:28.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a68bcf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 0x7f1a6419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:28.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a68bcf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 0x7f1a6419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57974/0 (socket says 172.21.15.70:57974) 2024-04-02T04:44:28.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a68bcf700 1 -- 172.21.15.70:0/1138855501 learned_addr learned my addr 172.21.15.70:0/1138855501 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:28.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a693d0700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64068580 0x7f1a6419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:28.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a68bcf700 1 -- 172.21.15.70:0/1138855501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64068580 msgr2=0x7f1a6419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:28.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a68bcf700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64068580 0x7f1a6419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:28.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a68bcf700 1 -- 172.21.15.70:0/1138855501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600095f0 con 0x7f1a64106cd0 2024-04-02T04:44:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a693d0700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64068580 0x7f1a6419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:44:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a68bcf700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 0x7f1a6419cc20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400b4b0 tx=0x7f1a5400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a5a7fc700 1 -- 172.21.15.70:0/1138855501 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a54011800 con 0x7f1a64106cd0 2024-04-02T04:44:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a5a7fc700 1 -- 172.21.15.70:0/1138855501 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a54011e10 con 0x7f1a64106cd0 2024-04-02T04:44:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a64196a40 con 0x7f1a64106cd0 2024-04-02T04:44:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a5a7fc700 1 -- 172.21.15.70:0/1138855501 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a5400f2e0 con 0x7f1a64106cd0 2024-04-02T04:44:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.625+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a64196f90 con 0x7f1a64106cd0 2024-04-02T04:44:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.626+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a641091d0 con 0x7f1a64106cd0 2024-04-02T04:44:28.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.627+0000 7f1a5a7fc700 1 -- 172.21.15.70:0/1138855501 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1a54010380 con 0x7f1a64106cd0 2024-04-02T04:44:28.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.627+0000 7f1a5a7fc700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a5006c4e0 0x7f1a5006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:28.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.627+0000 7f1a5a7fc700 1 -- 172.21.15.70:0/1138855501 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1a5408b130 con 0x7f1a64106cd0 2024-04-02T04:44:28.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.627+0000 7f1a693d0700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a5006c4e0 0x7f1a5006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:28.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.628+0000 7f1a693d0700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a5006c4e0 0x7f1a5006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000b5c0 tx=0x7f1a60005480 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:28.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.632+0000 7f1a5a7fc700 1 -- 172.21.15.70:0/1138855501 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1a540596e0 con 0x7f1a64106cd0 2024-04-02T04:44:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.816+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a64197210 con 0x7f1a5006c4e0 2024-04-02T04:44:28.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.818+0000 7f1a5a7fc700 1 -- 172.21.15.70:0/1138855501 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27958 (secure 0 0 0) 0x7f1a64197210 con 0x7f1a5006c4e0 2024-04-02T04:44:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:44:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a5006c4e0 msgr2=0x7f1a5006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a5006c4e0 0x7f1a5006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000b5c0 tx=0x7f1a60005480 comp rx=0 tx=0).stop 2024-04-02T04:44:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 msgr2=0x7f1a6419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 0x7f1a6419cc20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400b4b0 tx=0x7f1a5400b870 comp rx=0 tx=0).stop 2024-04-02T04:44:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 shutdown_connections 2024-04-02T04:44:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f1a5006c4e0 0x7f1a5006e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64068580 0x7f1a6419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:28.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 --2- 172.21.15.70:0/1138855501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a64106cd0 0x7f1a6419cc20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:28.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 >> 172.21.15.70:0/1138855501 conn(0x7f1a64075670 msgr2=0x7f1a640fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:28.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 shutdown_connections 2024-04-02T04:44:28.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:28.820+0000 7f1a6b634700 1 -- 172.21.15.70:0/1138855501 wait complete. 2024-04-02T04:44:28.825 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-04-02T04:44:29.431 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:29 smithi070 ceph-mon[29212]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-04-02T04:44:28.236646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001958"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":184,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-02T04:43:32.166314+0000","last_change":"2024-04-02T04:43:05.509572+0000","last_active":"2024-04-02T04:43:32.166314+0000","last_peered":"2024-04-02T04:43:32.166314+0000","last_clean":"2024-04-02T04:43:32.166314+0000","last_became_active":"2024-04-02T04:43:05.509143+0000","last_became_peered":"2024-04-02T04:43:05.509143+0000","last_unstale":"2024-04-02T04:43:32.166314+0000","last_undegraded":"2024-04-02T04:43:32.166314+0000","last_fullsized":"2024-04-02T04:43:32.166314+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T04:42:18.696201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T04:42:18.696201+0000","last_clean_scrub_stamp":"2024-04-02T04:42:18.696201+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:18:01.223562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.377},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Tue Apr 2 04:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Apr 2 04:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.437}]},{"osd":3,"last update":"Tue Apr 2 04:44:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.376},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39200000000000002}]},{"osd":1,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Apr 2 04:44:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:44:23 2024","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.35699999999999998,"15min":0.36699999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48199999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.307,"5min":0.36299999999999999,"15min":0.373},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.28000000000000003}]},{"osd":1,"last update":"Tue Apr 2 04:44:23 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.307},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.38100000000000001,"15min":0.39000000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.29599999999999999}]},{"osd":3,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.48199999999999998}]},{"osd":4,"last update":"Tue Apr 2 04:44:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.34699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Apr 2 04:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.315,"15min":0.307},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.192},{"interface":"front","average":{"1min":0.376,"5min":0.34699999999999998,"15min":0.34200000000000003},"min":{"1min":0.221,"5min":0.218,"15min":0.218},"max":{"1min":0.51200000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.23499999999999999}]},{"osd":2,"last update":"Tue Apr 2 04:44:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.38,"15min":0.38800000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48499999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.38300000000000001,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.49399999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.26500000000000001}]},{"osd":3,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":6.1269999999999998,"5min":6.1269999999999998,"15min":6.1269999999999998},"last":0.40100000000000002}]},{"osd":4,"last update":"Tue Apr 2 04:44:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:43:50 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.30099999999999999,"15min":0.29899999999999999},"min":{"1min":0.20799999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.34100000000000003,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45400000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.23599999999999999}]},{"osd":2,"last update":"Tue Apr 2 04:44:23 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.36699999999999999,"15min":0.373},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.45500000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.318}]},{"osd":3,"last update":"Tue Apr 2 04:43:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Apr 2 04:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T04:44:29.452 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-02T04:44:29.453 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T04:44:29.453 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-02T04:44:29.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-04-02T04:44:29.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:29 smithi179 ceph-mon[36603]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:30.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:30 smithi179 ceph-mon[36603]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:44:30.762 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:30 smithi070 ceph-mon[29212]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:44:31.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 -- 172.21.15.70:0/3099317972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 msgr2=0x7fd0a8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:31.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 --2- 172.21.15.70:0/3099317972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 0x7fd0a8105fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd098009910 tx=0x7fd098009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:31.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 -- 172.21.15.70:0/3099317972 shutdown_connections 2024-04-02T04:44:31.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 --2- 172.21.15.70:0/3099317972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a80ffc20 0x7fd0a8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:31.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 --2- 172.21.15.70:0/3099317972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 0x7fd0a8105fb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:31.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 -- 172.21.15.70:0/3099317972 >> 172.21.15.70:0/3099317972 conn(0x7fd0a8075670 msgr2=0x7fd0a8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:31.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 -- 172.21.15.70:0/3099317972 shutdown_connections 2024-04-02T04:44:31.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 -- 172.21.15.70:0/3099317972 wait complete. 2024-04-02T04:44:31.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.329+0000 7fd0ace97700 1 Processor -- start 2024-04-02T04:44:31.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0ace97700 1 -- start start 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0ace97700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a80ffc20 0x7fd0a8197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0ace97700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 0x7fd0a8197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0ace97700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8198360 con 0x7fd0a8105bd0 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0ace97700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81984d0 con 0x7fd0a80ffc20 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0a5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 0x7fd0a8197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0a5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 0x7fd0a8197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57994/0 (socket says 172.21.15.70:57994) 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0a5d9b700 1 -- 172.21.15.70:0/1445810304 learned_addr learned my addr 172.21.15.70:0/1445810304 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0a659c700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a80ffc20 0x7fd0a8197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:31.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0a5d9b700 1 -- 172.21.15.70:0/1445810304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a80ffc20 msgr2=0x7fd0a8197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0a5d9b700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a80ffc20 0x7fd0a8197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0a5d9b700 1 -- 172.21.15.70:0/1445810304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980095f0 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0a5d9b700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 0x7fd0a8197c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b4b0 tx=0x7fd09c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.330+0000 7fd0977fe700 1 -- 172.21.15.70:0/1445810304 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09c011800 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.331+0000 7fd0977fe700 1 -- 172.21.15.70:0/1445810304 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd09c011e10 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.331+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a81a07b0 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.331+0000 7fd0977fe700 1 -- 172.21.15.70:0/1445810304 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09c00f2e0 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.331+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a81a0d00 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.331+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a81080d0 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.332+0000 7fd0977fe700 1 -- 172.21.15.70:0/1445810304 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd09c010380 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.332+0000 7fd0977fe700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd09006c4e0 0x7fd09006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.332+0000 7fd0977fe700 1 -- 172.21.15.70:0/1445810304 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd09c08b0b0 con 0x7fd0a8105bd0 2024-04-02T04:44:31.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.332+0000 7fd0a659c700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd09006c4e0 0x7fd09006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:31.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.333+0000 7fd0a659c700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd09006c4e0 0x7fd09006e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd098006010 tx=0x7fd09801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:31.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.337+0000 7fd0977fe700 1 -- 172.21.15.70:0/1445810304 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd09c055bd0 con 0x7fd0a8105bd0 2024-04-02T04:44:31.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:31 smithi070 ceph-mon[29212]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:31.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:31 smithi179 ceph-mon[36603]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:31.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.567+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd0a8068b90 con 0x7fd0a8105bd0 2024-04-02T04:44:31.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.568+0000 7fd0977fe700 1 -- 172.21.15.70:0/1445810304 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd09c0591f0 con 0x7fd0a8105bd0 2024-04-02T04:44:31.572 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:44:31.572 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd09006c4e0 msgr2=0x7fd09006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd09006c4e0 0x7fd09006e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd098006010 tx=0x7fd09801a040 comp rx=0 tx=0).stop 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 msgr2=0x7fd0a8197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 0x7fd0a8197c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b4b0 tx=0x7fd09c00b7c0 comp rx=0 tx=0).stop 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 shutdown_connections 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fd09006c4e0 0x7fd09006e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a80ffc20 0x7fd0a8197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 --2- 172.21.15.70:0/1445810304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0a8105bd0 0x7fd0a8197c80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 >> 172.21.15.70:0/1445810304 conn(0x7fd0a8075670 msgr2=0x7fd0a80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 shutdown_connections 2024-04-02T04:44:31.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:31.570+0000 7fd0ace97700 1 -- 172.21.15.70:0/1445810304 wait complete. 2024-04-02T04:44:32.241 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-02T04:44:32.241 INFO:tasks.cephadm:Setup complete, yielding 2024-04-02T04:44:32.241 INFO:teuthology.run_tasks:Running task print... 2024-04-02T04:44:32.276 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-04-02T04:44:32.277 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:44:32.311 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:44:32.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-02T04:44:32.520 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:32 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1445810304' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T04:44:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:32 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1445810304' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T04:44:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:33 smithi179 ceph-mon[36603]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:33.583 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:33 smithi070 ceph-mon[29212]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 -- 172.21.15.70:0/1354073113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e40ffc10 msgr2=0x7f72e4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 --2- 172.21.15.70:0/1354073113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e40ffc10 0x7f72e4100030 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009910 tx=0x7f72d4009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 -- 172.21.15.70:0/1354073113 shutdown_connections 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 --2- 172.21.15.70:0/1354073113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e4100e10 0x7f72e4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 --2- 172.21.15.70:0/1354073113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e40ffc10 0x7f72e4100030 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 -- 172.21.15.70:0/1354073113 >> 172.21.15.70:0/1354073113 conn(0x7f72e40fb190 msgr2=0x7f72e40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 -- 172.21.15.70:0/1354073113 shutdown_connections 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 -- 172.21.15.70:0/1354073113 wait complete. 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.205+0000 7f72e9eb6700 1 Processor -- start 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e9eb6700 1 -- start start 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e9eb6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e40ffc10 0x7f72e41964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e9eb6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e4100e10 0x7f72e4196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e9eb6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4197050 con 0x7f72e4100e10 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e9eb6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4197190 con 0x7f72e40ffc10 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e4100e10 0x7f72e4196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e4100e10 0x7f72e4196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58018/0 (socket says 172.21.15.70:58018) 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e2ffd700 1 -- 172.21.15.70:0/2876411911 learned_addr learned my addr 172.21.15.70:0/2876411911 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e37fe700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e40ffc10 0x7f72e41964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e2ffd700 1 -- 172.21.15.70:0/2876411911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e40ffc10 msgr2=0x7f72e41964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e2ffd700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e40ffc10 0x7f72e41964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e2ffd700 1 -- 172.21.15.70:0/2876411911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40095f0 con 0x7f72e4100e10 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e2ffd700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e4100e10 0x7f72e4196a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d8d0 tx=0x7f72d800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.206+0000 7f72e0ff9700 1 -- 172.21.15.70:0/2876411911 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d80098a0 con 0x7f72e4100e10 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.207+0000 7f72e0ff9700 1 -- 172.21.15.70:0/2876411911 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f72d8010460 con 0x7f72e4100e10 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.207+0000 7f72e9eb6700 1 -- 172.21.15.70:0/2876411911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e419bc40 con 0x7f72e4100e10 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.207+0000 7f72e0ff9700 1 -- 172.21.15.70:0/2876411911 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d800f5f0 con 0x7f72e4100e10 2024-04-02T04:44:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.207+0000 7f72e9eb6700 1 -- 172.21.15.70:0/2876411911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e419c190 con 0x7f72e4100e10 2024-04-02T04:44:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.208+0000 7f72ca7fc700 1 -- 172.21.15.70:0/2876411911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e4108700 con 0x7f72e4100e10 2024-04-02T04:44:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.208+0000 7f72e0ff9700 1 -- 172.21.15.70:0/2876411911 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f72d8009a00 con 0x7f72e4100e10 2024-04-02T04:44:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.209+0000 7f72e0ff9700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f72cc06c4e0 0x7f72cc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.209+0000 7f72e37fe700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f72cc06c4e0 0x7f72cc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.209+0000 7f72e0ff9700 1 -- 172.21.15.70:0/2876411911 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f72d808b2f0 con 0x7f72e4100e10 2024-04-02T04:44:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.209+0000 7f72e37fe700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f72cc06c4e0 0x7f72cc06e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72d4006010 tx=0x7f72d400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.213+0000 7f72e0ff9700 1 -- 172.21.15.70:0/2876411911 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f72d8059920 con 0x7f72e4100e10 2024-04-02T04:44:34.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.431+0000 7f72ca7fc700 1 -- 172.21.15.70:0/2876411911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f72e4108890 con 0x7f72e4100e10 2024-04-02T04:44:34.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.436+0000 7f72e0ff9700 1 -- 172.21.15.70:0/2876411911 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f72d80594b0 con 0x7f72e4100e10 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 -- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f72cc06c4e0 msgr2=0x7f72cc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f72cc06c4e0 0x7f72cc06e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72d4006010 tx=0x7f72d400b540 comp rx=0 tx=0).stop 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 -- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e4100e10 msgr2=0x7f72e4196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e4100e10 0x7f72e4196a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d8d0 tx=0x7f72d800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 -- 172.21.15.70:0/2876411911 shutdown_connections 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f72cc06c4e0 0x7f72cc06e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e40ffc10 0x7f72e41964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 --2- 172.21.15.70:0/2876411911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72e4100e10 0x7f72e4196a30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 -- 172.21.15.70:0/2876411911 >> 172.21.15.70:0/2876411911 conn(0x7f72e40fb190 msgr2=0x7f72e4068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 -- 172.21.15.70:0/2876411911 shutdown_connections 2024-04-02T04:44:34.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:34.439+0000 7f72ca7fc700 1 -- 172.21.15.70:0/2876411911 wait complete. 2024-04-02T04:44:34.454 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:34.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:35.119 INFO:teuthology.run_tasks:Running task print... 2024-04-02T04:44:35.128 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-02T04:44:35.129 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:44:35.151 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:44:35.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-04-02T04:44:35.355 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:35.701 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:35.702 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2876411911' entity='client.admin' 2024-04-02T04:44:35.702 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:35.702 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:35.702 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:35.702 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:35.702 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:35.702 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:35.702 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:35 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:35.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:35.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2876411911' entity='client.admin' 2024-04-02T04:44:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:35 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:37.016 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:36 smithi070 ceph-mon[29212]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:37.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:36 smithi179 ceph-mon[36603]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.113+0000 7f3191f74700 1 -- 172.21.15.70:0/2565481252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c101740 msgr2=0x7f318c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.113+0000 7f3191f74700 1 --2- 172.21.15.70:0/2565481252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c101740 0x7f318c103b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f317c009910 tx=0x7f317c009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.113+0000 7f3191f74700 1 -- 172.21.15.70:0/2565481252 shutdown_connections 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.113+0000 7f3191f74700 1 --2- 172.21.15.70:0/2565481252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c104070 0x7f318c106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.113+0000 7f3191f74700 1 --2- 172.21.15.70:0/2565481252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c101740 0x7f318c103b30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.113+0000 7f3191f74700 1 -- 172.21.15.70:0/2565481252 >> 172.21.15.70:0/2565481252 conn(0x7f318c0fb150 msgr2=0x7f318c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.114+0000 7f3191f74700 1 -- 172.21.15.70:0/2565481252 shutdown_connections 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.114+0000 7f3191f74700 1 -- 172.21.15.70:0/2565481252 wait complete. 2024-04-02T04:44:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.114+0000 7f3191f74700 1 Processor -- start 2024-04-02T04:44:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.114+0000 7f3191f74700 1 -- start start 2024-04-02T04:44:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3191f74700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c101740 0x7f318c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3191f74700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c104070 0x7f318c196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3191f74700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c197050 con 0x7f318c104070 2024-04-02T04:44:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3191f74700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c197190 con 0x7f318c101740 2024-04-02T04:44:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f318affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c104070 0x7f318c196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f318affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c104070 0x7f318c196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40212/0 (socket says 172.21.15.70:40212) 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f318affd700 1 -- 172.21.15.70:0/1436016731 learned_addr learned my addr 172.21.15.70:0/1436016731 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f318b7fe700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c101740 0x7f318c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f318affd700 1 -- 172.21.15.70:0/1436016731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c101740 msgr2=0x7f318c1964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f318affd700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c101740 0x7f318c1964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f318affd700 1 -- 172.21.15.70:0/1436016731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0095f0 con 0x7f318c104070 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f318affd700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c104070 0x7f318c196a30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f318000b4b0 tx=0x7f318000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3188ff9700 1 -- 172.21.15.70:0/1436016731 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3180011800 con 0x7f318c104070 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3188ff9700 1 -- 172.21.15.70:0/1436016731 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3180011e10 con 0x7f318c104070 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c19bc40 con 0x7f318c104070 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3188ff9700 1 -- 172.21.15.70:0/1436016731 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f318000f2e0 con 0x7f318c104070 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.115+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c19c190 con 0x7f318c104070 2024-04-02T04:44:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.116+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c1906a0 con 0x7f318c104070 2024-04-02T04:44:37.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.117+0000 7f3188ff9700 1 -- 172.21.15.70:0/1436016731 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f3180010380 con 0x7f318c104070 2024-04-02T04:44:37.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.117+0000 7f3188ff9700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f317406c4e0 0x7f317406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:37.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.117+0000 7f318b7fe700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f317406c4e0 0x7f317406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:37.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.117+0000 7f3188ff9700 1 -- 172.21.15.70:0/1436016731 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f318008b050 con 0x7f318c104070 2024-04-02T04:44:37.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.118+0000 7f318b7fe700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f317406c4e0 0x7f317406e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f317c0059d0 tx=0x7f317c005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.122+0000 7f3188ff9700 1 -- 172.21.15.70:0/1436016731 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3180059680 con 0x7f318c104070 2024-04-02T04:44:37.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.326+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f318c0611d0 con 0x7f317406c4e0 2024-04-02T04:44:37.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.327+0000 7f3188ff9700 1 -- 172.21.15.70:0/1436016731 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f318c0611d0 con 0x7f317406c4e0 2024-04-02T04:44:37.331 INFO:teuthology.orchestra.run.smithi070.stdout:Backend: cephadm 2024-04-02T04:44:37.331 INFO:teuthology.orchestra.run.smithi070.stdout:Available: Yes 2024-04-02T04:44:37.331 INFO:teuthology.orchestra.run.smithi070.stdout:Paused: No 2024-04-02T04:44:37.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.329+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f317406c4e0 msgr2=0x7f317406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:37.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.329+0000 7f3191f74700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f317406c4e0 0x7f317406e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f317c0059d0 tx=0x7f317c005920 comp rx=0 tx=0).stop 2024-04-02T04:44:37.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.329+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c104070 msgr2=0x7f318c196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:37.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.329+0000 7f3191f74700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c104070 0x7f318c196a30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f318000b4b0 tx=0x7f318000b870 comp rx=0 tx=0).stop 2024-04-02T04:44:37.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.330+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 shutdown_connections 2024-04-02T04:44:37.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.330+0000 7f3191f74700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f317406c4e0 0x7f317406e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:37.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.330+0000 7f3191f74700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f318c101740 0x7f318c1964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:37.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.330+0000 7f3191f74700 1 --2- 172.21.15.70:0/1436016731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f318c104070 0x7f318c196a30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:37.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.330+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 >> 172.21.15.70:0/1436016731 conn(0x7f318c0fb150 msgr2=0x7f318c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:37.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.330+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 shutdown_connections 2024-04-02T04:44:37.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:37.330+0000 7f3191f74700 1 -- 172.21.15.70:0/1436016731 wait complete. 2024-04-02T04:44:38.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:37 smithi070 ceph-mon[29212]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:38.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:37 smithi179 ceph-mon[36603]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:38.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-04-02T04:44:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:39.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:38 smithi070 ceph-mon[29212]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:39.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:38 smithi179 ceph-mon[36603]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.959+0000 7fe052b77700 1 -- 172.21.15.70:0/406809799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 msgr2=0x7fe04c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.959+0000 7fe052b77700 1 --2- 172.21.15.70:0/406809799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 0x7fe04c102310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe038009910 tx=0x7fe038009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:39.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.959+0000 7fe052b77700 1 -- 172.21.15.70:0/406809799 shutdown_connections 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.959+0000 7fe052b77700 1 --2- 172.21.15.70:0/406809799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 0x7fe04c102310 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.959+0000 7fe052b77700 1 --2- 172.21.15.70:0/406809799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe04c100cb0 0x7fe04c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.959+0000 7fe052b77700 1 -- 172.21.15.70:0/406809799 >> 172.21.15.70:0/406809799 conn(0x7fe04c0fc230 msgr2=0x7fe04c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.959+0000 7fe052b77700 1 -- 172.21.15.70:0/406809799 shutdown_connections 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.959+0000 7fe052b77700 1 -- 172.21.15.70:0/406809799 wait complete. 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe052b77700 1 Processor -- start 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe052b77700 1 -- start start 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe052b77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe04c100cb0 0x7fe04c196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe052b77700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 0x7fe04c196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe052b77700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c1970b0 con 0x7fe04c101eb0 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe052b77700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c1971f0 con 0x7fe04c100cb0 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe04bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 0x7fe04c196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:39.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe04bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 0x7fe04c196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40222/0 (socket says 172.21.15.70:40222) 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe04bfff700 1 -- 172.21.15.70:0/1576856574 learned_addr learned my addr 172.21.15.70:0/1576856574 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe050913700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe04c100cb0 0x7fe04c196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe04bfff700 1 -- 172.21.15.70:0/1576856574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe04c100cb0 msgr2=0x7fe04c196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe04bfff700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe04c100cb0 0x7fe04c196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.960+0000 7fe04bfff700 1 -- 172.21.15.70:0/1576856574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0380095f0 con 0x7fe04c101eb0 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.961+0000 7fe04bfff700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 0x7fe04c196a90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b5c0 tx=0x7fe03800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.961+0000 7fe049ffb700 1 -- 172.21.15.70:0/1576856574 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe03801d070 con 0x7fe04c101eb0 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.961+0000 7fe049ffb700 1 -- 172.21.15.70:0/1576856574 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe038022470 con 0x7fe04c101eb0 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.961+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c19bc40 con 0x7fe04c101eb0 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.961+0000 7fe049ffb700 1 -- 172.21.15.70:0/1576856574 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe03800f690 con 0x7fe04c101eb0 2024-04-02T04:44:39.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.961+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c19c130 con 0x7fe04c101eb0 2024-04-02T04:44:39.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.962+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c1098b0 con 0x7fe04c101eb0 2024-04-02T04:44:39.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.962+0000 7fe049ffb700 1 -- 172.21.15.70:0/1576856574 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe038004b70 con 0x7fe04c101eb0 2024-04-02T04:44:39.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.963+0000 7fe049ffb700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:39.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.963+0000 7fe049ffb700 1 -- 172.21.15.70:0/1576856574 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe03808cb80 con 0x7fe04c101eb0 2024-04-02T04:44:39.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.963+0000 7fe050913700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:39.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.963+0000 7fe050913700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe040006fd0 tx=0x7fe040008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:39.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:39.967+0000 7fe049ffb700 1 -- 172.21.15.70:0/1576856574 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe03805b260 con 0x7fe04c101eb0 2024-04-02T04:44:40.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.166+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe04c106800 con 0x7fe03c06c4e0 2024-04-02T04:44:40.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.175+0000 7fe049ffb700 1 -- 172.21.15.70:0/1576856574 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7fe04c106800 con 0x7fe03c06c4e0 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (3m) 2m ago 5m 25.5M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (5m) 2m ago 5m 7067k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (4m) 70s ago 4m 6975k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (5m) 2m ago 5m 6665k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (4m) 70s ago 4m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (3m) 2m ago 4m 75.5M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:9283,8765,8443 running (6m) 2m ago 6m 481M - 18.2.1 5be31c24972a 3299b9e265dd 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (4m) 70s ago 4m 440M - 18.2.1 5be31c24972a c82df176868c 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (6m) 2m ago 6m 42.5M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (4m) 70s ago 4m 45.1M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (5m) 2m ago 5m 14.2M - 1.5.0 0da6a335fe13 bc06c0b59df5 2024-04-02T04:44:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (4m) 70s ago 4m 17.2M - 1.5.0 0da6a335fe13 6fe44da459b3 2024-04-02T04:44:40.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (3m) 2m ago 3m 41.9M 3946M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:44:40.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (3m) 2m ago 3m 40.5M 3946M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:44:40.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (2m) 2m ago 2m 40.2M 3946M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:44:40.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (2m) 70s ago 2m 44.1M 4970M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:44:40.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (101s) 70s ago 100s 38.5M 4970M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:44:40.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (75s) 70s ago 74s 10.8M 4970M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:44:40.180 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (3m) 2m ago 4m 38.1M - 2.43.0 a07b618ecd1d b1f1f05601f5 2024-04-02T04:44:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe03c06c4e0 msgr2=0x7fe03c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe040006fd0 tx=0x7fe040008040 comp rx=0 tx=0).stop 2024-04-02T04:44:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 msgr2=0x7fe04c196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 0x7fe04c196a90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b5c0 tx=0x7fe03800be20 comp rx=0 tx=0).stop 2024-04-02T04:44:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 shutdown_connections 2024-04-02T04:44:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe03c06c4e0 0x7fe03c06e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe04c100cb0 0x7fe04c196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:40.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 --2- 172.21.15.70:0/1576856574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe04c101eb0 0x7fe04c196a90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.177+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 >> 172.21.15.70:0/1576856574 conn(0x7fe04c0fc230 msgr2=0x7fe04c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.178+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 shutdown_connections 2024-04-02T04:44:40.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:40.178+0000 7fe052b77700 1 -- 172.21.15.70:0/1576856574 wait complete. 2024-04-02T04:44:40.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-04-02T04:44:41.042 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:41.502 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:41 smithi070 ceph-mon[29212]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:41.502 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:41 smithi070 ceph-mon[29212]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:41.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:41 smithi179 ceph-mon[36603]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:41 smithi179 ceph-mon[36603]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:42.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 -- 172.21.15.70:0/4140587478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c100dc0 msgr2=0x7f028c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 --2- 172.21.15.70:0/4140587478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c100dc0 0x7f028c101220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0278009910 tx=0x7f0278009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 -- 172.21.15.70:0/4140587478 shutdown_connections 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 --2- 172.21.15.70:0/4140587478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c100dc0 0x7f028c101220 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 --2- 172.21.15.70:0/4140587478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f028c0ffbc0 0x7f028c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 -- 172.21.15.70:0/4140587478 >> 172.21.15.70:0/4140587478 conn(0x7f028c0fb160 msgr2=0x7f028c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 -- 172.21.15.70:0/4140587478 shutdown_connections 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 -- 172.21.15.70:0/4140587478 wait complete. 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.789+0000 7f0292e94700 1 Processor -- start 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0292e94700 1 -- start start 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0292e94700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c0ffbc0 0x7f028c196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0292e94700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f028c100dc0 0x7f028c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:42.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0292e94700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c1973d0 con 0x7f028c0ffbc0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0292e94700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c197510 con 0x7f028c100dc0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0290c30700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c0ffbc0 0x7f028c196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0290c30700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c0ffbc0 0x7f028c196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40244/0 (socket says 172.21.15.70:40244) 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0290c30700 1 -- 172.21.15.70:0/218340678 learned_addr learned my addr 172.21.15.70:0/218340678 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f028bfff700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f028c100dc0 0x7f028c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0290c30700 1 -- 172.21.15.70:0/218340678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f028c100dc0 msgr2=0x7f028c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0290c30700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f028c100dc0 0x7f028c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0290c30700 1 -- 172.21.15.70:0/218340678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780095f0 con 0x7f028c0ffbc0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0290c30700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c0ffbc0 0x7f028c196870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f028000b480 tx=0x7f028000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0289ffb700 1 -- 172.21.15.70:0/218340678 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0280011800 con 0x7f028c0ffbc0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0289ffb700 1 -- 172.21.15.70:0/218340678 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0280011e10 con 0x7f028c0ffbc0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c19bfc0 con 0x7f028c0ffbc0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.790+0000 7f0289ffb700 1 -- 172.21.15.70:0/218340678 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0280010ea0 con 0x7f028c0ffbc0 2024-04-02T04:44:42.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.791+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c19c480 con 0x7f028c0ffbc0 2024-04-02T04:44:42.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.792+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c068af0 con 0x7f028c0ffbc0 2024-04-02T04:44:42.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.792+0000 7f0289ffb700 1 -- 172.21.15.70:0/218340678 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0280010380 con 0x7f028c0ffbc0 2024-04-02T04:44:42.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.792+0000 7f0289ffb700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f027c06c4e0 0x7f027c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:42.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.793+0000 7f0289ffb700 1 -- 172.21.15.70:0/218340678 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f028008b060 con 0x7f028c0ffbc0 2024-04-02T04:44:42.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.793+0000 7f028bfff700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f027c06c4e0 0x7f027c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:42.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.793+0000 7f028bfff700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f027c06c4e0 0x7f027c06e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0278006010 tx=0x7f027800b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:42.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.797+0000 7f0289ffb700 1 -- 172.21.15.70:0/218340678 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0280059610 con 0x7f028c0ffbc0 2024-04-02T04:44:42.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.990+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f028c069560 con 0x7f027c06c4e0 2024-04-02T04:44:42.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.995+0000 7f0289ffb700 1 -- 172.21.15.70:0/218340678 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1300 (secure 0 0 0) 0x7f028c069560 con 0x7f027c06c4e0 2024-04-02T04:44:42.998 INFO:teuthology.orchestra.run.smithi070.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-02T04:44:42.998 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-04-02T04:44:42.998 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter 2/2 2m ago 6m * 2024-04-02T04:44:42.998 INFO:teuthology.orchestra.run.smithi070.stdout:crash 2/2 2m ago 6m * 2024-04-02T04:44:42.999 INFO:teuthology.orchestra.run.smithi070.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-04-02T04:44:42.999 INFO:teuthology.orchestra.run.smithi070.stdout:mgr 2/2 2m ago 6m count:2 2024-04-02T04:44:42.999 INFO:teuthology.orchestra.run.smithi070.stdout:mon 2/2 2m ago 5m smithi070:172.21.15.70=smithi070;smithi179:172.21.15.179=smithi179;count:2 2024-04-02T04:44:42.999 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-04-02T04:44:42.999 INFO:teuthology.orchestra.run.smithi070.stdout:osd 6 2m ago - 2024-04-02T04:44:42.999 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-04-02T04:44:43.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f027c06c4e0 msgr2=0x7f027c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:43.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f027c06c4e0 0x7f027c06e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0278006010 tx=0x7f027800b560 comp rx=0 tx=0).stop 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c0ffbc0 msgr2=0x7f028c196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c0ffbc0 0x7f028c196870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f028000b480 tx=0x7f028000b790 comp rx=0 tx=0).stop 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 shutdown_connections 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f027c06c4e0 0x7f027c06e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f028c0ffbc0 0x7f028c196870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 --2- 172.21.15.70:0/218340678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f028c100dc0 0x7f028c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 >> 172.21.15.70:0/218340678 conn(0x7f028c0fb160 msgr2=0x7f028c068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 shutdown_connections 2024-04-02T04:44:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:42.997+0000 7f0292e94700 1 -- 172.21.15.70:0/218340678 wait complete. 2024-04-02T04:44:43.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:43 smithi179 ceph-mon[36603]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:43.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:43.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:43 smithi070 ceph-mon[29212]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:43.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:43.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-04-02T04:44:43.867 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:44.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:44 smithi179 ceph-mon[36603]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:44.601 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:44 smithi070 ceph-mon[29212]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.404+0000 7fb517b4a700 1 -- 172.21.15.70:0/1173219132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb510104100 msgr2=0x7fb5101064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.404+0000 7fb517b4a700 1 --2- 172.21.15.70:0/1173219132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb510104100 0x7fb5101064f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb500009910 tx=0x7fb500009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.404+0000 7fb517b4a700 1 -- 172.21.15.70:0/1173219132 shutdown_connections 2024-04-02T04:44:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.404+0000 7fb517b4a700 1 --2- 172.21.15.70:0/1173219132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb510104100 0x7fb5101064f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.404+0000 7fb517b4a700 1 --2- 172.21.15.70:0/1173219132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5101017d0 0x7fb510103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.404+0000 7fb517b4a700 1 -- 172.21.15.70:0/1173219132 >> 172.21.15.70:0/1173219132 conn(0x7fb5100fb120 msgr2=0x7fb5100fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.404+0000 7fb517b4a700 1 -- 172.21.15.70:0/1173219132 shutdown_connections 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.404+0000 7fb517b4a700 1 -- 172.21.15.70:0/1173219132 wait complete. 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb517b4a700 1 Processor -- start 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb517b4a700 1 -- start start 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb517b4a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5101017d0 0x7fb510196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb517b4a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb510104100 0x7fb510196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb517b4a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb510197090 con 0x7fb5101017d0 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb517b4a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5101971d0 con 0x7fb510104100 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb5158e6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5101017d0 0x7fb510196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb5158e6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5101017d0 0x7fb510196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40252/0 (socket says 172.21.15.70:40252) 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb5158e6700 1 -- 172.21.15.70:0/3713056307 learned_addr learned my addr 172.21.15.70:0/3713056307 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb5150e5700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb510104100 0x7fb510196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb5158e6700 1 -- 172.21.15.70:0/3713056307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb510104100 msgr2=0x7fb510196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb5158e6700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb510104100 0x7fb510196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.405+0000 7fb5158e6700 1 -- 172.21.15.70:0/3713056307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5000095f0 con 0x7fb5101017d0 2024-04-02T04:44:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.406+0000 7fb5158e6700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5101017d0 0x7fb510196530 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00ba50 tx=0x7fb50c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.406+0000 7fb506ffd700 1 -- 172.21.15.70:0/3713056307 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50c00ca00 con 0x7fb5101017d0 2024-04-02T04:44:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.406+0000 7fb506ffd700 1 -- 172.21.15.70:0/3713056307 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb50c00cb60 con 0x7fb5101017d0 2024-04-02T04:44:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.406+0000 7fb506ffd700 1 -- 172.21.15.70:0/3713056307 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50c007580 con 0x7fb5101017d0 2024-04-02T04:44:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.406+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51019bc80 con 0x7fb5101017d0 2024-04-02T04:44:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.406+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51019c180 con 0x7fb5101017d0 2024-04-02T04:44:45.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.407+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5101906a0 con 0x7fb5101017d0 2024-04-02T04:44:45.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.407+0000 7fb506ffd700 1 -- 172.21.15.70:0/3713056307 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb50c00ccd0 con 0x7fb5101017d0 2024-04-02T04:44:45.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.407+0000 7fb506ffd700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb4fc06c4e0 0x7fb4fc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:45.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.408+0000 7fb5150e5700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb4fc06c4e0 0x7fb4fc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:45.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.408+0000 7fb506ffd700 1 -- 172.21.15.70:0/3713056307 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb50c006e40 con 0x7fb5101017d0 2024-04-02T04:44:45.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.408+0000 7fb5150e5700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb4fc06c4e0 0x7fb4fc06e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb5000059d0 tx=0x7fb500005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.412+0000 7fb506ffd700 1 -- 172.21.15.70:0/3713056307 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb50c059000 con 0x7fb5101017d0 2024-04-02T04:44:45.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:45 smithi070 ceph-mon[29212]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:45.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:45 smithi179 ceph-mon[36603]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:45.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.609+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5100611d0 con 0x7fb4fc06c4e0 2024-04-02T04:44:45.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.610+0000 7fb506ffd700 1 -- 172.21.15.70:0/3713056307 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fb5100611d0 con 0x7fb4fc06c4e0 2024-04-02T04:44:45.613 INFO:teuthology.orchestra.run.smithi070.stdout:HOST ADDR LABELS STATUS 2024-04-02T04:44:45.613 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070 172.21.15.70 2024-04-02T04:44:45.613 INFO:teuthology.orchestra.run.smithi070.stdout:smithi179 172.21.15.179 2024-04-02T04:44:45.613 INFO:teuthology.orchestra.run.smithi070.stdout:2 hosts in cluster 2024-04-02T04:44:45.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb4fc06c4e0 msgr2=0x7fb4fc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:45.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb4fc06c4e0 0x7fb4fc06e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb5000059d0 tx=0x7fb500005920 comp rx=0 tx=0).stop 2024-04-02T04:44:45.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5101017d0 msgr2=0x7fb510196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:45.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5101017d0 0x7fb510196530 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00ba50 tx=0x7fb50c00be10 comp rx=0 tx=0).stop 2024-04-02T04:44:45.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 shutdown_connections 2024-04-02T04:44:45.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb4fc06c4e0 0x7fb4fc06e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:45.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5101017d0 0x7fb510196530 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:45.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 --2- 172.21.15.70:0/3713056307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb510104100 0x7fb510196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:45.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 >> 172.21.15.70:0/3713056307 conn(0x7fb5100fb120 msgr2=0x7fb5100fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:45.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 shutdown_connections 2024-04-02T04:44:45.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:45.612+0000 7fb517b4a700 1 -- 172.21.15.70:0/3713056307 wait complete. 2024-04-02T04:44:46.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-04-02T04:44:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:46.919 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:46 smithi070 ceph-mon[29212]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:46.919 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:46 smithi070 ceph-mon[29212]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:46 smithi179 ceph-mon[36603]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:46 smithi179 ceph-mon[36603]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.138+0000 7f637c502700 1 -- 172.21.15.70:0/2328830371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 msgr2=0x7f6374101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.138+0000 7f637c502700 1 --2- 172.21.15.70:0/2328830371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 0x7f6374101220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6364009910 tx=0x7f6364009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.138+0000 7f637c502700 1 -- 172.21.15.70:0/2328830371 shutdown_connections 2024-04-02T04:44:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.138+0000 7f637c502700 1 --2- 172.21.15.70:0/2328830371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 0x7f6374101220 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.138+0000 7f637c502700 1 --2- 172.21.15.70:0/2328830371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63740ffbc0 0x7f63740fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.138+0000 7f637c502700 1 -- 172.21.15.70:0/2328830371 >> 172.21.15.70:0/2328830371 conn(0x7f63740fb160 msgr2=0x7f63740fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f637c502700 1 -- 172.21.15.70:0/2328830371 shutdown_connections 2024-04-02T04:44:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f637c502700 1 -- 172.21.15.70:0/2328830371 wait complete. 2024-04-02T04:44:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f637c502700 1 Processor -- start 2024-04-02T04:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f637c502700 1 -- start start 2024-04-02T04:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f637c502700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63740ffbc0 0x7f6374069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f637c502700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 0x7f637419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f637c502700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637419bb90 con 0x7f6374100dc0 2024-04-02T04:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f637c502700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637419bcd0 con 0x7f63740ffbc0 2024-04-02T04:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f6379a9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 0x7f637419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f6379a9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 0x7f637419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42112/0 (socket says 172.21.15.70:42112) 2024-04-02T04:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.139+0000 7f6379a9d700 1 -- 172.21.15.70:0/3805751602 learned_addr learned my addr 172.21.15.70:0/3805751602 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f637a29e700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63740ffbc0 0x7f6374069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f6379a9d700 1 -- 172.21.15.70:0/3805751602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63740ffbc0 msgr2=0x7f6374069970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f6379a9d700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63740ffbc0 0x7f6374069970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f6379a9d700 1 -- 172.21.15.70:0/3805751602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63640095f0 con 0x7f6374100dc0 2024-04-02T04:44:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f6379a9d700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 0x7f637419b5c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f636400b5c0 tx=0x7f636400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f636b7fe700 1 -- 172.21.15.70:0/3805751602 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f636401d070 con 0x7f6374100dc0 2024-04-02T04:44:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f636b7fe700 1 -- 172.21.15.70:0/3805751602 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6364022470 con 0x7f6374100dc0 2024-04-02T04:44:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637419bf50 con 0x7f6374100dc0 2024-04-02T04:44:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f636b7fe700 1 -- 172.21.15.70:0/3805751602 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f636400f690 con 0x7f6374100dc0 2024-04-02T04:44:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.140+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637419c410 con 0x7f6374100dc0 2024-04-02T04:44:48.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.141+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637418f5d0 con 0x7f6374100dc0 2024-04-02T04:44:48.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.142+0000 7f636b7fe700 1 -- 172.21.15.70:0/3805751602 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f6364004b70 con 0x7f6374100dc0 2024-04-02T04:44:48.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.142+0000 7f636b7fe700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f636006c4e0 0x7f636006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:48.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.142+0000 7f636b7fe700 1 -- 172.21.15.70:0/3805751602 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f636408cbb0 con 0x7f6374100dc0 2024-04-02T04:44:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.142+0000 7f637a29e700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f636006c4e0 0x7f636006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.143+0000 7f637a29e700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f636006c4e0 0x7f636006e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6370006fd0 tx=0x7f6370008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.147+0000 7f636b7fe700 1 -- 172.21.15.70:0/3805751602 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6364093080 con 0x7f6374100dc0 2024-04-02T04:44:48.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.346+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f63740611d0 con 0x7f636006c4e0 2024-04-02T04:44:48.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.348+0000 7f636b7fe700 1 -- 172.21.15.70:0/3805751602 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f63740611d0 con 0x7f636006c4e0 2024-04-02T04:44:48.352 INFO:teuthology.orchestra.run.smithi070.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-02T04:44:48.352 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002P480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-02T04:44:48.352 INFO:teuthology.orchestra.run.smithi070.stdout:smithi179 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009J400BGN 372G No 74s ago Has a FileSystem, LVM detected 2024-04-02T04:44:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f636006c4e0 msgr2=0x7f636006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f636006c4e0 0x7f636006e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6370006fd0 tx=0x7f6370008040 comp rx=0 tx=0).stop 2024-04-02T04:44:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 msgr2=0x7f637419b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 0x7f637419b5c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f636400b5c0 tx=0x7f636400be20 comp rx=0 tx=0).stop 2024-04-02T04:44:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 shutdown_connections 2024-04-02T04:44:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f636006c4e0 0x7f636006e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63740ffbc0 0x7f6374069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:48.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 --2- 172.21.15.70:0/3805751602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6374100dc0 0x7f637419b5c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:48.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 >> 172.21.15.70:0/3805751602 conn(0x7f63740fb160 msgr2=0x7f63741040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:48.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 shutdown_connections 2024-04-02T04:44:48.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:48.350+0000 7f637c502700 1 -- 172.21.15.70:0/3805751602 wait complete. 2024-04-02T04:44:49.101 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:44:49.112 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:44:49.112 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-02T04:44:49.321 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:49.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:49 smithi070 ceph-mon[29212]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:49.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:49 smithi070 ceph-mon[29212]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:49.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:49 smithi179 ceph-mon[36603]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:49.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:49 smithi179 ceph-mon[36603]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.957+0000 7f94583f9700 1 -- 172.21.15.70:0/4039346761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 msgr2=0x7f9450106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.957+0000 7f94583f9700 1 --2- 172.21.15.70:0/4039346761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 0x7f9450106460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9440009910 tx=0x7f9440009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.957+0000 7f94583f9700 1 -- 172.21.15.70:0/4039346761 shutdown_connections 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.957+0000 7f94583f9700 1 --2- 172.21.15.70:0/4039346761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 0x7f9450106460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.957+0000 7f94583f9700 1 --2- 172.21.15.70:0/4039346761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9450101740 0x7f9450103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.957+0000 7f94583f9700 1 -- 172.21.15.70:0/4039346761 >> 172.21.15.70:0/4039346761 conn(0x7f94500fb150 msgr2=0x7f94500fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.957+0000 7f94583f9700 1 -- 172.21.15.70:0/4039346761 shutdown_connections 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.957+0000 7f94583f9700 1 -- 172.21.15.70:0/4039346761 wait complete. 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f94583f9700 1 Processor -- start 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f94583f9700 1 -- start start 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f94583f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9450101740 0x7f94501010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f94583f9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 0x7f94500ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f94583f9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500ffc30 con 0x7f9450104070 2024-04-02T04:44:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f94583f9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500ffd70 con 0x7f9450101740 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f9455994700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 0x7f94500ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f9455994700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 0x7f94500ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42118/0 (socket says 172.21.15.70:42118) 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.958+0000 7f9455994700 1 -- 172.21.15.70:0/608785782 learned_addr learned my addr 172.21.15.70:0/608785782 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f9455994700 1 -- 172.21.15.70:0/608785782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9450101740 msgr2=0x7f94501010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f9456195700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9450101740 0x7f94501010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f9455994700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9450101740 0x7f94501010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f9455994700 1 -- 172.21.15.70:0/608785782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400095f0 con 0x7f9450104070 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f9455994700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 0x7f94500ff6f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9440006010 tx=0x7f944000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f94477fe700 1 -- 172.21.15.70:0/608785782 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f944001d070 con 0x7f9450104070 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94500ffff0 con 0x7f9450104070 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f94477fe700 1 -- 172.21.15.70:0/608785782 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f944000f460 con 0x7f9450104070 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f94477fe700 1 -- 172.21.15.70:0/608785782 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9440021640 con 0x7f9450104070 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.959+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94501004e0 con 0x7f9450104070 2024-04-02T04:44:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.960+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94501906a0 con 0x7f9450104070 2024-04-02T04:44:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.960+0000 7f94477fe700 1 -- 172.21.15.70:0/608785782 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f944000fab0 con 0x7f9450104070 2024-04-02T04:44:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.961+0000 7f94477fe700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f943c06c4e0 0x7f943c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.961+0000 7f94477fe700 1 -- 172.21.15.70:0/608785782 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f944008c970 con 0x7f9450104070 2024-04-02T04:44:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.961+0000 7f9456195700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f943c06c4e0 0x7f943c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:50.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.961+0000 7f9456195700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f943c06c4e0 0x7f943c06e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9448007dc0 tx=0x7f94480072f0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:50.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:50.965+0000 7f94477fe700 1 -- 172.21.15.70:0/608785782 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f944005b040 con 0x7f9450104070 2024-04-02T04:44:51.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:51.181+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f94500611d0 con 0x7f943c06c4e0 2024-04-02T04:44:51.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:51 smithi070 ceph-mon[29212]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:51.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:51 smithi179 ceph-mon[36603]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:52 smithi070 ceph-mon[29212]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-02T04:44:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-02T04:44:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:52 smithi070 ceph-mon[29212]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T04:44:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:52 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-02T04:44:52.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.812+0000 7f94477fe700 1 -- 172.21.15.70:0/608785782 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f94500611d0 con 0x7f943c06c4e0 2024-04-02T04:44:52.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f943c06c4e0 msgr2=0x7f943c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f943c06c4e0 0x7f943c06e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9448007dc0 tx=0x7f94480072f0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 msgr2=0x7f94500ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 0x7f94500ff6f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9440006010 tx=0x7f944000be20 comp rx=0 tx=0).stop 2024-04-02T04:44:52.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 shutdown_connections 2024-04-02T04:44:52.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f943c06c4e0 0x7f943c06e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9450101740 0x7f94501010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 --2- 172.21.15.70:0/608785782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9450104070 0x7f94500ff6f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 >> 172.21.15.70:0/608785782 conn(0x7f94500fb150 msgr2=0x7f94500fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:52.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 shutdown_connections 2024-04-02T04:44:52.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:52.815+0000 7f94583f9700 1 -- 172.21.15.70:0/608785782 wait complete. 2024-04-02T04:44:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:52 smithi179 ceph-mon[36603]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-02T04:44:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-02T04:44:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:52 smithi179 ceph-mon[36603]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T04:44:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:52 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-02T04:44:53.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:52 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[29208]: 2024-04-02T04:44:52.787+0000 7efef974e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:44:53.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T04:44:53.758 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: pgmap v177: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:53.758 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T04:44:53.758 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-02T04:44:53.758 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T04:44:53.758 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-02T04:44:53.758 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: osdmap e44: 6 total, 6 up, 6 in 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: fsmap cephfs:0 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.mnzrwt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.mnzrwt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:44:53.759 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:53 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: pgmap v177: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T04:44:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-02T04:44:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T04:44:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-02T04:44:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:44:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T04:44:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: osdmap e44: 6 total, 6 up, 6 in 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: fsmap cephfs:0 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.mnzrwt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.mnzrwt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:44:53.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:53 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:54.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:54 smithi070 ceph-mon[29212]: Saving service mds.cephfs spec with placement count:4 2024-04-02T04:44:54.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:54 smithi070 ceph-mon[29212]: Deploying daemon mds.cephfs.smithi070.mnzrwt on smithi070 2024-04-02T04:44:54.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:54 smithi070 ceph-mon[29212]: osdmap e45: 6 total, 6 up, 6 in 2024-04-02T04:44:54.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:54 smithi179 ceph-mon[36603]: Saving service mds.cephfs spec with placement count:4 2024-04-02T04:44:54.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:54 smithi179 ceph-mon[36603]: Deploying daemon mds.cephfs.smithi070.mnzrwt on smithi070 2024-04-02T04:44:54.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:54 smithi179 ceph-mon[36603]: osdmap e45: 6 total, 6 up, 6 in 2024-04-02T04:44:54.845 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:55.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:55 smithi070 ceph-mon[29212]: pgmap v181: 65 pgs: 20 creating+peering, 6 active+clean, 39 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:55.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:55 smithi070 ceph-mon[29212]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T04:44:55.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:55 smithi070 ceph-mon[29212]: osdmap e46: 6 total, 6 up, 6 in 2024-04-02T04:44:55.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:55 smithi179 ceph-mon[36603]: pgmap v181: 65 pgs: 20 creating+peering, 6 active+clean, 39 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:55.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:55 smithi179 ceph-mon[36603]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T04:44:55.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:55 smithi179 ceph-mon[36603]: osdmap e46: 6 total, 6 up, 6 in 2024-04-02T04:44:56.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.665+0000 7f999d2e3700 1 -- 172.21.15.70:0/1155919391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 msgr2=0x7f99981010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.665+0000 7f999d2e3700 1 --2- 172.21.15.70:0/1155919391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 0x7f99981010d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9988009910 tx=0x7f9988009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.665+0000 7f999d2e3700 1 -- 172.21.15.70:0/1155919391 shutdown_connections 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.665+0000 7f999d2e3700 1 --2- 172.21.15.70:0/1155919391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9998101eb0 0x7f9998102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.665+0000 7f999d2e3700 1 --2- 172.21.15.70:0/1155919391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 0x7f99981010d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.665+0000 7f999d2e3700 1 -- 172.21.15.70:0/1155919391 >> 172.21.15.70:0/1155919391 conn(0x7f99980fc230 msgr2=0x7f99980fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.665+0000 7f999d2e3700 1 -- 172.21.15.70:0/1155919391 shutdown_connections 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.666+0000 7f999d2e3700 1 -- 172.21.15.70:0/1155919391 wait complete. 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.666+0000 7f999d2e3700 1 Processor -- start 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.666+0000 7f999d2e3700 1 -- start start 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.666+0000 7f999d2e3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 0x7f9998196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.666+0000 7f999d2e3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9998101eb0 0x7f9998196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.666+0000 7f999d2e3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99981970c0 con 0x7f9998100cb0 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.666+0000 7f999d2e3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998197200 con 0x7f9998101eb0 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f9996ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 0x7f9998196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f9996ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 0x7f9998196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40036/0 (socket says 172.21.15.70:40036) 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f9996ffd700 1 -- 172.21.15.70:0/2865622934 learned_addr learned my addr 172.21.15.70:0/2865622934 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f99967fc700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9998101eb0 0x7f9998196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f9996ffd700 1 -- 172.21.15.70:0/2865622934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9998101eb0 msgr2=0x7f9998196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f9996ffd700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9998101eb0 0x7f9998196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f9996ffd700 1 -- 172.21.15.70:0/2865622934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99880095f0 con 0x7f9998100cb0 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f9996ffd700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 0x7f9998196560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f998800bda0 tx=0x7f998800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f997ffff700 1 -- 172.21.15.70:0/2865622934 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998801d070 con 0x7f9998100cb0 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f997ffff700 1 -- 172.21.15.70:0/2865622934 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9988004c50 con 0x7f9998100cb0 2024-04-02T04:44:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999819bc50 con 0x7f9998100cb0 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f997ffff700 1 -- 172.21.15.70:0/2865622934 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998800f690 con 0x7f9998100cb0 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.667+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999819c140 con 0x7f9998100cb0 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.668+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9998109890 con 0x7f9998100cb0 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.669+0000 7f997ffff700 1 -- 172.21.15.70:0/2865622934 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f99880229b0 con 0x7f9998100cb0 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.669+0000 7f997ffff700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f998006c4e0 0x7f998006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.669+0000 7f997ffff700 1 -- 172.21.15.70:0/2865622934 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f998808cf80 con 0x7f9998100cb0 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.669+0000 7f99967fc700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f998006c4e0 0x7f998006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.669+0000 7f99967fc700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f998006c4e0 0x7f998006e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f998c006fd0 tx=0x7f998c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.674+0000 7f997ffff700 1 -- 172.21.15.70:0/2865622934 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f998805b2a0 con 0x7f9998100cb0 2024-04-02T04:44:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.912+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9998109a80 con 0x7f9998100cb0 2024-04-02T04:44:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.912+0000 7f997ffff700 1 -- 172.21.15.70:0/2865622934 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f998805ae30 con 0x7f9998100cb0 2024-04-02T04:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout:e2 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 2 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:44:52.788984+0000 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:in 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:up {} 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 0 2024-04-02T04:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f998006c4e0 msgr2=0x7f998006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f998006c4e0 0x7f998006e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f998c006fd0 tx=0x7f998c008040 comp rx=0 tx=0).stop 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 msgr2=0x7f9998196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 0x7f9998196560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f998800bda0 tx=0x7f998800be80 comp rx=0 tx=0).stop 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 shutdown_connections 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f998006c4e0 0x7f998006e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9998100cb0 0x7f9998196560 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 --2- 172.21.15.70:0/2865622934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9998101eb0 0x7f9998196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 >> 172.21.15.70:0/2865622934 conn(0x7f99980fc230 msgr2=0x7f99981050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 shutdown_connections 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:56.915+0000 7f999d2e3700 1 -- 172.21.15.70:0/2865622934 wait complete. 2024-04-02T04:44:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 2 2024-04-02T04:44:56.921 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:56 smithi070 ceph-mon[29212]: pgmap v183: 65 pgs: 20 creating+peering, 36 active+clean, 9 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:57.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:56 smithi179 ceph-mon[36603]: pgmap v183: 65 pgs: 20 creating+peering, 36 active+clean, 9 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:57.916 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:57 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2865622934' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:44:57.916 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:57 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:57.916 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:57 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:57.916 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:57 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:57.917 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:57 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.mzfimf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:44:57.917 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:57 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.mzfimf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:44:57.917 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:57 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:57.917 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:57 smithi070 ceph-mon[29212]: Deploying daemon mds.cephfs.smithi179.mzfimf on smithi179 2024-04-02T04:44:57.939 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:44:57.948 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:44:57.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-02T04:44:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:57 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2865622934' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:44:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:57 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:57 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:57 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:44:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:57 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.mzfimf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:44:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:57 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.mzfimf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:44:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:57 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:58.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:57 smithi179 ceph-mon[36603]: Deploying daemon mds.cephfs.smithi179.mzfimf on smithi179 2024-04-02T04:44:58.157 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] up:boot 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: daemon mds.cephfs.smithi070.mnzrwt assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: Cluster is now healthy 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: fsmap cephfs:0 1 up:standby 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:creating} 2024-04-02T04:44:59.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: daemon mds.cephfs.smithi070.mnzrwt is now active in filesystem cephfs as rank 0 2024-04-02T04:44:59.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:59.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:58 smithi070 ceph-mon[29212]: pgmap v184: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:59.066 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] up:boot 2024-04-02T04:44:59.066 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: daemon mds.cephfs.smithi070.mnzrwt assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-02T04:44:59.066 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:44:59.066 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T04:44:59.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: Cluster is now healthy 2024-04-02T04:44:59.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: fsmap cephfs:0 1 up:standby 2024-04-02T04:44:59.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:44:59.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:creating} 2024-04-02T04:44:59.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: daemon mds.cephfs.smithi070.mnzrwt is now active in filesystem cephfs as rank 0 2024-04-02T04:44:59.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:59.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:58 smithi179 ceph-mon[36603]: pgmap v184: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:44:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.758+0000 7ff31aea6700 1 -- 172.21.15.70:0/14053062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3140ffa80 msgr2=0x7ff3140ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.758+0000 7ff31aea6700 1 --2- 172.21.15.70:0/14053062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3140ffa80 0x7ff3140ffea0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff308009910 tx=0x7ff308009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.758+0000 7ff31aea6700 1 -- 172.21.15.70:0/14053062 shutdown_connections 2024-04-02T04:44:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.758+0000 7ff31aea6700 1 --2- 172.21.15.70:0/14053062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff314100c80 0x7ff3141010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.758+0000 7ff31aea6700 1 --2- 172.21.15.70:0/14053062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3140ffa80 0x7ff3140ffea0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.758+0000 7ff31aea6700 1 -- 172.21.15.70:0/14053062 >> 172.21.15.70:0/14053062 conn(0x7ff3140fb020 msgr2=0x7ff3140fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.758+0000 7ff31aea6700 1 -- 172.21.15.70:0/14053062 shutdown_connections 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.758+0000 7ff31aea6700 1 -- 172.21.15.70:0/14053062 wait complete. 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff31aea6700 1 Processor -- start 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff31aea6700 1 -- start start 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff31aea6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3140ffa80 0x7ff314103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff31aea6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff314100c80 0x7ff314102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff31aea6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141025a0 con 0x7ff314100c80 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff31aea6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141026e0 con 0x7ff3140ffa80 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff313fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff314100c80 0x7ff314102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff313fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff314100c80 0x7ff314102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40078/0 (socket says 172.21.15.70:40078) 2024-04-02T04:44:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff313fff700 1 -- 172.21.15.70:0/4194867636 learned_addr learned my addr 172.21.15.70:0/4194867636 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.759+0000 7ff318c42700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3140ffa80 0x7ff314103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff313fff700 1 -- 172.21.15.70:0/4194867636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3140ffa80 msgr2=0x7ff314103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff313fff700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3140ffa80 0x7ff314103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff313fff700 1 -- 172.21.15.70:0/4194867636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080095f0 con 0x7ff314100c80 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff313fff700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff314100c80 0x7ff314102060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff30000e920 tx=0x7ff30000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff311ffb700 1 -- 172.21.15.70:0/4194867636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff30000c9d0 con 0x7ff314100c80 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff311ffb700 1 -- 172.21.15.70:0/4194867636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff300010430 con 0x7ff314100c80 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff311ffb700 1 -- 172.21.15.70:0/4194867636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff300018650 con 0x7ff314100c80 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3141029c0 con 0x7ff314100c80 2024-04-02T04:44:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.760+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31419e360 con 0x7ff314100c80 2024-04-02T04:44:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.761+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3141085e0 con 0x7ff314100c80 2024-04-02T04:44:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.762+0000 7ff311ffb700 1 -- 172.21.15.70:0/4194867636 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff3000105a0 con 0x7ff314100c80 2024-04-02T04:44:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.762+0000 7ff311ffb700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff30406c4e0 0x7ff30406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.762+0000 7ff311ffb700 1 -- 172.21.15.70:0/4194867636 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff300014070 con 0x7ff314100c80 2024-04-02T04:44:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.762+0000 7ff318c42700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff30406c4e0 0x7ff30406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.763+0000 7ff318c42700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff30406c4e0 0x7ff30406e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b6f0 tx=0x7ff308005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:44:59.767+0000 7ff311ffb700 1 -- 172.21.15.70:0/4194867636 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff3000520e0 con 0x7ff314100c80 2024-04-02T04:45:00.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.008+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7ff31419e910 con 0x7ff314100c80 2024-04-02T04:45:00.013 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:59 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] up:active 2024-04-02T04:45:00.013 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:44:59 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 2024-04-02T04:45:00.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:59 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] up:active 2024-04-02T04:45:00.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:44:59 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 2024-04-02T04:45:00.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.845+0000 7ff311ffb700 1 -- 172.21.15.70:0/4194867636 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff300059db0 con 0x7ff314100c80 2024-04-02T04:45:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff30406c4e0 msgr2=0x7ff30406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff30406c4e0 0x7ff30406e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b6f0 tx=0x7ff308005940 comp rx=0 tx=0).stop 2024-04-02T04:45:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff314100c80 msgr2=0x7ff314102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff314100c80 0x7ff314102060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff30000e920 tx=0x7ff30000ec30 comp rx=0 tx=0).stop 2024-04-02T04:45:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 shutdown_connections 2024-04-02T04:45:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7ff30406c4e0 0x7ff30406e9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3140ffa80 0x7ff314103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 --2- 172.21.15.70:0/4194867636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff314100c80 0x7ff314102060 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 >> 172.21.15.70:0/4194867636 conn(0x7ff3140fb020 msgr2=0x7ff314068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 shutdown_connections 2024-04-02T04:45:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:00.847+0000 7ff31aea6700 1 -- 172.21.15.70:0/4194867636 wait complete. 2024-04-02T04:45:01.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:boot 2024-04-02T04:45:01.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 1 up:standby 2024-04-02T04:45:01.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 1 up:standby 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.zvraae", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.zvraae", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: Deploying daemon mds.cephfs.smithi070.zvraae on smithi070 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/4194867636' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-02T04:45:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:00 smithi070 ceph-mon[29212]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 2 op/s 2024-04-02T04:45:01.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:boot 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 1 up:standby 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 1 up:standby 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.zvraae", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.zvraae", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: Deploying daemon mds.cephfs.smithi070.zvraae on smithi070 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/4194867636' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-02T04:45:01.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:00 smithi179 ceph-mon[36603]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 2 op/s 2024-04-02T04:45:02.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:01 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/4194867636' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-02T04:45:02.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:01 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 1 up:standby 2024-04-02T04:45:02.059 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:45:02.069 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:45:02.069 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-02T04:45:02.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:01 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/4194867636' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-02T04:45:02.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:01 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 1 up:standby 2024-04-02T04:45:02.726 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:03.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:02 smithi070 ceph-mon[29212]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 2 op/s 2024-04-02T04:45:03.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:02 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:03.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:02 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:03.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:02 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:03.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:02 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.atrmwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:03.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:02 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.atrmwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:45:03.160 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:02 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:03.214 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:02 smithi179 ceph-mon[36603]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 2 op/s 2024-04-02T04:45:03.214 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:02 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:03.214 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:02 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:03.214 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:02 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:03.215 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:02 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.atrmwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:03.215 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:02 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.atrmwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:45:03.215 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:02 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:04.062 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:03 smithi070 ceph-mon[29212]: Deploying daemon mds.cephfs.smithi179.atrmwz on smithi179 2024-04-02T04:45:04.062 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:03 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:boot 2024-04-02T04:45:04.063 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:03 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 2 up:standby 2024-04-02T04:45:04.063 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:03 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:45:04.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:03 smithi179 ceph-mon[36603]: Deploying daemon mds.cephfs.smithi179.atrmwz on smithi179 2024-04-02T04:45:04.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:03 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:boot 2024-04-02T04:45:04.222 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:03 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 2 up:standby 2024-04-02T04:45:04.222 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:03 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:45:04.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.327+0000 7feaeddba700 1 -- 172.21.15.70:0/3436639970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae80ffba0 msgr2=0x7feae80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:04.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.327+0000 7feaeddba700 1 --2- 172.21.15.70:0/3436639970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae80ffba0 0x7feae80fffc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fead0009910 tx=0x7fead0009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:04.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.327+0000 7feaeddba700 1 -- 172.21.15.70:0/3436639970 shutdown_connections 2024-04-02T04:45:04.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.327+0000 7feaeddba700 1 --2- 172.21.15.70:0/3436639970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feae8100da0 0x7feae8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:04.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.327+0000 7feaeddba700 1 --2- 172.21.15.70:0/3436639970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae80ffba0 0x7feae80fffc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:04.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.327+0000 7feaeddba700 1 -- 172.21.15.70:0/3436639970 >> 172.21.15.70:0/3436639970 conn(0x7feae80fb120 msgr2=0x7feae80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.327+0000 7feaeddba700 1 -- 172.21.15.70:0/3436639970 shutdown_connections 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.327+0000 7feaeddba700 1 -- 172.21.15.70:0/3436639970 wait complete. 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.328+0000 7feaeddba700 1 Processor -- start 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.328+0000 7feaeddba700 1 -- start start 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.328+0000 7feaeddba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feae80ffba0 0x7feae81953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.328+0000 7feaeddba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae8100da0 0x7feae8195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feaeddba700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae8195f30 con 0x7feae8100da0 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feaeddba700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae8196070 con 0x7feae80ffba0 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae8100da0 0x7feae8195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae8100da0 0x7feae8195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40108/0 (socket says 172.21.15.70:40108) 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae6ffd700 1 -- 172.21.15.70:0/476107466 learned_addr learned my addr 172.21.15.70:0/476107466 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:04.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae77fe700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feae80ffba0 0x7feae81953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae6ffd700 1 -- 172.21.15.70:0/476107466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feae80ffba0 msgr2=0x7feae81953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae6ffd700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feae80ffba0 0x7feae81953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae6ffd700 1 -- 172.21.15.70:0/476107466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead00095f0 con 0x7feae8100da0 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae6ffd700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae8100da0 0x7feae8195910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fead800d8d0 tx=0x7fead800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae4ff9700 1 -- 172.21.15.70:0/476107466 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead80098a0 con 0x7feae8100da0 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae4ff9700 1 -- 172.21.15.70:0/476107466 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fead8010460 con 0x7feae8100da0 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae819ab20 con 0x7feae8100da0 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feae4ff9700 1 -- 172.21.15.70:0/476107466 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead800f5f0 con 0x7feae8100da0 2024-04-02T04:45:04.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.329+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae819b070 con 0x7feae8100da0 2024-04-02T04:45:04.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.331+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae8108700 con 0x7feae8100da0 2024-04-02T04:45:04.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.331+0000 7feae4ff9700 1 -- 172.21.15.70:0/476107466 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fead8009a00 con 0x7feae8100da0 2024-04-02T04:45:04.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.331+0000 7feae4ff9700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fead406c4e0 0x7fead406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:04.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.332+0000 7feae77fe700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fead406c4e0 0x7fead406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:04.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.332+0000 7feae4ff9700 1 -- 172.21.15.70:0/476107466 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fead808b630 con 0x7feae8100da0 2024-04-02T04:45:04.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.332+0000 7feae77fe700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fead406c4e0 0x7fead406e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fead0006010 tx=0x7fead000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:04.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.336+0000 7feae4ff9700 1 -- 172.21.15.70:0/476107466 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fead8059920 con 0x7feae8100da0 2024-04-02T04:45:04.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.596+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7feae819b820 con 0x7feae8100da0 2024-04-02T04:45:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.857+0000 7feae4ff9700 1 -- 172.21.15.70:0/476107466 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fead80594b0 con 0x7feae8100da0 2024-04-02T04:45:04.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fead406c4e0 msgr2=0x7fead406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:04.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fead406c4e0 0x7fead406e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fead0006010 tx=0x7fead000b540 comp rx=0 tx=0).stop 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae8100da0 msgr2=0x7feae8195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae8100da0 0x7feae8195910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fead800d8d0 tx=0x7fead800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 shutdown_connections 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fead406c4e0 0x7fead406e9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feae80ffba0 0x7feae81953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 --2- 172.21.15.70:0/476107466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feae8100da0 0x7feae8195910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 >> 172.21.15.70:0/476107466 conn(0x7feae80fb120 msgr2=0x7feae8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 shutdown_connections 2024-04-02T04:45:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:04.859+0000 7feaeddba700 1 -- 172.21.15.70:0/476107466 wait complete. 2024-04-02T04:45:05.117 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:04 smithi179 ceph-mon[36603]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-02T04:45:05.117 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:04 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/476107466' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-02T04:45:05.219 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:04 smithi070 ceph-mon[29212]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-02T04:45:05.219 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:04 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/476107466' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-02T04:45:05.561 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:45:05.574 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:45:05.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-02T04:45:05.804 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:06.181 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:05 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/476107466' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-02T04:45:06.182 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:05 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 2 up:standby 2024-04-02T04:45:06.182 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.182 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.182 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.182 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.182 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.182 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:05 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:06.220 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:05 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/476107466' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-02T04:45:06.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:05 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 2 up:standby 2024-04-02T04:45:06.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:06.221 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:05 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:06.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:06 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:boot 2024-04-02T04:45:06.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:06 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] up:active 2024-04-02T04:45:06.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:06 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:45:06.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:06 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:45:06.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:06 smithi070 ceph-mon[29212]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:45:07.086 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:06 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:boot 2024-04-02T04:45:07.086 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:06 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] up:active 2024-04-02T04:45:07.086 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:06 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:45:07.086 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:06 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:45:07.086 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:06 smithi179 ceph-mon[36603]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:45:07.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:07 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:standby 2024-04-02T04:45:07.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:07 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:45:07.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:07 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:07.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:07 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:08.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.090+0000 7f32cfa42700 1 -- 172.21.15.70:0/2172522202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 msgr2=0x7f32c8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:08.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.090+0000 7f32cfa42700 1 --2- 172.21.15.70:0/2172522202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 0x7f32c8101270 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009910 tx=0x7f32b8009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:08.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.090+0000 7f32cfa42700 1 -- 172.21.15.70:0/2172522202 shutdown_connections 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.090+0000 7f32cfa42700 1 --2- 172.21.15.70:0/2172522202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 0x7f32c8101270 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.090+0000 7f32cfa42700 1 --2- 172.21.15.70:0/2172522202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c80ffc10 0x7f32c8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.090+0000 7f32cfa42700 1 -- 172.21.15.70:0/2172522202 >> 172.21.15.70:0/2172522202 conn(0x7f32c80fb190 msgr2=0x7f32c80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.090+0000 7f32cfa42700 1 -- 172.21.15.70:0/2172522202 shutdown_connections 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.090+0000 7f32cfa42700 1 -- 172.21.15.70:0/2172522202 wait complete. 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32cfa42700 1 Processor -- start 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32cfa42700 1 -- start start 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32cfa42700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c80ffc10 0x7f32c81964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32cfa42700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 0x7f32c81969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32cfa42700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8197010 con 0x7f32c8100e10 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32cfa42700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8197150 con 0x7f32c80ffc10 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32ccfdd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 0x7f32c81969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32ccfdd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 0x7f32c81969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51536/0 (socket says 172.21.15.70:51536) 2024-04-02T04:45:08.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32ccfdd700 1 -- 172.21.15.70:0/1851173512 learned_addr learned my addr 172.21.15.70:0/1851173512 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32cd7de700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c80ffc10 0x7f32c81964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32ccfdd700 1 -- 172.21.15.70:0/1851173512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c80ffc10 msgr2=0x7f32c81964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32ccfdd700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c80ffc10 0x7f32c81964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32ccfdd700 1 -- 172.21.15.70:0/1851173512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80095f0 con 0x7f32c8100e10 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.091+0000 7f32ccfdd700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 0x7f32c81969f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f32b800b5c0 tx=0x7f32b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.092+0000 7f32be7fc700 1 -- 172.21.15.70:0/1851173512 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b801d070 con 0x7f32c8100e10 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.092+0000 7f32be7fc700 1 -- 172.21.15.70:0/1851173512 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32b800f460 con 0x7f32c8100e10 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.092+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c819bba0 con 0x7f32c8100e10 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.092+0000 7f32be7fc700 1 -- 172.21.15.70:0/1851173512 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b8021640 con 0x7f32c8100e10 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.092+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c819c090 con 0x7f32c8100e10 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.093+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8068dd0 con 0x7f32c8100e10 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.093+0000 7f32be7fc700 1 -- 172.21.15.70:0/1851173512 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f32b800f5d0 con 0x7f32c8100e10 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.093+0000 7f32be7fc700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f32b406c4e0 0x7f32b406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:08.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.094+0000 7f32be7fc700 1 -- 172.21.15.70:0/1851173512 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f32b808cb70 con 0x7f32c8100e10 2024-04-02T04:45:08.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.094+0000 7f32cd7de700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f32b406c4e0 0x7f32b406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:08.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.094+0000 7f32cd7de700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f32b406c4e0 0x7f32b406e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32c4005950 tx=0x7f32c40058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:08.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.098+0000 7f32be7fc700 1 -- 172.21.15.70:0/1851173512 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f32b805ade0 con 0x7f32c8100e10 2024-04-02T04:45:08.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:07 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:standby 2024-04-02T04:45:08.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:07 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:45:08.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:07 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:08.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:07 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:08.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.346+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f32c8068fc0 con 0x7f32c8100e10 2024-04-02T04:45:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.646+0000 7f32be7fc700 1 -- 172.21.15.70:0/1851173512 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v13) v1 ==== 133+0+0 (secure 0 0 0) 0x7f32b805a970 con 0x7f32c8100e10 2024-04-02T04:45:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f32b406c4e0 msgr2=0x7f32b406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f32b406c4e0 0x7f32b406e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32c4005950 tx=0x7f32c40058e0 comp rx=0 tx=0).stop 2024-04-02T04:45:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 msgr2=0x7f32c81969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 0x7f32c81969f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f32b800b5c0 tx=0x7f32b800be20 comp rx=0 tx=0).stop 2024-04-02T04:45:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 shutdown_connections 2024-04-02T04:45:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f32b406c4e0 0x7f32b406e9a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c80ffc10 0x7f32c81964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 --2- 172.21.15.70:0/1851173512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32c8100e10 0x7f32c81969f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 >> 172.21.15.70:0/1851173512 conn(0x7f32c80fb190 msgr2=0x7f32c8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 shutdown_connections 2024-04-02T04:45:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:08.648+0000 7f32cfa42700 1 -- 172.21.15.70:0/1851173512 wait complete. 2024-04-02T04:45:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:inline data disabled 2024-04-02T04:45:09.522 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:09 smithi070 ceph-mon[29212]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:45:09.522 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:09 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:09.522 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:09 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1851173512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-02T04:45:09.522 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:09 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:standby 2024-04-02T04:45:09.522 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:09 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1851173512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-02T04:45:09.522 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:09 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:45:09.543 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:45:09.554 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:45:09.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T04:45:09.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:09 smithi179 ceph-mon[36603]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:45:09.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:09 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:09.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:09 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1851173512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-02T04:45:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:09 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:standby 2024-04-02T04:45:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:09 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1851173512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-02T04:45:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:09 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:45:10.275 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:10.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:10 smithi070 ceph-mon[29212]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:standby 2024-04-02T04:45:10.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:10 smithi070 ceph-mon[29212]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:45:10.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:10 smithi070 ceph-mon[29212]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:45:11.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:10 smithi179 ceph-mon[36603]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:standby 2024-04-02T04:45:11.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:10 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:45:11.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:10 smithi179 ceph-mon[36603]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:45:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.522+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/1121552480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4101750 msgr2=0x7f4ae4103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.522+0000 7f4ae8b4c700 1 --2- 172.21.15.70:0/1121552480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4101750 0x7f4ae4103b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009910 tx=0x7f4ad4009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.523+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/1121552480 shutdown_connections 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.523+0000 7f4ae8b4c700 1 --2- 172.21.15.70:0/1121552480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae4104080 0x7f4ae4106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.523+0000 7f4ae8b4c700 1 --2- 172.21.15.70:0/1121552480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4101750 0x7f4ae4103b40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.523+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/1121552480 >> 172.21.15.70:0/1121552480 conn(0x7f4ae40fb160 msgr2=0x7f4ae40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.523+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/1121552480 shutdown_connections 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.523+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/1121552480 wait complete. 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.523+0000 7f4ae8b4c700 1 Processor -- start 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.523+0000 7f4ae8b4c700 1 -- start start 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae8b4c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae4101750 0x7f4ae4196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae8b4c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4104080 0x7f4ae4196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae8b4c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4197380 con 0x7f4ae4104080 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae8b4c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41974c0 con 0x7f4ae4101750 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae1d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4104080 0x7f4ae4196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae1d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4104080 0x7f4ae4196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51558/0 (socket says 172.21.15.70:51558) 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae1d9b700 1 -- 172.21.15.70:0/2595428693 learned_addr learned my addr 172.21.15.70:0/2595428693 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae259c700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae4101750 0x7f4ae4196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae1d9b700 1 -- 172.21.15.70:0/2595428693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae4101750 msgr2=0x7f4ae4196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae1d9b700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae4101750 0x7f4ae4196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae1d9b700 1 -- 172.21.15.70:0/2595428693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad40095f0 con 0x7f4ae4104080 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae1d9b700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4104080 0x7f4ae4196d60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b4b0 tx=0x7f4ad800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ad37fe700 1 -- 172.21.15.70:0/2595428693 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad8011800 con 0x7f4ae4104080 2024-04-02T04:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae419bf70 con 0x7f4ae4104080 2024-04-02T04:45:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ad37fe700 1 -- 172.21.15.70:0/2595428693 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ad8011e10 con 0x7f4ae4104080 2024-04-02T04:45:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ad37fe700 1 -- 172.21.15.70:0/2595428693 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad800f2e0 con 0x7f4ae4104080 2024-04-02T04:45:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.524+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae419c490 con 0x7f4ae4104080 2024-04-02T04:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.526+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae41906b0 con 0x7f4ae4104080 2024-04-02T04:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.526+0000 7f4ad37fe700 1 -- 172.21.15.70:0/2595428693 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4ad800f440 con 0x7f4ae4104080 2024-04-02T04:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.527+0000 7f4ad37fe700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4acc06c4e0 0x7f4acc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.527+0000 7f4ad37fe700 1 -- 172.21.15.70:0/2595428693 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4ad808b2d0 con 0x7f4ae4104080 2024-04-02T04:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.527+0000 7f4ae259c700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4acc06c4e0 0x7f4acc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.527+0000 7f4ae259c700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4acc06c4e0 0x7f4acc06e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006010 tx=0x7f4ad400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.531+0000 7f4ad37fe700 1 -- 172.21.15.70:0/2595428693 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4ad80595c0 con 0x7f4ae4104080 2024-04-02T04:45:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.777+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ae4068af0 con 0x7f4ae4104080 2024-04-02T04:45:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.777+0000 7f4ad37fe700 1 -- 172.21.15.70:0/2595428693 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1826 (secure 0 0 0) 0x7f4ad8059150 con 0x7f4ae4104080 2024-04-02T04:45:12.781 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:45:12.781 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:45:12.781 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:45:12.782 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:45:12.783 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:12.784 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4acc06c4e0 msgr2=0x7f4acc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4acc06c4e0 0x7f4acc06e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006010 tx=0x7f4ad400b540 comp rx=0 tx=0).stop 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4104080 msgr2=0x7f4ae4196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4104080 0x7f4ae4196d60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b4b0 tx=0x7f4ad800b7c0 comp rx=0 tx=0).stop 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 shutdown_connections 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f4acc06c4e0 0x7f4acc06e9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ae4101750 0x7f4ae4196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 --2- 172.21.15.70:0/2595428693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ae4104080 0x7f4ae4196d60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 >> 172.21.15.70:0/2595428693 conn(0x7f4ae40fb160 msgr2=0x7f4ae40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 shutdown_connections 2024-04-02T04:45:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:12.780+0000 7f4ae8b4c700 1 -- 172.21.15.70:0/2595428693 wait complete. 2024-04-02T04:45:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:45:13.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:13 smithi070 ceph-mon[29212]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 2 op/s 2024-04-02T04:45:13.463 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:13 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2595428693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:45:13.463 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:13.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-02T04:45:13.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:13 smithi179 ceph-mon[36603]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 2 op/s 2024-04-02T04:45:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:13 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2595428693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:45:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:15.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:15 smithi070 ceph-mon[29212]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 1 op/s 2024-04-02T04:45:15.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:15.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:15.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:15.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:45:15.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:15.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:15 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:15.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:15 smithi179 ceph-mon[36603]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 1 op/s 2024-04-02T04:45:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:45:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:15 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:15.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.639+0000 7fc195d0d700 1 -- 172.21.15.70:0/3434362696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 msgr2=0x7fc1901010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:15.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.639+0000 7fc195d0d700 1 --2- 172.21.15.70:0/3434362696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 0x7fc1901010d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc180009910 tx=0x7fc180009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:15.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.639+0000 7fc195d0d700 1 -- 172.21.15.70:0/3434362696 shutdown_connections 2024-04-02T04:45:15.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.639+0000 7fc195d0d700 1 --2- 172.21.15.70:0/3434362696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc190101eb0 0x7fc190102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:15.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.639+0000 7fc195d0d700 1 --2- 172.21.15.70:0/3434362696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 0x7fc1901010d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:15.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.639+0000 7fc195d0d700 1 -- 172.21.15.70:0/3434362696 >> 172.21.15.70:0/3434362696 conn(0x7fc1900fc230 msgr2=0x7fc1900fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:15.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.639+0000 7fc195d0d700 1 -- 172.21.15.70:0/3434362696 shutdown_connections 2024-04-02T04:45:15.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.639+0000 7fc195d0d700 1 -- 172.21.15.70:0/3434362696 wait complete. 2024-04-02T04:45:15.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.640+0000 7fc195d0d700 1 Processor -- start 2024-04-02T04:45:15.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.640+0000 7fc195d0d700 1 -- start start 2024-04-02T04:45:15.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.640+0000 7fc195d0d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 0x7fc1901964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.640+0000 7fc195d0d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc190101eb0 0x7fc190196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.640+0000 7fc195d0d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc190197020 con 0x7fc190100cb0 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.640+0000 7fc195d0d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc190197160 con 0x7fc190101eb0 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.640+0000 7fc18f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 0x7fc1901964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.640+0000 7fc18f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 0x7fc1901964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51568/0 (socket says 172.21.15.70:51568) 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18f7fe700 1 -- 172.21.15.70:0/2006764010 learned_addr learned my addr 172.21.15.70:0/2006764010 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18effd700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc190101eb0 0x7fc190196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18f7fe700 1 -- 172.21.15.70:0/2006764010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc190101eb0 msgr2=0x7fc190196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18f7fe700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc190101eb0 0x7fc190196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18f7fe700 1 -- 172.21.15.70:0/2006764010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800095f0 con 0x7fc190100cb0 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18effd700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc190101eb0 0x7fc190196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18f7fe700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 0x7fc1901964c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc18000bda0 tx=0x7fc18000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:15.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18cff9700 1 -- 172.21.15.70:0/2006764010 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc18001d070 con 0x7fc190100cb0 2024-04-02T04:45:15.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18cff9700 1 -- 172.21.15.70:0/2006764010 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc18000f460 con 0x7fc190100cb0 2024-04-02T04:45:15.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19019bbb0 con 0x7fc190100cb0 2024-04-02T04:45:15.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc18cff9700 1 -- 172.21.15.70:0/2006764010 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc180021640 con 0x7fc190100cb0 2024-04-02T04:45:15.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.641+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19019c0a0 con 0x7fc190100cb0 2024-04-02T04:45:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.642+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc190109810 con 0x7fc190100cb0 2024-04-02T04:45:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.643+0000 7fc18cff9700 1 -- 172.21.15.70:0/2006764010 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc18000fa00 con 0x7fc190100cb0 2024-04-02T04:45:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.643+0000 7fc18cff9700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc17806c4e0 0x7fc17806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.643+0000 7fc18effd700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc17806c4e0 0x7fc17806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.643+0000 7fc18cff9700 1 -- 172.21.15.70:0/2006764010 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc18008c9f0 con 0x7fc190100cb0 2024-04-02T04:45:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.644+0000 7fc18effd700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc17806c4e0 0x7fc17806e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc184005fd0 tx=0x7fc184005ee0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.648+0000 7fc18cff9700 1 -- 172.21.15.70:0/2006764010 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc18008ee60 con 0x7fc190100cb0 2024-04-02T04:45:15.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.900+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc190109a00 con 0x7fc190100cb0 2024-04-02T04:45:15.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.901+0000 7fc18cff9700 1 -- 172.21.15.70:0/2006764010 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fc18005ac60 con 0x7fc190100cb0 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc17806c4e0 msgr2=0x7fc17806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc17806c4e0 0x7fc17806e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc184005fd0 tx=0x7fc184005ee0 comp rx=0 tx=0).stop 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 msgr2=0x7fc1901964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 0x7fc1901964c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc18000bda0 tx=0x7fc18000be80 comp rx=0 tx=0).stop 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 shutdown_connections 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fc17806c4e0 0x7fc17806e9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc190100cb0 0x7fc1901964c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 --2- 172.21.15.70:0/2006764010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc190101eb0 0x7fc190196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 >> 172.21.15.70:0/2006764010 conn(0x7fc1900fc230 msgr2=0x7fc1901050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 shutdown_connections 2024-04-02T04:45:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:15.903+0000 7fc195d0d700 1 -- 172.21.15.70:0/2006764010 wait complete. 2024-04-02T04:45:15.907 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:45:15.916 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:45:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:16 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2006764010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:45:17.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:16 smithi070 ceph-mon[29212]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:17.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-02T04:45:17.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:16 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2006764010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:45:17.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:16 smithi179 ceph-mon[36603]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:17.934 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:18.287 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:18.287 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:18 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:18.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:18 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:18.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:18 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:19.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:19 smithi070 ceph-mon[29212]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:19.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:19 smithi179 ceph-mon[36603]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:20.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.287+0000 7fcce32dd700 1 -- 172.21.15.70:0/367225609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1017a0 msgr2=0x7fccdc103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.287+0000 7fcce32dd700 1 --2- 172.21.15.70:0/367225609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1017a0 0x7fccdc103b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009910 tx=0x7fccd8009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.287+0000 7fcce32dd700 1 -- 172.21.15.70:0/367225609 shutdown_connections 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.287+0000 7fcce32dd700 1 --2- 172.21.15.70:0/367225609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccdc1040d0 0x7fccdc1064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.287+0000 7fcce32dd700 1 --2- 172.21.15.70:0/367225609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1017a0 0x7fccdc103b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.287+0000 7fcce32dd700 1 -- 172.21.15.70:0/367225609 >> 172.21.15.70:0/367225609 conn(0x7fccdc0fb190 msgr2=0x7fccdc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.287+0000 7fcce32dd700 1 -- 172.21.15.70:0/367225609 shutdown_connections 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.287+0000 7fcce32dd700 1 -- 172.21.15.70:0/367225609 wait complete. 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce32dd700 1 Processor -- start 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce32dd700 1 -- start start 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce32dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccdc1017a0 0x7fccdc196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce32dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1040d0 0x7fccdc1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce32dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc196fc0 con 0x7fccdc1040d0 2024-04-02T04:45:20.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce32dd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc197100 con 0x7fccdc1017a0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce0878700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1040d0 0x7fccdc1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce0878700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1040d0 0x7fccdc1969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57750/0 (socket says 172.21.15.70:57750) 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce0878700 1 -- 172.21.15.70:0/633623103 learned_addr learned my addr 172.21.15.70:0/633623103 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce1079700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccdc1017a0 0x7fccdc196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce0878700 1 -- 172.21.15.70:0/633623103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccdc1017a0 msgr2=0x7fccdc196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce0878700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccdc1017a0 0x7fccdc196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce0878700 1 -- 172.21.15.70:0/633623103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80095f0 con 0x7fccdc1040d0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.288+0000 7fcce0878700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1040d0 0x7fccdc1969a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00d8d0 tx=0x7fcccc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.289+0000 7fccd27fc700 1 -- 172.21.15.70:0/633623103 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcccc0098a0 con 0x7fccdc1040d0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.289+0000 7fccd27fc700 1 -- 172.21.15.70:0/633623103 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcccc010460 con 0x7fccdc1040d0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.289+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccdc19bbb0 con 0x7fccdc1040d0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.289+0000 7fccd27fc700 1 -- 172.21.15.70:0/633623103 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcccc00f5f0 con 0x7fccdc1040d0 2024-04-02T04:45:20.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.289+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc19c100 con 0x7fccdc1040d0 2024-04-02T04:45:20.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.289+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccdc190610 con 0x7fccdc1040d0 2024-04-02T04:45:20.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.290+0000 7fccd27fc700 1 -- 172.21.15.70:0/633623103 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fcccc009a00 con 0x7fccdc1040d0 2024-04-02T04:45:20.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.290+0000 7fccd27fc700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fccc806c4e0 0x7fccc806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:20.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.290+0000 7fccd27fc700 1 -- 172.21.15.70:0/633623103 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fcccc08b6d0 con 0x7fccdc1040d0 2024-04-02T04:45:20.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.290+0000 7fcce1079700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fccc806c4e0 0x7fccc806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:20.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.291+0000 7fcce1079700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fccc806c4e0 0x7fccc806e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fccd8006010 tx=0x7fccd800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:20.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.295+0000 7fccd27fc700 1 -- 172.21.15.70:0/633623103 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcccc059940 con 0x7fccdc1040d0 2024-04-02T04:45:20.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.580+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fccdc02cc70 con 0x7fccdc1040d0 2024-04-02T04:45:20.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.580+0000 7fccd27fc700 1 -- 172.21.15.70:0/633623103 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v14) v1 ==== 78+0+83 (secure 0 0 0) 0x7fcccc0594d0 con 0x7fccdc1040d0 2024-04-02T04:45:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.582+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fccc806c4e0 msgr2=0x7fccc806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.582+0000 7fcce32dd700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fccc806c4e0 0x7fccc806e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fccd8006010 tx=0x7fccd800b540 comp rx=0 tx=0).stop 2024-04-02T04:45:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.582+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1040d0 msgr2=0x7fccdc1969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.582+0000 7fcce32dd700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1040d0 0x7fccdc1969a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00d8d0 tx=0x7fcccc00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:45:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.582+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 shutdown_connections 2024-04-02T04:45:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.582+0000 7fcce32dd700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fccc806c4e0 0x7fccc806e9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.582+0000 7fcce32dd700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccdc1017a0 0x7fccdc196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.582+0000 7fcce32dd700 1 --2- 172.21.15.70:0/633623103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccdc1040d0 0x7fccdc1969a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.583+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 >> 172.21.15.70:0/633623103 conn(0x7fccdc0fb190 msgr2=0x7fccdc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.583+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 shutdown_connections 2024-04-02T04:45:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:20.583+0000 7fcce32dd700 1 -- 172.21.15.70:0/633623103 wait complete. 2024-04-02T04:45:20.596 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:45:21.265 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-02T04:45:21.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T04:45:21.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 -- 172.21.15.70:0/3304870832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c101430 msgr2=0x7f2c4c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 --2- 172.21.15.70:0/3304870832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c101430 0x7f2c4c1018b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009880 tx=0x7f2c4002bce0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 -- 172.21.15.70:0/3304870832 shutdown_connections 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 --2- 172.21.15.70:0/3304870832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c101430 0x7f2c4c1018b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 --2- 172.21.15.70:0/3304870832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c0fe3c0 0x7f2c4c0fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 -- 172.21.15.70:0/3304870832 >> 172.21.15.70:0/3304870832 conn(0x7f2c4c0f9cd0 msgr2=0x7f2c4c0fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 -- 172.21.15.70:0/3304870832 shutdown_connections 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 -- 172.21.15.70:0/3304870832 wait complete. 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 Processor -- start 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.409+0000 7f2c52705640 1 -- start start 2024-04-02T04:45:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c52705640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c0fe3c0 0x7f2c4c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c52705640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c101430 0x7f2c4c195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c52705640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c1963f0 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c52705640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c196560 con 0x7f2c4c101430 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c0fe3c0 0x7f2c4c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c0fe3c0 0x7f2c4c1958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57780/0 (socket says 172.21.15.70:57780) 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c4bfff640 1 -- 172.21.15.70:0/3252726884 learned_addr learned my addr 172.21.15.70:0/3252726884 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c4b7fe640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c101430 0x7f2c4c195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c4bfff640 1 -- 172.21.15.70:0/3252726884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c101430 msgr2=0x7f2c4c195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c4bfff640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c101430 0x7f2c4c195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c4bfff640 1 -- 172.21.15.70:0/3252726884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40009560 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c4bfff640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c0fe3c0 0x7f2c4c1958e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00d8d0 tx=0x7f2c3c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c497fa640 1 -- 172.21.15.70:0/3252726884 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c3c009a30 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c497fa640 1 -- 172.21.15.70:0/3252726884 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2c3c009b90 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c19aff0 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c497fa640 1 -- 172.21.15.70:0/3252726884 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c3c00f680 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.410+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c19b4c0 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.411+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c108e70 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.412+0000 7f2c497fa640 1 -- 172.21.15.70:0/3252726884 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f2c3c009d00 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.412+0000 7f2c497fa640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2c3406c510 0x7f2c3406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:21.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.412+0000 7f2c4b7fe640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2c3406c510 0x7f2c3406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:21.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.412+0000 7f2c497fa640 1 -- 172.21.15.70:0/3252726884 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2c3c08b600 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.413+0000 7f2c4b7fe640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2c3406c510 0x7f2c3406e9d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2c40006010 tx=0x7f2c4002c700 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:21.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.417+0000 7f2c497fa640 1 -- 172.21.15.70:0/3252726884 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2c3c055e90 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:21 smithi070 ceph-mon[29212]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:21.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:21 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/633623103' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-02T04:45:21.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:21 smithi179 ceph-mon[36603]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:21 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/633623103' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-02T04:45:21.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.589+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c4c19b7a0 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.590+0000 7f2c497fa640 1 -- 172.21.15.70:0/3252726884 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15010 (secure 0 0 0) 0x7f2c3c0594b0 con 0x7f2c4c0fe3c0 2024-04-02T04:45:21.593 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:45:21.593 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":46,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:37:49.993595+0000","modified":"2024-04-02T04:44:54.799219+0000","last_up_change":"2024-04-02T04:43:32.163505+0000","last_in_change":"2024-04-02T04:43:12.453452+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:42:18.484513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-02T04:44:51.427296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-02T04:44:52.024713+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b5e5510-c4db-4073-b470-c777b557d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6803","nonce":1163584079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6805","nonce":1163584079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6809","nonce":1163584079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1163584079},{"type":"v1","addr":"172.21.15.70:6807","nonce":1163584079}]},"public_addr":"172.21.15.70:6803/1163584079","cluster_addr":"172.21.15.70:6805/1163584079","heartbeat_back_addr":"172.21.15.70:6809/1163584079","heartbeat_front_addr":"172.21.15.70:6807/1163584079","state":["exists","up"]},{"osd":1,"uuid":"b751ceaf-b734-4280-ba22-82683b9776ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6811","nonce":3081297537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6813","nonce":3081297537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6817","nonce":3081297537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3081297537},{"type":"v1","addr":"172.21.15.70:6815","nonce":3081297537}]},"public_addr":"172.21.15.70:6811/3081297537","cluster_addr":"172.21.15.70:6813/3081297537","heartbeat_back_addr":"172.21.15.70:6817/3081297537","heartbeat_front_addr":"172.21.15.70:6815/3081297537","state":["exists","up"]},{"osd":2,"uuid":"32080ab6-7809-4523-aab7-ca7a6d263ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6819","nonce":428679321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6821","nonce":428679321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6825","nonce":428679321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":428679321},{"type":"v1","addr":"172.21.15.70:6823","nonce":428679321}]},"public_addr":"172.21.15.70:6819/428679321","cluster_addr":"172.21.15.70:6821/428679321","heartbeat_back_addr":"172.21.15.70:6825/428679321","heartbeat_front_addr":"172.21.15.70:6823/428679321","state":["exists","up"]},{"osd":3,"uuid":"0d71a0bb-5e7a-4339-afe7-06174dea4986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6801","nonce":1047738021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6803","nonce":1047738021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6807","nonce":1047738021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1047738021},{"type":"v1","addr":"172.21.15.179:6805","nonce":1047738021}]},"public_addr":"172.21.15.179:6801/1047738021","cluster_addr":"172.21.15.179:6803/1047738021","heartbeat_back_addr":"172.21.15.179:6807/1047738021","heartbeat_front_addr":"172.21.15.179:6805/1047738021","state":["exists","up"]},{"osd":4,"uuid":"754e6f09-2a8e-42e2-a998-5f5a1addcb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6809","nonce":2251755550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6811","nonce":2251755550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6815","nonce":2251755550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2251755550},{"type":"v1","addr":"172.21.15.179:6813","nonce":2251755550}]},"public_addr":"172.21.15.179:6809/2251755550","cluster_addr":"172.21.15.179:6811/2251755550","heartbeat_back_addr":"172.21.15.179:6815/2251755550","heartbeat_front_addr":"172.21.15.179:6813/2251755550","state":["exists","up"]},{"osd":5,"uuid":"46a73805-a770-4ed1-96f4-e64cf88810eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6817","nonce":2668632021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6819","nonce":2668632021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6823","nonce":2668632021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2668632021},{"type":"v1","addr":"172.21.15.179:6821","nonce":2668632021}]},"public_addr":"172.21.15.179:6817/2668632021","cluster_addr":"172.21.15.179:6819/2668632021","heartbeat_back_addr":"172.21.15.179:6823/2668632021","heartbeat_front_addr":"172.21.15.179:6821/2668632021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:17.145441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:44.622049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:14.717560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:38.493463+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:04.319117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:30.171864+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/110321404":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/2937797857":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6801/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1721127786":"2024-04-03T04:38:51.244832+0000","172.21.15.70:6800/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/4005781191":"2024-04-03T04:38:18.397033+0000","172.21.15.70:6800/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/4149690148":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/2446527261":"2024-04-03T04:38:18.397033+0000","172.21.15.70:6801/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:6801/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/3512109322":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/2463094374":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/940716355":"2024-04-03T04:38:18.397033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:45:21.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2c3406c510 msgr2=0x7f2c3406e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:21.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2c3406c510 0x7f2c3406e9d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2c40006010 tx=0x7f2c4002c700 comp rx=0 tx=0).stop 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c0fe3c0 msgr2=0x7f2c4c1958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c0fe3c0 0x7f2c4c1958e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00d8d0 tx=0x7f2c3c00dd90 comp rx=0 tx=0).stop 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 shutdown_connections 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2c3406c510 0x7f2c3406e9d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c101430 0x7f2c4c195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 --2- 172.21.15.70:0/3252726884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c4c0fe3c0 0x7f2c4c1958e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 >> 172.21.15.70:0/3252726884 conn(0x7f2c4c0f9cd0 msgr2=0x7f2c4c104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 shutdown_connections 2024-04-02T04:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.592+0000 7f2c52705640 1 -- 172.21.15.70:0/3252726884 wait complete. 2024-04-02T04:45:21.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:42:18.484513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-02T04:44:51.427296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-02T04:44:52.024713+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T04:45:21.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T04:45:21.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 -- 172.21.15.70:0/2011367050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c103440 msgr2=0x7f9c8c105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 --2- 172.21.15.70:0/2011367050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c103440 0x7f9c8c105830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9c800098b0 tx=0x7f9c8002bcd0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 -- 172.21.15.70:0/2011367050 shutdown_connections 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 --2- 172.21.15.70:0/2011367050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c103440 0x7f9c8c105830 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 --2- 172.21.15.70:0/2011367050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c8c100b10 0x7f9c8c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 -- 172.21.15.70:0/2011367050 >> 172.21.15.70:0/2011367050 conn(0x7f9c8c0fa4e0 msgr2=0x7f9c8c0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 -- 172.21.15.70:0/2011367050 shutdown_connections 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 -- 172.21.15.70:0/2011367050 wait complete. 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 Processor -- start 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.718+0000 7f9c926af640 1 -- start start 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c926af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c100b10 0x7f9c8c1959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c926af640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c8c103440 0x7f9c8c195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:21.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c926af640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c1964d0 con 0x7f9c8c100b10 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c926af640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c196640 con 0x7f9c8c103440 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c8bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c100b10 0x7f9c8c1959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c8bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c100b10 0x7f9c8c1959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57804/0 (socket says 172.21.15.70:57804) 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c8bfff640 1 -- 172.21.15.70:0/3403396434 learned_addr learned my addr 172.21.15.70:0/3403396434 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c8b7fe640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c8c103440 0x7f9c8c195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c8bfff640 1 -- 172.21.15.70:0/3403396434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c8c103440 msgr2=0x7f9c8c195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c8bfff640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c8c103440 0x7f9c8c195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c8bfff640 1 -- 172.21.15.70:0/3403396434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c80009560 con 0x7f9c8c100b10 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c8bfff640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c100b10 0x7f9c8c1959c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00d6f0 tx=0x7f9c7c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c897fa640 1 -- 172.21.15.70:0/3403396434 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7c009a30 con 0x7f9c8c100b10 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.719+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c19b0d0 con 0x7f9c8c100b10 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.720+0000 7f9c897fa640 1 -- 172.21.15.70:0/3403396434 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9c7c0054d0 con 0x7f9c8c100b10 2024-04-02T04:45:21.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.720+0000 7f9c897fa640 1 -- 172.21.15.70:0/3403396434 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7c010460 con 0x7f9c8c100b10 2024-04-02T04:45:21.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.720+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c19b600 con 0x7f9c8c100b10 2024-04-02T04:45:21.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.721+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c108de0 con 0x7f9c8c100b10 2024-04-02T04:45:21.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.721+0000 7f9c897fa640 1 -- 172.21.15.70:0/3403396434 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f9c7c009b90 con 0x7f9c8c100b10 2024-04-02T04:45:21.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.722+0000 7f9c897fa640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f9c7406c450 0x7f9c7406e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:21.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.722+0000 7f9c897fa640 1 -- 172.21.15.70:0/3403396434 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f9c7c08b3c0 con 0x7f9c8c100b10 2024-04-02T04:45:21.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.722+0000 7f9c8b7fe640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f9c7406c450 0x7f9c7406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:21.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.722+0000 7f9c8b7fe640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f9c7406c450 0x7f9c7406e910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009880 tx=0x7f9c80005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:21.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.730+0000 7f9c897fa640 1 -- 172.21.15.70:0/3403396434 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9c7c055ba0 con 0x7f9c8c100b10 2024-04-02T04:45:21.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.900+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c8c19b820 con 0x7f9c8c100b10 2024-04-02T04:45:21.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.901+0000 7f9c897fa640 1 -- 172.21.15.70:0/3403396434 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c7c0591c0 con 0x7f9c8c100b10 2024-04-02T04:45:21.904 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-04-02T04:45:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f9c7406c450 msgr2=0x7f9c7406e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f9c7406c450 0x7f9c7406e910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009880 tx=0x7f9c80005fd0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c100b10 msgr2=0x7f9c8c1959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c100b10 0x7f9c8c1959c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00d6f0 tx=0x7f9c7c00dbb0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 shutdown_connections 2024-04-02T04:45:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f9c7406c450 0x7f9c7406e910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c8c103440 0x7f9c8c195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 --2- 172.21.15.70:0/3403396434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8c100b10 0x7f9c8c1959c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 >> 172.21.15.70:0/3403396434 conn(0x7f9c8c0fa4e0 msgr2=0x7f9c8c0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 shutdown_connections 2024-04-02T04:45:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:21.903+0000 7f9c926af640 1 -- 172.21.15.70:0/3403396434 wait complete. 2024-04-02T04:45:21.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-02T04:45:22.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.026+0000 7fbdf97e4640 1 -- 172.21.15.70:0/240980606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 msgr2=0x7fbdf4102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.026+0000 7fbdf97e4640 1 --2- 172.21.15.70:0/240980606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 0x7fbdf4102f00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbde40098b0 tx=0x7fbde402bcd0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.026+0000 7fbdf97e4640 1 -- 172.21.15.70:0/240980606 shutdown_connections 2024-04-02T04:45:22.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.026+0000 7fbdf97e4640 1 --2- 172.21.15.70:0/240980606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf4103440 0x7fbdf4105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.026+0000 7fbdf97e4640 1 --2- 172.21.15.70:0/240980606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 0x7fbdf4102f00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.026+0000 7fbdf97e4640 1 -- 172.21.15.70:0/240980606 >> 172.21.15.70:0/240980606 conn(0x7fbdf40fa4e0 msgr2=0x7fbdf40fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:22.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf97e4640 1 -- 172.21.15.70:0/240980606 shutdown_connections 2024-04-02T04:45:22.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf97e4640 1 -- 172.21.15.70:0/240980606 wait complete. 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf97e4640 1 Processor -- start 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf97e4640 1 -- start start 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf97e4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 0x7fbdf4100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf97e4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf4103440 0x7fbdf40feac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf97e4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40ff000 con 0x7fbdf4100b10 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf97e4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40ff170 con 0x7fbdf4103440 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 0x7fbdf4100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 0x7fbdf4100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57818/0 (socket says 172.21.15.70:57818) 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdf2ffd640 1 -- 172.21.15.70:0/3996273472 learned_addr learned my addr 172.21.15.70:0/3996273472 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.027+0000 7fbdebfff640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf4103440 0x7fbdf40feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf2ffd640 1 -- 172.21.15.70:0/3996273472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf4103440 msgr2=0x7fbdf40feac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.033 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf2ffd640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf4103440 0x7fbdf40feac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf2ffd640 1 -- 172.21.15.70:0/3996273472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4009560 con 0x7fbdf4100b10 2024-04-02T04:45:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf2ffd640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 0x7fbdf4100480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009880 tx=0x7fbde4004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf0ff9640 1 -- 172.21.15.70:0/3996273472 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde4005000 con 0x7fbdf4100b10 2024-04-02T04:45:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf40ff3f0 con 0x7fbdf4100b10 2024-04-02T04:45:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf0ff9640 1 -- 172.21.15.70:0/3996273472 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbde4005610 con 0x7fbdf4100b10 2024-04-02T04:45:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf0ff9640 1 -- 172.21.15.70:0/3996273472 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde4034440 con 0x7fbdf4100b10 2024-04-02T04:45:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.028+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf40ff8f0 con 0x7fbdf4100b10 2024-04-02T04:45:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.029+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf4108e00 con 0x7fbdf4100b10 2024-04-02T04:45:22.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.029+0000 7fbdf0ff9640 1 -- 172.21.15.70:0/3996273472 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbde4003880 con 0x7fbdf4100b10 2024-04-02T04:45:22.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.030+0000 7fbdf0ff9640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fbde006c450 0x7fbde006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.030+0000 7fbdebfff640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fbde006c450 0x7fbde006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.030+0000 7fbdf0ff9640 1 -- 172.21.15.70:0/3996273472 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbde40add70 con 0x7fbdf4100b10 2024-04-02T04:45:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.030+0000 7fbdebfff640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fbde006c450 0x7fbde006e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbddc005fd0 tx=0x7fbddc005850 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.034+0000 7fbdf0ff9640 1 -- 172.21.15.70:0/3996273472 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbde4078550 con 0x7fbdf4100b10 2024-04-02T04:45:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.206+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fbdf419d9c0 con 0x7fbdf4100b10 2024-04-02T04:45:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.206+0000 7fbdf0ff9640 1 -- 172.21.15.70:0/3996273472 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fbde407bb70 con 0x7fbdf4100b10 2024-04-02T04:45:22.209 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fbde006c450 msgr2=0x7fbde006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fbde006c450 0x7fbde006e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbddc005fd0 tx=0x7fbddc005850 comp rx=0 tx=0).stop 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 msgr2=0x7fbdf4100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 0x7fbdf4100480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009880 tx=0x7fbde4004e30 comp rx=0 tx=0).stop 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 shutdown_connections 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fbde006c450 0x7fbde006e910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf4103440 0x7fbdf40feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 --2- 172.21.15.70:0/3996273472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf4100b10 0x7fbdf4100480 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 >> 172.21.15.70:0/3996273472 conn(0x7fbdf40fa4e0 msgr2=0x7fbdf40fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 shutdown_connections 2024-04-02T04:45:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.208+0000 7fbdf97e4640 1 -- 172.21.15.70:0/3996273472 wait complete. 2024-04-02T04:45:22.222 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-02T04:45:22.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.326+0000 7fcca5120640 1 -- 172.21.15.70:0/3895032287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 msgr2=0x7fcca00ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.326+0000 7fcca5120640 1 --2- 172.21.15.70:0/3895032287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 0x7fcca00ff3b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009880 tx=0x7fcc9802bce0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.326+0000 7fcca5120640 1 -- 172.21.15.70:0/3895032287 shutdown_connections 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.326+0000 7fcca5120640 1 --2- 172.21.15.70:0/3895032287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca0100180 0x7fcca0100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.326+0000 7fcca5120640 1 --2- 172.21.15.70:0/3895032287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 0x7fcca00ff3b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.326+0000 7fcca5120640 1 -- 172.21.15.70:0/3895032287 >> 172.21.15.70:0/3895032287 conn(0x7fcca00fa4e0 msgr2=0x7fcca00fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.326+0000 7fcca5120640 1 -- 172.21.15.70:0/3895032287 shutdown_connections 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.326+0000 7fcca5120640 1 -- 172.21.15.70:0/3895032287 wait complete. 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcca5120640 1 Processor -- start 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcca5120640 1 -- start start 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcca5120640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 0x7fcca0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcca5120640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca0100180 0x7fcca01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcca5120640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca01986a0 con 0x7fcca00fef90 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcca5120640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca0198810 con 0x7fcca0100180 2024-04-02T04:45:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcc9ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 0x7fcca0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcc9ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 0x7fcca0197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57844/0 (socket says 172.21.15.70:57844) 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcc9ed9d640 1 -- 172.21.15.70:0/1910089814 learned_addr learned my addr 172.21.15.70:0/1910089814 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.327+0000 7fcc9e59c640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca0100180 0x7fcca01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcc9ed9d640 1 -- 172.21.15.70:0/1910089814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca0100180 msgr2=0x7fcca01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcc9ed9d640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca0100180 0x7fcca01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcc9ed9d640 1 -- 172.21.15.70:0/1910089814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc98009560 con 0x7fcca00fef90 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcc9ed9d640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 0x7fcca0197b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009230 tx=0x7fcc98004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcc87fff640 1 -- 172.21.15.70:0/1910089814 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc98005000 con 0x7fcca00fef90 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca019d240 con 0x7fcca00fef90 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcc87fff640 1 -- 172.21.15.70:0/1910089814 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcc98005610 con 0x7fcca00fef90 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcc87fff640 1 -- 172.21.15.70:0/1910089814 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc98034440 con 0x7fcca00fef90 2024-04-02T04:45:22.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.328+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca019d740 con 0x7fcca00fef90 2024-04-02T04:45:22.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.329+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca0107d20 con 0x7fcca00fef90 2024-04-02T04:45:22.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.329+0000 7fcc87fff640 1 -- 172.21.15.70:0/1910089814 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fcc98034640 con 0x7fcca00fef90 2024-04-02T04:45:22.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.330+0000 7fcc87fff640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcc8806c450 0x7fcc8806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.330+0000 7fcc87fff640 1 -- 172.21.15.70:0/1910089814 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fcc980adcf0 con 0x7fcca00fef90 2024-04-02T04:45:22.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.330+0000 7fcc9e59c640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcc8806c450 0x7fcc8806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.330+0000 7fcc9e59c640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcc8806c450 0x7fcc8806e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcca01990a0 tx=0x7fcc9400b390 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:22.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.334+0000 7fcc87fff640 1 -- 172.21.15.70:0/1910089814 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcc98078580 con 0x7fcca00fef90 2024-04-02T04:45:22.503 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:22 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3252726884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:45:22.503 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:22 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3403396434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:45:22.503 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:22 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3996273472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:45:22.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.520+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fcca019d960 con 0x7fcca00fef90 2024-04-02T04:45:22.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.520+0000 7fcc87fff640 1 -- 172.21.15.70:0/1910089814 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcc9807bba0 con 0x7fcca00fef90 2024-04-02T04:45:22.523 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-04-02T04:45:22.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcc8806c450 msgr2=0x7fcc8806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcc8806c450 0x7fcc8806e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcca01990a0 tx=0x7fcc9400b390 comp rx=0 tx=0).stop 2024-04-02T04:45:22.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 msgr2=0x7fcca0197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 0x7fcca0197b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009230 tx=0x7fcc98004e30 comp rx=0 tx=0).stop 2024-04-02T04:45:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 shutdown_connections 2024-04-02T04:45:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fcc8806c450 0x7fcc8806e910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca0100180 0x7fcca01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 --2- 172.21.15.70:0/1910089814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcca00fef90 0x7fcca0197b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.523+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 >> 172.21.15.70:0/1910089814 conn(0x7fcca00fa4e0 msgr2=0x7fcca0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.524+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 shutdown_connections 2024-04-02T04:45:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.524+0000 7fcca5120640 1 -- 172.21.15.70:0/1910089814 wait complete. 2024-04-02T04:45:22.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T04:45:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:22 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3252726884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:45:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:22 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3403396434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:45:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:22 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3996273472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:45:22.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.648+0000 7faa2ece0640 1 -- 172.21.15.70:0/689162875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa280fefb0 msgr2=0x7faa280ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.648+0000 7faa2ece0640 1 --2- 172.21.15.70:0/689162875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa280fefb0 0x7faa280ff3d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009880 tx=0x7faa1c02bce0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.648+0000 7faa2ece0640 1 -- 172.21.15.70:0/689162875 shutdown_connections 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.648+0000 7faa2ece0640 1 --2- 172.21.15.70:0/689162875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa281001a0 0x7faa28100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.648+0000 7faa2ece0640 1 --2- 172.21.15.70:0/689162875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa280fefb0 0x7faa280ff3d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.648+0000 7faa2ece0640 1 -- 172.21.15.70:0/689162875 >> 172.21.15.70:0/689162875 conn(0x7faa280fa4e0 msgr2=0x7faa280fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.648+0000 7faa2ece0640 1 -- 172.21.15.70:0/689162875 shutdown_connections 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.648+0000 7faa2ece0640 1 -- 172.21.15.70:0/689162875 wait complete. 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa2ece0640 1 Processor -- start 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa2ece0640 1 -- start start 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa2ece0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa280fefb0 0x7faa28067ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa2ece0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa281001a0 0x7faa2819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa2ece0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2819ac00 con 0x7faa281001a0 2024-04-02T04:45:22.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa2ece0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2819ad40 con 0x7faa280fefb0 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa281001a0 0x7faa2819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa281001a0 0x7faa2819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57864/0 (socket says 172.21.15.70:57864) 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa27fff640 1 -- 172.21.15.70:0/3598320338 learned_addr learned my addr 172.21.15.70:0/3598320338 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa2ca7c640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa280fefb0 0x7faa28067ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa27fff640 1 -- 172.21.15.70:0/3598320338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa280fefb0 msgr2=0x7faa28067ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa27fff640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa280fefb0 0x7faa28067ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa27fff640 1 -- 172.21.15.70:0/3598320338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c009560 con 0x7faa281001a0 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.649+0000 7faa27fff640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa281001a0 0x7faa2819a630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa1400e970 tx=0x7faa1400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.650+0000 7faa25ffb640 1 -- 172.21.15.70:0/3598320338 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa1400cc90 con 0x7faa281001a0 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.650+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2819aff0 con 0x7faa281001a0 2024-04-02T04:45:22.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.650+0000 7faa25ffb640 1 -- 172.21.15.70:0/3598320338 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faa1400cdf0 con 0x7faa281001a0 2024-04-02T04:45:22.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.650+0000 7faa25ffb640 1 -- 172.21.15.70:0/3598320338 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa14010430 con 0x7faa281001a0 2024-04-02T04:45:22.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.650+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2819b520 con 0x7faa281001a0 2024-04-02T04:45:22.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.651+0000 7faa25ffb640 1 -- 172.21.15.70:0/3598320338 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7faa14004f90 con 0x7faa281001a0 2024-04-02T04:45:22.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.652+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa28107bb0 con 0x7faa281001a0 2024-04-02T04:45:22.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.652+0000 7faa25ffb640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faa1806c510 0x7faa1806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:22.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.652+0000 7faa25ffb640 1 -- 172.21.15.70:0/3598320338 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7faa14014070 con 0x7faa281001a0 2024-04-02T04:45:22.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.652+0000 7faa2ca7c640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faa1806c510 0x7faa1806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:22.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.653+0000 7faa2ca7c640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faa1806c510 0x7faa1806e9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa1c000c00 tx=0x7faa1c005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:22.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.657+0000 7faa25ffb640 1 -- 172.21.15.70:0/3598320338 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faa140564d0 con 0x7faa281001a0 2024-04-02T04:45:22.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.874+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa28107dc0 con 0x7faa281001a0 2024-04-02T04:45:22.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.875+0000 7faa25ffb640 1 -- 172.21.15.70:0/3598320338 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4740 (secure 0 0 0) 0x7faa14059af0 con 0x7faa281001a0 2024-04-02T04:45:22.879 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:45:22.880 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14480,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6829/2586361462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2586361462},{"type":"v1","addr":"172.21.15.70:6829","nonce":2586361462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:45:08.643957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6827/1351191990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1351191990},{"type":"v1","addr":"172.21.15.70:6827","nonce":1351191990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-02T04:45:22.882 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:45:22.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faa1806c510 msgr2=0x7faa1806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faa1806c510 0x7faa1806e9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa1c000c00 tx=0x7faa1c005fd0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa281001a0 msgr2=0x7faa2819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:22.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa281001a0 0x7faa2819a630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa1400e970 tx=0x7faa1400ed70 comp rx=0 tx=0).stop 2024-04-02T04:45:22.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 shutdown_connections 2024-04-02T04:45:22.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7faa1806c510 0x7faa1806e9d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa281001a0 0x7faa2819a630 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 --2- 172.21.15.70:0/3598320338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa280fefb0 0x7faa28067ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:22.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 >> 172.21.15.70:0/3598320338 conn(0x7faa280fa4e0 msgr2=0x7faa281033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:22.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 shutdown_connections 2024-04-02T04:45:22.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:22.877+0000 7faa2ece0640 1 -- 172.21.15.70:0/3598320338 wait complete. 2024-04-02T04:45:22.892 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 13, 'max_mds': 1, 'flags': 18} 2024-04-02T04:45:22.892 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-02T04:45:22.944 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-02T04:45:22.944 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-02T04:45:22.944 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-02T04:45:22.944 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T04:45:22.945 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:22.945 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-02T04:45:22.945 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T04:45:22.945 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:22.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:22.945 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-04-02T04:45:22.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:22.965 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link delete ceph-brx 2024-04-02T04:45:23.041 INFO:teuthology.orchestra.run.smithi179.stderr:Cannot find device "ceph-brx" 2024-04-02T04:45:23.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:45:23.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:23.044 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-04-02T04:45:23.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:23.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2024-04-02T04:45:23.129 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2024-04-02T04:45:23.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:45:23.130 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-02T04:45:23.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T04:45:23.384 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:23.505 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:23 smithi070 ceph-mon[29212]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:23.505 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:23 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1910089814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:45:23.505 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:23 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:23.505 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:23 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:23.505 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:23 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:23.505 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:23 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:45:23.505 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:23 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:23.506 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:23 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3598320338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:45:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:23 smithi179 ceph-mon[36603]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:23 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1910089814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:45:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:23 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:23 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:23 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:23 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:45:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:23 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:45:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:23 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3598320338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:45:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.980+0000 7f45432b8700 1 -- 172.21.15.70:0/174531547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 msgr2=0x7f453c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.980+0000 7f45432b8700 1 --2- 172.21.15.70:0/174531547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 0x7f453c105fb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f452c009910 tx=0x7f452c009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.980+0000 7f45432b8700 1 -- 172.21.15.70:0/174531547 shutdown_connections 2024-04-02T04:45:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.980+0000 7f45432b8700 1 --2- 172.21.15.70:0/174531547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f453c0ffc20 0x7f453c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.980+0000 7f45432b8700 1 --2- 172.21.15.70:0/174531547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 0x7f453c105fb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.980+0000 7f45432b8700 1 -- 172.21.15.70:0/174531547 >> 172.21.15.70:0/174531547 conn(0x7f453c075670 msgr2=0x7f453c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.980+0000 7f45432b8700 1 -- 172.21.15.70:0/174531547 shutdown_connections 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.980+0000 7f45432b8700 1 -- 172.21.15.70:0/174531547 wait complete. 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f45432b8700 1 Processor -- start 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f45432b8700 1 -- start start 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f45432b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f453c0ffc20 0x7f453c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f45432b8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 0x7f453c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f45432b8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c19d230 con 0x7f453c105bd0 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f45432b8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c1966e0 con 0x7f453c0ffc20 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f4540853700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 0x7f453c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f4540853700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 0x7f453c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57880/0 (socket says 172.21.15.70:57880) 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f4540853700 1 -- 172.21.15.70:0/532746389 learned_addr learned my addr 172.21.15.70:0/532746389 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f4540853700 1 -- 172.21.15.70:0/532746389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f453c0ffc20 msgr2=0x7f453c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f4541054700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f453c0ffc20 0x7f453c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f4540853700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f453c0ffc20 0x7f453c19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f4540853700 1 -- 172.21.15.70:0/532746389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0095f0 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.981+0000 7f4540853700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 0x7f453c19cba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f453800d900 tx=0x7f453800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.982+0000 7f45327fc700 1 -- 172.21.15.70:0/532746389 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4538004a00 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.982+0000 7f45327fc700 1 -- 172.21.15.70:0/532746389 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f45380054d0 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.982+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c1969c0 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.982+0000 7f45327fc700 1 -- 172.21.15.70:0/532746389 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4538009d60 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.982+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c196f10 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.983+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c04f2e0 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.983+0000 7f45327fc700 1 -- 172.21.15.70:0/532746389 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4538005020 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.983+0000 7f45327fc700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f452806c4e0 0x7f452806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.983+0000 7f4541054700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f452806c4e0 0x7f452806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.983+0000 7f45327fc700 1 -- 172.21.15.70:0/532746389 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f453808b920 con 0x7f453c105bd0 2024-04-02T04:45:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.984+0000 7f4541054700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f452806c4e0 0x7f452806e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b5c0 tx=0x7f452c005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:25.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:24.988+0000 7f45327fc700 1 -- 172.21.15.70:0/532746389 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4538059c10 con 0x7f453c105bd0 2024-04-02T04:45:25.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.221+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f453c197bf0 con 0x7f453c105bd0 2024-04-02T04:45:25.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.221+0000 7f45327fc700 1 -- 172.21.15.70:0/532746389 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4538017020 con 0x7f453c105bd0 2024-04-02T04:45:25.225 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f452806c4e0 msgr2=0x7f452806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f452806c4e0 0x7f452806e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b5c0 tx=0x7f452c005960 comp rx=0 tx=0).stop 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 msgr2=0x7f453c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 0x7f453c19cba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f453800d900 tx=0x7f453800dcc0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 shutdown_connections 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f452806c4e0 0x7f452806e9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f453c0ffc20 0x7f453c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 --2- 172.21.15.70:0/532746389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f453c105bd0 0x7f453c19cba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 >> 172.21.15.70:0/532746389 conn(0x7f453c075670 msgr2=0x7f453c0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 shutdown_connections 2024-04-02T04:45:25.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:25.223+0000 7f45432b8700 1 -- 172.21.15.70:0/532746389 wait complete. 2024-04-02T04:45:25.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:25 smithi179 ceph-mon[36603]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:25.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:25 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/532746389' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T04:45:25.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:25 smithi070 ceph-mon[29212]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:25.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:25 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/532746389' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T04:45:26.040 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T04:45:26.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T04:45:26.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2024-04-02T04:45:26.040 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T04:45:26.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:26.040 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T04:45:26.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T04:45:26.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T04:45:26.041 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T04:45:26.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:26.041 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2024-04-02T04:45:26.059 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T04:45:26.059 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T04:45:26.059 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T04:45:26.059 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T04:45:26.060 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:45:26.061 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-02T04:45:26.061 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42279sec preferred_lft 42279sec 2024-04-02T04:45:26.061 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2024-04-02T04:45:26.061 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:45:26.061 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T04:45:26.061 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:26.061 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-04-02T04:45:26.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2024-04-02T04:45:26.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2024-04-02T04:45:26.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2024-04-02T04:45:26.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T04:45:26.062 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-04-02T04:45:26.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:26.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:26.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:26.276 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T04:45:26.303 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-04-02T04:45:26.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:26.305 DEBUG:teuthology.orchestra.run.smithi070:> route 2024-04-02T04:45:26.377 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2024-04-02T04:45:26.377 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T04:45:26.377 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-02T04:45:26.378 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-02T04:45:26.378 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T04:45:26.378 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:26.378 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-04-02T04:45:26.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-02T04:45:26.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-02T04:45:26.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-02T04:45:26.378 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-04-02T04:45:26.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:26.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:26.677 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-04-02T04:45:26.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:26.693 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2024-04-02T04:45:26.748 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:26.748 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-04-02T04:45:26.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:45:26.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T04:45:26.748 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-04-02T04:45:26.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:26.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:26 smithi070 ceph-mon[29212]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:26.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:26.871 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T04:45:26.872 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:26.872 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-04-02T04:45:26.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T04:45:26.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T04:45:26.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T04:45:26.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T04:45:26.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T04:45:26.873 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-04-02T04:45:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:27.012 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:27.012 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-04-02T04:45:27.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2024-04-02T04:45:27.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T04:45:27.012 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-04-02T04:45:27.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:27.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:26 smithi179 ceph-mon[36603]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:27.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:27.093 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-02T04:45:27.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T04:45:27.094 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:27.108 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T04:45:27.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T04:45:27.109 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:27.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2024-04-02T04:45:27.231 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:45:27.287 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2024-04-02T04:45:27.288 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2024-04-02T04:45:27.288 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2024-04-02T04:45:27.288 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2024-04-02T04:45:27.288 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2024-04-02T04:45:27.288 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2024-04-02T04:45:27.288 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/bpf 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2024-04-02T04:45:27.289 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:45:27.290 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:45:27.290 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:45:27.290 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:45:27.290 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2024-04-02T04:45:27.290 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2024-04-02T04:45:27.290 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:45:27.290 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/a68607e22ae575bea0d390496a049ea1920a5d9d733d963346bbf5e923e26b93/merged 2024-04-02T04:45:27.290 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/4527df920ced1b805d9ebefb56183a5a7a6f8601923757dd040eb40f571739cc/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/0 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/3d64ab3f0143e99e427eeeebf6770133e68b193619588bd3c444eac010d3326b/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/66893dd8ad47066d025e11241cd1d66c620ee5d1ce7af81a27aaa503105aa501/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/cc02c2be186501616fa987d5a53be1d5e3e9de145ab5d21705b6b9c683ebc947/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/b5a221008bede69dc9323aa7380bf34f6f3877da46ecc6c226990a50b9607275/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/39cfbd909d01e94035edd88829581d80e3cbab3bc5547ae7b42e2c00a824cb8d/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/d8ee4c78d03241c87e92fce16730411e51a56af04a592e7b8cd8448d0ea19026/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/fdff7081669244bb2c14a8844e1d17762da20d700658b59c1f744e196d135680/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/15c8f46d90ed1269c6a634d8975876c30edf87895fe676d05b3af0247561f4ae/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/c8f74c6601f73294faf268657d9583db3c55136707e65df2afdbda5f1319986a/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/91da51e13f203163daccc5e22069533efdc0d649e07dabc38fb89f4679272116/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/b276d219eed13b4da7b97973b17ccc3fff6d9044e0dfd9a32dc9a91deef4cc4a/merged 2024-04-02T04:45:27.291 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2024-04-02T04:45:27.292 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:45:27.292 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:45:27.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:27.292 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2024-04-02T04:45:27.343 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T04:45:27.343 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-02T04:45:27.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2024-04-02T04:45:27.412 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:45:27.468 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2024-04-02T04:45:27.468 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2024-04-02T04:45:27.468 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2024-04-02T04:45:27.468 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2024-04-02T04:45:27.468 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2024-04-02T04:45:27.468 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2024-04-02T04:45:27.469 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2024-04-02T04:45:27.469 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2024-04-02T04:45:27.469 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2024-04-02T04:45:27.469 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/bpf 2024-04-02T04:45:27.469 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2024-04-02T04:45:27.469 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2024-04-02T04:45:27.469 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:27.469 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:45:27.470 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/a68607e22ae575bea0d390496a049ea1920a5d9d733d963346bbf5e923e26b93/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/4527df920ced1b805d9ebefb56183a5a7a6f8601923757dd040eb40f571739cc/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/0 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/3d64ab3f0143e99e427eeeebf6770133e68b193619588bd3c444eac010d3326b/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/66893dd8ad47066d025e11241cd1d66c620ee5d1ce7af81a27aaa503105aa501/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/cc02c2be186501616fa987d5a53be1d5e3e9de145ab5d21705b6b9c683ebc947/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/b5a221008bede69dc9323aa7380bf34f6f3877da46ecc6c226990a50b9607275/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/39cfbd909d01e94035edd88829581d80e3cbab3bc5547ae7b42e2c00a824cb8d/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/d8ee4c78d03241c87e92fce16730411e51a56af04a592e7b8cd8448d0ea19026/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/fdff7081669244bb2c14a8844e1d17762da20d700658b59c1f744e196d135680/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/15c8f46d90ed1269c6a634d8975876c30edf87895fe676d05b3af0247561f4ae/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/c8f74c6601f73294faf268657d9583db3c55136707e65df2afdbda5f1319986a/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/91da51e13f203163daccc5e22069533efdc0d649e07dabc38fb89f4679272116/merged 2024-04-02T04:45:27.471 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/b276d219eed13b4da7b97973b17ccc3fff6d9044e0dfd9a32dc9a91deef4cc4a/merged 2024-04-02T04:45:27.472 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2024-04-02T04:45:27.472 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:45:27.472 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:45:27.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:27.472 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2024-04-02T04:45:27.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2024-04-02T04:45:27.485+0000 7fdcfc047480 -1 init, newargv = 0x555c97425ce0 newargc=15 2024-04-02T04:45:27.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[77817]: starting ceph client 2024-04-02T04:45:27.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[77817]: starting fuse 2024-04-02T04:45:27.504 INFO:teuthology.orchestra.run.smithi070.stdout:159 2024-04-02T04:45:27.504 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [159] 2024-04-02T04:45:27.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2024-04-02T04:45:27.505 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2024-04-02T04:45:27.505 DEBUG:teuthology.orchestra.run.smithi070:> import re 2024-04-02T04:45:27.505 DEBUG:teuthology.orchestra.run.smithi070:> import os 2024-04-02T04:45:27.505 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2024-04-02T04:45:27.505 DEBUG:teuthology.orchestra.run.smithi070:> 2024-04-02T04:45:27.505 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2024-04-02T04:45:27.505 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-02T04:45:27.505 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> return f 2024-04-02T04:45:27.506 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T04:45:27.507 DEBUG:teuthology.orchestra.run.smithi070:> 2024-04-02T04:45:27.507 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2024-04-02T04:45:27.507 DEBUG:teuthology.orchestra.run.smithi070:> ' 2024-04-02T04:45:27.625 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.77817.asok 2024-04-02T04:45:27.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:27.635 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77817.asok 2024-04-02T04:45:27.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:27.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77817.asok status 2024-04-02T04:45:27.770 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:45:27.770 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2024-04-02T04:45:27.770 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-04-02T04:45:27.770 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-02T04:45:27.770 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2024-04-02T04:45:27.770 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2024-04-02T04:45:27.770 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "77817", 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 14532, 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 14532 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2024-04-02T04:45:27.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3290270869 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3290270869 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.14532 192.168.0.1:0/3290270869", 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3290270869", 2024-04-02T04:45:27.772 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2024-04-02T04:45:27.773 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 13, 2024-04-02T04:45:27.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 46, 2024-04-02T04:45:27.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2024-04-02T04:45:27.773 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2024-04-02T04:45:27.773 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2024-04-02T04:45:27.773 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:45:27.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T04:45:27.985 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:28.455 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:28.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:29.557 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:29 smithi070 ceph-mon[29212]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:29.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:29 smithi179 ceph-mon[36603]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.855+0000 7fca2d52f700 1 -- 172.21.15.70:0/3121537345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 msgr2=0x7fca28105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.855+0000 7fca2d52f700 1 --2- 172.21.15.70:0/3121537345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 0x7fca28105fb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fca2000b0a0 tx=0x7fca2000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.855+0000 7fca2d52f700 1 -- 172.21.15.70:0/3121537345 shutdown_connections 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.855+0000 7fca2d52f700 1 --2- 172.21.15.70:0/3121537345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca280ffc20 0x7fca28100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.855+0000 7fca2d52f700 1 --2- 172.21.15.70:0/3121537345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 0x7fca28105fb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.855+0000 7fca2d52f700 1 -- 172.21.15.70:0/3121537345 >> 172.21.15.70:0/3121537345 conn(0x7fca28075670 msgr2=0x7fca28075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.855+0000 7fca2d52f700 1 -- 172.21.15.70:0/3121537345 shutdown_connections 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.855+0000 7fca2d52f700 1 -- 172.21.15.70:0/3121537345 wait complete. 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca2d52f700 1 Processor -- start 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca2d52f700 1 -- start start 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca2d52f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca280ffc20 0x7fca28101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca2d52f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 0x7fca28101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca2d52f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca281022c0 con 0x7fca28105bd0 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca2d52f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca28102400 con 0x7fca280ffc20 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca267fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 0x7fca28101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca267fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 0x7fca28101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52424/0 (socket says 172.21.15.70:52424) 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca267fc700 1 -- 172.21.15.70:0/2197555110 learned_addr learned my addr 172.21.15.70:0/2197555110 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca26ffd700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca280ffc20 0x7fca28101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca267fc700 1 -- 172.21.15.70:0/2197555110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca280ffc20 msgr2=0x7fca28101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca267fc700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca280ffc20 0x7fca28101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca267fc700 1 -- 172.21.15.70:0/2197555110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20009d00 con 0x7fca28105bd0 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.856+0000 7fca26ffd700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca280ffc20 0x7fca28101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.857+0000 7fca267fc700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 0x7fca28101d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fca1000d8d0 tx=0x7fca1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.857+0000 7fca1ffff700 1 -- 172.21.15.70:0/2197555110 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca10009940 con 0x7fca28105bd0 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.857+0000 7fca1ffff700 1 -- 172.21.15.70:0/2197555110 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fca10010460 con 0x7fca28105bd0 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.857+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca281a2a20 con 0x7fca28105bd0 2024-04-02T04:45:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.857+0000 7fca1ffff700 1 -- 172.21.15.70:0/2197555110 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1000f5f0 con 0x7fca28105bd0 2024-04-02T04:45:29.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.857+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca281a2e30 con 0x7fca28105bd0 2024-04-02T04:45:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.858+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2804f2e0 con 0x7fca28105bd0 2024-04-02T04:45:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.858+0000 7fca1ffff700 1 -- 172.21.15.70:0/2197555110 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fca100105d0 con 0x7fca28105bd0 2024-04-02T04:45:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.859+0000 7fca1ffff700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fca1406c4e0 0x7fca1406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.859+0000 7fca1ffff700 1 -- 172.21.15.70:0/2197555110 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fca1008b830 con 0x7fca28105bd0 2024-04-02T04:45:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.859+0000 7fca26ffd700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fca1406c4e0 0x7fca1406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.859+0000 7fca26ffd700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fca1406c4e0 0x7fca1406e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fca20009480 tx=0x7fca20006000 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:29.863+0000 7fca1ffff700 1 -- 172.21.15.70:0/2197555110 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fca10056010 con 0x7fca28105bd0 2024-04-02T04:45:30.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.093+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca281a30c0 con 0x7fca28105bd0 2024-04-02T04:45:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.094+0000 7fca1ffff700 1 -- 172.21.15.70:0/2197555110 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7fca10059630 con 0x7fca28105bd0 2024-04-02T04:45:30.098 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fca1406c4e0 msgr2=0x7fca1406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fca1406c4e0 0x7fca1406e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fca20009480 tx=0x7fca20006000 comp rx=0 tx=0).stop 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 msgr2=0x7fca28101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 0x7fca28101d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fca1000d8d0 tx=0x7fca1000dc90 comp rx=0 tx=0).stop 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 shutdown_connections 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fca1406c4e0 0x7fca1406e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca280ffc20 0x7fca28101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 --2- 172.21.15.70:0/2197555110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca28105bd0 0x7fca28101d80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 >> 172.21.15.70:0/2197555110 conn(0x7fca28075670 msgr2=0x7fca280686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 shutdown_connections 2024-04-02T04:45:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:30.096+0000 7fca2d52f700 1 -- 172.21.15.70:0/2197555110 wait complete. 2024-04-02T04:45:30.463 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:30 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2197555110' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T04:45:30.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:30 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2197555110' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T04:45:30.771 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T04:45:30.771 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T04:45:30.772 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-04-02T04:45:30.772 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-02T04:45:30.772 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:30.772 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T04:45:30.772 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T04:45:30.772 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T04:45:30.772 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-02T04:45:30.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:30.772 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-04-02T04:45:30.791 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T04:45:30.791 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T04:45:30.791 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T04:45:30.791 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:45:30.791 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-04-02T04:45:30.791 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42286sec preferred_lft 42286sec 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-04-02T04:45:30.792 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:45:30.793 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T04:45:30.793 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:30.793 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-04-02T04:45:30.793 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add name ceph-brx type bridge 2024-04-02T04:45:30.793 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip addr flush dev ceph-brx 2024-04-02T04:45:30.793 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set ceph-brx up 2024-04-02T04:45:30.793 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T04:45:30.793 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-04-02T04:45:30.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:30.991 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:30.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:30.998 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T04:45:31.026 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-04-02T04:45:31.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:31.027 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-04-02T04:45:31.095 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-04-02T04:45:31.095 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T04:45:31.095 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-02T04:45:31.095 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-02T04:45:31.095 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T04:45:31.096 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:31.096 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-04-02T04:45:31.096 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-02T04:45:31.096 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-02T04:45:31.096 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-02T04:45:31.096 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-04-02T04:45:31.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:31.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:31.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:31.379 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-04-02T04:45:31.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:31.395 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list-id 2024-04-02T04:45:31.450 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:31.450 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-04-02T04:45:31.450 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:45:31.450 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-02T04:45:31.450 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-04-02T04:45:31.465 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:31 smithi179 ceph-mon[36603]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:45:31.514 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:31.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:31.549 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-02T04:45:31.549 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:31.549 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-04-02T04:45:31.549 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-02T04:45:31.549 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T04:45:31.549 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-02T04:45:31.549 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-02T04:45:31.549 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-02T04:45:31.550 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-04-02T04:45:31.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:31.685 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:31.694 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:45:31.694 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-04-02T04:45:31.694 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-04-02T04:45:31.694 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T04:45:31.694 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-04-02T04:45:31.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:45:31.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:31.776 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-02T04:45:31.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T04:45:31.777 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:31.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:31 smithi070 ceph-mon[29212]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:45:31.791 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-02T04:45:31.791 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T04:45:31.791 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:31.845 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-04-02T04:45:31.914 DEBUG:teuthology.orchestra.run.smithi179:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:45:31.969 INFO:teuthology.orchestra.run.smithi179.stdout:/proc 2024-04-02T04:45:31.969 INFO:teuthology.orchestra.run.smithi179.stdout:/sys 2024-04-02T04:45:31.969 INFO:teuthology.orchestra.run.smithi179.stdout:/dev 2024-04-02T04:45:31.969 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/security 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/shm 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/pts 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/run 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/cgroup 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/pstore 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/bpf 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/ 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/selinux 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/mqueue 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/hugepages 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/debug 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/tracing 2024-04-02T04:45:31.970 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/fuse/connections 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/config 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/run/user/1000 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/ceph 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/run/user/0 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/4a56564da5f8feb8ef67af4b5ca57ff9f832819c0ce94d097f5cca6c4dbcfa32/merged 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/350ee2daef5c22ee76d0d66ee24bc2f7bc61461abf55a1413efb3e691777fd05/merged 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/bc7861290b641880e3370b20475c1cde55ebccfc7d4e866b0e4c645f4cf609e1/merged 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/b63f07ddf3ba99c3450e5b27025a4685348378d0a6836f0ab1970ee5671cf1b8/merged 2024-04-02T04:45:31.971 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/2f2b0241b01fafae6d55940932aa19f4c9d0d709e87bb825363de5043fe5d636/merged 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/2fc850815b1650404c7fa20bc9f7a0d39b688533d73102531a286b8357e57ef5/merged 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/c0c5a42b590c79201850abd1713fa66e117dfd69449a2fab21d830acd07feb75/merged 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/ad325e0e5b4a9daaf6884f0391cd93b45194919a855490bb563953a04432d437/merged 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/3c72b17d87c0004f934348f272046ac3f6d667fd1bf06984a135f37e2fc2bafe/merged 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/1cb00866cec868afadea5ed65c9872bde4e87c28c8000e338d192c932955e436/merged 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:45:31.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:31.972 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-04-02T04:45:32.024 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T04:45:32.024 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-02T04:45:32.067 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-04-02T04:45:32.093 DEBUG:teuthology.orchestra.run.smithi179:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:45:32.149 INFO:teuthology.orchestra.run.smithi179.stdout:/proc 2024-04-02T04:45:32.149 INFO:teuthology.orchestra.run.smithi179.stdout:/sys 2024-04-02T04:45:32.149 INFO:teuthology.orchestra.run.smithi179.stdout:/dev 2024-04-02T04:45:32.149 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/security 2024-04-02T04:45:32.149 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/shm 2024-04-02T04:45:32.149 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/pts 2024-04-02T04:45:32.149 INFO:teuthology.orchestra.run.smithi179.stdout:/run 2024-04-02T04:45:32.149 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/cgroup 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/pstore 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/bpf 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/ 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/selinux 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/mqueue 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/hugepages 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/debug 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/tracing 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/fuse/connections 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/config 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:45:32.150 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/run/user/1000 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/ceph 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/run/user/0 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/4a56564da5f8feb8ef67af4b5ca57ff9f832819c0ce94d097f5cca6c4dbcfa32/merged 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/350ee2daef5c22ee76d0d66ee24bc2f7bc61461abf55a1413efb3e691777fd05/merged 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/bc7861290b641880e3370b20475c1cde55ebccfc7d4e866b0e4c645f4cf609e1/merged 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/b63f07ddf3ba99c3450e5b27025a4685348378d0a6836f0ab1970ee5671cf1b8/merged 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/2f2b0241b01fafae6d55940932aa19f4c9d0d709e87bb825363de5043fe5d636/merged 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/2fc850815b1650404c7fa20bc9f7a0d39b688533d73102531a286b8357e57ef5/merged 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/c0c5a42b590c79201850abd1713fa66e117dfd69449a2fab21d830acd07feb75/merged 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/ad325e0e5b4a9daaf6884f0391cd93b45194919a855490bb563953a04432d437/merged 2024-04-02T04:45:32.151 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/3c72b17d87c0004f934348f272046ac3f6d667fd1bf06984a135f37e2fc2bafe/merged 2024-04-02T04:45:32.152 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/1cb00866cec868afadea5ed65c9872bde4e87c28c8000e338d192c932955e436/merged 2024-04-02T04:45:32.152 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns 2024-04-02T04:45:32.152 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:45:32.152 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:45:32.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:32.152 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-04-02T04:45:32.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi179.stderr:2024-04-02T04:45:32.201+0000 7f4736041480 -1 init, newargv = 0x55aa9aee64f0 newargc=15 2024-04-02T04:45:32.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi179.stderr:ceph-fuse[62638]: starting ceph client 2024-04-02T04:45:32.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi179.stderr:ceph-fuse[62638]: starting fuse 2024-04-02T04:45:33.305 DEBUG:teuthology.orchestra.run.smithi179:> sudo modprobe fuse 2024-04-02T04:45:33.341 DEBUG:teuthology.orchestra.run.smithi179:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:45:33.399 INFO:teuthology.orchestra.run.smithi179.stdout:/proc 2024-04-02T04:45:33.399 INFO:teuthology.orchestra.run.smithi179.stdout:/sys 2024-04-02T04:45:33.399 INFO:teuthology.orchestra.run.smithi179.stdout:/dev 2024-04-02T04:45:33.399 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/security 2024-04-02T04:45:33.399 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/shm 2024-04-02T04:45:33.399 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/pts 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/run 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/cgroup 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/pstore 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/bpf 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/ 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/selinux 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/mqueue 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/dev/hugepages 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/debug 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/tracing 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/fs/fuse/connections 2024-04-02T04:45:33.400 INFO:teuthology.orchestra.run.smithi179.stdout:/sys/kernel/config 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/run/user/1000 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/ceph 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/run/user/0 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/4a56564da5f8feb8ef67af4b5ca57ff9f832819c0ce94d097f5cca6c4dbcfa32/merged 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/350ee2daef5c22ee76d0d66ee24bc2f7bc61461abf55a1413efb3e691777fd05/merged 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/bc7861290b641880e3370b20475c1cde55ebccfc7d4e866b0e4c645f4cf609e1/merged 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/b63f07ddf3ba99c3450e5b27025a4685348378d0a6836f0ab1970ee5671cf1b8/merged 2024-04-02T04:45:33.401 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/2f2b0241b01fafae6d55940932aa19f4c9d0d709e87bb825363de5043fe5d636/merged 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/2fc850815b1650404c7fa20bc9f7a0d39b688533d73102531a286b8357e57ef5/merged 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/c0c5a42b590c79201850abd1713fa66e117dfd69449a2fab21d830acd07feb75/merged 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/ad325e0e5b4a9daaf6884f0391cd93b45194919a855490bb563953a04432d437/merged 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/3c72b17d87c0004f934348f272046ac3f6d667fd1bf06984a135f37e2fc2bafe/merged 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/var/lib/containers/storage/overlay/1cb00866cec868afadea5ed65c9872bde4e87c28c8000e338d192c932955e436/merged 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:45:33.402 INFO:teuthology.orchestra.run.smithi179.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:33.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:33.403 DEBUG:teuthology.orchestra.run.smithi179:> ls /sys/fs/fuse/connections 2024-04-02T04:45:33.461 INFO:teuthology.orchestra.run.smithi179.stdout:126 2024-04-02T04:45:33.461 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [126] 2024-04-02T04:45:33.461 DEBUG:teuthology.orchestra.run.smithi179:> sudo stdin-killer -- python3 -c ' 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> import glob 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> import re 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> import os 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> import subprocess 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> def _find_admin_socket(client_name): 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> files = glob.glob(asok_path) 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> # Given a non-glob path, it better be there 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> if "*" not in asok_path: 2024-04-02T04:45:33.462 DEBUG:teuthology.orchestra.run.smithi179:> assert(len(files) == 1) 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> return files[0] 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> for f in files: 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> contents = proc_f.read() 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> if mountpoint in contents: 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> return f 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> print(_find_admin_socket("client.1")) 2024-04-02T04:45:33.463 DEBUG:teuthology.orchestra.run.smithi179:> ' 2024-04-02T04:45:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:33 smithi179 ceph-mon[36603]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:45:33.581 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:33 smithi070 ceph-mon[29212]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:45:33.588 INFO:teuthology.orchestra.run.smithi179.stdout:/var/run/ceph/ceph-client.1.62638.asok 2024-04-02T04:45:33.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-02T04:45:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:45:33.599 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.62638.asok 2024-04-02T04:45:33.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:33.599 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.62638.asok status 2024-04-02T04:45:33.737 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-04-02T04:45:33.737 INFO:teuthology.orchestra.run.smithi179.stdout: "metadata": { 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "entity_id": "1", 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "hostname": "smithi179", 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "pid": "62638", 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "root": "/" 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_count": 0, 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "dentry_pinned_count": 0, 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "id": 24309, 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "inst": { 2024-04-02T04:45:33.738 INFO:teuthology.orchestra.run.smithi179.stdout: "name": { 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "client", 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "num": 24309 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2957635722 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": { 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "type": "v1", 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "addr": "192.168.0.1:0", 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: "nonce": 2957635722 2024-04-02T04:45:33.739 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout: "inst_str": "client.24309 192.168.0.1:0/2957635722", 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout: "addr_str": "192.168.0.1:0/2957635722", 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout: "inode_count": 1, 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_epoch": 13, 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch": 46, 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_epoch_barrier": 0, 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout: "blocklisted": false, 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout: "fs_name": "cephfs" 2024-04-02T04:45:33.740 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-04-02T04:45:33.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:33.746 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2024-04-02T04:45:33.746 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:33.765 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2024-04-02T04:45:33.765 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:33.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:33.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:33.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:33.839 DEBUG:teuthology.orchestra.run.smithi179:> stat --file-system '--printf=%T 2024-04-02T04:45:33.839 DEBUG:teuthology.orchestra.run.smithi179:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:33.854 INFO:teuthology.orchestra.run.smithi179.stdout:fuseblk 2024-04-02T04:45:33.854 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:33.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:45:33.855 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:33.924 INFO:teuthology.run_tasks:Running task print... 2024-04-02T04:45:33.936 INFO:teuthology.task.print:**** done client 2024-04-02T04:45:33.936 INFO:teuthology.run_tasks:Running task parallel... 2024-04-02T04:45:33.989 INFO:teuthology.task.parallel:starting parallel... 2024-04-02T04:45:33.989 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-02T04:45:33.990 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-02T04:45:33.991 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:45:33.991 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-02T04:45:33.991 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-02T04:45:33.991 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-02T04:45:33.995 INFO:tasks.workunit:Pulling workunits from ref 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:45:33.996 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-02T04:45:33.996 INFO:tasks.workunit:timeout=3h 2024-04-02T04:45:33.996 INFO:tasks.workunit:cleanup=True 2024-04-02T04:45:33.996 DEBUG:teuthology.orchestra.run.smithi070:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:34.014 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-02T04:45:34.014 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 9fh/159d Inode: 1 Links: 2 2024-04-02T04:45:34.014 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:45:34.014 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-02T04:45:34.014 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-02T04:45:34.015 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-02 04:44:57.789524987 +0000 2024-04-02T04:45:34.015 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-02 04:45:33.833339158 +0000 2024-04-02T04:45:34.015 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2024-04-02T04:45:34.015 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-02T04:45:34.015 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-02T04:45:34.088 DEBUG:teuthology.orchestra.run.smithi179:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:34.105 INFO:teuthology.orchestra.run.smithi179.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:34.105 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-02T04:45:34.106 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 7eh/126d Inode: 1 Links: 3 2024-04-02T04:45:34.106 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:45:34.106 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-02T04:45:34.106 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-02T04:45:34.106 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-02 04:45:34.079880497 +0000 2024-04-02T04:45:34.106 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-02 04:45:34.079880497 +0000 2024-04-02T04:45:34.106 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-04-02T04:45:34.106 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-02T04:45:34.107 DEBUG:teuthology.orchestra.run.smithi179:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-02T04:45:34.178 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:45:34.179 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:45:34.194 INFO:tasks.workunit.client.0.smithi070.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-02T04:45:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:34.234 INFO:tasks.workunit.client.1.smithi179.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-02T04:45:35.570 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:35 smithi070 ceph-mon[29212]: pgmap v202: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:45:35.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:35 smithi179 ceph-mon[36603]: pgmap v202: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 -- 172.21.15.70:0/3967442443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0934101f50 msgr2=0x7f09341023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 --2- 172.21.15.70:0/3967442443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0934101f50 0x7f09341023b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0924009910 tx=0x7f0924009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 -- 172.21.15.70:0/3967442443 shutdown_connections 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 --2- 172.21.15.70:0/3967442443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0934101f50 0x7f09341023b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 --2- 172.21.15.70:0/3967442443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09340fef70 0x7f09340ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 -- 172.21.15.70:0/3967442443 >> 172.21.15.70:0/3967442443 conn(0x7f09340fa8b0 msgr2=0x7f09340fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 -- 172.21.15.70:0/3967442443 shutdown_connections 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 -- 172.21.15.70:0/3967442443 wait complete. 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 Processor -- start 2024-04-02T04:45:35.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 -- start start 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340fef70 0x7f09341018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0934101f50 0x7f09340fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934100440 con 0x7f09340fef70 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.951+0000 7f093b779700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934100580 con 0x7f0934101f50 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f0939515700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340fef70 0x7f09341018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f0939515700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340fef70 0x7f09341018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52456/0 (socket says 172.21.15.70:52456) 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f0939515700 1 -- 172.21.15.70:0/212997993 learned_addr learned my addr 172.21.15.70:0/212997993 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f0939515700 1 -- 172.21.15.70:0/212997993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0934101f50 msgr2=0x7f09340fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f0938d14700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0934101f50 0x7f09340fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f0939515700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0934101f50 0x7f09340fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f0939515700 1 -- 172.21.15.70:0/212997993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240095f0 con 0x7f09340fef70 2024-04-02T04:45:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f0939515700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340fef70 0x7f09341018b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f093000c8f0 tx=0x7f093000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f092a7fc700 1 -- 172.21.15.70:0/212997993 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0930007a70 con 0x7f09340fef70 2024-04-02T04:45:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0934100860 con 0x7f09340fef70 2024-04-02T04:45:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f092a7fc700 1 -- 172.21.15.70:0/212997993 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0930007bd0 con 0x7f09340fef70 2024-04-02T04:45:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f092a7fc700 1 -- 172.21.15.70:0/212997993 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0930018630 con 0x7f09340fef70 2024-04-02T04:45:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.952+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093419e510 con 0x7f09340fef70 2024-04-02T04:45:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.953+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934109810 con 0x7f09340fef70 2024-04-02T04:45:35.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.954+0000 7f092a7fc700 1 -- 172.21.15.70:0/212997993 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f093000f450 con 0x7f09340fef70 2024-04-02T04:45:35.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.954+0000 7f092a7fc700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f092006c4e0 0x7f092006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:35.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.954+0000 7f092a7fc700 1 -- 172.21.15.70:0/212997993 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f093008b830 con 0x7f09340fef70 2024-04-02T04:45:35.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.954+0000 7f0938d14700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f092006c4e0 0x7f092006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:35.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.955+0000 7f0938d14700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f092006c4e0 0x7f092006e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f092400b5c0 tx=0x7f0924005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:35.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:35.959+0000 7f092a7fc700 1 -- 172.21.15.70:0/212997993 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0930059b50 con 0x7f09340fef70 2024-04-02T04:45:36.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.152+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f093419eac0 con 0x7f09340fef70 2024-04-02T04:45:36.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.152+0000 7f092a7fc700 1 -- 172.21.15.70:0/212997993 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f093419eac0 con 0x7f09340fef70 2024-04-02T04:45:36.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f092006c4e0 msgr2=0x7f092006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:36.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f092006c4e0 0x7f092006e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f092400b5c0 tx=0x7f0924005960 comp rx=0 tx=0).stop 2024-04-02T04:45:36.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340fef70 msgr2=0x7f09341018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:36.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340fef70 0x7f09341018b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f093000c8f0 tx=0x7f093000ccb0 comp rx=0 tx=0).stop 2024-04-02T04:45:36.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 shutdown_connections 2024-04-02T04:45:36.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f092006c4e0 0x7f092006e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:36.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340fef70 0x7f09341018b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:36.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 --2- 172.21.15.70:0/212997993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0934101f50 0x7f09340fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:36.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 >> 172.21.15.70:0/212997993 conn(0x7f09340fa8b0 msgr2=0x7f0934105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:36.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 shutdown_connections 2024-04-02T04:45:36.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:36.155+0000 7f093b779700 1 -- 172.21.15.70:0/212997993 wait complete. 2024-04-02T04:45:36.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-02T04:45:37.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:36 smithi179 ceph-mon[36603]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-02T04:45:37.093 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:37.215 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:36 smithi070 ceph-mon[29212]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-02T04:45:38.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.849+0000 7f189c57a700 1 -- 172.21.15.70:0/4192440556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894100dc0 msgr2=0x7f1894101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:38.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.849+0000 7f189c57a700 1 --2- 172.21.15.70:0/4192440556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894100dc0 0x7f1894101220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1884009910 tx=0x7f1884009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:38.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 -- 172.21.15.70:0/4192440556 shutdown_connections 2024-04-02T04:45:38.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 --2- 172.21.15.70:0/4192440556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1894100dc0 0x7f1894101220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:38.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 --2- 172.21.15.70:0/4192440556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18940ffbc0 0x7f18940fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 -- 172.21.15.70:0/4192440556 >> 172.21.15.70:0/4192440556 conn(0x7f18940fb160 msgr2=0x7f18940fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 -- 172.21.15.70:0/4192440556 shutdown_connections 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 -- 172.21.15.70:0/4192440556 wait complete. 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 Processor -- start 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 -- start start 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18940ffbc0 0x7f18940699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100dc0 0x7f189419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419bb80 con 0x7f18940ffbc0 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.850+0000 7f189c57a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419bcc0 con 0x7f1894100dc0 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189a316700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18940ffbc0 0x7f18940699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189a316700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18940ffbc0 0x7f18940699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51560/0 (socket says 172.21.15.70:51560) 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189a316700 1 -- 172.21.15.70:0/1789043453 learned_addr learned my addr 172.21.15.70:0/1789043453 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:38.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f1899b15700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100dc0 0x7f189419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189a316700 1 -- 172.21.15.70:0/1789043453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100dc0 msgr2=0x7f189419b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189a316700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100dc0 0x7f189419b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189a316700 1 -- 172.21.15.70:0/1789043453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18840095f0 con 0x7f18940ffbc0 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f1899b15700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100dc0 0x7f189419b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189a316700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18940ffbc0 0x7f18940699d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f189000b480 tx=0x7f189000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f188b7fe700 1 -- 172.21.15.70:0/1789043453 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18900118c0 con 0x7f18940ffbc0 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f188b7fe700 1 -- 172.21.15.70:0/1789043453 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1890010380 con 0x7f18940ffbc0 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189419bfa0 con 0x7f18940ffbc0 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f188b7fe700 1 -- 172.21.15.70:0/1789043453 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189000f4f0 con 0x7f18940ffbc0 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.851+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189419c460 con 0x7f18940ffbc0 2024-04-02T04:45:38.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.852+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189418f5d0 con 0x7f18940ffbc0 2024-04-02T04:45:38.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.853+0000 7f188b7fe700 1 -- 172.21.15.70:0/1789043453 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f18900104f0 con 0x7f18940ffbc0 2024-04-02T04:45:38.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.853+0000 7f188b7fe700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f188006c4e0 0x7f188006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:38.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.853+0000 7f188b7fe700 1 -- 172.21.15.70:0/1789043453 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f189008b3c0 con 0x7f18940ffbc0 2024-04-02T04:45:38.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.853+0000 7f1899b15700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f188006c4e0 0x7f188006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:38.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.854+0000 7f1899b15700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f188006c4e0 0x7f188006e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18840059d0 tx=0x7f1884005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:38.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:38.858+0000 7f188b7fe700 1 -- 172.21.15.70:0/1789043453 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f18900596b0 con 0x7f18940ffbc0 2024-04-02T04:45:39.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.048+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f1894068af0 con 0x7f18940ffbc0 2024-04-02T04:45:39.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.049+0000 7f188b7fe700 1 -- 172.21.15.70:0/1789043453 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f1890059240 con 0x7f18940ffbc0 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f188006c4e0 msgr2=0x7f188006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f188006c4e0 0x7f188006e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18840059d0 tx=0x7f1884005920 comp rx=0 tx=0).stop 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18940ffbc0 msgr2=0x7f18940699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18940ffbc0 0x7f18940699d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f189000b480 tx=0x7f189000b840 comp rx=0 tx=0).stop 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 shutdown_connections 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f188006c4e0 0x7f188006e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18940ffbc0 0x7f18940699d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 --2- 172.21.15.70:0/1789043453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100dc0 0x7f189419b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 >> 172.21.15.70:0/1789043453 conn(0x7f18940fb160 msgr2=0x7f18941040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 shutdown_connections 2024-04-02T04:45:39.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:39.051+0000 7f189c57a700 1 -- 172.21.15.70:0/1789043453 wait complete. 2024-04-02T04:45:39.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:39 smithi179 ceph-mon[36603]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-02T04:45:39.703 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:39 smithi070 ceph-mon[29212]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-02T04:45:39.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-02T04:45:40.039 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:41 smithi070 ceph-mon[29212]: pgmap v205: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-02T04:45:41.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:41 smithi179 ceph-mon[36603]: pgmap v205: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-02T04:45:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 -- 172.21.15.70:0/2463172081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f4100db0 msgr2=0x7fb0f4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 --2- 172.21.15.70:0/2463172081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f4100db0 0x7fb0f4101210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009910 tx=0x7fb0e4009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 -- 172.21.15.70:0/2463172081 shutdown_connections 2024-04-02T04:45:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 --2- 172.21.15.70:0/2463172081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f4100db0 0x7fb0f4101210 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:41.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 --2- 172.21.15.70:0/2463172081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f40ffbb0 0x7fb0f40fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:41.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 -- 172.21.15.70:0/2463172081 >> 172.21.15.70:0/2463172081 conn(0x7fb0f40fb150 msgr2=0x7fb0f40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:41.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 -- 172.21.15.70:0/2463172081 shutdown_connections 2024-04-02T04:45:41.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 -- 172.21.15.70:0/2463172081 wait complete. 2024-04-02T04:45:41.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 Processor -- start 2024-04-02T04:45:41.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.778+0000 7fb0fbee6700 1 -- start start 2024-04-02T04:45:41.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0fbee6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f40ffbb0 0x7fb0f41953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:41.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0fbee6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f4100db0 0x7fb0f4195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:41.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0fbee6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4195f50 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0fbee6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4196090 con 0x7fb0f4100db0 2024-04-02T04:45:41.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0f9c82700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f40ffbb0 0x7fb0f41953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:41.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0f9c82700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f40ffbb0 0x7fb0f41953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51580/0 (socket says 172.21.15.70:51580) 2024-04-02T04:45:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0f9c82700 1 -- 172.21.15.70:0/1184843294 learned_addr learned my addr 172.21.15.70:0/1184843294 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0f9481700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f4100db0 0x7fb0f4195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0f9c82700 1 -- 172.21.15.70:0/1184843294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f4100db0 msgr2=0x7fb0f4195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0f9c82700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f4100db0 0x7fb0f4195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0f9c82700 1 -- 172.21.15.70:0/1184843294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40095f0 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0f9c82700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f40ffbb0 0x7fb0f41953f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000d900 tx=0x7fb0f000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0eaffd700 1 -- 172.21.15.70:0/1184843294 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f00041f0 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0eaffd700 1 -- 172.21.15.70:0/1184843294 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb0f0004350 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f419ab40 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0eaffd700 1 -- 172.21.15.70:0/1184843294 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f0003d20 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.779+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f419b090 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.780+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f4108710 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.781+0000 7fb0eaffd700 1 -- 172.21.15.70:0/1184843294 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb0f0009730 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.781+0000 7fb0eaffd700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb0e006c4e0 0x7fb0e006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:41.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.782+0000 7fb0f9481700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb0e006c4e0 0x7fb0e006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:41.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.782+0000 7fb0eaffd700 1 -- 172.21.15.70:0/1184843294 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb0f008b6b0 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.782+0000 7fb0f9481700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb0e006c4e0 0x7fb0e006e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4006010 tx=0x7fb0e400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:41.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.786+0000 7fb0eaffd700 1 -- 172.21.15.70:0/1184843294 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb0f0059920 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.975+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fb0f4069490 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.975+0000 7fb0eaffd700 1 -- 172.21.15.70:0/1184843294 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fb0f0003e80 con 0x7fb0f40ffbb0 2024-04-02T04:45:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb0e006c4e0 msgr2=0x7fb0e006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb0e006c4e0 0x7fb0e006e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4006010 tx=0x7fb0e400b540 comp rx=0 tx=0).stop 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f40ffbb0 msgr2=0x7fb0f41953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f40ffbb0 0x7fb0f41953f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000d900 tx=0x7fb0f000dc10 comp rx=0 tx=0).stop 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 shutdown_connections 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fb0e006c4e0 0x7fb0e006e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f40ffbb0 0x7fb0f41953f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 --2- 172.21.15.70:0/1184843294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f4100db0 0x7fb0f4195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 >> 172.21.15.70:0/1184843294 conn(0x7fb0f40fb150 msgr2=0x7fb0f4103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 shutdown_connections 2024-04-02T04:45:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:41.977+0000 7fb0fbee6700 1 -- 172.21.15.70:0/1184843294 wait complete. 2024-04-02T04:45:42.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-02T04:45:43.001 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:45:43.450 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:43 smithi070 ceph-mon[29212]: pgmap v206: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-02T04:45:43.450 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:43.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:43 smithi179 ceph-mon[36603]: pgmap v206: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-02T04:45:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.650+0000 7f320c78b700 1 -- 172.21.15.70:0/2369626573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32040ffbc0 msgr2=0x7f32040fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.650+0000 7f320c78b700 1 --2- 172.21.15.70:0/2369626573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32040ffbc0 0x7f32040fffe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3200009910 tx=0x7f3200009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.650+0000 7f320c78b700 1 -- 172.21.15.70:0/2369626573 shutdown_connections 2024-04-02T04:45:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.650+0000 7f320c78b700 1 --2- 172.21.15.70:0/2369626573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3204100dc0 0x7f3204101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.650+0000 7f320c78b700 1 --2- 172.21.15.70:0/2369626573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32040ffbc0 0x7f32040fffe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.650+0000 7f320c78b700 1 -- 172.21.15.70:0/2369626573 >> 172.21.15.70:0/2369626573 conn(0x7f32040fb160 msgr2=0x7f32040fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.650+0000 7f320c78b700 1 -- 172.21.15.70:0/2369626573 shutdown_connections 2024-04-02T04:45:44.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.650+0000 7f320c78b700 1 -- 172.21.15.70:0/2369626573 wait complete. 2024-04-02T04:45:44.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f320c78b700 1 Processor -- start 2024-04-02T04:45:44.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f320c78b700 1 -- start start 2024-04-02T04:45:44.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f320c78b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32040ffbc0 0x7f32041989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:44.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f320c78b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3204100dc0 0x7f3204198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:44.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f320c78b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204199550 con 0x7f3204100dc0 2024-04-02T04:45:44.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f320c78b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204199690 con 0x7f32040ffbc0 2024-04-02T04:45:44.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f3209d26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3204100dc0 0x7f3204198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:44.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f3209d26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3204100dc0 0x7f3204198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51608/0 (socket says 172.21.15.70:51608) 2024-04-02T04:45:44.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f3209d26700 1 -- 172.21.15.70:0/1802807357 learned_addr learned my addr 172.21.15.70:0/1802807357 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:45:44.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f320a527700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32040ffbc0 0x7f32041989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:44.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f3209d26700 1 -- 172.21.15.70:0/1802807357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32040ffbc0 msgr2=0x7f32041989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f3209d26700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32040ffbc0 0x7f32041989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f3209d26700 1 -- 172.21.15.70:0/1802807357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000095f0 con 0x7f3204100dc0 2024-04-02T04:45:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.651+0000 7f3209d26700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3204100dc0 0x7f3204198f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f31f800b4b0 tx=0x7f31f800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.652+0000 7f31f77fe700 1 -- 172.21.15.70:0/1802807357 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31f8011800 con 0x7f3204100dc0 2024-04-02T04:45:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.652+0000 7f31f77fe700 1 -- 172.21.15.70:0/1802807357 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f31f8011e10 con 0x7f3204100dc0 2024-04-02T04:45:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.652+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320419e140 con 0x7f3204100dc0 2024-04-02T04:45:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.652+0000 7f31f77fe700 1 -- 172.21.15.70:0/1802807357 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31f800f2e0 con 0x7f3204100dc0 2024-04-02T04:45:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.652+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320419e690 con 0x7f3204100dc0 2024-04-02T04:45:44.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.653+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3204068af0 con 0x7f3204100dc0 2024-04-02T04:45:44.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.653+0000 7f31f77fe700 1 -- 172.21.15.70:0/1802807357 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f31f8010380 con 0x7f3204100dc0 2024-04-02T04:45:44.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.654+0000 7f31f77fe700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f31f006c4e0 0x7f31f006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:44.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.654+0000 7f320a527700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f31f006c4e0 0x7f31f006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:44.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.654+0000 7f31f77fe700 1 -- 172.21.15.70:0/1802807357 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f31f808b2e0 con 0x7f3204100dc0 2024-04-02T04:45:44.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.654+0000 7f320a527700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f31f006c4e0 0x7f31f006e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3200006010 tx=0x7f320000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:44.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.658+0000 7f31f77fe700 1 -- 172.21.15.70:0/1802807357 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f31f80595d0 con 0x7f3204100dc0 2024-04-02T04:45:44.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:45:44.878+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f3204069580 con 0x7f31f006c4e0 2024-04-02T04:45:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:45 smithi179 ceph-mon[36603]: pgmap v207: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-02T04:45:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:45 smithi070 ceph-mon[29212]: pgmap v207: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-02T04:45:46.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:46 smithi179 ceph-mon[36603]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:46 smithi070 ceph-mon[29212]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:47.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:47 smithi179 ceph-mon[36603]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-02T04:45:47.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:47 smithi070 ceph-mon[29212]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-02T04:45:49.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:49 smithi179 ceph-mon[36603]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-02T04:45:49.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:49 smithi070 ceph-mon[29212]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-02T04:45:51.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:51 smithi070 ceph-mon[29212]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-02T04:45:51.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:51 smithi179 ceph-mon[36603]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-02T04:45:53.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:53 smithi070 ceph-mon[29212]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-02T04:45:53.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:53 smithi179 ceph-mon[36603]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-02T04:45:55.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:55 smithi070 ceph-mon[29212]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-02T04:45:55.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:55 smithi179 ceph-mon[36603]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-02T04:45:57.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:56 smithi179 ceph-mon[36603]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-02T04:45:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:56 smithi070 ceph-mon[29212]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-02T04:45:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:58.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:59.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:45:59 smithi179 ceph-mon[36603]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:45:59.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:45:59 smithi070 ceph-mon[29212]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:01.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:01 smithi179 ceph-mon[36603]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:01.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:01 smithi070 ceph-mon[29212]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:03.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:03 smithi179 ceph-mon[36603]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:03.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:03 smithi070 ceph-mon[29212]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:05.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:05 smithi179 ceph-mon[36603]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:05.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:05 smithi070 ceph-mon[29212]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:07.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:06 smithi179 ceph-mon[36603]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:07.281 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:06 smithi070 ceph-mon[29212]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:09.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:09 smithi179 ceph-mon[36603]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:09.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:09 smithi070 ceph-mon[29212]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:11.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:11 smithi179 ceph-mon[36603]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:11.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:11 smithi070 ceph-mon[29212]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:13.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:13 smithi179 ceph-mon[36603]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:13.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:13 smithi070 ceph-mon[29212]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:13.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:15.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:15 smithi070 ceph-mon[29212]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:15.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:15 smithi179 ceph-mon[36603]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:17.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:16 smithi179 ceph-mon[36603]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:17.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:16 smithi070 ceph-mon[29212]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:19.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:19 smithi179 ceph-mon[36603]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:19.698 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:19 smithi070 ceph-mon[29212]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:21.003 INFO:teuthology.orchestra.run.smithi070.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:46:21.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:20.998+0000 7f31f77fe700 1 -- 172.21.15.70:0/1802807357 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f3204069580 con 0x7f31f006c4e0 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.000+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f31f006c4e0 msgr2=0x7f31f006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f31f006c4e0 0x7f31f006e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3200006010 tx=0x7f320000b540 comp rx=0 tx=0).stop 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3204100dc0 msgr2=0x7f3204198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3204100dc0 0x7f3204198f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f31f800b4b0 tx=0x7f31f800b7c0 comp rx=0 tx=0).stop 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 shutdown_connections 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f31f006c4e0 0x7f31f006e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32040ffbc0 0x7f32041989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 --2- 172.21.15.70:0/1802807357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3204100dc0 0x7f3204198f30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 >> 172.21.15.70:0/1802807357 conn(0x7f32040fb160 msgr2=0x7f3204068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 shutdown_connections 2024-04-02T04:46:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:21.001+0000 7f320c78b700 1 -- 172.21.15.70:0/1802807357 wait complete. 2024-04-02T04:46:21.498 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:21 smithi070 ceph-mon[29212]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:21.498 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:21 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:46:21.498 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:21 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:21.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:21 smithi179 ceph-mon[36603]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:21 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:46:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:21 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:21.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-02T04:46:22.228 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:46:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:22 smithi179 ceph-mon[36603]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:46:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:46:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:22 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:46:22.609 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:22 smithi070 ceph-mon[29212]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:46:22.609 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:22.609 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:46:22.609 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:22 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:46:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:23 smithi179 ceph-mon[36603]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:46:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:23 smithi179 ceph-mon[36603]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:23.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:23 smithi070 ceph-mon[29212]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:46:23.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:23 smithi070 ceph-mon[29212]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:25.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:25 smithi070 ceph-mon[29212]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:25.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:25 smithi179 ceph-mon[36603]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:27.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:26 smithi179 ceph-mon[36603]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:27.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:26 smithi070 ceph-mon[29212]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:28.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:28.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:29.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:29 smithi179 ceph-mon[36603]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:29.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:29 smithi070 ceph-mon[29212]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:31.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:31 smithi179 ceph-mon[36603]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:31.661 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:31 smithi070 ceph-mon[29212]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.260+0000 7f082dfe7700 1 -- 172.21.15.70:0/35396143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828104100 msgr2=0x7f08281064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.260+0000 7f082dfe7700 1 --2- 172.21.15.70:0/35396143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828104100 0x7f08281064f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f081c009910 tx=0x7f081c009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.260+0000 7f082dfe7700 1 -- 172.21.15.70:0/35396143 shutdown_connections 2024-04-02T04:46:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.260+0000 7f082dfe7700 1 --2- 172.21.15.70:0/35396143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828104100 0x7f08281064f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.260+0000 7f082dfe7700 1 --2- 172.21.15.70:0/35396143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08281017d0 0x7f0828103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.260+0000 7f082dfe7700 1 -- 172.21.15.70:0/35396143 >> 172.21.15.70:0/35396143 conn(0x7f08280fb160 msgr2=0x7f08280fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.260+0000 7f082dfe7700 1 -- 172.21.15.70:0/35396143 shutdown_connections 2024-04-02T04:46:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.260+0000 7f082dfe7700 1 -- 172.21.15.70:0/35396143 wait complete. 2024-04-02T04:46:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.261+0000 7f082dfe7700 1 Processor -- start 2024-04-02T04:46:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.261+0000 7f082dfe7700 1 -- start start 2024-04-02T04:46:33.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.261+0000 7f082dfe7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08281017d0 0x7f0828101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:33.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.261+0000 7f082dfe7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08280ff780 0x7f08280ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.261+0000 7f082dfe7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082819a5f0 con 0x7f08280ff780 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.261+0000 7f082dfe7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082819a760 con 0x7f08281017d0 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f08277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08281017d0 0x7f0828101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f08277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08281017d0 0x7f0828101130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:52084/0 (socket says 172.21.15.70:52084) 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f08277fe700 1 -- 172.21.15.70:0/1078776683 learned_addr learned my addr 172.21.15.70:0/1078776683 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f0826ffd700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08280ff780 0x7f08280ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f08277fe700 1 -- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08280ff780 msgr2=0x7f08280ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f08277fe700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08280ff780 0x7f08280ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f08277fe700 1 -- 172.21.15.70:0/1078776683 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0095f0 con 0x7f08281017d0 2024-04-02T04:46:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f0826ffd700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08280ff780 0x7f08280ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:46:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f08277fe700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08281017d0 0x7f0828101130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f081800e920 tx=0x7f081800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f0824ff9700 1 -- 172.21.15.70:0/1078776683 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0818009790 con 0x7f08281017d0 2024-04-02T04:46:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f0824ff9700 1 -- 172.21.15.70:0/1078776683 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f081800cba0 con 0x7f08281017d0 2024-04-02T04:46:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0828100140 con 0x7f08281017d0 2024-04-02T04:46:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.262+0000 7f0824ff9700 1 -- 172.21.15.70:0/1078776683 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0818018630 con 0x7f08281017d0 2024-04-02T04:46:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.263+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08281a31f0 con 0x7f08281017d0 2024-04-02T04:46:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.264+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0828190630 con 0x7f08281017d0 2024-04-02T04:46:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.264+0000 7f0824ff9700 1 -- 172.21.15.70:0/1078776683 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f081800cd10 con 0x7f08281017d0 2024-04-02T04:46:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.265+0000 7f0824ff9700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f081006c4e0 0x7f081006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.265+0000 7f0824ff9700 1 -- 172.21.15.70:0/1078776683 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0818014070 con 0x7f08281017d0 2024-04-02T04:46:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.265+0000 7f0826ffd700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f081006c4e0 0x7f081006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.266+0000 7f0826ffd700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f081006c4e0 0x7f081006e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f081c000c00 tx=0x7f081c009f70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:33.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.274+0000 7f0824ff9700 1 -- 172.21.15.70:0/1078776683 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f08180520e0 con 0x7f08281017d0 2024-04-02T04:46:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.508+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f082802ce70 con 0x7f081006c4e0 2024-04-02T04:46:33.512 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:33 smithi070 ceph-mon[29212]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:33.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.510+0000 7f0824ff9700 1 -- 172.21.15.70:0/1078776683 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f082802ce70 con 0x7f081006c4e0 2024-04-02T04:46:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f081006c4e0 msgr2=0x7f081006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f081006c4e0 0x7f081006e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f081c000c00 tx=0x7f081c009f70 comp rx=0 tx=0).stop 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08281017d0 msgr2=0x7f0828101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08281017d0 0x7f0828101130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f081800e920 tx=0x7f081800ece0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 shutdown_connections 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f081006c4e0 0x7f081006e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08281017d0 0x7f0828101130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 --2- 172.21.15.70:0/1078776683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08280ff780 0x7f08280ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.512+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 >> 172.21.15.70:0/1078776683 conn(0x7f08280fb160 msgr2=0x7f08280fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.513+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 shutdown_connections 2024-04-02T04:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.513+0000 7f082dfe7700 1 -- 172.21.15.70:0/1078776683 wait complete. 2024-04-02T04:46:33.531 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:46:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:33 smithi179 ceph-mon[36603]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:33.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.673+0000 7f5bc2251700 1 -- 172.21.15.70:0/1176480364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc100e10 msgr2=0x7f5bbc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:33.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.673+0000 7f5bc2251700 1 --2- 172.21.15.70:0/1176480364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc100e10 0x7f5bbc101270 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009910 tx=0x7f5bb0009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:33.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.673+0000 7f5bc2251700 1 -- 172.21.15.70:0/1176480364 shutdown_connections 2024-04-02T04:46:33.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.673+0000 7f5bc2251700 1 --2- 172.21.15.70:0/1176480364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc100e10 0x7f5bbc101270 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.673+0000 7f5bc2251700 1 --2- 172.21.15.70:0/1176480364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc0ffc10 0x7f5bbc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.673+0000 7f5bc2251700 1 -- 172.21.15.70:0/1176480364 >> 172.21.15.70:0/1176480364 conn(0x7f5bbc0fb190 msgr2=0x7f5bbc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:33.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.674+0000 7f5bc2251700 1 -- 172.21.15.70:0/1176480364 shutdown_connections 2024-04-02T04:46:33.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.674+0000 7f5bc2251700 1 -- 172.21.15.70:0/1176480364 wait complete. 2024-04-02T04:46:33.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.675+0000 7f5bc2251700 1 Processor -- start 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.675+0000 7f5bc2251700 1 -- start start 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bc2251700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc0ffc10 0x7f5bbc1953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bc2251700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc100e10 0x7f5bbc1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bc2251700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc195f00 con 0x7f5bbc0ffc10 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bc2251700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc196040 con 0x7f5bbc100e10 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bbb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc0ffc10 0x7f5bbc1953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bbb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc0ffc10 0x7f5bbc1953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55920/0 (socket says 172.21.15.70:55920) 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bbb7fe700 1 -- 172.21.15.70:0/3864700804 learned_addr learned my addr 172.21.15.70:0/3864700804 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:46:33.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bbaffd700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc100e10 0x7f5bbc1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:33.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bbaffd700 1 -- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc0ffc10 msgr2=0x7f5bbc1953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:33.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bbaffd700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc0ffc10 0x7f5bbc1953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:33.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bbaffd700 1 -- 172.21.15.70:0/3864700804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb00095f0 con 0x7f5bbc100e10 2024-04-02T04:46:33.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.676+0000 7f5bbb7fe700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc0ffc10 0x7f5bbc1953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:46:33.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.677+0000 7f5bbaffd700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc100e10 0x7f5bbc1958e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:33.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.677+0000 7f5bb8ff9700 1 -- 172.21.15.70:0/3864700804 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb001d070 con 0x7f5bbc100e10 2024-04-02T04:46:33.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.677+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc068580 con 0x7f5bbc100e10 2024-04-02T04:46:33.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.677+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc068a70 con 0x7f5bbc100e10 2024-04-02T04:46:33.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.677+0000 7f5bb8ff9700 1 -- 172.21.15.70:0/3864700804 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5bb0005dc0 con 0x7f5bbc100e10 2024-04-02T04:46:33.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.677+0000 7f5bb8ff9700 1 -- 172.21.15.70:0/3864700804 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bb0017400 con 0x7f5bbc100e10 2024-04-02T04:46:33.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.678+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba8005320 con 0x7f5bbc100e10 2024-04-02T04:46:33.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.679+0000 7f5bb8ff9700 1 -- 172.21.15.70:0/3864700804 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5bb0003680 con 0x7f5bbc100e10 2024-04-02T04:46:33.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.680+0000 7f5bb8ff9700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5ba406c4e0 0x7f5ba406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:33.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.680+0000 7f5bb8ff9700 1 -- 172.21.15.70:0/3864700804 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5bb0044020 con 0x7f5bbc100e10 2024-04-02T04:46:33.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.680+0000 7f5bbb7fe700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5ba406c4e0 0x7f5ba406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:33.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.681+0000 7f5bbb7fe700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5ba406c4e0 0x7f5ba406e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5bac005950 tx=0x7f5bac010040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:33.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:33.688+0000 7f5bb8ff9700 1 -- 172.21.15.70:0/3864700804 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5bb005b1f0 con 0x7f5bbc100e10 2024-04-02T04:46:34.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.061+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ba8000bf0 con 0x7f5ba406c4e0 2024-04-02T04:46:34.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.062+0000 7f5bb8ff9700 1 -- 172.21.15.70:0/3864700804 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f5ba8000bf0 con 0x7f5ba406c4e0 2024-04-02T04:46:34.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5ba406c4e0 msgr2=0x7f5ba406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5ba406c4e0 0x7f5ba406e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5bac005950 tx=0x7f5bac010040 comp rx=0 tx=0).stop 2024-04-02T04:46:34.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc100e10 msgr2=0x7f5bbc1958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc100e10 0x7f5bbc1958e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb000f690 comp rx=0 tx=0).stop 2024-04-02T04:46:34.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 shutdown_connections 2024-04-02T04:46:34.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f5ba406c4e0 0x7f5ba406e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bbc0ffc10 0x7f5bbc1953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 --2- 172.21.15.70:0/3864700804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc100e10 0x7f5bbc1958e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.064+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 >> 172.21.15.70:0/3864700804 conn(0x7f5bbc0fb190 msgr2=0x7f5bbc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:34.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.065+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 shutdown_connections 2024-04-02T04:46:34.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.065+0000 7f5bc2251700 1 -- 172.21.15.70:0/3864700804 wait complete. 2024-04-02T04:46:34.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.174+0000 7fe8df868700 1 -- 172.21.15.70:0/2510993049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 msgr2=0x7fe8d8100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.174+0000 7fe8df868700 1 --2- 172.21.15.70:0/2510993049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 0x7fe8d8100390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009910 tx=0x7fe8c8009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:34.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.174+0000 7fe8df868700 1 -- 172.21.15.70:0/2510993049 shutdown_connections 2024-04-02T04:46:34.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.174+0000 7fe8df868700 1 --2- 172.21.15.70:0/2510993049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 0x7fe8d8100390 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.174+0000 7fe8df868700 1 --2- 172.21.15.70:0/2510993049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d80ff5b0 0x7fe8d80ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.174+0000 7fe8df868700 1 -- 172.21.15.70:0/2510993049 >> 172.21.15.70:0/2510993049 conn(0x7fe8d80fb150 msgr2=0x7fe8d80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.174+0000 7fe8df868700 1 -- 172.21.15.70:0/2510993049 shutdown_connections 2024-04-02T04:46:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.174+0000 7fe8df868700 1 -- 172.21.15.70:0/2510993049 wait complete. 2024-04-02T04:46:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.175+0000 7fe8df868700 1 Processor -- start 2024-04-02T04:46:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.175+0000 7fe8df868700 1 -- start start 2024-04-02T04:46:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8df868700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d80ff5b0 0x7fe8d81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8df868700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 0x7fe8d8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8dd604700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d80ff5b0 0x7fe8d81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8dd604700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d80ff5b0 0x7fe8d81986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:52114/0 (socket says 172.21.15.70:52114) 2024-04-02T04:46:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8dce03700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 0x7fe8d8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8dce03700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 0x7fe8d8198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55930/0 (socket says 172.21.15.70:55930) 2024-04-02T04:46:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8dd604700 1 -- 172.21.15.70:0/1137944295 learned_addr learned my addr 172.21.15.70:0/1137944295 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:46:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8df868700 1 -- 172.21.15.70:0/1137944295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8199210 con 0x7fe8d80fff10 2024-04-02T04:46:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.176+0000 7fe8df868700 1 -- 172.21.15.70:0/1137944295 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8199350 con 0x7fe8d80ff5b0 2024-04-02T04:46:34.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.177+0000 7fe8dce03700 1 -- 172.21.15.70:0/1137944295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d80ff5b0 msgr2=0x7fe8d81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.177+0000 7fe8dce03700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d80ff5b0 0x7fe8d81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.177+0000 7fe8dce03700 1 -- 172.21.15.70:0/1137944295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c80095f0 con 0x7fe8d80fff10 2024-04-02T04:46:34.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.177+0000 7fe8dd604700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d80ff5b0 0x7fe8d81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:46:34.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.177+0000 7fe8dce03700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 0x7fe8d8198bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009910 tx=0x7fe8c800fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:34.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.178+0000 7fe8ce7fc700 1 -- 172.21.15.70:0/1137944295 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8c801d070 con 0x7fe8d80fff10 2024-04-02T04:46:34.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.178+0000 7fe8ce7fc700 1 -- 172.21.15.70:0/1137944295 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe8c800fe70 con 0x7fe8d80fff10 2024-04-02T04:46:34.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.178+0000 7fe8ce7fc700 1 -- 172.21.15.70:0/1137944295 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8c8017630 con 0x7fe8d80fff10 2024-04-02T04:46:34.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.178+0000 7fe8df868700 1 -- 172.21.15.70:0/1137944295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d819dda0 con 0x7fe8d80fff10 2024-04-02T04:46:34.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.178+0000 7fe8df868700 1 -- 172.21.15.70:0/1137944295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d819e290 con 0x7fe8d80fff10 2024-04-02T04:46:34.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.179+0000 7fe8c3fff700 1 -- 172.21.15.70:0/1137944295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b80052f0 con 0x7fe8d80fff10 2024-04-02T04:46:34.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.182+0000 7fe8ce7fc700 1 -- 172.21.15.70:0/1137944295 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe8c8003680 con 0x7fe8d80fff10 2024-04-02T04:46:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.182+0000 7fe8ce7fc700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe8c406c530 0x7fe8c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.183+0000 7fe8ce7fc700 1 -- 172.21.15.70:0/1137944295 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fe8c808cff0 con 0x7fe8d80fff10 2024-04-02T04:46:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.183+0000 7fe8dd604700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe8c406c530 0x7fe8c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:34.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.183+0000 7fe8dd604700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe8c406c530 0x7fe8c406e9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009cf0 tx=0x7fe8d4009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.188+0000 7fe8ce7fc700 1 -- 172.21.15.70:0/1137944295 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe8c805b370 con 0x7fe8d80fff10 2024-04-02T04:46:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.469+0000 7fe8c3fff700 1 -- 172.21.15.70:0/1137944295 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8b8000bc0 con 0x7fe8c406c530 2024-04-02T04:46:34.474 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:34 smithi070 ceph-mon[29212]: from='client.24321 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:34.474 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:46:34.474 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:34.474 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:34 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:46:34.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.480+0000 7fe8ce7fc700 1 -- 172.21.15.70:0/1137944295 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7fe8b8000bc0 con 0x7fe8c406c530 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (5m) 71s ago 7m 24.5M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (7m) 71s ago 7m 7457k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (6m) 77s ago 6m 7184k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (7m) 71s ago 7m 6665k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (6m) 77s ago 6m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (5m) 71s ago 6m 85.6M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (98s) 71s ago 97s 16.5M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (92s) 71s ago 91s 14.6M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (89s) 77s ago 89s 13.0M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (95s) 77s ago 94s 14.7M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:9283,8765,8443 running (8m) 71s ago 8m 489M - 18.2.1 5be31c24972a 3299b9e265dd 2024-04-02T04:46:34.488 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (6m) 77s ago 6m 441M - 18.2.1 5be31c24972a c82df176868c 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (8m) 71s ago 8m 44.4M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (6m) 77s ago 6m 40.6M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 71s ago 7m 14.3M - 1.5.0 0da6a335fe13 bc06c0b59df5 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (6m) 77s ago 6m 17.0M - 1.5.0 0da6a335fe13 6fe44da459b3 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (5m) 71s ago 5m 47.9M 3946M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (4m) 71s ago 4m 48.2M 3946M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (4m) 71s ago 4m 46.5M 3946M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (4m) 77s ago 4m 48.5M 4970M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (3m) 77s ago 3m 43.5M 4970M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (3m) 77s ago 3m 42.6M 4970M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:46:34.489 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (5m) 71s ago 6m 56.3M - 2.43.0 a07b618ecd1d b1f1f05601f5 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 -- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe8c406c530 msgr2=0x7fe8c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe8c406c530 0x7fe8c406e9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009cf0 tx=0x7fe8d4009260 comp rx=0 tx=0).stop 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 -- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 msgr2=0x7fe8d8198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 0x7fe8d8198bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009910 tx=0x7fe8c800fb00 comp rx=0 tx=0).stop 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 -- 172.21.15.70:0/1137944295 shutdown_connections 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fe8c406c530 0x7fe8c406e9f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d80ff5b0 0x7fe8d81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 --2- 172.21.15.70:0/1137944295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8d80fff10 0x7fe8d8198bf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 -- 172.21.15.70:0/1137944295 >> 172.21.15.70:0/1137944295 conn(0x7fe8d80fb150 msgr2=0x7fe8d8107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:34.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 -- 172.21.15.70:0/1137944295 shutdown_connections 2024-04-02T04:46:34.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.483+0000 7fe8c3fff700 1 -- 172.21.15.70:0/1137944295 wait complete. 2024-04-02T04:46:34.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.600+0000 7fdaf1892700 1 -- 172.21.15.70:0/1279725150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 msgr2=0x7fdaec103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.600+0000 7fdaf1892700 1 --2- 172.21.15.70:0/1279725150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 0x7fdaec103bc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009910 tx=0x7fdadc009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:34.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.600+0000 7fdaf1892700 1 -- 172.21.15.70:0/1279725150 shutdown_connections 2024-04-02T04:46:34.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.600+0000 7fdaf1892700 1 --2- 172.21.15.70:0/1279725150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104100 0x7fdaec1064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.600+0000 7fdaf1892700 1 --2- 172.21.15.70:0/1279725150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 0x7fdaec103bc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.600+0000 7fdaf1892700 1 -- 172.21.15.70:0/1279725150 >> 172.21.15.70:0/1279725150 conn(0x7fdaec0fb120 msgr2=0x7fdaec0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:34.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.600+0000 7fdaf1892700 1 -- 172.21.15.70:0/1279725150 shutdown_connections 2024-04-02T04:46:34.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.600+0000 7fdaf1892700 1 -- 172.21.15.70:0/1279725150 wait complete. 2024-04-02T04:46:34.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.601+0000 7fdaf1892700 1 Processor -- start 2024-04-02T04:46:34.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.601+0000 7fdaf1892700 1 -- start start 2024-04-02T04:46:34.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.601+0000 7fdaf1892700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 0x7fdaec1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.601+0000 7fdaf1892700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104100 0x7fdaec1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.601+0000 7fdaf1892700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec197010 con 0x7fdaec1017d0 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.601+0000 7fdaf1892700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec197150 con 0x7fdaec104100 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaea7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104100 0x7fdaec1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 0x7fdaec1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaea7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104100 0x7fdaec1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:52136/0 (socket says 172.21.15.70:52136) 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaea7fc700 1 -- 172.21.15.70:0/42995329 learned_addr learned my addr 172.21.15.70:0/42995329 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaea7fc700 1 -- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 msgr2=0x7fdaec1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaea7fc700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 0x7fdaec1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaea7fc700 1 -- 172.21.15.70:0/42995329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc0095f0 con 0x7fdaec104100 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaeaffd700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 0x7fdaec1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaea7fc700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104100 0x7fdaec1969f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e920 tx=0x7fdae000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaf0890700 1 -- 172.21.15.70:0/42995329 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdae000c9d0 con 0x7fdaec104100 2024-04-02T04:46:34.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.602+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec0ff620 con 0x7fdaec104100 2024-04-02T04:46:34.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.603+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec0ffb70 con 0x7fdaec104100 2024-04-02T04:46:34.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.603+0000 7fdaf0890700 1 -- 172.21.15.70:0/42995329 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdae0010430 con 0x7fdaec104100 2024-04-02T04:46:34.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.603+0000 7fdaf0890700 1 -- 172.21.15.70:0/42995329 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdae0018650 con 0x7fdaec104100 2024-04-02T04:46:34.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.604+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad8005320 con 0x7fdaec104100 2024-04-02T04:46:34.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.604+0000 7fdaf0890700 1 -- 172.21.15.70:0/42995329 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fdae000cb30 con 0x7fdaec104100 2024-04-02T04:46:34.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.605+0000 7fdaf0890700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fdad406c4e0 0x7fdad406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:34.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.605+0000 7fdaf0890700 1 -- 172.21.15.70:0/42995329 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fdae0014070 con 0x7fdaec104100 2024-04-02T04:46:34.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.605+0000 7fdaeaffd700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fdad406c4e0 0x7fdad406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:34.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.605+0000 7fdaeaffd700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fdad406c4e0 0x7fdad406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00b6f0 tx=0x7fdadc005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:34.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.613+0000 7fdaf0890700 1 -- 172.21.15.70:0/42995329 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdae00520e0 con 0x7fdaec104100 2024-04-02T04:46:34.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:34 smithi179 ceph-mon[36603]: from='client.24321 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:34.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:46:34.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:34.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:34 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:46:34.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.892+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdad8006200 con 0x7fdaec104100 2024-04-02T04:46:34.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.892+0000 7fdaf0890700 1 -- 172.21.15.70:0/42995329 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fdae0059c30 con 0x7fdaec104100 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:46:34.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:46:34.899 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:46:34.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:46:34.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:46:34.899 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:46:34.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-04-02T04:46:34.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:46:34.899 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.895+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fdad406c4e0 msgr2=0x7fdad406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.895+0000 7fdaf1892700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fdad406c4e0 0x7fdad406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00b6f0 tx=0x7fdadc005a90 comp rx=0 tx=0).stop 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.895+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104100 msgr2=0x7fdaec1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.895+0000 7fdaf1892700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104100 0x7fdaec1969f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e920 tx=0x7fdae000ec30 comp rx=0 tx=0).stop 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.896+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 shutdown_connections 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.896+0000 7fdaf1892700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7fdad406c4e0 0x7fdad406e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.896+0000 7fdaf1892700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaec1017d0 0x7fdaec1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.896+0000 7fdaf1892700 1 --2- 172.21.15.70:0/42995329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104100 0x7fdaec1969f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.896+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 >> 172.21.15.70:0/42995329 conn(0x7fdaec0fb120 msgr2=0x7fdaec0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.896+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 shutdown_connections 2024-04-02T04:46:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:34.896+0000 7fdaf1892700 1 -- 172.21.15.70:0/42995329 wait complete. 2024-04-02T04:46:35.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.005+0000 7f949ad82700 1 -- 172.21.15.70:0/2531559945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9494101f40 msgr2=0x7f94941023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:35.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.005+0000 7f949ad82700 1 --2- 172.21.15.70:0/2531559945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9494101f40 0x7f94941023a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9480009910 tx=0x7f9480009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:35.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.006+0000 7f949ad82700 1 -- 172.21.15.70:0/2531559945 shutdown_connections 2024-04-02T04:46:35.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.006+0000 7f949ad82700 1 --2- 172.21.15.70:0/2531559945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9494101f40 0x7f94941023a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:35.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.006+0000 7f949ad82700 1 --2- 172.21.15.70:0/2531559945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94940fef60 0x7f94940ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:35.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.006+0000 7f949ad82700 1 -- 172.21.15.70:0/2531559945 >> 172.21.15.70:0/2531559945 conn(0x7f94940fa8c0 msgr2=0x7f94940fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:35.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.006+0000 7f949ad82700 1 -- 172.21.15.70:0/2531559945 shutdown_connections 2024-04-02T04:46:35.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.006+0000 7f949ad82700 1 -- 172.21.15.70:0/2531559945 wait complete. 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.006+0000 7f949ad82700 1 Processor -- start 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.006+0000 7f949ad82700 1 -- start start 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f949ad82700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940fef60 0x7f94941986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f949ad82700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9494101f40 0x7f9494198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f949ad82700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494199220 con 0x7f94940fef60 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f949ad82700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494199360 con 0x7f9494101f40 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9493fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9494101f40 0x7f9494198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9493fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9494101f40 0x7f9494198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:52154/0 (socket says 172.21.15.70:52154) 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9493fff700 1 -- 172.21.15.70:0/3357989617 learned_addr learned my addr 172.21.15.70:0/3357989617 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9498b1e700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940fef60 0x7f94941986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9498b1e700 1 -- 172.21.15.70:0/3357989617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9494101f40 msgr2=0x7f9494198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9498b1e700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9494101f40 0x7f9494198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:35.012 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9498b1e700 1 -- 172.21.15.70:0/3357989617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800095f0 con 0x7f94940fef60 2024-04-02T04:46:35.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9493fff700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9494101f40 0x7f9494198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:46:35.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.007+0000 7f9498b1e700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940fef60 0x7f94941986c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f948800bf10 tx=0x7f9488009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:35.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.008+0000 7f9491ffb700 1 -- 172.21.15.70:0/3357989617 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9488014cb0 con 0x7f94940fef60 2024-04-02T04:46:35.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.008+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949418eed0 con 0x7f94940fef60 2024-04-02T04:46:35.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.008+0000 7f9491ffb700 1 -- 172.21.15.70:0/3357989617 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9488014e10 con 0x7f94940fef60 2024-04-02T04:46:35.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.008+0000 7f9491ffb700 1 -- 172.21.15.70:0/3357989617 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9488015050 con 0x7f94940fef60 2024-04-02T04:46:35.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.008+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949418f420 con 0x7f94940fef60 2024-04-02T04:46:35.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.009+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9494106020 con 0x7f94940fef60 2024-04-02T04:46:35.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.011+0000 7f9491ffb700 1 -- 172.21.15.70:0/3357989617 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f9488004000 con 0x7f94940fef60 2024-04-02T04:46:35.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.011+0000 7f9491ffb700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f948406c4e0 0x7f948406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:35.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.011+0000 7f9491ffb700 1 -- 172.21.15.70:0/3357989617 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f9488093730 con 0x7f94940fef60 2024-04-02T04:46:35.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.011+0000 7f9493fff700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f948406c4e0 0x7f948406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:35.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.012+0000 7f9493fff700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f948406c4e0 0x7f948406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9480006010 tx=0x7f948000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:35.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.020+0000 7f9491ffb700 1 -- 172.21.15.70:0/3357989617 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f94880619a0 con 0x7f94940fef60 2024-04-02T04:46:35.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.235+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94940611d0 con 0x7f948406c4e0 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.237+0000 7f9491ffb700 1 -- 172.21.15.70:0/3357989617 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f94940611d0 con 0x7f948406c4e0 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "0/2 daemons upgraded", 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae image on host smithi179", 2024-04-02T04:46:35.241 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:46:35.242 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:46:35.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f948406c4e0 msgr2=0x7f948406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:35.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f948406c4e0 0x7f948406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9480006010 tx=0x7f948000b540 comp rx=0 tx=0).stop 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940fef60 msgr2=0x7f94941986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940fef60 0x7f94941986c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f948800bf10 tx=0x7f9488009520 comp rx=0 tx=0).stop 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 shutdown_connections 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f948406c4e0 0x7f948406e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940fef60 0x7f94941986c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 --2- 172.21.15.70:0/3357989617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9494101f40 0x7f9494198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.239+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 >> 172.21.15.70:0/3357989617 conn(0x7f94940fa8c0 msgr2=0x7f9494103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.240+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 shutdown_connections 2024-04-02T04:46:35.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:46:35.240+0000 7f949ad82700 1 -- 172.21.15.70:0/3357989617 wait complete. 2024-04-02T04:46:35.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:35 smithi070 ceph-mon[29212]: Upgrade: Target is version 19.0.0-1979-g6cec793d (unknown) 2024-04-02T04:46:35.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:35 smithi070 ceph-mon[29212]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae'] 2024-04-02T04:46:35.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:35 smithi070 ceph-mon[29212]: Upgrade: Need to upgrade myself (mgr.smithi070.igidvr) 2024-04-02T04:46:35.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:35 smithi070 ceph-mon[29212]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:35.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:35 smithi070 ceph-mon[29212]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:35.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:35 smithi070 ceph-mon[29212]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae on smithi179 2024-04-02T04:46:35.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:35 smithi070 ceph-mon[29212]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:35.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:35 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/42995329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:35.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:35 smithi179 ceph-mon[36603]: Upgrade: Target is version 19.0.0-1979-g6cec793d (unknown) 2024-04-02T04:46:35.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:35 smithi179 ceph-mon[36603]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae'] 2024-04-02T04:46:35.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:35 smithi179 ceph-mon[36603]: Upgrade: Need to upgrade myself (mgr.smithi070.igidvr) 2024-04-02T04:46:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:35 smithi179 ceph-mon[36603]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:35 smithi179 ceph-mon[36603]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:35 smithi179 ceph-mon[36603]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae on smithi179 2024-04-02T04:46:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:35 smithi179 ceph-mon[36603]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:35 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/42995329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:36.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:36 smithi070 ceph-mon[29212]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:36.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:36 smithi179 ceph-mon[36603]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:37.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:37 smithi070 ceph-mon[29212]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:37.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:37 smithi179 ceph-mon[36603]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:39.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:39 smithi070 ceph-mon[29212]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:39.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:39 smithi179 ceph-mon[36603]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:41.040 INFO:tasks.workunit.client.1.smithi179.stderr:Updating files: 91% (11414/12435) Updating files: 92% (11441/12435) Updating files: 93% (11565/12435) Updating files: 94% (11689/12435) Updating files: 95% (11814/12435) Updating files: 96% (11938/12435) Updating files: 97% (12062/12435) Updating files: 98% (12187/12435) Updating files: 99% (12311/12435) Updating files: 100% (12435/12435) Updating files: 100% (12435/12435), done. 2024-04-02T04:46:41.250 INFO:tasks.workunit.client.1.smithi179.stderr:Note: switching to '6cec793d9073f0594e2364b9b687ecaf22424cca'. 2024-04-02T04:46:41.250 INFO:tasks.workunit.client.1.smithi179.stderr: 2024-04-02T04:46:41.250 INFO:tasks.workunit.client.1.smithi179.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T04:46:41.250 INFO:tasks.workunit.client.1.smithi179.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T04:46:41.250 INFO:tasks.workunit.client.1.smithi179.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T04:46:41.250 INFO:tasks.workunit.client.1.smithi179.stderr: 2024-04-02T04:46:41.250 INFO:tasks.workunit.client.1.smithi179.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr: 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr: git switch -c 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr: 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr:Or undo this operation with: 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr: 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr: git switch - 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr: 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr: 2024-04-02T04:46:41.251 INFO:tasks.workunit.client.1.smithi179.stderr:HEAD is now at 6cec793d907 Merge pull request #56601 from zdover23/wip-doc-2024-04-01-backport-56600-to-squid 2024-04-02T04:46:41.254 DEBUG:teuthology.orchestra.run.smithi179:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-02T04:46:41.317 INFO:tasks.workunit.client.1.smithi179.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T04:46:41.318 INFO:tasks.workunit.client.1.smithi179.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-02T04:46:41.319 INFO:tasks.workunit.client.1.smithi179.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T04:46:41.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:41 smithi070 ceph-mon[29212]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:41.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:41 smithi179 ceph-mon[36603]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:42.104 INFO:tasks.workunit.client.1.smithi179.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T04:46:42.175 INFO:tasks.workunit.client.1.smithi179.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T04:46:42.216 INFO:tasks.workunit.client.1.smithi179.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-02T04:46:42.217 INFO:tasks.workunit.client.1.smithi179.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-02T04:46:42.217 INFO:tasks.workunit.client.1.smithi179.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T04:46:42.258 INFO:tasks.workunit.client.1.smithi179.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-02T04:46:42.261 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-02T04:46:42.261 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-02T04:46:42.318 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-02T04:46:42.320 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-02T04:46:42.320 DEBUG:teuthology.orchestra.run.smithi179:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-02T04:46:42.384 INFO:tasks.workunit.client.1.smithi179.stderr:+ mkdir -p fsstress 2024-04-02T04:46:42.386 INFO:tasks.workunit.client.1.smithi179.stderr:+ pushd fsstress 2024-04-02T04:46:42.387 INFO:tasks.workunit.client.1.smithi179.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-02T04:46:42.387 INFO:tasks.workunit.client.1.smithi179.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-02T04:46:42.662 INFO:tasks.workunit.client.0.smithi070.stderr:Note: switching to '6cec793d9073f0594e2364b9b687ecaf22424cca'. 2024-04-02T04:46:42.662 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-04-02T04:46:42.662 INFO:tasks.workunit.client.0.smithi070.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T04:46:42.662 INFO:tasks.workunit.client.0.smithi070.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T04:46:42.662 INFO:tasks.workunit.client.0.smithi070.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T04:46:42.662 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-04-02T04:46:42.662 INFO:tasks.workunit.client.0.smithi070.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T04:46:42.662 INFO:tasks.workunit.client.0.smithi070.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr: git switch -c 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr:Or undo this operation with: 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr: git switch - 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-04-02T04:46:42.663 INFO:tasks.workunit.client.0.smithi070.stderr:HEAD is now at 6cec793d907 Merge pull request #56601 from zdover23/wip-doc-2024-04-01-backport-56600-to-squid 2024-04-02T04:46:42.666 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-02T04:46:42.736 INFO:tasks.workunit.client.0.smithi070.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T04:46:42.738 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T04:46:42.738 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T04:46:43.310 INFO:tasks.workunit.client.1.smithi179.stderr:+ tar xzf ltp-full.tgz 2024-04-02T04:46:43.480 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T04:46:43.549 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T04:46:43.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:43 smithi179 ceph-mon[36603]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:43.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:43 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:43.594 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T04:46:43.595 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T04:46:43.595 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T04:46:43.640 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T04:46:43.643 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T04:46:43.644 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-02T04:46:43.699 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-02T04:46:43.700 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-02T04:46:43.701 DEBUG:teuthology.orchestra.run.smithi070:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-02T04:46:43.765 INFO:tasks.workunit.client.0.smithi070.stderr:+ mkdir -p fsstress 2024-04-02T04:46:43.768 INFO:tasks.workunit.client.0.smithi070.stderr:+ pushd fsstress 2024-04-02T04:46:43.769 INFO:tasks.workunit.client.0.smithi070.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-02T04:46:43.769 INFO:tasks.workunit.client.0.smithi070.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-02T04:46:43.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:43 smithi070 ceph-mon[29212]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:43.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:43 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:44.681 INFO:tasks.workunit.client.0.smithi070.stderr:+ tar xzf ltp-full.tgz 2024-04-02T04:46:45.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:45 smithi179 ceph-mon[36603]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:45 smithi070 ceph-mon[29212]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:46:47.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:46 smithi179 ceph-mon[36603]: pgmap v238: 65 pgs: 65 active+clean; 9.2 MiB data, 182 MiB used, 536 GiB / 536 GiB avail; 755 KiB/s wr, 26 op/s 2024-04-02T04:46:47.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:46 smithi070 ceph-mon[29212]: pgmap v238: 65 pgs: 65 active+clean; 9.2 MiB data, 182 MiB used, 536 GiB / 536 GiB avail; 755 KiB/s wr, 26 op/s 2024-04-02T04:46:49.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:49 smithi179 ceph-mon[36603]: pgmap v239: 65 pgs: 65 active+clean; 28 MiB data, 251 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s wr, 128 op/s 2024-04-02T04:46:49.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:49 smithi070 ceph-mon[29212]: pgmap v239: 65 pgs: 65 active+clean; 28 MiB data, 251 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s wr, 128 op/s 2024-04-02T04:46:51.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:51 smithi070 ceph-mon[29212]: pgmap v240: 65 pgs: 65 active+clean; 34 MiB data, 294 MiB used, 536 GiB / 536 GiB avail; 2.9 MiB/s wr, 214 op/s 2024-04-02T04:46:51.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:51 smithi179 ceph-mon[36603]: pgmap v240: 65 pgs: 65 active+clean; 34 MiB data, 294 MiB used, 536 GiB / 536 GiB avail; 2.9 MiB/s wr, 214 op/s 2024-04-02T04:46:53.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:53 smithi070 ceph-mon[29212]: pgmap v241: 65 pgs: 65 active+clean; 35 MiB data, 314 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s wr, 236 op/s 2024-04-02T04:46:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:53 smithi179 ceph-mon[36603]: pgmap v241: 65 pgs: 65 active+clean; 35 MiB data, 314 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s wr, 236 op/s 2024-04-02T04:46:55.626 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:55 smithi070 ceph-mon[29212]: pgmap v242: 65 pgs: 65 active+clean; 43 MiB data, 357 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s wr, 338 op/s 2024-04-02T04:46:55.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:55 smithi179 ceph-mon[36603]: pgmap v242: 65 pgs: 65 active+clean; 43 MiB data, 357 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s wr, 338 op/s 2024-04-02T04:46:57.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:56 smithi179 ceph-mon[36603]: pgmap v243: 65 pgs: 65 active+clean; 55 MiB data, 411 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s wr, 430 op/s 2024-04-02T04:46:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:56 smithi070 ceph-mon[29212]: pgmap v243: 65 pgs: 65 active+clean; 55 MiB data, 411 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s wr, 430 op/s 2024-04-02T04:46:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:58 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:58.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:58 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:59.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:46:59 smithi070 ceph-mon[29212]: pgmap v244: 65 pgs: 65 active+clean; 67 MiB data, 468 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s wr, 500 op/s 2024-04-02T04:46:59.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:46:59 smithi179 ceph-mon[36603]: pgmap v244: 65 pgs: 65 active+clean; 67 MiB data, 468 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s wr, 500 op/s 2024-04-02T04:47:01.722 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:01 smithi179 ceph-mon[36603]: pgmap v245: 65 pgs: 65 active+clean; 84 MiB data, 498 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 473 op/s 2024-04-02T04:47:01.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:01 smithi070 ceph-mon[29212]: pgmap v245: 65 pgs: 65 active+clean; 84 MiB data, 498 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 473 op/s 2024-04-02T04:47:03.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:03 smithi070 ceph-mon[29212]: pgmap v246: 65 pgs: 65 active+clean; 85 MiB data, 503 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s wr, 405 op/s 2024-04-02T04:47:03.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:03 smithi179 ceph-mon[36603]: pgmap v246: 65 pgs: 65 active+clean; 85 MiB data, 503 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s wr, 405 op/s 2024-04-02T04:47:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.394+0000 7f2c039f8700 1 -- 172.21.15.70:0/2094999005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 msgr2=0x7f2bfc100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.394+0000 7f2c039f8700 1 --2- 172.21.15.70:0/2094999005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 0x7f2bfc100150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009910 tx=0x7f2bec009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.394+0000 7f2c039f8700 1 -- 172.21.15.70:0/2094999005 shutdown_connections 2024-04-02T04:47:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.394+0000 7f2c039f8700 1 --2- 172.21.15.70:0/2094999005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 0x7f2bfc100150 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.394+0000 7f2c039f8700 1 --2- 172.21.15.70:0/2094999005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bfc0fef60 0x7f2bfc0ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.394+0000 7f2c039f8700 1 -- 172.21.15.70:0/2094999005 >> 172.21.15.70:0/2094999005 conn(0x7f2bfc0fa8c0 msgr2=0x7f2bfc0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.395+0000 7f2c039f8700 1 -- 172.21.15.70:0/2094999005 shutdown_connections 2024-04-02T04:47:05.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.395+0000 7f2c039f8700 1 -- 172.21.15.70:0/2094999005 wait complete. 2024-04-02T04:47:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c039f8700 1 Processor -- start 2024-04-02T04:47:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c039f8700 1 -- start start 2024-04-02T04:47:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c039f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bfc0fef60 0x7f2bfc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c039f8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 0x7f2bfc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c039f8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc10e4d0 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c039f8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc10e610 con 0x7f2bfc0fef60 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c00f93700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 0x7f2bfc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c00f93700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 0x7f2bfc10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57590/0 (socket says 172.21.15.70:57590) 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c00f93700 1 -- 172.21.15.70:0/1417529009 learned_addr learned my addr 172.21.15.70:0/1417529009 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c01794700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bfc0fef60 0x7f2bfc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c00f93700 1 -- 172.21.15.70:0/1417529009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bfc0fef60 msgr2=0x7f2bfc10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c00f93700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bfc0fef60 0x7f2bfc10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c00f93700 1 -- 172.21.15.70:0/1417529009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0095f0 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.396+0000 7f2c01794700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bfc0fef60 0x7f2bfc10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.397+0000 7f2c00f93700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 0x7f2bfc10deb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00f690 tx=0x7f2bec00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.397+0000 7f2bf27fc700 1 -- 172.21.15.70:0/1417529009 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bec01d070 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.397+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc1aa990 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.397+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc1aae80 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.398+0000 7f2bf27fc700 1 -- 172.21.15.70:0/1417529009 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2bec005dc0 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.398+0000 7f2bf27fc700 1 -- 172.21.15.70:0/1417529009 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bec022470 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.399+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be0005320 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.399+0000 7f2bf27fc700 1 -- 172.21.15.70:0/1417529009 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f2bec0225d0 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.400+0000 7f2bf27fc700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2be806c530 0x7f2be806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:05.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.400+0000 7f2bf27fc700 1 -- 172.21.15.70:0/1417529009 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2bec08cc90 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.400+0000 7f2c01794700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2be806c530 0x7f2be806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:05.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.401+0000 7f2c01794700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2be806c530 0x7f2be806e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8011490 tx=0x7f2bf8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:05.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.406+0000 7f2bf27fc700 1 -- 172.21.15.70:0/1417529009 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2bec05aec0 con 0x7f2bfc0ffcd0 2024-04-02T04:47:05.708 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:05 smithi070 ceph-mon[29212]: pgmap v247: 65 pgs: 65 active+clean; 93 MiB data, 506 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s wr, 494 op/s 2024-04-02T04:47:05.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.705+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2be0000bf0 con 0x7f2be806c530 2024-04-02T04:47:05.709 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.706+0000 7f2bf27fc700 1 -- 172.21.15.70:0/1417529009 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f2be0000bf0 con 0x7f2be806c530 2024-04-02T04:47:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2be806c530 msgr2=0x7f2be806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2be806c530 0x7f2be806e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8011490 tx=0x7f2bf8009260 comp rx=0 tx=0).stop 2024-04-02T04:47:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 msgr2=0x7f2bfc10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 0x7f2bfc10deb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00f690 tx=0x7f2bec00f6c0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 shutdown_connections 2024-04-02T04:47:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f2be806c530 0x7f2be806e9f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bfc0fef60 0x7f2bfc10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 --2- 172.21.15.70:0/1417529009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bfc0ffcd0 0x7f2bfc10deb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 >> 172.21.15.70:0/1417529009 conn(0x7f2bfc0fa8c0 msgr2=0x7f2bfc102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 shutdown_connections 2024-04-02T04:47:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.709+0000 7f2c039f8700 1 -- 172.21.15.70:0/1417529009 wait complete. 2024-04-02T04:47:05.726 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:47:05.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:05 smithi179 ceph-mon[36603]: pgmap v247: 65 pgs: 65 active+clean; 93 MiB data, 506 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s wr, 494 op/s 2024-04-02T04:47:05.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.854+0000 7f6d3ee10700 1 -- 172.21.15.70:0/2515587158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 msgr2=0x7f6d38100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:05.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.854+0000 7f6d3ee10700 1 --2- 172.21.15.70:0/2515587158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 0x7f6d38100030 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009910 tx=0x7f6d2c009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:05.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.854+0000 7f6d3ee10700 1 -- 172.21.15.70:0/2515587158 shutdown_connections 2024-04-02T04:47:05.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.854+0000 7f6d3ee10700 1 --2- 172.21.15.70:0/2515587158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d38100e10 0x7f6d38101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.854+0000 7f6d3ee10700 1 --2- 172.21.15.70:0/2515587158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 0x7f6d38100030 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.854+0000 7f6d3ee10700 1 -- 172.21.15.70:0/2515587158 >> 172.21.15.70:0/2515587158 conn(0x7f6d380fb190 msgr2=0x7f6d380fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:05.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.854+0000 7f6d3ee10700 1 -- 172.21.15.70:0/2515587158 shutdown_connections 2024-04-02T04:47:05.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.854+0000 7f6d3ee10700 1 -- 172.21.15.70:0/2515587158 wait complete. 2024-04-02T04:47:05.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.855+0000 7f6d3ee10700 1 Processor -- start 2024-04-02T04:47:05.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.855+0000 7f6d3ee10700 1 -- start start 2024-04-02T04:47:05.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.855+0000 7f6d3ee10700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 0x7f6d3819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:05.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.855+0000 7f6d3ee10700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d38100e10 0x7f6d3819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:05.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.855+0000 7f6d3ee10700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3819d630 con 0x7f6d380ffc10 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.855+0000 7f6d3ee10700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3819d770 con 0x7f6d38100e10 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.855+0000 7f6d3cbac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 0x7f6d3819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d3cbac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 0x7f6d3819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57612/0 (socket says 172.21.15.70:57612) 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d3cbac700 1 -- 172.21.15.70:0/3093636218 learned_addr learned my addr 172.21.15.70:0/3093636218 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d37fff700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d38100e10 0x7f6d3819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d3cbac700 1 -- 172.21.15.70:0/3093636218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d38100e10 msgr2=0x7f6d3819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d3cbac700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d38100e10 0x7f6d3819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d3cbac700 1 -- 172.21.15.70:0/3093636218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c0095f0 con 0x7f6d380ffc10 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d3cbac700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 0x7f6d3819cad0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c000c00 tx=0x7f6d2c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d35ffb700 1 -- 172.21.15.70:0/3093636218 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d2c01d070 con 0x7f6d380ffc10 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d381a21c0 con 0x7f6d380ffc10 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d35ffb700 1 -- 172.21.15.70:0/3093636218 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6d2c005dc0 con 0x7f6d380ffc10 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.856+0000 7f6d35ffb700 1 -- 172.21.15.70:0/3093636218 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d2c022470 con 0x7f6d380ffc10 2024-04-02T04:47:05.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.857+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d381a26b0 con 0x7f6d380ffc10 2024-04-02T04:47:05.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.857+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d381075f0 con 0x7f6d380ffc10 2024-04-02T04:47:05.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.858+0000 7f6d35ffb700 1 -- 172.21.15.70:0/3093636218 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f6d2c003810 con 0x7f6d380ffc10 2024-04-02T04:47:05.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.859+0000 7f6d35ffb700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f6d2806c4e0 0x7f6d2806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:05.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.859+0000 7f6d35ffb700 1 -- 172.21.15.70:0/3093636218 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6d2c08ccb0 con 0x7f6d380ffc10 2024-04-02T04:47:05.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.859+0000 7f6d37fff700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f6d2806c4e0 0x7f6d2806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:05.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.860+0000 7f6d37fff700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f6d2806c4e0 0x7f6d2806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009c60 tx=0x7f6d24009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:05.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:05.864+0000 7f6d35ffb700 1 -- 172.21.15.70:0/3093636218 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6d2c05af20 con 0x7f6d380ffc10 2024-04-02T04:47:06.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.179+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d381a2db0 con 0x7f6d2806c4e0 2024-04-02T04:47:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.181+0000 7f6d35ffb700 1 -- 172.21.15.70:0/3093636218 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6d381a2db0 con 0x7f6d2806c4e0 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.183+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f6d2806c4e0 msgr2=0x7f6d2806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.183+0000 7f6d3ee10700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f6d2806c4e0 0x7f6d2806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009c60 tx=0x7f6d24009260 comp rx=0 tx=0).stop 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.183+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 msgr2=0x7f6d3819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.183+0000 7f6d3ee10700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 0x7f6d3819cad0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c000c00 tx=0x7f6d2c00f690 comp rx=0 tx=0).stop 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.183+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 shutdown_connections 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.184+0000 7f6d3ee10700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f6d2806c4e0 0x7f6d2806e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.184+0000 7f6d3ee10700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d380ffc10 0x7f6d3819cad0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.184+0000 7f6d3ee10700 1 --2- 172.21.15.70:0/3093636218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d38100e10 0x7f6d3819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.184+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 >> 172.21.15.70:0/3093636218 conn(0x7f6d380fb190 msgr2=0x7f6d38104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.184+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 shutdown_connections 2024-04-02T04:47:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.184+0000 7f6d3ee10700 1 -- 172.21.15.70:0/3093636218 wait complete. 2024-04-02T04:47:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.331+0000 7f4961c5b700 1 -- 172.21.15.70:0/1640366309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c100cb0 msgr2=0x7f495c1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.331+0000 7f4961c5b700 1 --2- 172.21.15.70:0/1640366309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c100cb0 0x7f495c1010d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f494c009910 tx=0x7f494c009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.331+0000 7f4961c5b700 1 -- 172.21.15.70:0/1640366309 shutdown_connections 2024-04-02T04:47:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.331+0000 7f4961c5b700 1 --2- 172.21.15.70:0/1640366309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f495c101eb0 0x7f495c102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.331+0000 7f4961c5b700 1 --2- 172.21.15.70:0/1640366309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c100cb0 0x7f495c1010d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.331+0000 7f4961c5b700 1 -- 172.21.15.70:0/1640366309 >> 172.21.15.70:0/1640366309 conn(0x7f495c0fc230 msgr2=0x7f495c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:06.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.332+0000 7f4961c5b700 1 -- 172.21.15.70:0/1640366309 shutdown_connections 2024-04-02T04:47:06.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.332+0000 7f4961c5b700 1 -- 172.21.15.70:0/1640366309 wait complete. 2024-04-02T04:47:06.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.332+0000 7f4961c5b700 1 Processor -- start 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.332+0000 7f4961c5b700 1 -- start start 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.333+0000 7f4961c5b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f495c100cb0 0x7f495c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.333+0000 7f4961c5b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c101eb0 0x7f495c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.333+0000 7f4961c5b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c1991c0 con 0x7f495c101eb0 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.333+0000 7f4961c5b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c199300 con 0x7f495c100cb0 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.333+0000 7f495b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f495c100cb0 0x7f495c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.333+0000 7f495affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c101eb0 0x7f495c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.333+0000 7f495affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c101eb0 0x7f495c198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57626/0 (socket says 172.21.15.70:57626) 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.333+0000 7f495affd700 1 -- 172.21.15.70:0/3975739391 learned_addr learned my addr 172.21.15.70:0/3975739391 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.334+0000 7f495affd700 1 -- 172.21.15.70:0/3975739391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f495c100cb0 msgr2=0x7f495c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:06.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.334+0000 7f495affd700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f495c100cb0 0x7f495c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.334+0000 7f495affd700 1 -- 172.21.15.70:0/3975739391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c0095f0 con 0x7f495c101eb0 2024-04-02T04:47:06.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.334+0000 7f495b7fe700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f495c100cb0 0x7f495c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:47:06.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.334+0000 7f495affd700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c101eb0 0x7f495c198ba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f495000e920 tx=0x7f495000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:06.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.335+0000 7f4958ff9700 1 -- 172.21.15.70:0/3975739391 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495000c9d0 con 0x7f495c101eb0 2024-04-02T04:47:06.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.335+0000 7f4958ff9700 1 -- 172.21.15.70:0/3975739391 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4950010430 con 0x7f495c101eb0 2024-04-02T04:47:06.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.335+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495c19ddb0 con 0x7f495c101eb0 2024-04-02T04:47:06.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.335+0000 7f4958ff9700 1 -- 172.21.15.70:0/3975739391 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4950018650 con 0x7f495c101eb0 2024-04-02T04:47:06.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.335+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c19e2d0 con 0x7f495c101eb0 2024-04-02T04:47:06.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.336+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c10ba10 con 0x7f495c101eb0 2024-04-02T04:47:06.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.337+0000 7f4958ff9700 1 -- 172.21.15.70:0/3975739391 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f49500105a0 con 0x7f495c101eb0 2024-04-02T04:47:06.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.338+0000 7f4958ff9700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f494406c600 0x7f494406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:06.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.338+0000 7f4958ff9700 1 -- 172.21.15.70:0/3975739391 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4950014070 con 0x7f495c101eb0 2024-04-02T04:47:06.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.338+0000 7f495b7fe700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f494406c600 0x7f494406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:06.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.339+0000 7f495b7fe700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f494406c600 0x7f494406eac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f494c00b6f0 tx=0x7f494c009f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:06.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.344+0000 7f4958ff9700 1 -- 172.21.15.70:0/3975739391 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f495005a190 con 0x7f495c101eb0 2024-04-02T04:47:06.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.664+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f495c106800 con 0x7f494406c600 2024-04-02T04:47:06.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.681+0000 7f4958ff9700 1 -- 172.21.15.70:0/3975739391 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3456 (secure 0 0 0) 0x7f495c106800 con 0x7f494406c600 2024-04-02T04:47:06.684 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:47:06.684 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (6m) 103s ago 7m 24.5M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (7m) 103s ago 7m 7457k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (6m) 109s ago 6m 7184k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (7m) 103s ago 7m 6665k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (6m) 109s ago 6m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (6m) 103s ago 7m 85.6M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (2m) 103s ago 2m 16.5M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (2m) 103s ago 2m 14.6M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (2m) 109s ago 2m 13.0M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (2m) 109s ago 2m 14.7M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:9283,8765,8443 running (9m) 103s ago 9m 489M - 18.2.1 5be31c24972a 3299b9e265dd 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (6m) 109s ago 6m 441M - 18.2.1 5be31c24972a c82df176868c 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (9m) 103s ago 9m 44.4M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (6m) 109s ago 6m 40.6M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:47:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 103s ago 7m 14.3M - 1.5.0 0da6a335fe13 bc06c0b59df5 2024-04-02T04:47:06.686 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (6m) 109s ago 6m 17.0M - 1.5.0 0da6a335fe13 6fe44da459b3 2024-04-02T04:47:06.686 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (5m) 103s ago 5m 47.9M 3946M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:47:06.686 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (5m) 103s ago 5m 48.2M 3946M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:47:06.686 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (4m) 103s ago 4m 46.5M 3946M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:47:06.686 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (4m) 109s ago 4m 48.5M 4970M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:47:06.686 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (4m) 109s ago 4m 43.5M 4970M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:47:06.686 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (3m) 109s ago 3m 42.6M 4970M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:47:06.686 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (5m) 103s ago 7m 56.3M - 2.43.0 a07b618ecd1d b1f1f05601f5 2024-04-02T04:47:06.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f494406c600 msgr2=0x7f494406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:06.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f494406c600 0x7f494406eac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f494c00b6f0 tx=0x7f494c009f90 comp rx=0 tx=0).stop 2024-04-02T04:47:06.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c101eb0 msgr2=0x7f495c198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:06.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c101eb0 0x7f495c198ba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f495000e920 tx=0x7f495000ec30 comp rx=0 tx=0).stop 2024-04-02T04:47:06.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 shutdown_connections 2024-04-02T04:47:06.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f494406c600 0x7f494406eac0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f495c100cb0 0x7f495c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 --2- 172.21.15.70:0/3975739391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f495c101eb0 0x7f495c198ba0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 >> 172.21.15.70:0/3975739391 conn(0x7f495c0fc230 msgr2=0x7f495c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:06.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 shutdown_connections 2024-04-02T04:47:06.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.684+0000 7f4961c5b700 1 -- 172.21.15.70:0/3975739391 wait complete. 2024-04-02T04:47:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.833+0000 7f498895d700 1 -- 172.21.15.70:0/3700242849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984100cb0 msgr2=0x7f49841010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.833+0000 7f498895d700 1 --2- 172.21.15.70:0/3700242849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984100cb0 0x7f49841010d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4974009910 tx=0x7f4974009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.833+0000 7f498895d700 1 -- 172.21.15.70:0/3700242849 shutdown_connections 2024-04-02T04:47:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.833+0000 7f498895d700 1 --2- 172.21.15.70:0/3700242849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4984101eb0 0x7f4984102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.833+0000 7f498895d700 1 --2- 172.21.15.70:0/3700242849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984100cb0 0x7f49841010d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.833+0000 7f498895d700 1 -- 172.21.15.70:0/3700242849 >> 172.21.15.70:0/3700242849 conn(0x7f49840fc230 msgr2=0x7f49840fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.833+0000 7f498895d700 1 -- 172.21.15.70:0/3700242849 shutdown_connections 2024-04-02T04:47:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.833+0000 7f498895d700 1 -- 172.21.15.70:0/3700242849 wait complete. 2024-04-02T04:47:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.834+0000 7f498895d700 1 Processor -- start 2024-04-02T04:47:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.834+0000 7f498895d700 1 -- start start 2024-04-02T04:47:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.834+0000 7f498895d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4984100cb0 0x7f49841964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.834+0000 7f498895d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984101eb0 0x7f4984196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f4981d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984101eb0 0x7f4984196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f4981d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984101eb0 0x7f4984196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37066/0 (socket says 172.21.15.70:37066) 2024-04-02T04:47:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f4981d9b700 1 -- 172.21.15.70:0/2305689004 learned_addr learned my addr 172.21.15.70:0/2305689004 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4984197050 con 0x7f4984101eb0 2024-04-02T04:47:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4984197190 con 0x7f4984100cb0 2024-04-02T04:47:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f498259c700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4984100cb0 0x7f49841964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f4981d9b700 1 -- 172.21.15.70:0/2305689004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4984100cb0 msgr2=0x7f49841964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f4981d9b700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4984100cb0 0x7f49841964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f4981d9b700 1 -- 172.21.15.70:0/2305689004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740095f0 con 0x7f4984101eb0 2024-04-02T04:47:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.835+0000 7f4981d9b700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984101eb0 0x7f4984196a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f497800e920 tx=0x7f497800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.836+0000 7f49737fe700 1 -- 172.21.15.70:0/2305689004 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f497800c9d0 con 0x7f4984101eb0 2024-04-02T04:47:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.836+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498419bc40 con 0x7f4984101eb0 2024-04-02T04:47:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.836+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498419c190 con 0x7f4984101eb0 2024-04-02T04:47:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.836+0000 7f49737fe700 1 -- 172.21.15.70:0/2305689004 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4978010430 con 0x7f4984101eb0 2024-04-02T04:47:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.836+0000 7f49737fe700 1 -- 172.21.15.70:0/2305689004 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4978018650 con 0x7f4984101eb0 2024-04-02T04:47:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.837+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4964005320 con 0x7f4984101eb0 2024-04-02T04:47:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.837+0000 7f49737fe700 1 -- 172.21.15.70:0/2305689004 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f497800cb30 con 0x7f4984101eb0 2024-04-02T04:47:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.838+0000 7f49737fe700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f496c0708f0 0x7f496c072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.838+0000 7f49737fe700 1 -- 172.21.15.70:0/2305689004 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4978014070 con 0x7f4984101eb0 2024-04-02T04:47:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.838+0000 7f498259c700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f496c0708f0 0x7f496c072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.838+0000 7f498259c700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f496c0708f0 0x7f496c072db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4974009fd0 tx=0x7f4974005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:06.845+0000 7f49737fe700 1 -- 172.21.15.70:0/2305689004 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f49780520e0 con 0x7f4984101eb0 2024-04-02T04:47:07.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:06 smithi070 ceph-mon[29212]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:07.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:06 smithi070 ceph-mon[29212]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:07.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:06 smithi070 ceph-mon[29212]: pgmap v248: 65 pgs: 65 active+clean; 104 MiB data, 552 MiB used, 536 GiB / 536 GiB avail; 5.4 MiB/s wr, 488 op/s 2024-04-02T04:47:07.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:06 smithi179 ceph-mon[36603]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:07.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:06 smithi179 ceph-mon[36603]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:07.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:06 smithi179 ceph-mon[36603]: pgmap v248: 65 pgs: 65 active+clean; 104 MiB data, 552 MiB used, 536 GiB / 536 GiB avail; 5.4 MiB/s wr, 488 op/s 2024-04-02T04:47:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.284+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4964006200 con 0x7f4984101eb0 2024-04-02T04:47:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.284+0000 7f49737fe700 1 -- 172.21.15.70:0/2305689004 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f4978059c50 con 0x7f4984101eb0 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:47:07.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:47:07.289 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:47:07.289 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-04-02T04:47:07.289 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:47:07.289 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:47:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.287+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f496c0708f0 msgr2=0x7f496c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.287+0000 7f498895d700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f496c0708f0 0x7f496c072db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4974009fd0 tx=0x7f4974005fd0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.287+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984101eb0 msgr2=0x7f4984196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.287+0000 7f498895d700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984101eb0 0x7f4984196a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f497800e920 tx=0x7f497800ec30 comp rx=0 tx=0).stop 2024-04-02T04:47:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.288+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 shutdown_connections 2024-04-02T04:47:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.288+0000 7f498895d700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f496c0708f0 0x7f496c072db0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.288+0000 7f498895d700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4984100cb0 0x7f49841964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.288+0000 7f498895d700 1 --2- 172.21.15.70:0/2305689004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4984101eb0 0x7f4984196a30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.288+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 >> 172.21.15.70:0/2305689004 conn(0x7f49840fc230 msgr2=0x7f49841050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.288+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 shutdown_connections 2024-04-02T04:47:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.288+0000 7f498895d700 1 -- 172.21.15.70:0/2305689004 wait complete. 2024-04-02T04:47:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.416+0000 7f0fbc426700 1 -- 172.21.15.70:0/3952091490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 msgr2=0x7f0fb4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.416+0000 7f0fbc426700 1 --2- 172.21.15.70:0/3952091490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 0x7f0fb4100030 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009910 tx=0x7f0fb0009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.416+0000 7f0fbc426700 1 -- 172.21.15.70:0/3952091490 shutdown_connections 2024-04-02T04:47:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.416+0000 7f0fbc426700 1 --2- 172.21.15.70:0/3952091490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4100e10 0x7f0fb4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.416+0000 7f0fbc426700 1 --2- 172.21.15.70:0/3952091490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 0x7f0fb4100030 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.416+0000 7f0fbc426700 1 -- 172.21.15.70:0/3952091490 >> 172.21.15.70:0/3952091490 conn(0x7f0fb40fb190 msgr2=0x7f0fb40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.417+0000 7f0fbc426700 1 -- 172.21.15.70:0/3952091490 shutdown_connections 2024-04-02T04:47:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.417+0000 7f0fbc426700 1 -- 172.21.15.70:0/3952091490 wait complete. 2024-04-02T04:47:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.417+0000 7f0fbc426700 1 Processor -- start 2024-04-02T04:47:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.417+0000 7f0fbc426700 1 -- start start 2024-04-02T04:47:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.417+0000 7f0fbc426700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 0x7f0fb4103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.417+0000 7f0fbc426700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4100e10 0x7f0fb41021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fbc426700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4102730 con 0x7f0fb40ffc10 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fbc426700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4102870 con 0x7f0fb4100e10 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fb99c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4100e10 0x7f0fb41021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fba1c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 0x7f0fb4103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fb99c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4100e10 0x7f0fb41021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:35520/0 (socket says 172.21.15.70:35520) 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fb99c1700 1 -- 172.21.15.70:0/2291847010 learned_addr learned my addr 172.21.15.70:0/2291847010 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fba1c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 0x7f0fb4103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37096/0 (socket says 172.21.15.70:37096) 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fb99c1700 1 -- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 msgr2=0x7f0fb4103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fb99c1700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 0x7f0fb4103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fb99c1700 1 -- 172.21.15.70:0/2291847010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00095f0 con 0x7f0fb4100e10 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fba1c2700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 0x7f0fb4103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:47:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fb99c1700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4100e10 0x7f0fb41021f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400e920 tx=0x7f0fa400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.418+0000 7f0fab7fe700 1 -- 172.21.15.70:0/2291847010 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa400c9d0 con 0x7f0fb4100e10 2024-04-02T04:47:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.419+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb4102b50 con 0x7f0fb4100e10 2024-04-02T04:47:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.419+0000 7f0fab7fe700 1 -- 172.21.15.70:0/2291847010 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0fa4010430 con 0x7f0fb4100e10 2024-04-02T04:47:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.419+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb419e550 con 0x7f0fb4100e10 2024-04-02T04:47:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.419+0000 7f0fab7fe700 1 -- 172.21.15.70:0/2291847010 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa4018650 con 0x7f0fb4100e10 2024-04-02T04:47:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.420+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f98005320 con 0x7f0fb4100e10 2024-04-02T04:47:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.420+0000 7f0fab7fe700 1 -- 172.21.15.70:0/2291847010 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0fa400cb30 con 0x7f0fb4100e10 2024-04-02T04:47:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.420+0000 7f0fab7fe700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0fa006c4e0 0x7f0fa006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.421+0000 7f0fab7fe700 1 -- 172.21.15.70:0/2291847010 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0fa4014070 con 0x7f0fb4100e10 2024-04-02T04:47:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.421+0000 7f0fba1c2700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0fa006c4e0 0x7f0fa006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.421+0000 7f0fba1c2700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0fa006c4e0 0x7f0fa006e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009fd0 tx=0x7f0fb0005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.425+0000 7f0fab7fe700 1 -- 172.21.15.70:0/2291847010 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0fa40520e0 con 0x7f0fb4100e10 2024-04-02T04:47:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.719+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 --> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f98000bf0 con 0x7f0fa006c4e0 2024-04-02T04:47:07.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.720+0000 7f0fab7fe700 1 -- 172.21.15.70:0/2291847010 <== mgr.14213 v2:172.21.15.70:6800/2814127860 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0f98000bf0 con 0x7f0fa006c4e0 2024-04-02T04:47:07.723 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:47:07.723 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:47:07.723 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:47:07.723 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:47:07.723 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-04-02T04:47:07.723 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "0/2 daemons upgraded", 2024-04-02T04:47:07.724 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae image on host smithi179", 2024-04-02T04:47:07.724 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:47:07.724 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:47:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0fa006c4e0 msgr2=0x7f0fa006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0fa006c4e0 0x7f0fa006e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009fd0 tx=0x7f0fb0005a90 comp rx=0 tx=0).stop 2024-04-02T04:47:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4100e10 msgr2=0x7f0fb41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4100e10 0x7f0fb41021f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400e920 tx=0x7f0fa400ec30 comp rx=0 tx=0).stop 2024-04-02T04:47:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 shutdown_connections 2024-04-02T04:47:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:6800/2814127860,v1:172.21.15.70:6801/2814127860] conn(0x7f0fa006c4e0 0x7f0fa006e9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fb40ffc10 0x7f0fb4103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 --2- 172.21.15.70:0/2291847010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4100e10 0x7f0fb41021f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 >> 172.21.15.70:0/2291847010 conn(0x7f0fb40fb190 msgr2=0x7f0fb4068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 shutdown_connections 2024-04-02T04:47:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:07.723+0000 7f0fbc426700 1 -- 172.21.15.70:0/2291847010 wait complete. 2024-04-02T04:47:08.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:07 smithi070 ceph-mon[29212]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:08.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:07 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2305689004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:08.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:07 smithi179 ceph-mon[36603]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:08.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:07 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2305689004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:09.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:08 smithi179 ceph-mon[36603]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:09.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:08 smithi179 ceph-mon[36603]: pgmap v249: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s wr, 486 op/s 2024-04-02T04:47:09.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:08 smithi070 ceph-mon[29212]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:09.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:08 smithi070 ceph-mon[29212]: pgmap v249: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s wr, 486 op/s 2024-04-02T04:47:10.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:10 smithi179 ceph-mon[36603]: Upgrade: Updating mgr.smithi179.labmvs 2024-04-02T04:47:10.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:10.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:47:10.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:47:10.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:10 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:10.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:10 smithi179 ceph-mon[36603]: Deploying daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:47:10.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:10 smithi179 ceph-mon[36603]: pgmap v250: 65 pgs: 65 active+clean; 114 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 4.2 MiB/s wr, 454 op/s 2024-04-02T04:47:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:10 smithi070 ceph-mon[29212]: Upgrade: Updating mgr.smithi179.labmvs 2024-04-02T04:47:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:47:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:47:11.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:10 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:11.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:10 smithi070 ceph-mon[29212]: Deploying daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:47:11.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:10 smithi070 ceph-mon[29212]: pgmap v250: 65 pgs: 65 active+clean; 114 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 4.2 MiB/s wr, 454 op/s 2024-04-02T04:47:13.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:13 smithi179 ceph-mon[36603]: pgmap v251: 65 pgs: 65 active+clean; 115 MiB data, 627 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s wr, 394 op/s 2024-04-02T04:47:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:13 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:13.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:13 smithi070 ceph-mon[29212]: pgmap v251: 65 pgs: 65 active+clean; 115 MiB data, 627 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s wr, 394 op/s 2024-04-02T04:47:13.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:13 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:14.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:14 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:14.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:14 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:14.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:14 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:15.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:14 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:15.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:14 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:15.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:14 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:15.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:15 smithi179 ceph-mon[36603]: pgmap v252: 65 pgs: 65 active+clean; 119 MiB data, 664 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s wr, 474 op/s 2024-04-02T04:47:15.889 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:15 smithi070 ceph-mon[29212]: pgmap v252: 65 pgs: 65 active+clean; 119 MiB data, 664 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s wr, 474 op/s 2024-04-02T04:47:17.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:16 smithi179 ceph-mon[36603]: pgmap v253: 65 pgs: 65 active+clean; 131 MiB data, 688 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s wr, 451 op/s 2024-04-02T04:47:17.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:16 smithi070 ceph-mon[29212]: pgmap v253: 65 pgs: 65 active+clean; 131 MiB data, 688 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s wr, 451 op/s 2024-04-02T04:47:19.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:19 smithi179 ceph-mon[36603]: pgmap v254: 65 pgs: 65 active+clean; 144 MiB data, 727 MiB used, 536 GiB / 536 GiB avail; 3.6 MiB/s wr, 454 op/s 2024-04-02T04:47:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:19 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:19.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:19 smithi070 ceph-mon[29212]: pgmap v254: 65 pgs: 65 active+clean; 144 MiB data, 727 MiB used, 536 GiB / 536 GiB avail; 3.6 MiB/s wr, 454 op/s 2024-04-02T04:47:19.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:19.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:19.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:19 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:20.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:20 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:20.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:20 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:21.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:21 smithi179 ceph-mon[36603]: pgmap v255: 65 pgs: 65 active+clean; 155 MiB data, 771 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s wr, 429 op/s 2024-04-02T04:47:21.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:21 smithi070 ceph-mon[29212]: pgmap v255: 65 pgs: 65 active+clean; 155 MiB data, 771 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s wr, 429 op/s 2024-04-02T04:47:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:22 smithi179 ceph-mon[36603]: Standby manager daemon smithi179.labmvs restarted 2024-04-02T04:47:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:22 smithi179 ceph-mon[36603]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:47:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:22 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:47:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:22 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:47:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:22 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:47:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:22 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:47:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:22 smithi070 ceph-mon[29212]: Standby manager daemon smithi179.labmvs restarted 2024-04-02T04:47:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:22 smithi070 ceph-mon[29212]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:47:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:22 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:47:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:22 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:47:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:22 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:47:22.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:22 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:47:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:23 smithi179 ceph-mon[36603]: pgmap v256: 65 pgs: 65 active+clean; 155 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 3.6 MiB/s wr, 381 op/s 2024-04-02T04:47:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:23 smithi179 ceph-mon[36603]: mgrmap e20: smithi070.igidvr(active, since 7m), standbys: smithi179.labmvs 2024-04-02T04:47:23.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:23 smithi070 ceph-mon[29212]: pgmap v256: 65 pgs: 65 active+clean; 155 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 3.6 MiB/s wr, 381 op/s 2024-04-02T04:47:23.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:23 smithi070 ceph-mon[29212]: mgrmap e20: smithi070.igidvr(active, since 7m), standbys: smithi179.labmvs 2024-04-02T04:47:25.535 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:25 smithi179 ceph-mon[36603]: pgmap v257: 65 pgs: 65 active+clean; 168 MiB data, 823 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s wr, 454 op/s 2024-04-02T04:47:25.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:25 smithi070 ceph-mon[29212]: pgmap v257: 65 pgs: 65 active+clean; 168 MiB data, 823 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s wr, 454 op/s 2024-04-02T04:47:27.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:26 smithi179 ceph-mon[36603]: pgmap v258: 65 pgs: 65 active+clean; 172 MiB data, 849 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s wr, 431 op/s 2024-04-02T04:47:27.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:26 smithi070 ceph-mon[29212]: pgmap v258: 65 pgs: 65 active+clean; 172 MiB data, 849 MiB used, 536 GiB / 536 GiB avail; 4.7 MiB/s wr, 431 op/s 2024-04-02T04:47:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:47:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr fail", "who": "smithi070.igidvr"}]: dispatch 2024-04-02T04:47:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: osdmap e47: 6 total, 6 up, 6 in 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "mgr fail", "who": "smithi070.igidvr"}]': finished 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: mgrmap e21: smithi179.labmvs(active, starting, since 0.00777615s) 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr metadata", "who": "smithi179.labmvs", "id": "smithi179.labmvs"}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:47:29.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: Manager daemon smithi179.labmvs is now available 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi179.labmvs/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi179.labmvs/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi179.labmvs/trash_purge_schedule"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi179.labmvs/trash_purge_schedule"}]: dispatch 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: Deploying cephadm binary to smithi179 2024-04-02T04:47:29.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:28 smithi070 ceph-mon[29212]: Deploying cephadm binary to smithi070 2024-04-02T04:47:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:47:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' 2024-04-02T04:47:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr fail", "who": "smithi070.igidvr"}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: osdmap e47: 6 total, 6 up, 6 in 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.14213 172.21.15.70:0/1919477535' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "mgr fail", "who": "smithi070.igidvr"}]': finished 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: mgrmap e21: smithi179.labmvs(active, starting, since 0.00777615s) 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr metadata", "who": "smithi179.labmvs", "id": "smithi179.labmvs"}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:47:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: Manager daemon smithi179.labmvs is now available 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi179.labmvs/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi179.labmvs/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi179.labmvs/trash_purge_schedule"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi179.labmvs/trash_purge_schedule"}]: dispatch 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: Deploying cephadm binary to smithi179 2024-04-02T04:47:29.074 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:28 smithi179 ceph-mon[36603]: Deploying cephadm binary to smithi070 2024-04-02T04:47:30.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:29 smithi070 ceph-mon[29212]: mgrmap e22: smithi179.labmvs(active, since 1.00806s) 2024-04-02T04:47:30.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:29 smithi070 ceph-mon[29212]: pgmap v3: 65 pgs: 65 active+clean; 197 MiB data, 917 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:47:30.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:29 smithi070 ceph-mon[29212]: [02/Apr/2024:04:47:29] ENGINE Bus STARTING 2024-04-02T04:47:30.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:29 smithi070 ceph-mon[29212]: [02/Apr/2024:04:47:29] ENGINE Serving on https://172.21.15.179:7150 2024-04-02T04:47:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:29 smithi070 ceph-mon[29212]: [02/Apr/2024:04:47:29] ENGINE Serving on http://172.21.15.179:8765 2024-04-02T04:47:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:29 smithi070 ceph-mon[29212]: [02/Apr/2024:04:47:29] ENGINE Bus STARTED 2024-04-02T04:47:30.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:29 smithi179 ceph-mon[36603]: mgrmap e22: smithi179.labmvs(active, since 1.00806s) 2024-04-02T04:47:30.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:29 smithi179 ceph-mon[36603]: pgmap v3: 65 pgs: 65 active+clean; 197 MiB data, 917 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:47:30.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:29 smithi179 ceph-mon[36603]: [02/Apr/2024:04:47:29] ENGINE Bus STARTING 2024-04-02T04:47:30.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:29 smithi179 ceph-mon[36603]: [02/Apr/2024:04:47:29] ENGINE Serving on https://172.21.15.179:7150 2024-04-02T04:47:30.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:29 smithi179 ceph-mon[36603]: [02/Apr/2024:04:47:29] ENGINE Serving on http://172.21.15.179:8765 2024-04-02T04:47:30.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:29 smithi179 ceph-mon[36603]: [02/Apr/2024:04:47:29] ENGINE Bus STARTED 2024-04-02T04:47:30.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:30 smithi070 ceph-mon[29212]: pgmap v4: 65 pgs: 65 active+clean; 197 MiB data, 917 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:47:31.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:30 smithi179 ceph-mon[36603]: pgmap v4: 65 pgs: 65 active+clean; 197 MiB data, 917 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:47:32.271 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:31 smithi179 ceph-mon[36603]: mgrmap e23: smithi179.labmvs(active, since 3s) 2024-04-02T04:47:32.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:31 smithi070 ceph-mon[29212]: mgrmap e23: smithi179.labmvs(active, since 3s) 2024-04-02T04:47:32.840 INFO:tasks.workunit.client.1.smithi179.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-02T04:47:32.846 INFO:tasks.workunit.client.1.smithi179.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-02T04:47:32.846 INFO:tasks.workunit.client.1.smithi179.stderr:+ make 2024-04-02T04:47:33.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:32 smithi070 ceph-mon[29212]: pgmap v5: 65 pgs: 65 active+clean; 197 MiB data, 917 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:47:33.193 INFO:tasks.workunit.client.1.smithi179.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-02T04:47:33.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:32 smithi179 ceph-mon[36603]: pgmap v5: 65 pgs: 65 active+clean; 197 MiB data, 917 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:47:33.617 INFO:tasks.workunit.client.0.smithi070.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-02T04:47:33.624 INFO:tasks.workunit.client.0.smithi070.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-02T04:47:33.624 INFO:tasks.workunit.client.0.smithi070.stderr:+ make 2024-04-02T04:47:33.720 INFO:tasks.workunit.client.0.smithi070.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-02T04:47:34.450 INFO:tasks.workunit.client.0.smithi070.stderr:++ readlink -f fsstress 2024-04-02T04:47:34.452 INFO:tasks.workunit.client.0.smithi070.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-02T04:47:34.452 INFO:tasks.workunit.client.0.smithi070.stderr:+ popd 2024-04-02T04:47:34.453 INFO:tasks.workunit.client.0.smithi070.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-02T04:47:34.453 INFO:tasks.workunit.client.0.smithi070.stderr:+ popd 2024-04-02T04:47:34.453 INFO:tasks.workunit.client.0.smithi070.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-02T04:47:34.454 INFO:tasks.workunit.client.0.smithi070.stderr:++ mktemp -d -p . 2024-04-02T04:47:34.457 INFO:tasks.workunit.client.0.smithi070.stderr:+ T=./tmp.9CbRg7ThXK 2024-04-02T04:47:34.457 INFO:tasks.workunit.client.0.smithi070.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.9CbRg7ThXK -l 1 -n 1000 -p 10 -v 2024-04-02T04:47:34.460 INFO:tasks.workunit.client.0.smithi070.stdout:seed = 1712157567 2024-04-02T04:47:34.464 INFO:tasks.workunit.client.0.smithi070.stdout:0/0: read - no filename 2024-04-02T04:47:34.464 INFO:tasks.workunit.client.0.smithi070.stdout:0/1: write - no filename 2024-04-02T04:47:34.465 INFO:tasks.workunit.client.0.smithi070.stdout:1/0: chown . 1593 1 2024-04-02T04:47:34.465 INFO:tasks.workunit.client.0.smithi070.stdout:1/1: write - no filename 2024-04-02T04:47:34.465 INFO:tasks.workunit.client.0.smithi070.stdout:1/2: dread - no filename 2024-04-02T04:47:34.466 INFO:tasks.workunit.client.0.smithi070.stdout:0/2: creat f0 x:0 0 0 2024-04-02T04:47:34.467 INFO:tasks.workunit.client.0.smithi070.stdout:2/0: unlink - no file 2024-04-02T04:47:34.467 INFO:tasks.workunit.client.0.smithi070.stdout:2/1: read - no filename 2024-04-02T04:47:34.467 INFO:tasks.workunit.client.0.smithi070.stdout:2/2: write - no filename 2024-04-02T04:47:34.467 INFO:tasks.workunit.client.0.smithi070.stdout:2/3: dread - no filename 2024-04-02T04:47:34.467 INFO:tasks.workunit.client.0.smithi070.stdout:2/4: write - no filename 2024-04-02T04:47:34.467 INFO:tasks.workunit.client.0.smithi070.stdout:2/5: readlink - no filename 2024-04-02T04:47:34.467 INFO:tasks.workunit.client.0.smithi070.stdout:2/6: dwrite - no filename 2024-04-02T04:47:34.467 INFO:tasks.workunit.client.0.smithi070.stdout:2/7: rmdir - no directory 2024-04-02T04:47:34.468 INFO:tasks.workunit.client.0.smithi070.stdout:1/3: mkdir d0 0 2024-04-02T04:47:34.468 INFO:tasks.workunit.client.0.smithi070.stdout:1/4: dwrite - no filename 2024-04-02T04:47:34.468 INFO:tasks.workunit.client.0.smithi070.stdout:1/5: stat d0 0 2024-04-02T04:47:34.468 INFO:tasks.workunit.client.0.smithi070.stdout:1/6: read - no filename 2024-04-02T04:47:34.468 INFO:tasks.workunit.client.0.smithi070.stdout:1/7: dwrite - no filename 2024-04-02T04:47:34.471 INFO:tasks.workunit.client.0.smithi070.stdout:2/8: creat f0 x:0 0 0 2024-04-02T04:47:34.471 INFO:tasks.workunit.client.0.smithi070.stdout:1/8: mkdir d0/d1 0 2024-04-02T04:47:34.472 INFO:tasks.workunit.client.0.smithi070.stdout:2/9: truncate f0 959703 0 2024-04-02T04:47:34.473 INFO:tasks.workunit.client.0.smithi070.stdout:4/0: dwrite - no filename 2024-04-02T04:47:34.474 INFO:tasks.workunit.client.0.smithi070.stdout:4/1: dread - no filename 2024-04-02T04:47:34.474 INFO:tasks.workunit.client.0.smithi070.stdout:4/2: write - no filename 2024-04-02T04:47:34.474 INFO:tasks.workunit.client.0.smithi070.stdout:4/3: rmdir - no directory 2024-04-02T04:47:34.478 INFO:tasks.workunit.client.0.smithi070.stdout:5/0: read - no filename 2024-04-02T04:47:34.479 INFO:tasks.workunit.client.0.smithi070.stdout:5/1: chown . 9454 1 2024-04-02T04:47:34.480 INFO:tasks.workunit.client.0.smithi070.stdout:4/4: mkdir d0 0 2024-04-02T04:47:34.480 INFO:tasks.workunit.client.0.smithi070.stdout:4/5: dread - no filename 2024-04-02T04:47:34.480 INFO:tasks.workunit.client.0.smithi070.stdout:5/2: chown . 207914350 1 2024-04-02T04:47:34.481 INFO:tasks.workunit.client.0.smithi070.stdout:5/3: chown . 12105130 1 2024-04-02T04:47:34.482 INFO:tasks.workunit.client.0.smithi070.stdout:6/0: rename - no filename 2024-04-02T04:47:34.485 INFO:tasks.workunit.client.0.smithi070.stdout:0/3: dwrite f0 [0,4194304] 0 2024-04-02T04:47:34.485 INFO:tasks.workunit.client.0.smithi070.stdout:7/0: dwrite - no filename 2024-04-02T04:47:34.485 INFO:tasks.workunit.client.0.smithi070.stdout:7/1: dread - no filename 2024-04-02T04:47:34.487 INFO:tasks.workunit.client.0.smithi070.stdout:5/4: mknod c0 0 2024-04-02T04:47:34.487 INFO:tasks.workunit.client.0.smithi070.stdout:5/5: fdatasync - no filename 2024-04-02T04:47:34.489 INFO:tasks.workunit.client.0.smithi070.stdout:0/4: write f0 [2051781,127440] 0 2024-04-02T04:47:34.489 INFO:tasks.workunit.client.0.smithi070.stdout:5/6: chown c0 331 1 2024-04-02T04:47:34.489 INFO:tasks.workunit.client.0.smithi070.stdout:5/7: write - no filename 2024-04-02T04:47:34.489 INFO:tasks.workunit.client.0.smithi070.stdout:5/8: dwrite - no filename 2024-04-02T04:47:34.490 INFO:tasks.workunit.client.0.smithi070.stdout:6/1: mkdir d0 0 2024-04-02T04:47:34.491 INFO:tasks.workunit.client.0.smithi070.stdout:2/10: dwrite f0 [0,4194304] 0 2024-04-02T04:47:34.492 INFO:tasks.workunit.client.0.smithi070.stdout:6/2: chown d0 0 1 2024-04-02T04:47:34.493 INFO:tasks.workunit.client.0.smithi070.stdout:6/3: stat d0 0 2024-04-02T04:47:34.493 INFO:tasks.workunit.client.0.smithi070.stdout:6/4: unlink - no file 2024-04-02T04:47:34.493 INFO:tasks.workunit.client.0.smithi070.stdout:6/5: readlink - no filename 2024-04-02T04:47:34.494 INFO:tasks.workunit.client.0.smithi070.stdout:4/6: rmdir d0 0 2024-04-02T04:47:34.494 INFO:tasks.workunit.client.0.smithi070.stdout:4/7: write - no filename 2024-04-02T04:47:34.495 INFO:tasks.workunit.client.0.smithi070.stdout:6/6: rename d0 to d0/d1 22 2024-04-02T04:47:34.500 INFO:tasks.workunit.client.0.smithi070.stdout:6/7: rename d0 to d0/d2 22 2024-04-02T04:47:34.500 INFO:tasks.workunit.client.0.smithi070.stdout:6/8: write - no filename 2024-04-02T04:47:34.503 INFO:tasks.workunit.client.0.smithi070.stdout:0/5: dread f0 [0,4194304] 0 2024-04-02T04:47:34.503 INFO:tasks.workunit.client.0.smithi070.stdout:5/9: symlink l1 0 2024-04-02T04:47:34.504 INFO:tasks.workunit.client.0.smithi070.stdout:5/10: write - no filename 2024-04-02T04:47:34.507 INFO:tasks.workunit.client.0.smithi070.stdout:2/11: dwrite f0 [0,4194304] 0 2024-04-02T04:47:34.512 INFO:tasks.workunit.client.0.smithi070.stdout:4/8: mkdir d1 0 2024-04-02T04:47:34.513 INFO:tasks.workunit.client.0.smithi070.stdout:7/2: mknod c0 0 2024-04-02T04:47:34.514 INFO:tasks.workunit.client.0.smithi070.stdout:7/3: chown c0 853804 1 2024-04-02T04:47:34.514 INFO:tasks.workunit.client.0.smithi070.stdout:7/4: chown c0 8 1 2024-04-02T04:47:34.514 INFO:tasks.workunit.client.0.smithi070.stdout:7/5: rmdir - no directory 2024-04-02T04:47:34.518 INFO:tasks.workunit.client.0.smithi070.stdout:5/11: creat f2 x:0 0 0 2024-04-02T04:47:34.518 INFO:tasks.workunit.client.0.smithi070.stdout:5/12: stat c0 0 2024-04-02T04:47:34.518 INFO:tasks.workunit.client.0.smithi070.stdout:8/0: creat f0 x:0 0 0 2024-04-02T04:47:34.519 INFO:tasks.workunit.client.0.smithi070.stdout:8/1: read - f0 zero size 2024-04-02T04:47:34.520 INFO:tasks.workunit.client.0.smithi070.stdout:5/13: write f2 [934848,123953] 0 2024-04-02T04:47:34.524 INFO:tasks.workunit.client.0.smithi070.stdout:6/9: mkdir d0/d3 0 2024-04-02T04:47:34.527 INFO:tasks.workunit.client.0.smithi070.stdout:5/14: dread f2 [0,4194304] 0 2024-04-02T04:47:34.527 INFO:tasks.workunit.client.0.smithi070.stdout:0/6: link f0 f1 0 2024-04-02T04:47:34.528 INFO:tasks.workunit.client.0.smithi070.stdout:8/2: dwrite f0 [0,4194304] 0 2024-04-02T04:47:34.528 INFO:tasks.workunit.client.0.smithi070.stdout:5/15: stat f2 0 2024-04-02T04:47:34.529 INFO:tasks.workunit.client.0.smithi070.stdout:9/0: mknod c0 0 2024-04-02T04:47:34.529 INFO:tasks.workunit.client.0.smithi070.stdout:9/1: fsync - no filename 2024-04-02T04:47:34.529 INFO:tasks.workunit.client.0.smithi070.stdout:9/2: dwrite - no filename 2024-04-02T04:47:34.530 INFO:tasks.workunit.client.0.smithi070.stdout:8/3: chown f0 506012 1 2024-04-02T04:47:34.530 INFO:tasks.workunit.client.0.smithi070.stdout:9/3: chown c0 1738 1 2024-04-02T04:47:34.530 INFO:tasks.workunit.client.0.smithi070.stdout:9/4: dread - no filename 2024-04-02T04:47:34.530 INFO:tasks.workunit.client.0.smithi070.stdout:9/5: dread - no filename 2024-04-02T04:47:34.531 INFO:tasks.workunit.client.0.smithi070.stdout:4/9: creat d1/f2 x:0 0 0 2024-04-02T04:47:34.531 INFO:tasks.workunit.client.0.smithi070.stdout:9/6: chown c0 228918 1 2024-04-02T04:47:34.531 INFO:tasks.workunit.client.0.smithi070.stdout:9/7: dread - no filename 2024-04-02T04:47:34.531 INFO:tasks.workunit.client.0.smithi070.stdout:9/8: write - no filename 2024-04-02T04:47:34.533 INFO:tasks.workunit.client.0.smithi070.stdout:6/10: symlink d0/l4 0 2024-04-02T04:47:34.535 INFO:tasks.workunit.client.0.smithi070.stdout:0/7: fdatasync f1 0 2024-04-02T04:47:34.537 INFO:tasks.workunit.client.0.smithi070.stdout:5/16: creat f3 x:0 0 0 2024-04-02T04:47:34.538 INFO:tasks.workunit.client.0.smithi070.stdout:8/4: dread f0 [0,4194304] 0 2024-04-02T04:47:34.540 INFO:tasks.workunit.client.0.smithi070.stdout:9/9: mknod c1 0 2024-04-02T04:47:34.540 INFO:tasks.workunit.client.0.smithi070.stdout:9/10: fdatasync - no filename 2024-04-02T04:47:34.541 INFO:tasks.workunit.client.0.smithi070.stdout:7/6: link c0 c1 0 2024-04-02T04:47:34.541 INFO:tasks.workunit.client.0.smithi070.stdout:9/11: stat c1 0 2024-04-02T04:47:34.543 INFO:tasks.workunit.client.0.smithi070.stdout:6/11: symlink d0/l5 0 2024-04-02T04:47:34.545 INFO:tasks.workunit.client.0.smithi070.stdout:4/10: dwrite d1/f2 [0,4194304] 0 2024-04-02T04:47:34.546 INFO:tasks.workunit.client.0.smithi070.stdout:5/17: dread f2 [0,4194304] 0 2024-04-02T04:47:34.549 INFO:tasks.workunit.client.0.smithi070.stdout:9/12: rename c1 to c2 0 2024-04-02T04:47:34.549 INFO:tasks.workunit.client.0.smithi070.stdout:9/13: write - no filename 2024-04-02T04:47:34.551 INFO:tasks.workunit.client.0.smithi070.stdout:4/11: unlink d1/f2 0 2024-04-02T04:47:34.551 INFO:tasks.workunit.client.0.smithi070.stdout:4/12: write - no filename 2024-04-02T04:47:34.556 INFO:tasks.workunit.client.0.smithi070.stdout:9/14: creat f3 x:0 0 0 2024-04-02T04:47:34.556 INFO:tasks.workunit.client.0.smithi070.stdout:6/12: getdents d0/d3 0 2024-04-02T04:47:34.559 INFO:tasks.workunit.client.0.smithi070.stdout:4/13: mkdir d1/d3 0 2024-04-02T04:47:34.559 INFO:tasks.workunit.client.0.smithi070.stdout:4/14: write - no filename 2024-04-02T04:47:34.560 INFO:tasks.workunit.client.0.smithi070.stdout:7/7: link c1 c2 0 2024-04-02T04:47:34.560 INFO:tasks.workunit.client.0.smithi070.stdout:7/8: fsync - no filename 2024-04-02T04:47:34.562 INFO:tasks.workunit.client.0.smithi070.stdout:4/15: symlink d1/l4 0 2024-04-02T04:47:34.562 INFO:tasks.workunit.client.0.smithi070.stdout:4/16: dwrite - no filename 2024-04-02T04:47:34.564 INFO:tasks.workunit.client.0.smithi070.stdout:6/13: mknod d0/d3/c6 0 2024-04-02T04:47:34.565 INFO:tasks.workunit.client.0.smithi070.stdout:4/17: symlink d1/l5 0 2024-04-02T04:47:34.566 INFO:tasks.workunit.client.0.smithi070.stdout:4/18: stat d1/d3 0 2024-04-02T04:47:34.567 INFO:tasks.workunit.client.0.smithi070.stdout:4/19: chown d1/l5 13328506 1 2024-04-02T04:47:34.567 INFO:tasks.workunit.client.0.smithi070.stdout:4/20: dread - no filename 2024-04-02T04:47:34.568 INFO:tasks.workunit.client.0.smithi070.stdout:6/14: rename d0/d3/c6 to d0/d3/c7 0 2024-04-02T04:47:34.568 INFO:tasks.workunit.client.0.smithi070.stdout:6/15: dread - no filename 2024-04-02T04:47:34.568 INFO:tasks.workunit.client.0.smithi070.stdout:6/16: dwrite - no filename 2024-04-02T04:47:34.569 INFO:tasks.workunit.client.0.smithi070.stdout:7/9: link c1 c3 0 2024-04-02T04:47:34.569 INFO:tasks.workunit.client.0.smithi070.stdout:7/10: dread - no filename 2024-04-02T04:47:34.572 INFO:tasks.workunit.client.0.smithi070.stdout:4/21: symlink d1/d3/l6 0 2024-04-02T04:47:34.572 INFO:tasks.workunit.client.0.smithi070.stdout:4/22: dwrite - no filename 2024-04-02T04:47:34.573 INFO:tasks.workunit.client.0.smithi070.stdout:7/11: unlink c1 0 2024-04-02T04:47:34.573 INFO:tasks.workunit.client.0.smithi070.stdout:7/12: dwrite - no filename 2024-04-02T04:47:34.577 INFO:tasks.workunit.client.0.smithi070.stdout:7/13: unlink c2 0 2024-04-02T04:47:34.577 INFO:tasks.workunit.client.0.smithi070.stdout:7/14: rmdir - no directory 2024-04-02T04:47:34.577 INFO:tasks.workunit.client.0.smithi070.stdout:4/23: link d1/l5 d1/d3/l7 0 2024-04-02T04:47:34.580 INFO:tasks.workunit.client.0.smithi070.stdout:4/24: mknod d1/c8 0 2024-04-02T04:47:34.581 INFO:tasks.workunit.client.0.smithi070.stdout:7/15: chown c0 694492 1 2024-04-02T04:47:34.583 INFO:tasks.workunit.client.0.smithi070.stdout:4/25: mkdir d1/d3/d9 0 2024-04-02T04:47:34.583 INFO:tasks.workunit.client.0.smithi070.stdout:4/26: dread - no filename 2024-04-02T04:47:34.586 INFO:tasks.workunit.client.0.smithi070.stdout:4/27: unlink d1/l5 0 2024-04-02T04:47:34.586 INFO:tasks.workunit.client.0.smithi070.stdout:4/28: write - no filename 2024-04-02T04:47:34.586 INFO:tasks.workunit.client.0.smithi070.stdout:4/29: dread - no filename 2024-04-02T04:47:34.587 INFO:tasks.workunit.client.0.smithi070.stdout:7/16: link c3 c4 0 2024-04-02T04:47:34.587 INFO:tasks.workunit.client.0.smithi070.stdout:7/17: write - no filename 2024-04-02T04:47:34.588 INFO:tasks.workunit.client.0.smithi070.stdout:4/30: creat d1/fa x:0 0 0 2024-04-02T04:47:34.589 INFO:tasks.workunit.client.0.smithi070.stdout:7/18: creat f5 x:0 0 0 2024-04-02T04:47:34.590 INFO:tasks.workunit.client.0.smithi070.stdout:4/31: creat d1/fb x:0 0 0 2024-04-02T04:47:34.593 INFO:tasks.workunit.client.0.smithi070.stdout:7/19: chown c0 443575 1 2024-04-02T04:47:34.596 INFO:tasks.workunit.client.0.smithi070.stdout:7/20: chown c4 15 1 2024-04-02T04:47:34.600 INFO:tasks.workunit.client.0.smithi070.stdout:4/32: dwrite d1/fb [0,4194304] 0 2024-04-02T04:47:34.600 INFO:tasks.workunit.client.0.smithi070.stdout:7/21: rename f5 to f6 0 2024-04-02T04:47:34.604 INFO:tasks.workunit.client.0.smithi070.stdout:4/33: creat d1/fc x:0 0 0 2024-04-02T04:47:34.605 INFO:tasks.workunit.client.0.smithi070.stdout:7/22: unlink c3 0 2024-04-02T04:47:34.606 INFO:tasks.workunit.client.0.smithi070.stdout:7/23: stat f6 0 2024-04-02T04:47:34.606 INFO:tasks.workunit.client.0.smithi070.stdout:7/24: chown f6 3226920 1 2024-04-02T04:47:34.613 INFO:tasks.workunit.client.0.smithi070.stdout:4/34: dwrite d1/fc [0,4194304] 0 2024-04-02T04:47:34.616 INFO:tasks.workunit.client.0.smithi070.stdout:4/35: creat d1/fd x:0 0 0 2024-04-02T04:47:34.618 INFO:tasks.workunit.client.0.smithi070.stdout:4/36: creat d1/fe x:0 0 0 2024-04-02T04:47:34.621 INFO:tasks.workunit.client.0.smithi070.stdout:4/37: creat d1/d3/d9/ff x:0 0 0 2024-04-02T04:47:34.622 INFO:tasks.workunit.client.0.smithi070.stdout:4/38: dread - d1/fd zero size 2024-04-02T04:47:34.624 INFO:tasks.workunit.client.0.smithi070.stdout:4/39: symlink d1/l10 0 2024-04-02T04:47:34.694 INFO:tasks.workunit.client.0.smithi070.stdout:1/9: getdents d0 0 2024-04-02T04:47:34.696 INFO:tasks.workunit.client.0.smithi070.stdout:1/10: creat d0/d1/f2 x:0 0 0 2024-04-02T04:47:34.696 INFO:tasks.workunit.client.0.smithi070.stdout:1/11: chown d0 77260184 1 2024-04-02T04:47:34.699 INFO:tasks.workunit.client.0.smithi070.stdout:1/12: creat d0/d1/f3 x:0 0 0 2024-04-02T04:47:34.700 INFO:tasks.workunit.client.0.smithi070.stdout:1/13: write d0/d1/f3 [347221,3507] 0 2024-04-02T04:47:34.703 INFO:tasks.workunit.client.0.smithi070.stdout:1/14: creat d0/f4 x:0 0 0 2024-04-02T04:47:34.703 INFO:tasks.workunit.client.0.smithi070.stdout:2/12: write f0 [5230117,95338] 0 2024-04-02T04:47:34.708 INFO:tasks.workunit.client.0.smithi070.stdout:1/15: unlink d0/d1/f3 0 2024-04-02T04:47:34.710 INFO:tasks.workunit.client.0.smithi070.stdout:2/13: unlink f0 0 2024-04-02T04:47:34.710 INFO:tasks.workunit.client.0.smithi070.stdout:2/14: write - no filename 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/15: write - no filename 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/16: chown . 21312 1 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/17: write - no filename 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/18: truncate - no filename 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/19: dwrite - no filename 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/20: unlink - no file 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/21: truncate - no filename 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/22: fsync - no filename 2024-04-02T04:47:34.711 INFO:tasks.workunit.client.0.smithi070.stdout:2/23: read - no filename 2024-04-02T04:47:34.712 INFO:tasks.workunit.client.0.smithi070.stdout:1/16: mkdir d0/d1/d5 0 2024-04-02T04:47:34.714 INFO:tasks.workunit.client.0.smithi070.stdout:2/24: symlink l1 0 2024-04-02T04:47:34.714 INFO:tasks.workunit.client.0.smithi070.stdout:1/17: write d0/f4 [20444,101854] 0 2024-04-02T04:47:34.715 INFO:tasks.workunit.client.0.smithi070.stdout:0/8: fsync f0 0 2024-04-02T04:47:34.716 INFO:tasks.workunit.client.0.smithi070.stdout:1/18: write d0/d1/f2 [810083,57856] 0 2024-04-02T04:47:34.719 INFO:tasks.workunit.client.0.smithi070.stdout:2/25: mkdir d2 0 2024-04-02T04:47:34.721 INFO:tasks.workunit.client.0.smithi070.stdout:0/9: dread f0 [0,4194304] 0 2024-04-02T04:47:34.721 INFO:tasks.workunit.client.0.smithi070.stdout:1/19: symlink d0/d1/d5/l6 0 2024-04-02T04:47:34.724 INFO:tasks.workunit.client.0.smithi070.stdout:1/20: truncate d0/f4 848543 0 2024-04-02T04:47:34.725 INFO:tasks.workunit.client.0.smithi070.stdout:5/18: fsync f3 0 2024-04-02T04:47:34.727 INFO:tasks.workunit.client.0.smithi070.stdout:1/21: write d0/d1/f2 [1309198,84338] 0 2024-04-02T04:47:34.727 INFO:tasks.workunit.client.0.smithi070.stdout:0/10: link f0 f2 0 2024-04-02T04:47:34.727 INFO:tasks.workunit.client.0.smithi070.stdout:5/19: truncate f3 634232 0 2024-04-02T04:47:34.731 INFO:tasks.workunit.client.0.smithi070.stdout:8/5: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:34.731 INFO:tasks.workunit.client.0.smithi070.stdout:8/6: readlink - no filename 2024-04-02T04:47:34.732 INFO:tasks.workunit.client.0.smithi070.stdout:9/15: chown c2 443353 1 2024-04-02T04:47:34.733 INFO:tasks.workunit.client.0.smithi070.stdout:1/22: chown d0/d1/d5/l6 200234787 1 2024-04-02T04:47:34.733 INFO:tasks.workunit.client.0.smithi070.stdout:5/20: unlink c0 0 2024-04-02T04:47:34.733 INFO:tasks.workunit.client.0.smithi070.stdout:9/16: truncate f3 386561 0 2024-04-02T04:47:34.734 INFO:tasks.workunit.client.0.smithi070.stdout:5/21: truncate f3 840480 0 2024-04-02T04:47:34.734 INFO:tasks.workunit.client.0.smithi070.stdout:5/22: write f3 [1596989,25914] 0 2024-04-02T04:47:34.734 INFO:tasks.workunit.client.0.smithi070.stdout:5/23: rmdir - no directory 2024-04-02T04:47:34.735 INFO:tasks.workunit.client.0.smithi070.stdout:0/11: rename f1 to f3 0 2024-04-02T04:47:34.735 INFO:tasks.workunit.client.0.smithi070.stdout:0/12: readlink - no filename 2024-04-02T04:47:34.739 INFO:tasks.workunit.client.0.smithi070.stdout:9/17: mkdir d4 0 2024-04-02T04:47:34.742 INFO:tasks.workunit.client.0.smithi070.stdout:9/18: dread f3 [0,4194304] 0 2024-04-02T04:47:34.743 INFO:tasks.workunit.client.0.smithi070.stdout:9/19: dread f3 [0,4194304] 0 2024-04-02T04:47:34.745 INFO:tasks.workunit.client.0.smithi070.stdout:9/20: write f3 [376590,121250] 0 2024-04-02T04:47:34.746 INFO:tasks.workunit.client.0.smithi070.stdout:9/21: read f3 [230070,42589] 0 2024-04-02T04:47:34.748 INFO:tasks.workunit.client.0.smithi070.stdout:9/22: dread f3 [0,4194304] 0 2024-04-02T04:47:34.748 INFO:tasks.workunit.client.0.smithi070.stdout:6/17: stat d0/d3/c7 0 2024-04-02T04:47:34.748 INFO:tasks.workunit.client.0.smithi070.stdout:6/18: dwrite - no filename 2024-04-02T04:47:34.754 INFO:tasks.workunit.client.0.smithi070.stdout:0/13: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:34.756 INFO:tasks.workunit.client.0.smithi070.stdout:0/14: write f2 [1413632,122314] 0 2024-04-02T04:47:34.758 INFO:tasks.workunit.client.0.smithi070.stdout:6/19: unlink d0/l5 0 2024-04-02T04:47:34.759 INFO:tasks.workunit.client.0.smithi070.stdout:6/20: chown d0/d3/c7 0 1 2024-04-02T04:47:34.759 INFO:tasks.workunit.client.0.smithi070.stdout:6/21: dread - no filename 2024-04-02T04:47:34.759 INFO:tasks.workunit.client.0.smithi070.stdout:6/22: dwrite - no filename 2024-04-02T04:47:34.763 INFO:tasks.workunit.client.0.smithi070.stdout:9/23: dwrite f3 [0,4194304] 0 2024-04-02T04:47:34.767 INFO:tasks.workunit.client.0.smithi070.stdout:8/7: fsync f0 0 2024-04-02T04:47:34.767 INFO:tasks.workunit.client.0.smithi070.stdout:8/8: rmdir - no directory 2024-04-02T04:47:34.768 INFO:tasks.workunit.client.0.smithi070.stdout:7/25: fsync f6 0 2024-04-02T04:47:34.768 INFO:tasks.workunit.client.0.smithi070.stdout:8/9: chown f0 52702 1 2024-04-02T04:47:34.769 INFO:tasks.workunit.client.0.smithi070.stdout:8/10: chown f0 46641144 1 2024-04-02T04:47:34.775 INFO:tasks.workunit.client.0.smithi070.stdout:8/11: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:34.862 INFO:tasks.workunit.client.0.smithi070.stdout:7/26: unlink f6 0 2024-04-02T04:47:34.866 INFO:tasks.workunit.client.0.smithi070.stdout:4/40: getdents d1 0 2024-04-02T04:47:34.867 INFO:tasks.workunit.client.0.smithi070.stdout:9/24: fsync f3 0 2024-04-02T04:47:34.867 INFO:tasks.workunit.client.0.smithi070.stdout:7/27: mkdir d7 0 2024-04-02T04:47:34.867 INFO:tasks.workunit.client.0.smithi070.stdout:7/28: truncate - no filename 2024-04-02T04:47:34.870 INFO:tasks.workunit.client.0.smithi070.stdout:8/12: link f0 f1 0 2024-04-02T04:47:34.872 INFO:tasks.workunit.client.0.smithi070.stdout:7/29: stat c0 0 2024-04-02T04:47:34.872 INFO:tasks.workunit.client.0.smithi070.stdout:7/30: chown c4 435259 1 2024-04-02T04:47:34.872 INFO:tasks.workunit.client.0.smithi070.stdout:7/31: fsync - no filename 2024-04-02T04:47:34.873 INFO:tasks.workunit.client.0.smithi070.stdout:7/32: truncate - no filename 2024-04-02T04:47:34.873 INFO:tasks.workunit.client.0.smithi070.stdout:7/33: dread - no filename 2024-04-02T04:47:34.875 INFO:tasks.workunit.client.0.smithi070.stdout:8/13: mkdir d2 0 2024-04-02T04:47:34.876 INFO:tasks.workunit.client.0.smithi070.stdout:8/14: mkdir d2/d3 0 2024-04-02T04:47:34.878 INFO:tasks.workunit.client.0.smithi070.stdout:8/15: mknod d2/c4 0 2024-04-02T04:47:34.880 INFO:tasks.workunit.client.0.smithi070.stdout:1/23: rename d0/d1/d5 to d0/d7 0 2024-04-02T04:47:34.881 INFO:tasks.workunit.client.0.smithi070.stdout:8/16: rename d2/c4 to d2/c5 0 2024-04-02T04:47:34.884 INFO:tasks.workunit.client.0.smithi070.stdout:6/23: rename d0/l4 to d0/d3/l8 0 2024-04-02T04:47:34.886 INFO:tasks.workunit.client.0.smithi070.stdout:0/15: getdents . 0 2024-04-02T04:47:34.887 INFO:tasks.workunit.client.0.smithi070.stdout:1/24: creat d0/d7/f8 x:0 0 0 2024-04-02T04:47:34.891 INFO:tasks.workunit.client.0.smithi070.stdout:6/24: creat d0/f9 x:0 0 0 2024-04-02T04:47:34.898 INFO:tasks.workunit.client.0.smithi070.stdout:1/25: mkdir d0/d9 0 2024-04-02T04:47:34.898 INFO:tasks.workunit.client.0.smithi070.stdout:8/17: dwrite f1 [4194304,4194304] 0 2024-04-02T04:47:34.903 INFO:tasks.workunit.client.0.smithi070.stdout:5/24: dwrite f2 [0,4194304] 0 2024-04-02T04:47:34.904 INFO:tasks.workunit.client.0.smithi070.stdout:0/16: dwrite f0 [0,4194304] 0 2024-04-02T04:47:34.906 INFO:tasks.workunit.client.0.smithi070.stdout:6/25: rename d0/f9 to d0/d3/fa 0 2024-04-02T04:47:34.909 INFO:tasks.workunit.client.0.smithi070.stdout:6/26: dread - d0/d3/fa zero size 2024-04-02T04:47:34.910 INFO:tasks.workunit.client.0.smithi070.stdout:6/27: dread - d0/d3/fa zero size 2024-04-02T04:47:34.912 INFO:tasks.workunit.client.0.smithi070.stdout:6/28: read - d0/d3/fa zero size 2024-04-02T04:47:34.914 INFO:tasks.workunit.client.0.smithi070.stdout:5/25: dwrite f2 [0,4194304] 0 2024-04-02T04:47:34.915 INFO:tasks.workunit.client.0.smithi070.stdout:5/26: chown l1 7697733 1 2024-04-02T04:47:34.915 INFO:tasks.workunit.client.0.smithi070.stdout:1/26: symlink d0/d7/la 0 2024-04-02T04:47:34.916 INFO:tasks.workunit.client.0.smithi070.stdout:1/27: stat d0 0 2024-04-02T04:47:34.920 INFO:tasks.workunit.client.0.smithi070.stdout:4/41: write d1/d3/d9/ff [369389,39074] 0 2024-04-02T04:47:34.923 INFO:tasks.workunit.client.0.smithi070.stdout:0/17: mkdir d4 0 2024-04-02T04:47:34.925 INFO:tasks.workunit.client.0.smithi070.stdout:6/29: creat d0/d3/fb x:0 0 0 2024-04-02T04:47:34.927 INFO:tasks.workunit.client.0.smithi070.stdout:5/27: creat f4 x:0 0 0 2024-04-02T04:47:34.928 INFO:tasks.workunit.client.0.smithi070.stdout:4/42: dwrite d1/fa [0,4194304] 0 2024-04-02T04:47:34.930 INFO:tasks.workunit.client.0.smithi070.stdout:0/18: unlink f2 0 2024-04-02T04:47:34.932 INFO:tasks.workunit.client.0.smithi070.stdout:6/30: creat d0/fc x:0 0 0 2024-04-02T04:47:34.932 INFO:tasks.workunit.client.0.smithi070.stdout:6/31: dread - d0/fc zero size 2024-04-02T04:47:34.933 INFO:tasks.workunit.client.0.smithi070.stdout:5/28: mkdir d5 0 2024-04-02T04:47:34.934 INFO:tasks.workunit.client.0.smithi070.stdout:5/29: chown f3 15 1 2024-04-02T04:47:34.938 INFO:tasks.workunit.client.0.smithi070.stdout:4/43: creat d1/d3/d9/f11 x:0 0 0 2024-04-02T04:47:34.939 INFO:tasks.workunit.client.0.smithi070.stdout:4/44: chown d1/c8 0 1 2024-04-02T04:47:34.939 INFO:tasks.workunit.client.0.smithi070.stdout:5/30: dread f3 [0,4194304] 0 2024-04-02T04:47:34.940 INFO:tasks.workunit.client.0.smithi070.stdout:5/31: fsync f3 0 2024-04-02T04:47:34.953 INFO:tasks.workunit.client.0.smithi070.stdout:5/32: creat d5/f6 x:0 0 0 2024-04-02T04:47:34.953 INFO:tasks.workunit.client.0.smithi070.stdout:6/32: link d0/d3/c7 d0/cd 0 2024-04-02T04:47:34.955 INFO:tasks.workunit.client.0.smithi070.stdout:5/33: mknod d5/c7 0 2024-04-02T04:47:34.956 INFO:tasks.workunit.client.0.smithi070.stdout:6/33: mkdir d0/d3/de 0 2024-04-02T04:47:34.957 INFO:tasks.workunit.client.0.smithi070.stdout:6/34: truncate d0/d3/fb 620228 0 2024-04-02T04:47:34.961 INFO:tasks.workunit.client.0.smithi070.stdout:6/35: rename d0/d3/de to d0/df 0 2024-04-02T04:47:34.961 INFO:tasks.workunit.client.0.smithi070.stdout:5/34: link d5/c7 d5/c8 0 2024-04-02T04:47:34.964 INFO:tasks.workunit.client.0.smithi070.stdout:5/35: mkdir d5/d9 0 2024-04-02T04:47:34.966 INFO:tasks.workunit.client.0.smithi070.stdout:5/36: creat d5/fa x:0 0 0 2024-04-02T04:47:34.969 INFO:tasks.workunit.client.0.smithi070.stdout:5/37: creat d5/d9/fb x:0 0 0 2024-04-02T04:47:34.975 INFO:tasks.workunit.client.0.smithi070.stdout:0/19: fdatasync f3 0 2024-04-02T04:47:34.980 INFO:tasks.workunit.client.0.smithi070.stdout:0/20: mkdir d4/d5 0 2024-04-02T04:47:34.981 INFO:tasks.workunit.client.0.smithi070.stdout:6/36: readlink d0/d3/l8 0 2024-04-02T04:47:34.984 INFO:tasks.workunit.client.0.smithi070.stdout:8/18: unlink d2/c5 0 2024-04-02T04:47:34.986 INFO:tasks.workunit.client.0.smithi070.stdout:0/21: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:34.987 INFO:tasks.workunit.client.0.smithi070.stdout:6/37: write d0/d3/fa [924787,90233] 0 2024-04-02T04:47:34.988 INFO:tasks.workunit.client.0.smithi070.stdout:6/38: stat d0/d3 0 2024-04-02T04:47:34.990 INFO:tasks.workunit.client.0.smithi070.stdout:8/19: dread f0 [0,4194304] 0 2024-04-02T04:47:34.991 INFO:tasks.workunit.client.0.smithi070.stdout:8/20: chown f0 0 1 2024-04-02T04:47:35.004 INFO:tasks.workunit.client.0.smithi070.stdout:6/39: rename d0/cd to d0/c10 0 2024-04-02T04:47:35.006 INFO:tasks.workunit.client.0.smithi070.stdout:8/21: mkdir d2/d3/d6 0 2024-04-02T04:47:35.008 INFO:tasks.workunit.client.0.smithi070.stdout:6/40: symlink d0/l11 0 2024-04-02T04:47:35.009 INFO:tasks.workunit.client.0.smithi070.stdout:8/22: dread f1 [0,4194304] 0 2024-04-02T04:47:35.021 INFO:tasks.workunit.client.0.smithi070.stdout:1/28: truncate d0/d1/f2 1335887 0 2024-04-02T04:47:35.024 INFO:tasks.workunit.client.0.smithi070.stdout:1/29: dread d0/f4 [0,4194304] 0 2024-04-02T04:47:35.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:34 smithi070 ceph-mon[29212]: pgmap v6: 65 pgs: 65 active+clean; 204 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s wr, 86 op/s 2024-04-02T04:47:35.031 INFO:tasks.workunit.client.0.smithi070.stdout:1/30: dwrite d0/d7/f8 [0,4194304] 0 2024-04-02T04:47:35.041 INFO:tasks.workunit.client.0.smithi070.stdout:1/31: getdents d0/d1 0 2024-04-02T04:47:35.071 INFO:tasks.workunit.client.0.smithi070.stdout:8/23: getdents d2 0 2024-04-02T04:47:35.075 INFO:tasks.workunit.client.0.smithi070.stdout:8/24: rename f0 to d2/d3/f7 0 2024-04-02T04:47:35.075 INFO:tasks.workunit.client.0.smithi070.stdout:8/25: chown f1 1381297096 1 2024-04-02T04:47:35.078 INFO:tasks.workunit.client.0.smithi070.stdout:8/26: dread d2/d3/f7 [4194304,4194304] 0 2024-04-02T04:47:35.079 INFO:tasks.workunit.client.0.smithi070.stdout:0/22: truncate f0 3529873 0 2024-04-02T04:47:35.079 INFO:tasks.workunit.client.0.smithi070.stdout:8/27: stat d2/d3 0 2024-04-02T04:47:35.081 INFO:tasks.workunit.client.0.smithi070.stdout:8/28: rmdir d2 39 2024-04-02T04:47:35.087 INFO:tasks.workunit.client.0.smithi070.stdout:8/29: rmdir d2/d3 39 2024-04-02T04:47:35.089 INFO:tasks.workunit.client.0.smithi070.stdout:8/30: chown d2/d3 851669 1 2024-04-02T04:47:35.090 INFO:tasks.workunit.client.0.smithi070.stdout:6/41: truncate d0/d3/fb 537775 0 2024-04-02T04:47:35.092 INFO:tasks.workunit.client.0.smithi070.stdout:8/31: fdatasync f1 0 2024-04-02T04:47:35.092 INFO:tasks.workunit.client.0.smithi070.stdout:8/32: chown d2/d3 5100 1 2024-04-02T04:47:35.092 INFO:tasks.workunit.client.0.smithi070.stdout:8/33: chown f1 173 1 2024-04-02T04:47:35.098 INFO:tasks.workunit.client.0.smithi070.stdout:8/34: mkdir d2/d8 0 2024-04-02T04:47:35.106 INFO:tasks.workunit.client.0.smithi070.stdout:8/35: link f1 d2/d3/f9 0 2024-04-02T04:47:35.110 INFO:tasks.workunit.client.0.smithi070.stdout:8/36: mkdir d2/d8/da 0 2024-04-02T04:47:35.114 INFO:tasks.workunit.client.0.smithi070.stdout:8/37: mkdir d2/d8/da/db 0 2024-04-02T04:47:35.122 INFO:tasks.workunit.client.0.smithi070.stdout:8/38: dwrite f1 [8388608,4194304] 0 2024-04-02T04:47:35.132 INFO:tasks.workunit.client.0.smithi070.stdout:8/39: dwrite d2/d3/f9 [8388608,4194304] 0 2024-04-02T04:47:35.132 INFO:tasks.workunit.client.0.smithi070.stdout:8/40: readlink - no filename 2024-04-02T04:47:35.132 INFO:tasks.workunit.client.0.smithi070.stdout:8/41: rename d2/d8 to d2/d8/dc 22 2024-04-02T04:47:35.140 INFO:tasks.workunit.client.0.smithi070.stdout:8/42: link f1 d2/d3/fd 0 2024-04-02T04:47:35.143 INFO:tasks.workunit.client.0.smithi070.stdout:8/43: dread d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:35.146 INFO:tasks.workunit.client.0.smithi070.stdout:8/44: mknod d2/d3/ce 0 2024-04-02T04:47:35.153 INFO:tasks.workunit.client.0.smithi070.stdout:8/45: dwrite d2/d3/f9 [4194304,4194304] 0 2024-04-02T04:47:35.158 INFO:tasks.workunit.client.0.smithi070.stdout:8/46: dread f1 [0,4194304] 0 2024-04-02T04:47:35.165 INFO:tasks.workunit.client.0.smithi070.stdout:0/23: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:35.166 INFO:tasks.workunit.client.0.smithi070.stdout:0/24: write f0 [6085731,10279] 0 2024-04-02T04:47:35.166 INFO:tasks.workunit.client.0.smithi070.stdout:8/47: dread d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:35.167 INFO:tasks.workunit.client.0.smithi070.stdout:0/25: write f0 [3905646,122892] 0 2024-04-02T04:47:35.168 INFO:tasks.workunit.client.0.smithi070.stdout:0/26: read f3 [611968,109901] 0 2024-04-02T04:47:35.174 INFO:tasks.workunit.client.0.smithi070.stdout:0/27: dwrite f3 [0,4194304] 0 2024-04-02T04:47:35.273 INFO:tasks.workunit.client.0.smithi070.stdout:0/28: dread f0 [4194304,4194304] 0 2024-04-02T04:47:35.274 INFO:tasks.workunit.client.0.smithi070.stdout:0/29: read f3 [3843066,6327] 0 2024-04-02T04:47:35.274 INFO:tasks.workunit.client.0.smithi070.stdout:0/30: chown d4/d5 104 1 2024-04-02T04:47:35.276 INFO:tasks.workunit.client.0.smithi070.stdout:1/32: truncate d0/d1/f2 713128 0 2024-04-02T04:47:35.291 INFO:tasks.workunit.client.0.smithi070.stdout:0/31: mkdir d4/d6 0 2024-04-02T04:47:35.293 INFO:tasks.workunit.client.0.smithi070.stdout:8/48: symlink d2/d8/da/db/lf 0 2024-04-02T04:47:35.295 INFO:tasks.workunit.client.0.smithi070.stdout:1/33: symlink d0/d1/lb 0 2024-04-02T04:47:35.298 INFO:tasks.workunit.client.0.smithi070.stdout:0/32: unlink f3 0 2024-04-02T04:47:35.299 INFO:tasks.workunit.client.0.smithi070.stdout:8/49: fdatasync d2/d3/f9 0 2024-04-02T04:47:35.300 INFO:tasks.workunit.client.0.smithi070.stdout:8/50: chown d2/d8/da 1415 1 2024-04-02T04:47:35.304 INFO:tasks.workunit.client.0.smithi070.stdout:0/33: symlink d4/l7 0 2024-04-02T04:47:35.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:34 smithi179 ceph-mon[36603]: pgmap v6: 65 pgs: 65 active+clean; 204 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s wr, 86 op/s 2024-04-02T04:47:35.329 INFO:tasks.workunit.client.0.smithi070.stdout:0/34: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:35.330 INFO:tasks.workunit.client.0.smithi070.stdout:0/35: write f0 [1904738,130114] 0 2024-04-02T04:47:35.331 INFO:tasks.workunit.client.0.smithi070.stdout:0/36: read f0 [4179937,44513] 0 2024-04-02T04:47:35.356 INFO:tasks.workunit.client.0.smithi070.stdout:1/34: read d0/d1/f2 [511758,75503] 0 2024-04-02T04:47:35.357 INFO:tasks.workunit.client.0.smithi070.stdout:1/35: chown d0/f4 765018947 1 2024-04-02T04:47:35.357 INFO:tasks.workunit.client.0.smithi070.stdout:1/36: stat d0/d9 0 2024-04-02T04:47:35.503 INFO:tasks.workunit.client.0.smithi070.stdout:1/37: creat d0/d9/fc x:0 0 0 2024-04-02T04:47:35.563 INFO:tasks.workunit.client.0.smithi070.stdout:0/37: truncate f0 5078570 0 2024-04-02T04:47:35.569 INFO:tasks.workunit.client.0.smithi070.stdout:1/38: dwrite d0/f4 [0,4194304] 0 2024-04-02T04:47:35.569 INFO:tasks.workunit.client.0.smithi070.stdout:0/38: fsync f0 0 2024-04-02T04:47:35.570 INFO:tasks.workunit.client.0.smithi070.stdout:1/39: fsync d0/d9/fc 0 2024-04-02T04:47:35.572 INFO:tasks.workunit.client.0.smithi070.stdout:1/40: write d0/d7/f8 [3143659,390] 0 2024-04-02T04:47:35.575 INFO:tasks.workunit.client.0.smithi070.stdout:0/39: link f0 d4/d6/f8 0 2024-04-02T04:47:35.576 INFO:tasks.workunit.client.0.smithi070.stdout:1/41: creat d0/d1/fd x:0 0 0 2024-04-02T04:47:35.578 INFO:tasks.workunit.client.0.smithi070.stdout:0/40: write f0 [4121231,44021] 0 2024-04-02T04:47:35.581 INFO:tasks.workunit.client.0.smithi070.stdout:0/41: creat d4/f9 x:0 0 0 2024-04-02T04:47:35.583 INFO:tasks.workunit.client.0.smithi070.stdout:0/42: fsync d4/d6/f8 0 2024-04-02T04:47:35.584 INFO:tasks.workunit.client.0.smithi070.stdout:1/42: dwrite d0/f4 [4194304,4194304] 0 2024-04-02T04:47:35.587 INFO:tasks.workunit.client.0.smithi070.stdout:0/43: rename d4/d6 to d4/d5/da 0 2024-04-02T04:47:35.590 INFO:tasks.workunit.client.0.smithi070.stdout:1/43: rename d0/d7/la to d0/d1/le 0 2024-04-02T04:47:35.591 INFO:tasks.workunit.client.0.smithi070.stdout:1/44: dread - d0/d9/fc zero size 2024-04-02T04:47:35.592 INFO:tasks.workunit.client.0.smithi070.stdout:1/45: stat d0/d9 0 2024-04-02T04:47:35.592 INFO:tasks.workunit.client.0.smithi070.stdout:0/44: rename d4/l7 to d4/d5/da/lb 0 2024-04-02T04:47:35.593 INFO:tasks.workunit.client.0.smithi070.stdout:1/46: symlink d0/d9/lf 0 2024-04-02T04:47:35.593 INFO:tasks.workunit.client.0.smithi070.stdout:0/45: mknod d4/d5/cc 0 2024-04-02T04:47:35.594 INFO:tasks.workunit.client.0.smithi070.stdout:0/46: chown d4 419976 1 2024-04-02T04:47:35.617 INFO:tasks.workunit.client.0.smithi070.stdout:1/47: rename d0/f4 to d0/d7/f10 0 2024-04-02T04:47:35.621 INFO:tasks.workunit.client.0.smithi070.stdout:0/47: truncate f0 4752037 0 2024-04-02T04:47:35.625 INFO:tasks.workunit.client.0.smithi070.stdout:1/48: rmdir d0 39 2024-04-02T04:47:35.629 INFO:tasks.workunit.client.0.smithi070.stdout:1/49: creat d0/d9/f11 x:0 0 0 2024-04-02T04:47:35.631 INFO:tasks.workunit.client.0.smithi070.stdout:0/48: dread d4/d5/da/f8 [0,4194304] 0 2024-04-02T04:47:35.634 INFO:tasks.workunit.client.0.smithi070.stdout:0/49: readlink d4/d5/da/lb 0 2024-04-02T04:47:35.634 INFO:tasks.workunit.client.0.smithi070.stdout:1/50: link d0/d7/l6 d0/d1/l12 0 2024-04-02T04:47:35.635 INFO:tasks.workunit.client.0.smithi070.stdout:1/51: chown d0/d1/lb 308385 1 2024-04-02T04:47:35.635 INFO:tasks.workunit.client.0.smithi070.stdout:1/52: chown d0/d1/f2 1644 1 2024-04-02T04:47:35.636 INFO:tasks.workunit.client.0.smithi070.stdout:1/53: chown d0/d1/lb 737660 1 2024-04-02T04:47:35.637 INFO:tasks.workunit.client.0.smithi070.stdout:0/50: dread d4/d5/da/f8 [0,4194304] 0 2024-04-02T04:47:35.641 INFO:tasks.workunit.client.0.smithi070.stdout:0/51: dwrite d4/d5/da/f8 [0,4194304] 0 2024-04-02T04:47:35.642 INFO:tasks.workunit.client.0.smithi070.stdout:0/52: chown d4/d5/da 879268264 1 2024-04-02T04:47:35.643 INFO:tasks.workunit.client.0.smithi070.stdout:0/53: write f0 [5358640,58826] 0 2024-04-02T04:47:35.643 INFO:tasks.workunit.client.0.smithi070.stdout:0/54: chown f0 2 1 2024-04-02T04:47:35.645 INFO:tasks.workunit.client.0.smithi070.stdout:0/55: symlink d4/d5/ld 0 2024-04-02T04:47:35.684 INFO:tasks.workunit.client.0.smithi070.stdout:0/56: fsync f0 0 2024-04-02T04:47:35.690 INFO:tasks.workunit.client.0.smithi070.stdout:0/57: dwrite d4/d5/da/f8 [4194304,4194304] 0 2024-04-02T04:47:35.695 INFO:tasks.workunit.client.0.smithi070.stdout:0/58: getdents d4/d5/da 0 2024-04-02T04:47:35.697 INFO:tasks.workunit.client.0.smithi070.stdout:0/59: rename d4/d5/da/lb to d4/d5/le 0 2024-04-02T04:47:35.699 INFO:tasks.workunit.client.0.smithi070.stdout:0/60: rmdir d4/d5/da 39 2024-04-02T04:47:35.701 INFO:tasks.workunit.client.0.smithi070.stdout:0/61: write d4/d5/da/f8 [2092426,10996] 0 2024-04-02T04:47:35.703 INFO:tasks.workunit.client.0.smithi070.stdout:0/62: mkdir d4/d5/df 0 2024-04-02T04:47:35.705 INFO:tasks.workunit.client.0.smithi070.stdout:0/63: mknod d4/d5/df/c10 0 2024-04-02T04:47:35.706 INFO:tasks.workunit.client.0.smithi070.stdout:0/64: write d4/d5/da/f8 [1276183,121423] 0 2024-04-02T04:47:35.707 INFO:tasks.workunit.client.0.smithi070.stdout:0/65: write d4/d5/da/f8 [8619480,1956] 0 2024-04-02T04:47:35.709 INFO:tasks.workunit.client.0.smithi070.stdout:0/66: chown d4/d5/df/c10 5 1 2024-04-02T04:47:35.710 INFO:tasks.workunit.client.0.smithi070.stdout:0/67: rmdir d4/d5 39 2024-04-02T04:47:35.711 INFO:tasks.workunit.client.0.smithi070.stdout:0/68: stat d4/d5/cc 0 2024-04-02T04:47:35.711 INFO:tasks.workunit.client.0.smithi070.stdout:0/69: stat d4/d5 0 2024-04-02T04:47:35.712 INFO:tasks.workunit.client.0.smithi070.stdout:0/70: chown d4/d5/ld 0 1 2024-04-02T04:47:35.713 INFO:tasks.workunit.client.0.smithi070.stdout:0/71: mknod d4/d5/df/c11 0 2024-04-02T04:47:35.714 INFO:tasks.workunit.client.0.smithi070.stdout:0/72: readlink d4/d5/ld 0 2024-04-02T04:47:35.714 INFO:tasks.workunit.client.0.smithi070.stdout:0/73: dread - d4/f9 zero size 2024-04-02T04:47:35.720 INFO:tasks.workunit.client.0.smithi070.stdout:0/74: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:35.725 INFO:tasks.workunit.client.0.smithi070.stdout:0/75: rename d4/d5/le to d4/d5/da/l12 0 2024-04-02T04:47:35.731 INFO:tasks.workunit.client.0.smithi070.stdout:0/76: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:35.734 INFO:tasks.workunit.client.0.smithi070.stdout:0/77: mkdir d4/d5/d13 0 2024-04-02T04:47:35.737 INFO:tasks.workunit.client.0.smithi070.stdout:0/78: creat d4/d5/f14 x:0 0 0 2024-04-02T04:47:35.739 INFO:tasks.workunit.client.0.smithi070.stdout:0/79: truncate d4/d5/f14 893861 0 2024-04-02T04:47:35.740 INFO:tasks.workunit.client.0.smithi070.stdout:0/80: write d4/d5/f14 [884345,72038] 0 2024-04-02T04:47:35.743 INFO:tasks.workunit.client.0.smithi070.stdout:0/81: creat d4/d5/f15 x:0 0 0 2024-04-02T04:47:35.744 INFO:tasks.workunit.client.0.smithi070.stdout:0/82: mkdir d4/d5/da/d16 0 2024-04-02T04:47:35.745 INFO:tasks.workunit.client.0.smithi070.stdout:0/83: fsync d4/d5/da/f8 0 2024-04-02T04:47:35.747 INFO:tasks.workunit.client.0.smithi070.stdout:0/84: dread d4/d5/da/f8 [8388608,4194304] 0 2024-04-02T04:47:35.747 INFO:tasks.workunit.client.0.smithi070.stdout:0/85: chown d4/d5/d13 13 1 2024-04-02T04:47:35.752 INFO:tasks.workunit.client.0.smithi070.stdout:0/86: dwrite d4/d5/da/f8 [8388608,4194304] 0 2024-04-02T04:47:35.755 INFO:tasks.workunit.client.0.smithi070.stdout:0/87: mkdir d4/d5/d13/d17 0 2024-04-02T04:47:35.765 INFO:tasks.workunit.client.0.smithi070.stdout:0/88: truncate d4/d5/f14 138189 0 2024-04-02T04:47:35.768 INFO:tasks.workunit.client.0.smithi070.stdout:0/89: getdents d4/d5/d13 0 2024-04-02T04:47:35.770 INFO:tasks.workunit.client.0.smithi070.stdout:0/90: mkdir d4/d5/df/d18 0 2024-04-02T04:47:35.771 INFO:tasks.workunit.client.0.smithi070.stdout:0/91: getdents d4/d5/da/d16 0 2024-04-02T04:47:35.772 INFO:tasks.workunit.client.0.smithi070.stdout:0/92: fsync f0 0 2024-04-02T04:47:35.772 INFO:tasks.workunit.client.0.smithi070.stdout:0/93: stat d4/d5/d13/d17 0 2024-04-02T04:47:35.777 INFO:tasks.workunit.client.0.smithi070.stdout:0/94: dread d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:35.783 INFO:tasks.workunit.client.0.smithi070.stdout:0/95: dwrite d4/f9 [0,4194304] 0 2024-04-02T04:47:35.784 INFO:tasks.workunit.client.0.smithi070.stdout:0/96: dread - d4/d5/f15 zero size 2024-04-02T04:47:35.787 INFO:tasks.workunit.client.0.smithi070.stdout:0/97: getdents d4/d5/d13 0 2024-04-02T04:47:35.788 INFO:tasks.workunit.client.0.smithi070.stdout:0/98: write d4/d5/f15 [609426,38745] 0 2024-04-02T04:47:35.796 INFO:tasks.workunit.client.0.smithi070.stdout:0/99: truncate d4/d5/f14 117421 0 2024-04-02T04:47:35.799 INFO:tasks.workunit.client.0.smithi070.stdout:0/100: dread d4/f9 [0,4194304] 0 2024-04-02T04:47:35.800 INFO:tasks.workunit.client.0.smithi070.stdout:0/101: write d4/d5/f15 [487956,50678] 0 2024-04-02T04:47:35.801 INFO:tasks.workunit.client.0.smithi070.stdout:0/102: write f0 [3981380,126337] 0 2024-04-02T04:47:35.804 INFO:tasks.workunit.client.0.smithi070.stdout:0/103: creat d4/d5/d13/f19 x:0 0 0 2024-04-02T04:47:35.805 INFO:tasks.workunit.client.0.smithi070.stdout:0/104: mkdir d4/d5/da/d1a 0 2024-04-02T04:47:35.807 INFO:tasks.workunit.client.0.smithi070.stdout:0/105: creat d4/f1b x:0 0 0 2024-04-02T04:47:35.807 INFO:tasks.workunit.client.0.smithi070.stdout:0/106: stat d4/f9 0 2024-04-02T04:47:35.814 INFO:tasks.workunit.client.0.smithi070.stdout:0/107: dwrite d4/f1b [0,4194304] 0 2024-04-02T04:47:35.820 INFO:tasks.workunit.client.0.smithi070.stdout:0/108: dwrite d4/f9 [0,4194304] 0 2024-04-02T04:47:35.824 INFO:tasks.workunit.client.0.smithi070.stdout:0/109: readlink d4/d5/da/l12 0 2024-04-02T04:47:35.829 INFO:tasks.workunit.client.0.smithi070.stdout:0/110: dread d4/d5/f15 [0,4194304] 0 2024-04-02T04:47:35.831 INFO:tasks.workunit.client.0.smithi070.stdout:0/111: getdents d4/d5/da/d1a 0 2024-04-02T04:47:35.833 INFO:tasks.workunit.client.0.smithi070.stdout:0/112: dread d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:35.834 INFO:tasks.workunit.client.0.smithi070.stdout:0/113: write d4/f1b [934058,30159] 0 2024-04-02T04:47:35.837 INFO:tasks.workunit.client.0.smithi070.stdout:0/114: unlink d4/d5/da/l12 0 2024-04-02T04:47:35.879 INFO:tasks.workunit.client.0.smithi070.stdout:0/115: rmdir d4/d5/da 39 2024-04-02T04:47:35.885 INFO:tasks.workunit.client.0.smithi070.stdout:0/116: dwrite d4/d5/d13/f19 [0,4194304] 0 2024-04-02T04:47:35.892 INFO:tasks.workunit.client.0.smithi070.stdout:0/117: creat d4/d5/da/d1a/f1c x:0 0 0 2024-04-02T04:47:35.892 INFO:tasks.workunit.client.0.smithi070.stdout:0/118: fdatasync d4/d5/da/d1a/f1c 0 2024-04-02T04:47:35.893 INFO:tasks.workunit.client.0.smithi070.stdout:0/119: fsync d4/f1b 0 2024-04-02T04:47:35.899 INFO:tasks.workunit.client.0.smithi070.stdout:0/120: symlink d4/l1d 0 2024-04-02T04:47:35.900 INFO:tasks.workunit.client.0.smithi070.stdout:0/121: dread d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:35.901 INFO:tasks.workunit.client.0.smithi070.stdout:0/122: dread d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:35.902 INFO:tasks.workunit.client.0.smithi070.stdout:0/123: write f0 [12112997,35354] 0 2024-04-02T04:47:35.903 INFO:tasks.workunit.client.0.smithi070.stdout:0/124: truncate d4/d5/da/d1a/f1c 894632 0 2024-04-02T04:47:35.904 INFO:tasks.workunit.client.0.smithi070.stdout:0/125: stat d4/d5/df/c10 0 2024-04-02T04:47:35.911 INFO:tasks.workunit.client.0.smithi070.stdout:0/126: rmdir d4/d5/da/d1a 39 2024-04-02T04:47:35.915 INFO:tasks.workunit.client.0.smithi070.stdout:0/127: dread f0 [8388608,4194304] 0 2024-04-02T04:47:35.918 INFO:tasks.workunit.client.1.smithi179.stderr:++ readlink -f fsstress 2024-04-02T04:47:35.919 INFO:tasks.workunit.client.0.smithi070.stdout:0/128: rmdir d4/d5 39 2024-04-02T04:47:35.920 INFO:tasks.workunit.client.1.smithi179.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-02T04:47:35.920 INFO:tasks.workunit.client.1.smithi179.stderr:+ popd 2024-04-02T04:47:35.921 INFO:tasks.workunit.client.1.smithi179.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-02T04:47:35.921 INFO:tasks.workunit.client.1.smithi179.stderr:+ popd 2024-04-02T04:47:35.921 INFO:tasks.workunit.client.1.smithi179.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-02T04:47:35.922 INFO:tasks.workunit.client.0.smithi070.stdout:0/129: stat d4/d5/da/f8 0 2024-04-02T04:47:35.922 INFO:tasks.workunit.client.1.smithi179.stderr:++ mktemp -d -p . 2024-04-02T04:47:35.925 INFO:tasks.workunit.client.0.smithi070.stdout:0/130: dread d4/f1b [0,4194304] 0 2024-04-02T04:47:35.929 INFO:tasks.workunit.client.0.smithi070.stdout:0/131: getdents d4/d5/df/d18 0 2024-04-02T04:47:35.931 INFO:tasks.workunit.client.0.smithi070.stdout:0/132: rename d4/d5/df/d18 to d4/d5/df/d18/d1e 22 2024-04-02T04:47:35.931 INFO:tasks.workunit.client.0.smithi070.stdout:0/133: chown d4 35 1 2024-04-02T04:47:35.931 INFO:tasks.workunit.client.0.smithi070.stdout:0/134: fdatasync d4/f9 0 2024-04-02T04:47:35.949 INFO:tasks.workunit.client.0.smithi070.stdout:0/135: truncate d4/f9 2488081 0 2024-04-02T04:47:35.949 INFO:tasks.workunit.client.0.smithi070.stdout:0/136: read d4/f1b [1194138,115188] 0 2024-04-02T04:47:35.954 INFO:tasks.workunit.client.0.smithi070.stdout:0/137: link d4/d5/f15 d4/d5/d13/f1f 0 2024-04-02T04:47:35.954 INFO:tasks.workunit.client.0.smithi070.stdout:0/138: stat d4/d5/da/d16 0 2024-04-02T04:47:35.961 INFO:tasks.workunit.client.0.smithi070.stdout:0/139: dwrite d4/f1b [0,4194304] 0 2024-04-02T04:47:35.968 INFO:tasks.workunit.client.0.smithi070.stdout:0/140: rename d4/d5/da/d1a/f1c to d4/d5/da/d1a/f20 0 2024-04-02T04:47:35.976 INFO:tasks.workunit.client.0.smithi070.stdout:0/141: dwrite f0 [0,4194304] 0 2024-04-02T04:47:36.072 INFO:tasks.workunit.client.0.smithi070.stdout:0/142: rename d4/d5/da/d16 to d4/d5/d21 0 2024-04-02T04:47:36.078 INFO:tasks.workunit.client.0.smithi070.stdout:0/143: link d4/d5/df/c10 d4/d5/d13/d17/c22 0 2024-04-02T04:47:36.078 INFO:tasks.workunit.client.0.smithi070.stdout:0/144: stat d4/d5/d13/d17 0 2024-04-02T04:47:36.080 INFO:tasks.workunit.client.0.smithi070.stdout:0/145: dread d4/d5/f15 [0,4194304] 0 2024-04-02T04:47:36.132 INFO:tasks.workunit.client.0.smithi070.stdout:0/146: dwrite d4/d5/f15 [0,4194304] 0 2024-04-02T04:47:36.133 INFO:tasks.workunit.client.0.smithi070.stdout:0/147: chown d4/d5/f15 0 1 2024-04-02T04:47:36.183 INFO:tasks.workunit.client.0.smithi070.stdout:0/148: truncate d4/d5/f15 1222843 0 2024-04-02T04:47:36.185 INFO:tasks.workunit.client.0.smithi070.stdout:0/149: fdatasync d4/d5/d13/f19 0 2024-04-02T04:47:36.191 INFO:tasks.workunit.client.0.smithi070.stdout:0/150: dwrite d4/d5/da/f8 [8388608,4194304] 0 2024-04-02T04:47:36.243 INFO:tasks.workunit.client.0.smithi070.stdout:0/151: dwrite d4/d5/d13/f1f [0,4194304] 0 2024-04-02T04:47:36.244 INFO:tasks.workunit.client.1.smithi179.stderr:+ T=./tmp.k9eQVjoRPQ 2024-04-02T04:47:36.244 INFO:tasks.workunit.client.1.smithi179.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.k9eQVjoRPQ -l 1 -n 1000 -p 10 -v 2024-04-02T04:47:36.248 INFO:tasks.workunit.client.1.smithi179.stdout:seed = 1711820581 2024-04-02T04:47:36.249 INFO:tasks.workunit.client.0.smithi070.stdout:0/152: getdents d4/d5/d13/d17 0 2024-04-02T04:47:36.254 INFO:tasks.workunit.client.0.smithi070.stdout:0/153: getdents d4 0 2024-04-02T04:47:36.255 INFO:tasks.workunit.client.1.smithi179.stdout:1/0: dwrite - no filename 2024-04-02T04:47:36.255 INFO:tasks.workunit.client.1.smithi179.stdout:1/1: stat - no entries 2024-04-02T04:47:36.255 INFO:tasks.workunit.client.1.smithi179.stdout:1/2: rename - no filename 2024-04-02T04:47:36.256 INFO:tasks.workunit.client.0.smithi070.stdout:0/154: mkdir d4/d5/d13/d23 0 2024-04-02T04:47:36.258 INFO:tasks.workunit.client.0.smithi070.stdout:0/155: write d4/d5/f15 [1335456,16047] 0 2024-04-02T04:47:36.258 INFO:tasks.workunit.client.1.smithi179.stdout:0/0: creat f0 x:0 0 0 2024-04-02T04:47:36.259 INFO:tasks.workunit.client.1.smithi179.stdout:2/0: link - no file 2024-04-02T04:47:36.259 INFO:tasks.workunit.client.1.smithi179.stdout:2/1: dwrite - no filename 2024-04-02T04:47:36.260 INFO:tasks.workunit.client.0.smithi070.stdout:0/156: symlink d4/d5/d21/l24 0 2024-04-02T04:47:36.262 INFO:tasks.workunit.client.1.smithi179.stdout:3/0: link - no file 2024-04-02T04:47:36.262 INFO:tasks.workunit.client.1.smithi179.stdout:3/1: dwrite - no filename 2024-04-02T04:47:36.262 INFO:tasks.workunit.client.1.smithi179.stdout:3/2: write - no filename 2024-04-02T04:47:36.262 INFO:tasks.workunit.client.1.smithi179.stdout:3/3: rmdir - no directory 2024-04-02T04:47:36.263 INFO:tasks.workunit.client.1.smithi179.stdout:3/4: write - no filename 2024-04-02T04:47:36.263 INFO:tasks.workunit.client.1.smithi179.stdout:3/5: write - no filename 2024-04-02T04:47:36.263 INFO:tasks.workunit.client.1.smithi179.stdout:3/6: dread - no filename 2024-04-02T04:47:36.263 INFO:tasks.workunit.client.1.smithi179.stdout:1/3: mknod c0 0 2024-04-02T04:47:36.263 INFO:tasks.workunit.client.1.smithi179.stdout:1/4: dwrite - no filename 2024-04-02T04:47:36.265 INFO:tasks.workunit.client.1.smithi179.stdout:4/0: dwrite - no filename 2024-04-02T04:47:36.265 INFO:tasks.workunit.client.1.smithi179.stdout:4/1: read - no filename 2024-04-02T04:47:36.265 INFO:tasks.workunit.client.1.smithi179.stdout:4/2: write - no filename 2024-04-02T04:47:36.266 INFO:tasks.workunit.client.1.smithi179.stdout:4/3: chown . 46348 1 2024-04-02T04:47:36.266 INFO:tasks.workunit.client.1.smithi179.stdout:4/4: dwrite - no filename 2024-04-02T04:47:36.266 INFO:tasks.workunit.client.1.smithi179.stdout:4/5: rmdir - no directory 2024-04-02T04:47:36.266 INFO:tasks.workunit.client.1.smithi179.stdout:4/6: dwrite - no filename 2024-04-02T04:47:36.266 INFO:tasks.workunit.client.1.smithi179.stdout:4/7: truncate - no filename 2024-04-02T04:47:36.266 INFO:tasks.workunit.client.1.smithi179.stdout:4/8: chown . 91 1 2024-04-02T04:47:36.267 INFO:tasks.workunit.client.0.smithi070.stdout:0/157: link d4/d5/f14 d4/f25 0 2024-04-02T04:47:36.268 INFO:tasks.workunit.client.1.smithi179.stdout:3/7: getdents . 0 2024-04-02T04:47:36.271 INFO:tasks.workunit.client.1.smithi179.stdout:1/5: link c0 c1 0 2024-04-02T04:47:36.272 INFO:tasks.workunit.client.1.smithi179.stdout:5/0: creat f0 x:0 0 0 2024-04-02T04:47:36.274 INFO:tasks.workunit.client.1.smithi179.stdout:3/8: getdents . 0 2024-04-02T04:47:36.274 INFO:tasks.workunit.client.1.smithi179.stdout:3/9: dwrite - no filename 2024-04-02T04:47:36.276 INFO:tasks.workunit.client.1.smithi179.stdout:5/1: rename f0 to f1 0 2024-04-02T04:47:36.276 INFO:tasks.workunit.client.1.smithi179.stdout:5/2: chown f1 109663 1 2024-04-02T04:47:36.276 INFO:tasks.workunit.client.1.smithi179.stdout:5/3: chown f1 4092 1 2024-04-02T04:47:36.277 INFO:tasks.workunit.client.1.smithi179.stdout:8/0: chown . 313 1 2024-04-02T04:47:36.277 INFO:tasks.workunit.client.1.smithi179.stdout:8/1: dread - no filename 2024-04-02T04:47:36.277 INFO:tasks.workunit.client.1.smithi179.stdout:8/2: dwrite - no filename 2024-04-02T04:47:36.277 INFO:tasks.workunit.client.1.smithi179.stdout:8/3: dwrite - no filename 2024-04-02T04:47:36.277 INFO:tasks.workunit.client.1.smithi179.stdout:6/0: creat f0 x:0 0 0 2024-04-02T04:47:36.278 INFO:tasks.workunit.client.1.smithi179.stdout:1/6: chown c0 15861400 1 2024-04-02T04:47:36.293 INFO:tasks.workunit.client.0.smithi070.stdout:0/158: dread d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:36.294 INFO:tasks.workunit.client.0.smithi070.stdout:0/159: mknod d4/d5/d13/d17/c26 0 2024-04-02T04:47:36.296 INFO:tasks.workunit.client.0.smithi070.stdout:0/160: mkdir d4/d5/df/d18/d27 0 2024-04-02T04:47:36.297 INFO:tasks.workunit.client.0.smithi070.stdout:0/161: write d4/f1b [1563675,67726] 0 2024-04-02T04:47:36.298 INFO:tasks.workunit.client.0.smithi070.stdout:0/162: read d4/d5/f15 [2883475,78814] 0 2024-04-02T04:47:36.304 INFO:tasks.workunit.client.0.smithi070.stdout:0/163: write d4/f25 [1090884,110331] 0 2024-04-02T04:47:36.307 INFO:tasks.workunit.client.0.smithi070.stdout:0/164: symlink d4/l28 0 2024-04-02T04:47:36.310 INFO:tasks.workunit.client.0.smithi070.stdout:0/165: rename d4/d5/df to d4/d5/da/d1a/d29 0 2024-04-02T04:47:36.315 INFO:tasks.workunit.client.0.smithi070.stdout:0/166: rename d4/d5/da/d1a to d4/d5/d13/d2a 0 2024-04-02T04:47:36.319 INFO:tasks.workunit.client.0.smithi070.stdout:0/167: write d4/d5/f15 [4245239,11540] 0 2024-04-02T04:47:36.319 INFO:tasks.workunit.client.0.smithi070.stdout:0/168: write f0 [10347218,114947] 0 2024-04-02T04:47:36.320 INFO:tasks.workunit.client.0.smithi070.stdout:0/169: readlink d4/d5/ld 0 2024-04-02T04:47:36.322 INFO:tasks.workunit.client.0.smithi070.stdout:0/170: creat d4/d5/d13/f2b x:0 0 0 2024-04-02T04:47:36.417 INFO:tasks.workunit.client.1.smithi179.stdout:3/10: mkdir d0 0 2024-04-02T04:47:36.417 INFO:tasks.workunit.client.1.smithi179.stdout:6/1: creat f1 x:0 0 0 2024-04-02T04:47:36.418 INFO:tasks.workunit.client.1.smithi179.stdout:0/1: fsync f0 0 2024-04-02T04:47:36.438 INFO:tasks.workunit.client.1.smithi179.stdout:9/0: mkdir d0 0 2024-04-02T04:47:36.438 INFO:tasks.workunit.client.1.smithi179.stdout:9/1: chown d0 121579 1 2024-04-02T04:47:36.439 INFO:tasks.workunit.client.1.smithi179.stdout:6/2: creat f2 x:0 0 0 2024-04-02T04:47:36.440 INFO:tasks.workunit.client.1.smithi179.stdout:1/7: chown c1 79587909 1 2024-04-02T04:47:36.442 INFO:tasks.workunit.client.1.smithi179.stdout:0/2: mkdir d1 0 2024-04-02T04:47:36.444 INFO:tasks.workunit.client.1.smithi179.stdout:6/3: mkdir d3 0 2024-04-02T04:47:36.447 INFO:tasks.workunit.client.1.smithi179.stdout:9/2: mkdir d0/d1 0 2024-04-02T04:47:36.447 INFO:tasks.workunit.client.1.smithi179.stdout:9/3: dread - no filename 2024-04-02T04:47:36.447 INFO:tasks.workunit.client.1.smithi179.stdout:9/4: read - no filename 2024-04-02T04:47:36.447 INFO:tasks.workunit.client.1.smithi179.stdout:9/5: chown d0/d1 339 1 2024-04-02T04:47:36.447 INFO:tasks.workunit.client.1.smithi179.stdout:9/6: write - no filename 2024-04-02T04:47:36.448 INFO:tasks.workunit.client.1.smithi179.stdout:3/11: rmdir d0 0 2024-04-02T04:47:36.449 INFO:tasks.workunit.client.1.smithi179.stdout:0/3: dwrite f0 [0,4194304] 0 2024-04-02T04:47:36.452 INFO:tasks.workunit.client.1.smithi179.stdout:6/4: creat d3/f4 x:0 0 0 2024-04-02T04:47:36.453 INFO:tasks.workunit.client.1.smithi179.stdout:6/5: stat d3/f4 0 2024-04-02T04:47:36.454 INFO:tasks.workunit.client.1.smithi179.stdout:0/4: dread f0 [0,4194304] 0 2024-04-02T04:47:36.454 INFO:tasks.workunit.client.1.smithi179.stdout:3/12: mknod c1 0 2024-04-02T04:47:36.458 INFO:tasks.workunit.client.1.smithi179.stdout:1/8: link c1 c2 0 2024-04-02T04:47:36.459 INFO:tasks.workunit.client.1.smithi179.stdout:9/7: symlink d0/d1/l2 0 2024-04-02T04:47:36.459 INFO:tasks.workunit.client.1.smithi179.stdout:9/8: write - no filename 2024-04-02T04:47:36.460 INFO:tasks.workunit.client.1.smithi179.stdout:9/9: chown d0 3 1 2024-04-02T04:47:36.461 INFO:tasks.workunit.client.1.smithi179.stdout:6/6: dwrite f2 [0,4194304] 0 2024-04-02T04:47:36.461 INFO:tasks.workunit.client.1.smithi179.stdout:6/7: readlink - no filename 2024-04-02T04:47:36.461 INFO:tasks.workunit.client.1.smithi179.stdout:3/13: creat f2 x:0 0 0 2024-04-02T04:47:36.462 INFO:tasks.workunit.client.1.smithi179.stdout:3/14: fdatasync f2 0 2024-04-02T04:47:36.462 INFO:tasks.workunit.client.1.smithi179.stdout:3/15: read - f2 zero size 2024-04-02T04:47:36.462 INFO:tasks.workunit.client.1.smithi179.stdout:3/16: dread - f2 zero size 2024-04-02T04:47:36.471 INFO:tasks.workunit.client.1.smithi179.stdout:1/9: mkdir d3 0 2024-04-02T04:47:36.471 INFO:tasks.workunit.client.1.smithi179.stdout:1/10: dwrite - no filename 2024-04-02T04:47:36.474 INFO:tasks.workunit.client.1.smithi179.stdout:9/10: rename d0/d1/l2 to d0/l3 0 2024-04-02T04:47:36.476 INFO:tasks.workunit.client.1.smithi179.stdout:6/8: mknod d3/c5 0 2024-04-02T04:47:36.481 INFO:tasks.workunit.client.1.smithi179.stdout:9/11: creat d0/d1/f4 x:0 0 0 2024-04-02T04:47:36.483 INFO:tasks.workunit.client.1.smithi179.stdout:6/9: symlink d3/l6 0 2024-04-02T04:47:36.486 INFO:tasks.workunit.client.1.smithi179.stdout:6/10: symlink d3/l7 0 2024-04-02T04:47:36.486 INFO:tasks.workunit.client.1.smithi179.stdout:0/5: write f0 [5179219,27499] 0 2024-04-02T04:47:36.487 INFO:tasks.workunit.client.1.smithi179.stdout:1/11: link c1 d3/c4 0 2024-04-02T04:47:36.488 INFO:tasks.workunit.client.1.smithi179.stdout:9/12: dwrite d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:36.492 INFO:tasks.workunit.client.1.smithi179.stdout:6/11: creat d3/f8 x:0 0 0 2024-04-02T04:47:36.498 INFO:tasks.workunit.client.1.smithi179.stdout:3/17: fsync f2 0 2024-04-02T04:47:36.500 INFO:tasks.workunit.client.1.smithi179.stdout:6/12: rename f1 to d3/f9 0 2024-04-02T04:47:36.502 INFO:tasks.workunit.client.1.smithi179.stdout:0/6: creat d1/f2 x:0 0 0 2024-04-02T04:47:36.502 INFO:tasks.workunit.client.1.smithi179.stdout:0/7: fsync f0 0 2024-04-02T04:47:36.503 INFO:tasks.workunit.client.1.smithi179.stdout:0/8: read - d1/f2 zero size 2024-04-02T04:47:36.503 INFO:tasks.workunit.client.1.smithi179.stdout:0/9: truncate d1/f2 949188 0 2024-04-02T04:47:36.504 INFO:tasks.workunit.client.1.smithi179.stdout:1/12: mkdir d3/d5 0 2024-04-02T04:47:36.504 INFO:tasks.workunit.client.1.smithi179.stdout:0/10: read f0 [237275,68520] 0 2024-04-02T04:47:36.504 INFO:tasks.workunit.client.1.smithi179.stdout:0/11: chown d1/f2 334143 1 2024-04-02T04:47:36.510 INFO:tasks.workunit.client.1.smithi179.stdout:3/18: rename f2 to f3 0 2024-04-02T04:47:36.510 INFO:tasks.workunit.client.1.smithi179.stdout:3/19: readlink - no filename 2024-04-02T04:47:36.510 INFO:tasks.workunit.client.1.smithi179.stdout:3/20: chown f3 46 1 2024-04-02T04:47:36.510 INFO:tasks.workunit.client.1.smithi179.stdout:3/21: truncate f3 173930 0 2024-04-02T04:47:36.521 INFO:tasks.workunit.client.1.smithi179.stdout:3/22: write f3 [82571,42210] 0 2024-04-02T04:47:36.523 INFO:tasks.workunit.client.1.smithi179.stdout:1/13: symlink d3/d5/l6 0 2024-04-02T04:47:36.523 INFO:tasks.workunit.client.1.smithi179.stdout:1/14: truncate - no filename 2024-04-02T04:47:36.528 INFO:tasks.workunit.client.1.smithi179.stdout:6/13: rename d3/f9 to d3/fa 0 2024-04-02T04:47:36.530 INFO:tasks.workunit.client.1.smithi179.stdout:9/13: getdents d0/d1 0 2024-04-02T04:47:36.532 INFO:tasks.workunit.client.1.smithi179.stdout:3/23: creat f4 x:0 0 0 2024-04-02T04:47:36.532 INFO:tasks.workunit.client.1.smithi179.stdout:3/24: chown c1 453928452 1 2024-04-02T04:47:36.533 INFO:tasks.workunit.client.1.smithi179.stdout:9/14: dread d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:36.533 INFO:tasks.workunit.client.1.smithi179.stdout:1/15: stat c2 0 2024-04-02T04:47:36.534 INFO:tasks.workunit.client.1.smithi179.stdout:1/16: truncate - no filename 2024-04-02T04:47:36.534 INFO:tasks.workunit.client.1.smithi179.stdout:1/17: chown c2 880401844 1 2024-04-02T04:47:36.534 INFO:tasks.workunit.client.1.smithi179.stdout:1/18: dread - no filename 2024-04-02T04:47:36.534 INFO:tasks.workunit.client.1.smithi179.stdout:1/19: dread - no filename 2024-04-02T04:47:36.538 INFO:tasks.workunit.client.1.smithi179.stdout:9/15: creat d0/f5 x:0 0 0 2024-04-02T04:47:36.541 INFO:tasks.workunit.client.1.smithi179.stdout:1/20: creat d3/d5/f7 x:0 0 0 2024-04-02T04:47:36.546 INFO:tasks.workunit.client.1.smithi179.stdout:9/16: mknod d0/d1/c6 0 2024-04-02T04:47:36.548 INFO:tasks.workunit.client.1.smithi179.stdout:1/21: symlink d3/l8 0 2024-04-02T04:47:36.549 INFO:tasks.workunit.client.1.smithi179.stdout:1/22: write d3/d5/f7 [664182,123396] 0 2024-04-02T04:47:36.551 INFO:tasks.workunit.client.1.smithi179.stdout:9/17: creat d0/d1/f7 x:0 0 0 2024-04-02T04:47:36.553 INFO:tasks.workunit.client.1.smithi179.stdout:0/12: truncate d1/f2 643649 0 2024-04-02T04:47:36.557 INFO:tasks.workunit.client.1.smithi179.stdout:0/13: symlink d1/l3 0 2024-04-02T04:47:36.557 INFO:tasks.workunit.client.1.smithi179.stdout:9/18: dread d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:36.558 INFO:tasks.workunit.client.1.smithi179.stdout:9/19: rename d0 to d0/d1/d8 22 2024-04-02T04:47:36.560 INFO:tasks.workunit.client.1.smithi179.stdout:0/14: rename d1/l3 to d1/l4 0 2024-04-02T04:47:36.564 INFO:tasks.workunit.client.1.smithi179.stdout:9/20: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:47:36.566 INFO:tasks.workunit.client.1.smithi179.stdout:9/21: mknod d0/c9 0 2024-04-02T04:47:36.574 INFO:tasks.workunit.client.1.smithi179.stdout:9/22: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:47:36.575 INFO:tasks.workunit.client.1.smithi179.stdout:9/23: write d0/d1/f7 [261079,29991] 0 2024-04-02T04:47:36.581 INFO:tasks.workunit.client.1.smithi179.stdout:9/24: dread d0/f5 [0,4194304] 0 2024-04-02T04:47:36.591 INFO:tasks.workunit.client.1.smithi179.stdout:9/25: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:47:36.591 INFO:tasks.workunit.client.1.smithi179.stdout:9/26: rename d0/d1 to d0/d1/da 22 2024-04-02T04:47:36.591 INFO:tasks.workunit.client.1.smithi179.stdout:9/27: stat d0 0 2024-04-02T04:47:36.598 INFO:tasks.workunit.client.1.smithi179.stdout:9/28: dwrite d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:36.602 INFO:tasks.workunit.client.1.smithi179.stdout:9/29: mkdir d0/d1/db 0 2024-04-02T04:47:36.603 INFO:tasks.workunit.client.1.smithi179.stdout:9/30: stat d0/c9 0 2024-04-02T04:47:36.606 INFO:tasks.workunit.client.1.smithi179.stdout:9/31: dread d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:36.608 INFO:tasks.workunit.client.1.smithi179.stdout:0/15: write d1/f2 [413405,21014] 0 2024-04-02T04:47:36.610 INFO:tasks.workunit.client.1.smithi179.stdout:9/32: mknod d0/d1/db/cc 0 2024-04-02T04:47:36.614 INFO:tasks.workunit.client.1.smithi179.stdout:9/33: dread d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:36.615 INFO:tasks.workunit.client.1.smithi179.stdout:0/16: truncate f0 97693 0 2024-04-02T04:47:36.619 INFO:tasks.workunit.client.1.smithi179.stdout:9/34: rmdir d0/d1/db 39 2024-04-02T04:47:36.619 INFO:tasks.workunit.client.1.smithi179.stdout:9/35: read d0/f5 [1670543,129831] 0 2024-04-02T04:47:36.626 INFO:tasks.workunit.client.1.smithi179.stdout:9/36: link d0/d1/f7 d0/fd 0 2024-04-02T04:47:36.652 INFO:tasks.workunit.client.0.smithi070.stdout:0/171: truncate d4/d5/da/f8 1179959 0 2024-04-02T04:47:36.652 INFO:tasks.workunit.client.0.smithi070.stdout:0/172: stat d4/l28 0 2024-04-02T04:47:36.682 INFO:tasks.workunit.client.1.smithi179.stdout:0/17: write f0 [1004631,26995] 0 2024-04-02T04:47:36.688 INFO:tasks.workunit.client.1.smithi179.stdout:0/18: dwrite d1/f2 [0,4194304] 0 2024-04-02T04:47:36.693 INFO:tasks.workunit.client.1.smithi179.stdout:0/19: dwrite f0 [0,4194304] 0 2024-04-02T04:47:36.694 INFO:tasks.workunit.client.1.smithi179.stdout:0/20: chown f0 50 1 2024-04-02T04:47:36.698 INFO:tasks.workunit.client.1.smithi179.stdout:0/21: creat d1/f5 x:0 0 0 2024-04-02T04:47:36.711 INFO:tasks.workunit.client.0.smithi070.stdout:0/173: unlink d4/d5/d13/d17/c26 0 2024-04-02T04:47:36.713 INFO:tasks.workunit.client.0.smithi070.stdout:0/174: mknod d4/d5/da/c2c 0 2024-04-02T04:47:36.714 INFO:tasks.workunit.client.0.smithi070.stdout:0/175: read d4/d5/d13/f19 [2495743,27664] 0 2024-04-02T04:47:36.715 INFO:tasks.workunit.client.1.smithi179.stdout:0/22: fsync d1/f2 0 2024-04-02T04:47:36.715 INFO:tasks.workunit.client.1.smithi179.stdout:0/23: chown d1 10139086 1 2024-04-02T04:47:36.716 INFO:tasks.workunit.client.1.smithi179.stdout:0/24: creat d1/f6 x:0 0 0 2024-04-02T04:47:36.721 INFO:tasks.workunit.client.0.smithi070.stdout:0/176: rename d4/d5/d13/d2a/d29/d18 to d4/d5/da/d2d 0 2024-04-02T04:47:36.727 INFO:tasks.workunit.client.0.smithi070.stdout:0/177: dwrite d4/d5/d13/f1f [0,4194304] 0 2024-04-02T04:47:36.729 INFO:tasks.workunit.client.0.smithi070.stdout:0/178: write d4/d5/f14 [966867,69987] 0 2024-04-02T04:47:36.735 INFO:tasks.workunit.client.0.smithi070.stdout:0/179: getdents d4/d5 0 2024-04-02T04:47:36.739 INFO:tasks.workunit.client.1.smithi179.stdout:0/25: getdents d1 0 2024-04-02T04:47:36.739 INFO:tasks.workunit.client.1.smithi179.stdout:0/26: readlink d1/l4 0 2024-04-02T04:47:36.740 INFO:tasks.workunit.client.0.smithi070.stdout:0/180: dwrite d4/d5/f15 [0,4194304] 0 2024-04-02T04:47:36.741 INFO:tasks.workunit.client.1.smithi179.stdout:0/27: mknod d1/c7 0 2024-04-02T04:47:36.744 INFO:tasks.workunit.client.1.smithi179.stdout:0/28: dread d1/f2 [0,4194304] 0 2024-04-02T04:47:36.744 INFO:tasks.workunit.client.1.smithi179.stdout:0/29: fdatasync d1/f5 0 2024-04-02T04:47:36.748 INFO:tasks.workunit.client.1.smithi179.stdout:0/30: link d1/f2 d1/f8 0 2024-04-02T04:47:36.754 INFO:tasks.workunit.client.0.smithi070.stdout:0/181: truncate d4/d5/d13/f1f 1913816 0 2024-04-02T04:47:36.756 INFO:tasks.workunit.client.1.smithi179.stdout:0/31: dwrite d1/f8 [0,4194304] 0 2024-04-02T04:47:36.763 INFO:tasks.workunit.client.1.smithi179.stdout:0/32: dwrite d1/f5 [0,4194304] 0 2024-04-02T04:47:36.763 INFO:tasks.workunit.client.1.smithi179.stdout:0/33: dread - d1/f6 zero size 2024-04-02T04:47:36.769 INFO:tasks.workunit.client.1.smithi179.stdout:0/34: getdents d1 0 2024-04-02T04:47:36.772 INFO:tasks.workunit.client.1.smithi179.stdout:0/35: rename d1/f8 to d1/f9 0 2024-04-02T04:47:36.773 INFO:tasks.workunit.client.1.smithi179.stdout:0/36: chown d1/f9 5 1 2024-04-02T04:47:36.777 INFO:tasks.workunit.client.0.smithi070.stdout:0/182: dwrite d4/d5/d13/f1f [0,4194304] 0 2024-04-02T04:47:36.778 INFO:tasks.workunit.client.0.smithi070.stdout:0/183: write d4/d5/f14 [2220613,43340] 0 2024-04-02T04:47:36.783 INFO:tasks.workunit.client.1.smithi179.stdout:0/37: dwrite d1/f2 [0,4194304] 0 2024-04-02T04:47:36.783 INFO:tasks.workunit.client.0.smithi070.stdout:0/184: link d4/d5/ld d4/d5/d13/d23/l2e 0 2024-04-02T04:47:36.784 INFO:tasks.workunit.client.1.smithi179.stdout:0/38: write f0 [2049741,35633] 0 2024-04-02T04:47:36.785 INFO:tasks.workunit.client.0.smithi070.stdout:0/185: write f0 [97127,66188] 0 2024-04-02T04:47:36.789 INFO:tasks.workunit.client.1.smithi179.stdout:0/39: mkdir d1/da 0 2024-04-02T04:47:36.789 INFO:tasks.workunit.client.1.smithi179.stdout:0/40: readlink d1/l4 0 2024-04-02T04:47:36.793 INFO:tasks.workunit.client.0.smithi070.stdout:0/186: dwrite d4/d5/d13/f2b [0,4194304] 0 2024-04-02T04:47:36.794 INFO:tasks.workunit.client.1.smithi179.stdout:0/41: dread f0 [0,4194304] 0 2024-04-02T04:47:36.799 INFO:tasks.workunit.client.0.smithi070.stdout:0/187: getdents d4/d5 0 2024-04-02T04:47:36.801 INFO:tasks.workunit.client.1.smithi179.stdout:0/42: dwrite d1/f6 [0,4194304] 0 2024-04-02T04:47:36.802 INFO:tasks.workunit.client.0.smithi070.stdout:0/188: creat d4/d5/d13/d2a/f2f x:0 0 0 2024-04-02T04:47:36.805 INFO:tasks.workunit.client.0.smithi070.stdout:0/189: unlink d4/d5/d13/d2a/f2f 0 2024-04-02T04:47:36.807 INFO:tasks.workunit.client.0.smithi070.stdout:0/190: readlink d4/d5/ld 0 2024-04-02T04:47:36.811 INFO:tasks.workunit.client.1.smithi179.stdout:0/43: dwrite d1/f5 [4194304,4194304] 0 2024-04-02T04:47:36.812 INFO:tasks.workunit.client.0.smithi070.stdout:0/191: dwrite d4/f25 [0,4194304] 0 2024-04-02T04:47:36.815 INFO:tasks.workunit.client.0.smithi070.stdout:0/192: mknod d4/d5/da/c30 0 2024-04-02T04:47:36.817 INFO:tasks.workunit.client.0.smithi070.stdout:0/193: getdents d4/d5/d13/d23 0 2024-04-02T04:47:36.820 INFO:tasks.workunit.client.0.smithi070.stdout:0/194: rename d4/d5/d21/l24 to d4/d5/d13/d23/l31 0 2024-04-02T04:47:36.821 INFO:tasks.workunit.client.0.smithi070.stdout:0/195: write d4/d5/f15 [1533942,17034] 0 2024-04-02T04:47:36.821 INFO:tasks.workunit.client.1.smithi179.stdout:0/44: dwrite d1/f2 [0,4194304] 0 2024-04-02T04:47:36.823 INFO:tasks.workunit.client.0.smithi070.stdout:0/196: creat d4/d5/d13/f32 x:0 0 0 2024-04-02T04:47:36.825 INFO:tasks.workunit.client.1.smithi179.stdout:0/45: mkdir d1/db 0 2024-04-02T04:47:36.826 INFO:tasks.workunit.client.0.smithi070.stdout:0/197: dread d4/d5/da/f8 [0,4194304] 0 2024-04-02T04:47:36.827 INFO:tasks.workunit.client.0.smithi070.stdout:0/198: readlink d4/d5/d13/d23/l2e 0 2024-04-02T04:47:36.832 INFO:tasks.workunit.client.1.smithi179.stdout:0/46: creat d1/db/fc x:0 0 0 2024-04-02T04:47:36.832 INFO:tasks.workunit.client.1.smithi179.stdout:0/47: chown d1 34 1 2024-04-02T04:47:36.834 INFO:tasks.workunit.client.1.smithi179.stdout:0/48: symlink d1/db/ld 0 2024-04-02T04:47:36.835 INFO:tasks.workunit.client.0.smithi070.stdout:0/199: dwrite d4/d5/d13/f1f [0,4194304] 0 2024-04-02T04:47:36.836 INFO:tasks.workunit.client.0.smithi070.stdout:0/200: truncate d4/d5/d13/f32 497740 0 2024-04-02T04:47:36.837 INFO:tasks.workunit.client.0.smithi070.stdout:0/201: chown d4/d5/da/c2c 15 1 2024-04-02T04:47:36.839 INFO:tasks.workunit.client.0.smithi070.stdout:0/202: symlink d4/d5/d21/l33 0 2024-04-02T04:47:36.842 INFO:tasks.workunit.client.0.smithi070.stdout:0/203: dread d4/d5/da/f8 [0,4194304] 0 2024-04-02T04:47:36.848 INFO:tasks.workunit.client.0.smithi070.stdout:0/204: dwrite d4/d5/d13/f2b [0,4194304] 0 2024-04-02T04:47:36.850 INFO:tasks.workunit.client.0.smithi070.stdout:0/205: symlink d4/d5/d13/d23/l34 0 2024-04-02T04:47:36.851 INFO:tasks.workunit.client.0.smithi070.stdout:0/206: fdatasync d4/d5/d13/f19 0 2024-04-02T04:47:36.854 INFO:tasks.workunit.client.0.smithi070.stdout:0/207: link d4/l28 d4/d5/d21/l35 0 2024-04-02T04:47:36.857 INFO:tasks.workunit.client.0.smithi070.stdout:0/208: creat d4/d5/d13/d23/f36 x:0 0 0 2024-04-02T04:47:36.858 INFO:tasks.workunit.client.0.smithi070.stdout:0/209: chown d4/d5/d13/d17/c22 173 1 2024-04-02T04:47:36.859 INFO:tasks.workunit.client.0.smithi070.stdout:0/210: rename d4/d5 to d4/d5/d13/d37 22 2024-04-02T04:47:36.962 INFO:tasks.workunit.client.0.smithi070.stdout:0/211: truncate d4/d5/da/f8 144674 0 2024-04-02T04:47:36.968 INFO:tasks.workunit.client.1.smithi179.stdout:0/49: rmdir d1/db 39 2024-04-02T04:47:36.969 INFO:tasks.workunit.client.0.smithi070.stdout:0/212: dwrite d4/f1b [0,4194304] 0 2024-04-02T04:47:36.976 INFO:tasks.workunit.client.0.smithi070.stdout:0/213: dwrite d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:37.035 INFO:tasks.workunit.client.0.smithi070.stdout:0/214: dread d4/d5/da/f8 [0,4194304] 0 2024-04-02T04:47:37.036 INFO:tasks.workunit.client.0.smithi070.stdout:0/215: chown d4/d5/f14 169440273 1 2024-04-02T04:47:37.039 INFO:tasks.workunit.client.0.smithi070.stdout:0/216: fsync d4/d5/da/f8 0 2024-04-02T04:47:37.042 INFO:tasks.workunit.client.0.smithi070.stdout:0/217: unlink d4/d5/da/c2c 0 2024-04-02T04:47:37.044 INFO:tasks.workunit.client.0.smithi070.stdout:0/218: dread d4/d5/d13/d2a/f20 [0,4194304] 0 2024-04-02T04:47:37.047 INFO:tasks.workunit.client.0.smithi070.stdout:0/219: symlink d4/d5/l38 0 2024-04-02T04:47:37.051 INFO:tasks.workunit.client.0.smithi070.stdout:0/220: symlink d4/l39 0 2024-04-02T04:47:37.058 INFO:tasks.workunit.client.0.smithi070.stdout:0/221: getdents d4/d5/d21 0 2024-04-02T04:47:37.061 INFO:tasks.workunit.client.0.smithi070.stdout:0/222: dread d4/d5/d13/f1f [0,4194304] 0 2024-04-02T04:47:37.064 INFO:tasks.workunit.client.0.smithi070.stdout:0/223: mkdir d4/d5/d13/d2a/d3a 0 2024-04-02T04:47:37.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:36 smithi179 ceph-mon[36603]: pgmap v7: 65 pgs: 65 active+clean; 222 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-04-02T04:47:37.095 INFO:tasks.workunit.client.0.smithi070.stdout:0/224: write d4/d5/d13/d2a/f20 [856314,42276] 0 2024-04-02T04:47:37.096 INFO:tasks.workunit.client.0.smithi070.stdout:0/225: read d4/d5/d13/f2b [3118585,128384] 0 2024-04-02T04:47:37.102 INFO:tasks.workunit.client.0.smithi070.stdout:0/226: getdents d4/d5/d21 0 2024-04-02T04:47:37.102 INFO:tasks.workunit.client.0.smithi070.stdout:0/227: chown d4/d5/d13/d23/l34 1 1 2024-04-02T04:47:37.115 INFO:tasks.workunit.client.1.smithi179.stdout:0/50: rmdir d1/db 39 2024-04-02T04:47:37.115 INFO:tasks.workunit.client.1.smithi179.stdout:0/51: chown d1/l4 12794 1 2024-04-02T04:47:37.117 INFO:tasks.workunit.client.1.smithi179.stdout:0/52: truncate d1/db/fc 441026 0 2024-04-02T04:47:37.120 INFO:tasks.workunit.client.1.smithi179.stdout:0/53: link d1/db/fc d1/fe 0 2024-04-02T04:47:37.121 INFO:tasks.workunit.client.1.smithi179.stdout:0/54: mkdir d1/da/df 0 2024-04-02T04:47:37.128 INFO:tasks.workunit.client.1.smithi179.stdout:0/55: dwrite d1/fe [0,4194304] 0 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:8/51: sync 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:5/38: sync 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:7/34: sync 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:3/0: sync 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:3/1: fsync - no filename 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:1/54: sync 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:6/42: sync 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:4/45: sync 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:9/25: sync 2024-04-02T04:47:37.148 INFO:tasks.workunit.client.0.smithi070.stdout:2/26: sync 2024-04-02T04:47:37.149 INFO:tasks.workunit.client.0.smithi070.stdout:9/26: read f3 [2462605,56090] 0 2024-04-02T04:47:37.149 INFO:tasks.workunit.client.0.smithi070.stdout:5/39: write f2 [2600793,16212] 0 2024-04-02T04:47:37.149 INFO:tasks.workunit.client.0.smithi070.stdout:8/52: fdatasync d2/d3/fd 0 2024-04-02T04:47:37.152 INFO:tasks.workunit.client.0.smithi070.stdout:5/40: truncate d5/fa 421148 0 2024-04-02T04:47:37.152 INFO:tasks.workunit.client.0.smithi070.stdout:1/55: readlink d0/d1/le 0 2024-04-02T04:47:37.152 INFO:tasks.workunit.client.0.smithi070.stdout:5/41: stat f4 0 2024-04-02T04:47:37.154 INFO:tasks.workunit.client.0.smithi070.stdout:4/46: dread d1/fc [0,4194304] 0 2024-04-02T04:47:37.155 INFO:tasks.workunit.client.0.smithi070.stdout:1/56: truncate d0/d1/fd 992581 0 2024-04-02T04:47:37.155 INFO:tasks.workunit.client.0.smithi070.stdout:5/42: dread d5/fa [0,4194304] 0 2024-04-02T04:47:37.156 INFO:tasks.workunit.client.0.smithi070.stdout:5/43: stat d5/d9 0 2024-04-02T04:47:37.156 INFO:tasks.workunit.client.0.smithi070.stdout:1/57: readlink d0/d9/lf 0 2024-04-02T04:47:37.156 INFO:tasks.workunit.client.0.smithi070.stdout:9/27: dread f3 [0,4194304] 0 2024-04-02T04:47:37.157 INFO:tasks.workunit.client.0.smithi070.stdout:5/44: fsync f4 0 2024-04-02T04:47:37.157 INFO:tasks.workunit.client.0.smithi070.stdout:2/27: rename l1 to d2/l3 0 2024-04-02T04:47:37.158 INFO:tasks.workunit.client.0.smithi070.stdout:3/2: creat f0 x:0 0 0 2024-04-02T04:47:37.160 INFO:tasks.workunit.client.0.smithi070.stdout:9/28: dread f3 [0,4194304] 0 2024-04-02T04:47:37.160 INFO:tasks.workunit.client.0.smithi070.stdout:8/53: unlink d2/d8/da/db/lf 0 2024-04-02T04:47:37.161 INFO:tasks.workunit.client.0.smithi070.stdout:6/43: symlink d0/l12 0 2024-04-02T04:47:37.163 INFO:tasks.workunit.client.0.smithi070.stdout:8/54: dread f1 [0,4194304] 0 2024-04-02T04:47:37.165 INFO:tasks.workunit.client.0.smithi070.stdout:7/35: creat d7/f8 x:0 0 0 2024-04-02T04:47:37.166 INFO:tasks.workunit.client.0.smithi070.stdout:4/47: creat d1/f12 x:0 0 0 2024-04-02T04:47:37.167 INFO:tasks.workunit.client.0.smithi070.stdout:4/48: chown d1 365 1 2024-04-02T04:47:37.168 INFO:tasks.workunit.client.0.smithi070.stdout:5/45: mknod d5/cc 0 2024-04-02T04:47:37.169 INFO:tasks.workunit.client.0.smithi070.stdout:5/46: dread - f4 zero size 2024-04-02T04:47:37.172 INFO:tasks.workunit.client.0.smithi070.stdout:2/28: creat d2/f4 x:0 0 0 2024-04-02T04:47:37.173 INFO:tasks.workunit.client.0.smithi070.stdout:3/3: creat f1 x:0 0 0 2024-04-02T04:47:37.173 INFO:tasks.workunit.client.0.smithi070.stdout:4/49: dwrite d1/fe [0,4194304] 0 2024-04-02T04:47:37.175 INFO:tasks.workunit.client.0.smithi070.stdout:9/29: rename c2 to d4/c5 0 2024-04-02T04:47:37.179 INFO:tasks.workunit.client.0.smithi070.stdout:7/36: mkdir d7/d9 0 2024-04-02T04:47:37.181 INFO:tasks.workunit.client.0.smithi070.stdout:2/29: dwrite d2/f4 [0,4194304] 0 2024-04-02T04:47:37.183 INFO:tasks.workunit.client.0.smithi070.stdout:3/4: mknod c2 0 2024-04-02T04:47:37.184 INFO:tasks.workunit.client.0.smithi070.stdout:3/5: fdatasync f0 0 2024-04-02T04:47:37.185 INFO:tasks.workunit.client.0.smithi070.stdout:3/6: dread - f0 zero size 2024-04-02T04:47:37.186 INFO:tasks.workunit.client.0.smithi070.stdout:9/30: rename c0 to d4/c6 0 2024-04-02T04:47:37.186 INFO:tasks.workunit.client.0.smithi070.stdout:3/7: dread - f0 zero size 2024-04-02T04:47:37.187 INFO:tasks.workunit.client.0.smithi070.stdout:9/31: stat f3 0 2024-04-02T04:47:37.190 INFO:tasks.workunit.client.0.smithi070.stdout:2/30: mkdir d2/d5 0 2024-04-02T04:47:37.191 INFO:tasks.workunit.client.0.smithi070.stdout:7/37: dwrite d7/f8 [0,4194304] 0 2024-04-02T04:47:37.192 INFO:tasks.workunit.client.0.smithi070.stdout:4/50: link d1/fd d1/d3/d9/f13 0 2024-04-02T04:47:37.192 INFO:tasks.workunit.client.0.smithi070.stdout:2/31: stat d2/d5 0 2024-04-02T04:47:37.194 INFO:tasks.workunit.client.0.smithi070.stdout:3/8: symlink l3 0 2024-04-02T04:47:37.195 INFO:tasks.workunit.client.0.smithi070.stdout:2/32: read d2/f4 [3570504,15100] 0 2024-04-02T04:47:37.195 INFO:tasks.workunit.client.0.smithi070.stdout:0/228: dwrite d4/d5/da/f8 [0,4194304] 0 2024-04-02T04:47:37.196 INFO:tasks.workunit.client.0.smithi070.stdout:3/9: readlink l3 0 2024-04-02T04:47:37.200 INFO:tasks.workunit.client.0.smithi070.stdout:3/10: creat f4 x:0 0 0 2024-04-02T04:47:37.200 INFO:tasks.workunit.client.0.smithi070.stdout:4/51: dread d1/d3/d9/ff [0,4194304] 0 2024-04-02T04:47:37.201 INFO:tasks.workunit.client.0.smithi070.stdout:0/229: chown d4/d5/d13/d2a/d29/c11 5565809 1 2024-04-02T04:47:37.203 INFO:tasks.workunit.client.0.smithi070.stdout:9/32: dwrite f3 [0,4194304] 0 2024-04-02T04:47:37.203 INFO:tasks.workunit.client.1.smithi179.stdout:0/56: getdents d1 0 2024-04-02T04:47:37.203 INFO:tasks.workunit.client.0.smithi070.stdout:0/230: chown d4/d5/d13/d17 49782854 1 2024-04-02T04:47:37.205 INFO:tasks.workunit.client.0.smithi070.stdout:7/38: dwrite d7/f8 [0,4194304] 0 2024-04-02T04:47:37.206 INFO:tasks.workunit.client.1.smithi179.stdout:0/57: rmdir d1 39 2024-04-02T04:47:37.210 INFO:tasks.workunit.client.0.smithi070.stdout:9/33: creat d4/f7 x:0 0 0 2024-04-02T04:47:37.211 INFO:tasks.workunit.client.0.smithi070.stdout:3/11: link f0 f5 0 2024-04-02T04:47:37.212 INFO:tasks.workunit.client.0.smithi070.stdout:9/34: write f3 [987392,98790] 0 2024-04-02T04:47:37.213 INFO:tasks.workunit.client.0.smithi070.stdout:4/52: dwrite d1/fb [0,4194304] 0 2024-04-02T04:47:37.220 INFO:tasks.workunit.client.1.smithi179.stdout:0/58: symlink d1/l10 0 2024-04-02T04:47:37.221 INFO:tasks.workunit.client.0.smithi070.stdout:3/12: mknod c6 0 2024-04-02T04:47:37.221 INFO:tasks.workunit.client.0.smithi070.stdout:3/13: dread - f5 zero size 2024-04-02T04:47:37.222 INFO:tasks.workunit.client.0.smithi070.stdout:3/14: chown c2 423 1 2024-04-02T04:47:37.222 INFO:tasks.workunit.client.0.smithi070.stdout:9/35: creat d4/f8 x:0 0 0 2024-04-02T04:47:37.223 INFO:tasks.workunit.client.0.smithi070.stdout:9/36: write f3 [1751170,120442] 0 2024-04-02T04:47:37.225 INFO:tasks.workunit.client.0.smithi070.stdout:4/53: mknod d1/c14 0 2024-04-02T04:47:37.226 INFO:tasks.workunit.client.0.smithi070.stdout:9/37: write d4/f7 [248084,119361] 0 2024-04-02T04:47:37.226 INFO:tasks.workunit.client.0.smithi070.stdout:9/38: readlink - no filename 2024-04-02T04:47:37.226 INFO:tasks.workunit.client.0.smithi070.stdout:7/39: creat d7/d9/fa x:0 0 0 2024-04-02T04:47:37.227 INFO:tasks.workunit.client.0.smithi070.stdout:7/40: read d7/f8 [2487389,78859] 0 2024-04-02T04:47:37.228 INFO:tasks.workunit.client.0.smithi070.stdout:7/41: fdatasync d7/d9/fa 0 2024-04-02T04:47:37.228 INFO:tasks.workunit.client.0.smithi070.stdout:7/42: chown d7/d9 198845 1 2024-04-02T04:47:37.229 INFO:tasks.workunit.client.0.smithi070.stdout:7/43: dread - d7/d9/fa zero size 2024-04-02T04:47:37.229 INFO:tasks.workunit.client.0.smithi070.stdout:7/44: stat d7/d9/fa 0 2024-04-02T04:47:37.235 INFO:tasks.workunit.client.0.smithi070.stdout:3/15: link f5 f7 0 2024-04-02T04:47:37.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:36 smithi070 ceph-mon[29212]: pgmap v7: 65 pgs: 65 active+clean; 222 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-04-02T04:47:37.238 INFO:tasks.workunit.client.0.smithi070.stdout:8/55: getdents d2/d8/da/db 0 2024-04-02T04:47:37.240 INFO:tasks.workunit.client.0.smithi070.stdout:7/45: rename d7/f8 to d7/d9/fb 0 2024-04-02T04:47:37.242 INFO:tasks.workunit.client.0.smithi070.stdout:8/56: mknod d2/d3/c10 0 2024-04-02T04:47:37.243 INFO:tasks.workunit.client.0.smithi070.stdout:7/46: dread - d7/d9/fa zero size 2024-04-02T04:47:37.243 INFO:tasks.workunit.client.0.smithi070.stdout:7/47: readlink - no filename 2024-04-02T04:47:37.243 INFO:tasks.workunit.client.0.smithi070.stdout:7/48: readlink - no filename 2024-04-02T04:47:37.244 INFO:tasks.workunit.client.0.smithi070.stdout:3/16: dwrite f1 [0,4194304] 0 2024-04-02T04:47:37.244 INFO:tasks.workunit.client.0.smithi070.stdout:8/57: read d2/d3/f9 [11819968,10932] 0 2024-04-02T04:47:37.245 INFO:tasks.workunit.client.0.smithi070.stdout:3/17: write f1 [2290399,78859] 0 2024-04-02T04:47:37.252 INFO:tasks.workunit.client.0.smithi070.stdout:8/58: dwrite d2/d3/f9 [4194304,4194304] 0 2024-04-02T04:47:37.255 INFO:tasks.workunit.client.0.smithi070.stdout:8/59: symlink d2/d3/d6/l11 0 2024-04-02T04:47:37.256 INFO:tasks.workunit.client.0.smithi070.stdout:8/60: chown d2/d3/f9 17 1 2024-04-02T04:47:37.258 INFO:tasks.workunit.client.0.smithi070.stdout:8/61: getdents d2/d8 0 2024-04-02T04:47:37.259 INFO:tasks.workunit.client.0.smithi070.stdout:8/62: rmdir d2/d8/da 39 2024-04-02T04:47:37.260 INFO:tasks.workunit.client.0.smithi070.stdout:8/63: write d2/d3/fd [11649892,23409] 0 2024-04-02T04:47:37.266 INFO:tasks.workunit.client.0.smithi070.stdout:8/64: dwrite d2/d3/f7 [0,4194304] 0 2024-04-02T04:47:37.267 INFO:tasks.workunit.client.0.smithi070.stdout:8/65: write d2/d3/fd [12355973,58715] 0 2024-04-02T04:47:37.271 INFO:tasks.workunit.client.0.smithi070.stdout:8/66: link d2/d3/c10 d2/c12 0 2024-04-02T04:47:37.315 INFO:tasks.workunit.client.0.smithi070.stdout:5/47: truncate d5/fa 467818 0 2024-04-02T04:47:37.315 INFO:tasks.workunit.client.0.smithi070.stdout:5/48: chown d5/f6 201756462 1 2024-04-02T04:47:37.316 INFO:tasks.workunit.client.0.smithi070.stdout:5/49: chown l1 2997 1 2024-04-02T04:47:37.316 INFO:tasks.workunit.client.0.smithi070.stdout:1/58: truncate d0/d7/f10 4826988 0 2024-04-02T04:47:37.319 INFO:tasks.workunit.client.0.smithi070.stdout:5/50: unlink d5/d9/fb 0 2024-04-02T04:47:37.321 INFO:tasks.workunit.client.0.smithi070.stdout:1/59: creat d0/d1/f13 x:0 0 0 2024-04-02T04:47:37.323 INFO:tasks.workunit.client.0.smithi070.stdout:5/51: symlink d5/d9/ld 0 2024-04-02T04:47:37.324 INFO:tasks.workunit.client.0.smithi070.stdout:1/60: creat d0/d9/f14 x:0 0 0 2024-04-02T04:47:37.325 INFO:tasks.workunit.client.0.smithi070.stdout:5/52: mknod d5/d9/ce 0 2024-04-02T04:47:37.326 INFO:tasks.workunit.client.0.smithi070.stdout:1/61: mknod d0/d7/c15 0 2024-04-02T04:47:37.328 INFO:tasks.workunit.client.0.smithi070.stdout:1/62: readlink d0/d7/l6 0 2024-04-02T04:47:37.329 INFO:tasks.workunit.client.0.smithi070.stdout:5/53: dread f3 [0,4194304] 0 2024-04-02T04:47:37.329 INFO:tasks.workunit.client.0.smithi070.stdout:1/63: stat d0/d9/lf 0 2024-04-02T04:47:37.330 INFO:tasks.workunit.client.0.smithi070.stdout:5/54: dread - d5/f6 zero size 2024-04-02T04:47:37.330 INFO:tasks.workunit.client.0.smithi070.stdout:5/55: dread - f4 zero size 2024-04-02T04:47:37.334 INFO:tasks.workunit.client.0.smithi070.stdout:5/56: rename f3 to d5/ff 0 2024-04-02T04:47:37.336 INFO:tasks.workunit.client.0.smithi070.stdout:5/57: mkdir d5/d9/d10 0 2024-04-02T04:47:37.338 INFO:tasks.workunit.client.0.smithi070.stdout:5/58: creat d5/d9/d10/f11 x:0 0 0 2024-04-02T04:47:37.340 INFO:tasks.workunit.client.0.smithi070.stdout:5/59: mknod d5/d9/c12 0 2024-04-02T04:47:37.340 INFO:tasks.workunit.client.0.smithi070.stdout:5/60: fdatasync f4 0 2024-04-02T04:47:37.340 INFO:tasks.workunit.client.0.smithi070.stdout:5/61: dread - d5/f6 zero size 2024-04-02T04:47:37.346 INFO:tasks.workunit.client.0.smithi070.stdout:5/62: dwrite f2 [0,4194304] 0 2024-04-02T04:47:37.347 INFO:tasks.workunit.client.0.smithi070.stdout:5/63: dread - d5/f6 zero size 2024-04-02T04:47:37.347 INFO:tasks.workunit.client.0.smithi070.stdout:5/64: stat f2 0 2024-04-02T04:47:37.354 INFO:tasks.workunit.client.0.smithi070.stdout:5/65: dwrite f4 [0,4194304] 0 2024-04-02T04:47:37.356 INFO:tasks.workunit.client.0.smithi070.stdout:5/66: symlink d5/d9/d10/l13 0 2024-04-02T04:47:37.358 INFO:tasks.workunit.client.0.smithi070.stdout:5/67: mkdir d5/d14 0 2024-04-02T04:47:37.370 INFO:tasks.workunit.client.0.smithi070.stdout:2/33: fdatasync d2/f4 0 2024-04-02T04:47:37.373 INFO:tasks.workunit.client.0.smithi070.stdout:6/44: dwrite d0/d3/fb [0,4194304] 0 2024-04-02T04:47:37.373 INFO:tasks.workunit.client.0.smithi070.stdout:3/18: fsync f4 0 2024-04-02T04:47:37.373 INFO:tasks.workunit.client.0.smithi070.stdout:2/34: write d2/f4 [4915416,123631] 0 2024-04-02T04:47:37.377 INFO:tasks.workunit.client.0.smithi070.stdout:3/19: rename f7 to f8 0 2024-04-02T04:47:37.379 INFO:tasks.workunit.client.0.smithi070.stdout:6/45: rename d0/fc to d0/d3/f13 0 2024-04-02T04:47:37.381 INFO:tasks.workunit.client.0.smithi070.stdout:2/35: mknod d2/c6 0 2024-04-02T04:47:37.384 INFO:tasks.workunit.client.0.smithi070.stdout:3/20: mknod c9 0 2024-04-02T04:47:37.386 INFO:tasks.workunit.client.0.smithi070.stdout:2/36: dread d2/f4 [0,4194304] 0 2024-04-02T04:47:37.386 INFO:tasks.workunit.client.0.smithi070.stdout:4/54: getdents d1 0 2024-04-02T04:47:37.388 INFO:tasks.workunit.client.1.smithi179.stdout:0/59: write d1/f2 [4596918,109942] 0 2024-04-02T04:47:37.389 INFO:tasks.workunit.client.0.smithi070.stdout:0/231: truncate d4/d5/da/f8 564164 0 2024-04-02T04:47:37.392 INFO:tasks.workunit.client.0.smithi070.stdout:3/21: creat fa x:0 0 0 2024-04-02T04:47:37.393 INFO:tasks.workunit.client.1.smithi179.stdout:0/60: dwrite d1/f2 [0,4194304] 0 2024-04-02T04:47:37.393 INFO:tasks.workunit.client.0.smithi070.stdout:6/46: getdents d0/df 0 2024-04-02T04:47:37.396 INFO:tasks.workunit.client.0.smithi070.stdout:0/232: write d4/d5/d13/d23/f36 [988845,98698] 0 2024-04-02T04:47:37.399 INFO:tasks.workunit.client.0.smithi070.stdout:7/49: unlink d7/d9/fb 0 2024-04-02T04:47:37.400 INFO:tasks.workunit.client.0.smithi070.stdout:4/55: dwrite d1/f12 [0,4194304] 0 2024-04-02T04:47:37.400 INFO:tasks.workunit.client.1.smithi179.stdout:0/61: dwrite d1/f2 [4194304,4194304] 0 2024-04-02T04:47:37.401 INFO:tasks.workunit.client.0.smithi070.stdout:7/50: chown d7/d9 30 1 2024-04-02T04:47:37.403 INFO:tasks.workunit.client.0.smithi070.stdout:2/37: dwrite d2/f4 [0,4194304] 0 2024-04-02T04:47:37.403 INFO:tasks.workunit.client.0.smithi070.stdout:3/22: rename f4 to fb 0 2024-04-02T04:47:37.407 INFO:tasks.workunit.client.0.smithi070.stdout:7/51: creat d7/fc x:0 0 0 2024-04-02T04:47:37.410 INFO:tasks.workunit.client.0.smithi070.stdout:7/52: write d7/d9/fa [1032584,107310] 0 2024-04-02T04:47:37.412 INFO:tasks.workunit.client.0.smithi070.stdout:3/23: unlink c9 0 2024-04-02T04:47:37.413 INFO:tasks.workunit.client.0.smithi070.stdout:3/24: readlink l3 0 2024-04-02T04:47:37.413 INFO:tasks.workunit.client.0.smithi070.stdout:8/67: truncate d2/d3/f7 2810886 0 2024-04-02T04:47:37.415 INFO:tasks.workunit.client.1.smithi179.stdout:0/62: creat d1/da/df/f11 x:0 0 0 2024-04-02T04:47:37.419 INFO:tasks.workunit.client.0.smithi070.stdout:7/53: mkdir d7/dd 0 2024-04-02T04:47:37.421 INFO:tasks.workunit.client.0.smithi070.stdout:3/25: mknod cc 0 2024-04-02T04:47:37.422 INFO:tasks.workunit.client.0.smithi070.stdout:6/47: dread d0/d3/fa [0,4194304] 0 2024-04-02T04:47:37.423 INFO:tasks.workunit.client.0.smithi070.stdout:3/26: truncate f0 75129 0 2024-04-02T04:47:37.425 INFO:tasks.workunit.client.1.smithi179.stdout:0/63: rename d1/f5 to d1/da/df/f12 0 2024-04-02T04:47:37.426 INFO:tasks.workunit.client.0.smithi070.stdout:1/64: getdents d0/d7 0 2024-04-02T04:47:37.426 INFO:tasks.workunit.client.0.smithi070.stdout:7/54: dwrite d7/fc [0,4194304] 0 2024-04-02T04:47:37.427 INFO:tasks.workunit.client.0.smithi070.stdout:7/55: rename d7 to d7/d9/de 22 2024-04-02T04:47:37.428 INFO:tasks.workunit.client.0.smithi070.stdout:1/65: write d0/d7/f8 [3385667,80562] 0 2024-04-02T04:47:37.430 INFO:tasks.workunit.client.0.smithi070.stdout:6/48: dread d0/d3/fb [0,4194304] 0 2024-04-02T04:47:37.431 INFO:tasks.workunit.client.0.smithi070.stdout:5/68: dread d5/ff [0,4194304] 0 2024-04-02T04:47:37.431 INFO:tasks.workunit.client.0.smithi070.stdout:6/49: truncate d0/d3/fa 2028780 0 2024-04-02T04:47:37.432 INFO:tasks.workunit.client.0.smithi070.stdout:5/69: fdatasync d5/f6 0 2024-04-02T04:47:37.437 INFO:tasks.workunit.client.0.smithi070.stdout:0/233: getdents d4/d5/d13/d17 0 2024-04-02T04:47:37.439 INFO:tasks.workunit.client.0.smithi070.stdout:1/66: creat d0/d1/f16 x:0 0 0 2024-04-02T04:47:37.440 INFO:tasks.workunit.client.0.smithi070.stdout:1/67: dread - d0/d9/f14 zero size 2024-04-02T04:47:37.443 INFO:tasks.workunit.client.0.smithi070.stdout:2/38: fsync d2/f4 0 2024-04-02T04:47:37.444 INFO:tasks.workunit.client.0.smithi070.stdout:7/56: mkdir d7/dd/df 0 2024-04-02T04:47:37.445 INFO:tasks.workunit.client.0.smithi070.stdout:1/68: symlink d0/d1/l17 0 2024-04-02T04:47:37.449 INFO:tasks.workunit.client.0.smithi070.stdout:2/39: rmdir d2 39 2024-04-02T04:47:37.450 INFO:tasks.workunit.client.0.smithi070.stdout:7/57: dwrite d7/d9/fa [0,4194304] 0 2024-04-02T04:47:37.452 INFO:tasks.workunit.client.0.smithi070.stdout:6/50: rmdir d0/df 0 2024-04-02T04:47:37.465 INFO:tasks.workunit.client.0.smithi070.stdout:8/68: dread d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:37.465 INFO:tasks.workunit.client.0.smithi070.stdout:6/51: rename d0/d3/c7 to d0/d3/c14 0 2024-04-02T04:47:37.466 INFO:tasks.workunit.client.0.smithi070.stdout:1/69: link d0/d1/l12 d0/d1/l18 0 2024-04-02T04:47:37.466 INFO:tasks.workunit.client.0.smithi070.stdout:1/70: chown d0/d7 4007 1 2024-04-02T04:47:37.471 INFO:tasks.workunit.client.0.smithi070.stdout:6/52: rename d0/d3/fa to d0/f15 0 2024-04-02T04:47:37.471 INFO:tasks.workunit.client.0.smithi070.stdout:6/53: chown d0/d3/f13 176045558 1 2024-04-02T04:47:37.476 INFO:tasks.workunit.client.0.smithi070.stdout:2/40: rmdir d2/d5 0 2024-04-02T04:47:37.477 INFO:tasks.workunit.client.0.smithi070.stdout:8/69: creat d2/d8/da/db/f13 x:0 0 0 2024-04-02T04:47:37.478 INFO:tasks.workunit.client.1.smithi179.stdout:0/64: mknod d1/c13 0 2024-04-02T04:47:37.479 INFO:tasks.workunit.client.0.smithi070.stdout:1/71: chown d0/d1/l18 99089 1 2024-04-02T04:47:37.480 INFO:tasks.workunit.client.0.smithi070.stdout:3/27: dwrite f5 [0,4194304] 0 2024-04-02T04:47:37.482 INFO:tasks.workunit.client.0.smithi070.stdout:2/41: rename d2/c6 to d2/c7 0 2024-04-02T04:47:37.483 INFO:tasks.workunit.client.1.smithi179.stdout:0/65: dread d1/db/fc [0,4194304] 0 2024-04-02T04:47:37.490 INFO:tasks.workunit.client.1.smithi179.stdout:0/66: unlink f0 0 2024-04-02T04:47:37.490 INFO:tasks.workunit.client.0.smithi070.stdout:3/28: dread f8 [0,4194304] 0 2024-04-02T04:47:37.493 INFO:tasks.workunit.client.0.smithi070.stdout:7/58: write d7/d9/fa [4671739,13794] 0 2024-04-02T04:47:37.494 INFO:tasks.workunit.client.0.smithi070.stdout:5/70: dwrite d5/fa [0,4194304] 0 2024-04-02T04:47:37.495 INFO:tasks.workunit.client.0.smithi070.stdout:7/59: fsync d7/fc 0 2024-04-02T04:47:37.495 INFO:tasks.workunit.client.0.smithi070.stdout:2/42: dwrite d2/f4 [0,4194304] 0 2024-04-02T04:47:37.496 INFO:tasks.workunit.client.0.smithi070.stdout:1/72: rename d0/d1 to d0/d7/d19 0 2024-04-02T04:47:37.501 INFO:tasks.workunit.client.0.smithi070.stdout:3/29: creat fd x:0 0 0 2024-04-02T04:47:37.503 INFO:tasks.workunit.client.0.smithi070.stdout:8/70: rename d2/d8/da to d2/d8/d14 0 2024-04-02T04:47:37.506 INFO:tasks.workunit.client.0.smithi070.stdout:7/60: rmdir d7 39 2024-04-02T04:47:37.507 INFO:tasks.workunit.client.1.smithi179.stdout:0/67: dwrite d1/f6 [4194304,4194304] 0 2024-04-02T04:47:37.511 INFO:tasks.workunit.client.0.smithi070.stdout:1/73: write d0/d7/d19/f2 [1184293,60797] 0 2024-04-02T04:47:37.511 INFO:tasks.workunit.client.0.smithi070.stdout:1/74: stat d0/d7 0 2024-04-02T04:47:37.512 INFO:tasks.workunit.client.0.smithi070.stdout:1/75: dread - d0/d9/f14 zero size 2024-04-02T04:47:37.512 INFO:tasks.workunit.client.1.smithi179.stdout:0/68: fsync d1/f9 0 2024-04-02T04:47:37.513 INFO:tasks.workunit.client.1.smithi179.stdout:0/69: stat d1 0 2024-04-02T04:47:37.514 INFO:tasks.workunit.client.1.smithi179.stdout:0/70: read d1/f6 [5483051,51593] 0 2024-04-02T04:47:37.515 INFO:tasks.workunit.client.0.smithi070.stdout:8/71: creat d2/d8/d14/f15 x:0 0 0 2024-04-02T04:47:37.520 INFO:tasks.workunit.client.1.smithi179.stdout:0/71: chown d1/da/df 155881285 1 2024-04-02T04:47:37.522 INFO:tasks.workunit.client.0.smithi070.stdout:2/43: link d2/f4 d2/f8 0 2024-04-02T04:47:37.523 INFO:tasks.workunit.client.0.smithi070.stdout:1/76: mkdir d0/d7/d1a 0 2024-04-02T04:47:37.524 INFO:tasks.workunit.client.0.smithi070.stdout:1/77: stat d0/d7/d19/le 0 2024-04-02T04:47:37.525 INFO:tasks.workunit.client.0.smithi070.stdout:7/61: dwrite d7/d9/fa [0,4194304] 0 2024-04-02T04:47:37.525 INFO:tasks.workunit.client.0.smithi070.stdout:7/62: stat d7/d9/fa 0 2024-04-02T04:47:37.528 INFO:tasks.workunit.client.0.smithi070.stdout:1/78: truncate d0/d7/d19/f13 478397 0 2024-04-02T04:47:37.528 INFO:tasks.workunit.client.0.smithi070.stdout:2/44: dread d2/f8 [0,4194304] 0 2024-04-02T04:47:37.531 INFO:tasks.workunit.client.0.smithi070.stdout:2/45: write d2/f4 [3233921,91383] 0 2024-04-02T04:47:37.532 INFO:tasks.workunit.client.0.smithi070.stdout:1/79: rename d0/d7/d19/fd to d0/d7/d1a/f1b 0 2024-04-02T04:47:37.533 INFO:tasks.workunit.client.0.smithi070.stdout:2/46: stat d2 0 2024-04-02T04:47:37.533 INFO:tasks.workunit.client.0.smithi070.stdout:8/72: dwrite d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:37.534 INFO:tasks.workunit.client.0.smithi070.stdout:7/63: dread d7/fc [0,4194304] 0 2024-04-02T04:47:37.535 INFO:tasks.workunit.client.0.smithi070.stdout:1/80: rmdir d0/d9 39 2024-04-02T04:47:37.535 INFO:tasks.workunit.client.0.smithi070.stdout:7/64: write d7/fc [1424121,39075] 0 2024-04-02T04:47:37.536 INFO:tasks.workunit.client.0.smithi070.stdout:5/71: write d5/ff [2058851,73736] 0 2024-04-02T04:47:37.537 INFO:tasks.workunit.client.0.smithi070.stdout:2/47: readlink d2/l3 0 2024-04-02T04:47:37.537 INFO:tasks.workunit.client.0.smithi070.stdout:7/65: stat d7 0 2024-04-02T04:47:37.544 INFO:tasks.workunit.client.0.smithi070.stdout:5/72: dwrite d5/ff [0,4194304] 0 2024-04-02T04:47:37.544 INFO:tasks.workunit.client.0.smithi070.stdout:7/66: dwrite d7/fc [0,4194304] 0 2024-04-02T04:47:37.549 INFO:tasks.workunit.client.0.smithi070.stdout:1/81: chown d0/d9 77905 1 2024-04-02T04:47:37.553 INFO:tasks.workunit.client.0.smithi070.stdout:2/48: creat d2/f9 x:0 0 0 2024-04-02T04:47:37.554 INFO:tasks.workunit.client.0.smithi070.stdout:8/73: fsync d2/d3/f9 0 2024-04-02T04:47:37.554 INFO:tasks.workunit.client.0.smithi070.stdout:2/49: write d2/f9 [354886,102574] 0 2024-04-02T04:47:37.555 INFO:tasks.workunit.client.0.smithi070.stdout:5/73: mknod d5/d9/d10/c15 0 2024-04-02T04:47:37.556 INFO:tasks.workunit.client.1.smithi179.stdout:0/72: write d1/db/fc [3188022,96079] 0 2024-04-02T04:47:37.558 INFO:tasks.workunit.client.0.smithi070.stdout:3/30: truncate f0 3770746 0 2024-04-02T04:47:37.560 INFO:tasks.workunit.client.1.smithi179.stdout:0/73: unlink d1/db/ld 0 2024-04-02T04:47:37.562 INFO:tasks.workunit.client.0.smithi070.stdout:8/74: creat d2/d3/d6/f16 x:0 0 0 2024-04-02T04:47:37.563 INFO:tasks.workunit.client.0.smithi070.stdout:7/67: fsync d7/fc 0 2024-04-02T04:47:37.564 INFO:tasks.workunit.client.0.smithi070.stdout:7/68: read d7/d9/fa [3236962,61637] 0 2024-04-02T04:47:37.564 INFO:tasks.workunit.client.1.smithi179.stdout:0/74: write d1/f2 [3593995,67036] 0 2024-04-02T04:47:37.564 INFO:tasks.workunit.client.0.smithi070.stdout:7/69: chown c4 9742289 1 2024-04-02T04:47:37.565 INFO:tasks.workunit.client.0.smithi070.stdout:3/31: creat fe x:0 0 0 2024-04-02T04:47:37.565 INFO:tasks.workunit.client.1.smithi179.stdout:0/75: write d1/f9 [3101407,122900] 0 2024-04-02T04:47:37.566 INFO:tasks.workunit.client.0.smithi070.stdout:1/82: creat d0/f1c x:0 0 0 2024-04-02T04:47:37.567 INFO:tasks.workunit.client.0.smithi070.stdout:8/75: creat d2/d3/f17 x:0 0 0 2024-04-02T04:47:37.568 INFO:tasks.workunit.client.0.smithi070.stdout:7/70: dread d7/fc [0,4194304] 0 2024-04-02T04:47:37.573 INFO:tasks.workunit.client.1.smithi179.stdout:0/76: dwrite d1/da/df/f11 [0,4194304] 0 2024-04-02T04:47:37.573 INFO:tasks.workunit.client.0.smithi070.stdout:2/50: getdents d2 0 2024-04-02T04:47:37.576 INFO:tasks.workunit.client.0.smithi070.stdout:7/71: dwrite d7/fc [4194304,4194304] 0 2024-04-02T04:47:37.578 INFO:tasks.workunit.client.1.smithi179.stdout:0/77: fdatasync d1/da/df/f12 0 2024-04-02T04:47:37.579 INFO:tasks.workunit.client.0.smithi070.stdout:8/76: fsync d2/d3/d6/f16 0 2024-04-02T04:47:37.580 INFO:tasks.workunit.client.0.smithi070.stdout:8/77: chown d2/d8/d14/db 47181846 1 2024-04-02T04:47:37.581 INFO:tasks.workunit.client.0.smithi070.stdout:8/78: stat d2/d3/d6 0 2024-04-02T04:47:37.581 INFO:tasks.workunit.client.0.smithi070.stdout:8/79: chown f1 1 1 2024-04-02T04:47:37.585 INFO:tasks.workunit.client.0.smithi070.stdout:2/51: getdents d2 0 2024-04-02T04:47:37.589 INFO:tasks.workunit.client.0.smithi070.stdout:7/72: creat d7/dd/df/f10 x:0 0 0 2024-04-02T04:47:37.591 INFO:tasks.workunit.client.1.smithi179.stdout:0/78: truncate d1/fe 1769829 0 2024-04-02T04:47:37.592 INFO:tasks.workunit.client.0.smithi070.stdout:8/80: truncate d2/d3/f7 1180996 0 2024-04-02T04:47:37.594 INFO:tasks.workunit.client.1.smithi179.stdout:0/79: creat d1/f14 x:0 0 0 2024-04-02T04:47:37.595 INFO:tasks.workunit.client.0.smithi070.stdout:7/73: truncate d7/dd/df/f10 559958 0 2024-04-02T04:47:37.597 INFO:tasks.workunit.client.1.smithi179.stdout:0/80: mkdir d1/d15 0 2024-04-02T04:47:37.597 INFO:tasks.workunit.client.0.smithi070.stdout:3/32: dwrite f0 [0,4194304] 0 2024-04-02T04:47:37.599 INFO:tasks.workunit.client.0.smithi070.stdout:8/81: creat d2/d8/d14/db/f18 x:0 0 0 2024-04-02T04:47:37.599 INFO:tasks.workunit.client.0.smithi070.stdout:1/83: dwrite d0/d7/f10 [0,4194304] 0 2024-04-02T04:47:37.600 INFO:tasks.workunit.client.0.smithi070.stdout:3/33: chown fe 2062 1 2024-04-02T04:47:37.601 INFO:tasks.workunit.client.0.smithi070.stdout:2/52: dwrite d2/f9 [0,4194304] 0 2024-04-02T04:47:37.602 INFO:tasks.workunit.client.0.smithi070.stdout:8/82: readlink d2/d3/d6/l11 0 2024-04-02T04:47:37.605 INFO:tasks.workunit.client.0.smithi070.stdout:2/53: creat d2/fa x:0 0 0 2024-04-02T04:47:37.609 INFO:tasks.workunit.client.0.smithi070.stdout:7/74: truncate d7/d9/fa 561831 0 2024-04-02T04:47:37.612 INFO:tasks.workunit.client.0.smithi070.stdout:8/83: mknod d2/c19 0 2024-04-02T04:47:37.613 INFO:tasks.workunit.client.0.smithi070.stdout:1/84: dwrite d0/d7/d19/f13 [0,4194304] 0 2024-04-02T04:47:37.614 INFO:tasks.workunit.client.0.smithi070.stdout:2/54: creat d2/fb x:0 0 0 2024-04-02T04:47:37.617 INFO:tasks.workunit.client.0.smithi070.stdout:1/85: chown d0/d9/lf 1451 1 2024-04-02T04:47:37.619 INFO:tasks.workunit.client.1.smithi179.stdout:0/81: truncate d1/da/df/f11 74696 0 2024-04-02T04:47:37.621 INFO:tasks.workunit.client.0.smithi070.stdout:7/75: dwrite d7/dd/df/f10 [0,4194304] 0 2024-04-02T04:47:37.627 INFO:tasks.workunit.client.0.smithi070.stdout:7/76: unlink c0 0 2024-04-02T04:47:37.629 INFO:tasks.workunit.client.0.smithi070.stdout:1/86: dwrite d0/f1c [0,4194304] 0 2024-04-02T04:47:37.629 INFO:tasks.workunit.client.0.smithi070.stdout:2/55: dwrite d2/f8 [0,4194304] 0 2024-04-02T04:47:37.631 INFO:tasks.workunit.client.0.smithi070.stdout:7/77: write d7/fc [1543124,21990] 0 2024-04-02T04:47:37.632 INFO:tasks.workunit.client.0.smithi070.stdout:2/56: write d2/f9 [2919969,69371] 0 2024-04-02T04:47:37.633 INFO:tasks.workunit.client.0.smithi070.stdout:2/57: fsync d2/fb 0 2024-04-02T04:47:37.635 INFO:tasks.workunit.client.0.smithi070.stdout:2/58: readlink d2/l3 0 2024-04-02T04:47:37.635 INFO:tasks.workunit.client.0.smithi070.stdout:2/59: chown d2/l3 1215 1 2024-04-02T04:47:37.637 INFO:tasks.workunit.client.0.smithi070.stdout:1/87: rmdir d0/d9 39 2024-04-02T04:47:37.639 INFO:tasks.workunit.client.0.smithi070.stdout:2/60: dread d2/f4 [0,4194304] 0 2024-04-02T04:47:37.640 INFO:tasks.workunit.client.0.smithi070.stdout:7/78: mkdir d7/d9/d11 0 2024-04-02T04:47:37.643 INFO:tasks.workunit.client.0.smithi070.stdout:1/88: write d0/d9/f11 [588053,81141] 0 2024-04-02T04:47:37.645 INFO:tasks.workunit.client.1.smithi179.stdout:0/82: truncate d1/f6 4858319 0 2024-04-02T04:47:37.645 INFO:tasks.workunit.client.1.smithi179.stdout:0/83: truncate d1/f14 751158 0 2024-04-02T04:47:37.646 INFO:tasks.workunit.client.0.smithi070.stdout:1/89: write d0/d7/d19/f13 [273031,130283] 0 2024-04-02T04:47:37.649 INFO:tasks.workunit.client.0.smithi070.stdout:8/84: dwrite d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:37.650 INFO:tasks.workunit.client.0.smithi070.stdout:2/61: getdents d2 0 2024-04-02T04:47:37.651 INFO:tasks.workunit.client.0.smithi070.stdout:2/62: write d2/fa [1041429,113876] 0 2024-04-02T04:47:37.658 INFO:tasks.workunit.client.0.smithi070.stdout:1/90: rename d0/d7/d19 to d0/d9/d1d 0 2024-04-02T04:47:37.658 INFO:tasks.workunit.client.0.smithi070.stdout:1/91: read d0/f1c [2593365,100319] 0 2024-04-02T04:47:37.659 INFO:tasks.workunit.client.0.smithi070.stdout:1/92: rename d0/d7 to d0/d7/d1a/d1e 22 2024-04-02T04:47:37.662 INFO:tasks.workunit.client.0.smithi070.stdout:1/93: dread d0/d7/f8 [0,4194304] 0 2024-04-02T04:47:37.663 INFO:tasks.workunit.client.0.smithi070.stdout:1/94: dread - d0/d9/d1d/f16 zero size 2024-04-02T04:47:37.711 INFO:tasks.workunit.client.0.smithi070.stdout:1/95: symlink d0/d9/d1d/l1f 0 2024-04-02T04:47:37.712 INFO:tasks.workunit.client.0.smithi070.stdout:7/79: dread d7/d9/fa [0,4194304] 0 2024-04-02T04:47:37.714 INFO:tasks.workunit.client.0.smithi070.stdout:2/63: link d2/f8 d2/fc 0 2024-04-02T04:47:37.719 INFO:tasks.workunit.client.0.smithi070.stdout:2/64: mknod d2/cd 0 2024-04-02T04:47:37.719 INFO:tasks.workunit.client.0.smithi070.stdout:2/65: dread - d2/fb zero size 2024-04-02T04:47:37.720 INFO:tasks.workunit.client.0.smithi070.stdout:2/66: chown d2/fa 4144261 1 2024-04-02T04:47:37.720 INFO:tasks.workunit.client.0.smithi070.stdout:2/67: truncate d2/fb 650950 0 2024-04-02T04:47:37.721 INFO:tasks.workunit.client.0.smithi070.stdout:7/80: mknod d7/d9/d11/c12 0 2024-04-02T04:47:37.722 INFO:tasks.workunit.client.0.smithi070.stdout:7/81: truncate d7/d9/fa 1145298 0 2024-04-02T04:47:37.723 INFO:tasks.workunit.client.0.smithi070.stdout:2/68: write d2/f8 [172517,73046] 0 2024-04-02T04:47:37.724 INFO:tasks.workunit.client.0.smithi070.stdout:2/69: dread d2/fb [0,4194304] 0 2024-04-02T04:47:37.725 INFO:tasks.workunit.client.0.smithi070.stdout:2/70: dread d2/fb [0,4194304] 0 2024-04-02T04:47:37.730 INFO:tasks.workunit.client.0.smithi070.stdout:9/39: sync 2024-04-02T04:47:37.730 INFO:tasks.workunit.client.0.smithi070.stdout:2/71: dread d2/fc [0,4194304] 0 2024-04-02T04:47:37.731 INFO:tasks.workunit.client.0.smithi070.stdout:1/96: rmdir d0/d9/d1d 39 2024-04-02T04:47:37.733 INFO:tasks.workunit.client.0.smithi070.stdout:9/40: dread d4/f7 [0,4194304] 0 2024-04-02T04:47:37.736 INFO:tasks.workunit.client.0.smithi070.stdout:1/97: creat d0/d7/f20 x:0 0 0 2024-04-02T04:47:37.737 INFO:tasks.workunit.client.0.smithi070.stdout:2/72: dwrite d2/f4 [0,4194304] 0 2024-04-02T04:47:37.739 INFO:tasks.workunit.client.0.smithi070.stdout:9/41: creat d4/f9 x:0 0 0 2024-04-02T04:47:37.742 INFO:tasks.workunit.client.0.smithi070.stdout:2/73: write d2/fb [1435829,110485] 0 2024-04-02T04:47:37.744 INFO:tasks.workunit.client.0.smithi070.stdout:1/98: creat d0/d9/d1d/f21 x:0 0 0 2024-04-02T04:47:37.745 INFO:tasks.workunit.client.0.smithi070.stdout:1/99: fsync d0/d9/fc 0 2024-04-02T04:47:37.747 INFO:tasks.workunit.client.0.smithi070.stdout:2/74: creat d2/fe x:0 0 0 2024-04-02T04:47:37.748 INFO:tasks.workunit.client.0.smithi070.stdout:1/100: mkdir d0/d9/d1d/d22 0 2024-04-02T04:47:37.750 INFO:tasks.workunit.client.0.smithi070.stdout:2/75: mknod d2/cf 0 2024-04-02T04:47:37.751 INFO:tasks.workunit.client.0.smithi070.stdout:1/101: readlink d0/d9/d1d/l12 0 2024-04-02T04:47:37.753 INFO:tasks.workunit.client.0.smithi070.stdout:2/76: mkdir d2/d10 0 2024-04-02T04:47:37.756 INFO:tasks.workunit.client.0.smithi070.stdout:1/102: mknod d0/d9/d1d/d22/c23 0 2024-04-02T04:47:37.757 INFO:tasks.workunit.client.0.smithi070.stdout:1/103: stat d0/d7/d1a/f1b 0 2024-04-02T04:47:37.759 INFO:tasks.workunit.client.0.smithi070.stdout:2/77: dwrite d2/fb [0,4194304] 0 2024-04-02T04:47:37.760 INFO:tasks.workunit.client.0.smithi070.stdout:1/104: creat d0/d7/d1a/f24 x:0 0 0 2024-04-02T04:47:37.761 INFO:tasks.workunit.client.0.smithi070.stdout:1/105: truncate d0/d7/f20 953236 0 2024-04-02T04:47:37.763 INFO:tasks.workunit.client.0.smithi070.stdout:2/78: mknod d2/d10/c11 0 2024-04-02T04:47:37.764 INFO:tasks.workunit.client.0.smithi070.stdout:2/79: chown d2/fa 556412 1 2024-04-02T04:47:37.765 INFO:tasks.workunit.client.0.smithi070.stdout:2/80: chown d2/c7 94286889 1 2024-04-02T04:47:37.765 INFO:tasks.workunit.client.0.smithi070.stdout:1/106: unlink d0/d9/f11 0 2024-04-02T04:47:37.769 INFO:tasks.workunit.client.0.smithi070.stdout:1/107: creat d0/d9/d1d/d22/f25 x:0 0 0 2024-04-02T04:47:37.770 INFO:tasks.workunit.client.0.smithi070.stdout:1/108: write d0/d9/f14 [936886,27399] 0 2024-04-02T04:47:37.773 INFO:tasks.workunit.client.0.smithi070.stdout:1/109: read d0/d9/f14 [925563,95157] 0 2024-04-02T04:47:37.791 INFO:tasks.workunit.client.0.smithi070.stdout:2/81: fsync d2/fc 0 2024-04-02T04:47:37.791 INFO:tasks.workunit.client.0.smithi070.stdout:2/82: chown d2/c7 105046734 1 2024-04-02T04:47:37.793 INFO:tasks.workunit.client.0.smithi070.stdout:2/83: mkdir d2/d12 0 2024-04-02T04:47:37.793 INFO:tasks.workunit.client.1.smithi179.stdout:0/84: write d1/fe [2302319,7160] 0 2024-04-02T04:47:37.795 INFO:tasks.workunit.client.1.smithi179.stdout:0/85: dread d1/f14 [0,4194304] 0 2024-04-02T04:47:37.798 INFO:tasks.workunit.client.1.smithi179.stdout:0/86: dread d1/f9 [4194304,4194304] 0 2024-04-02T04:47:37.799 INFO:tasks.workunit.client.1.smithi179.stdout:0/87: write d1/f2 [1120306,83324] 0 2024-04-02T04:47:37.800 INFO:tasks.workunit.client.0.smithi070.stdout:2/84: dwrite d2/fa [0,4194304] 0 2024-04-02T04:47:37.802 INFO:tasks.workunit.client.0.smithi070.stdout:2/85: write d2/fb [2852205,98868] 0 2024-04-02T04:47:37.802 INFO:tasks.workunit.client.0.smithi070.stdout:9/42: truncate f3 84776 0 2024-04-02T04:47:37.807 INFO:tasks.workunit.client.0.smithi070.stdout:9/43: mknod d4/ca 0 2024-04-02T04:47:37.810 INFO:tasks.workunit.client.0.smithi070.stdout:9/44: mknod d4/cb 0 2024-04-02T04:47:37.811 INFO:tasks.workunit.client.0.smithi070.stdout:2/86: creat d2/d12/f13 x:0 0 0 2024-04-02T04:47:37.813 INFO:tasks.workunit.client.0.smithi070.stdout:9/45: dread d4/f7 [0,4194304] 0 2024-04-02T04:47:37.813 INFO:tasks.workunit.client.0.smithi070.stdout:1/110: getdents d0/d9 0 2024-04-02T04:47:37.814 INFO:tasks.workunit.client.0.smithi070.stdout:1/111: write d0/d9/d1d/f21 [303697,8342] 0 2024-04-02T04:47:37.815 INFO:tasks.workunit.client.0.smithi070.stdout:2/87: unlink d2/f9 0 2024-04-02T04:47:37.819 INFO:tasks.workunit.client.0.smithi070.stdout:1/112: creat d0/d9/d1d/f26 x:0 0 0 2024-04-02T04:47:37.820 INFO:tasks.workunit.client.0.smithi070.stdout:2/88: rename d2/fe to d2/f14 0 2024-04-02T04:47:37.821 INFO:tasks.workunit.client.0.smithi070.stdout:2/89: dread - d2/d12/f13 zero size 2024-04-02T04:47:37.821 INFO:tasks.workunit.client.0.smithi070.stdout:2/90: write d2/f4 [1694098,43599] 0 2024-04-02T04:47:37.824 INFO:tasks.workunit.client.0.smithi070.stdout:2/91: symlink d2/l15 0 2024-04-02T04:47:37.825 INFO:tasks.workunit.client.0.smithi070.stdout:2/92: chown d2/cd 87441 1 2024-04-02T04:47:37.825 INFO:tasks.workunit.client.1.smithi179.stdout:0/88: truncate d1/db/fc 574903 0 2024-04-02T04:47:37.828 INFO:tasks.workunit.client.0.smithi070.stdout:2/93: dread d2/fc [0,4194304] 0 2024-04-02T04:47:37.830 INFO:tasks.workunit.client.1.smithi179.stdout:0/89: rename d1/l4 to d1/da/l16 0 2024-04-02T04:47:37.832 INFO:tasks.workunit.client.0.smithi070.stdout:2/94: rename d2/c7 to d2/c16 0 2024-04-02T04:47:37.833 INFO:tasks.workunit.client.1.smithi179.stdout:0/90: dread d1/da/df/f12 [0,4194304] 0 2024-04-02T04:47:37.833 INFO:tasks.workunit.client.0.smithi070.stdout:2/95: read d2/f8 [1814569,38991] 0 2024-04-02T04:47:37.834 INFO:tasks.workunit.client.1.smithi179.stdout:0/91: read d1/f14 [336856,35687] 0 2024-04-02T04:47:37.835 INFO:tasks.workunit.client.0.smithi070.stdout:2/96: unlink d2/cf 0 2024-04-02T04:47:37.837 INFO:tasks.workunit.client.0.smithi070.stdout:2/97: readlink d2/l15 0 2024-04-02T04:47:37.837 INFO:tasks.workunit.client.1.smithi179.stdout:0/92: unlink d1/f9 0 2024-04-02T04:47:37.838 INFO:tasks.workunit.client.0.smithi070.stdout:9/46: dwrite d4/f7 [0,4194304] 0 2024-04-02T04:47:37.839 INFO:tasks.workunit.client.0.smithi070.stdout:2/98: mknod d2/c17 0 2024-04-02T04:47:37.841 INFO:tasks.workunit.client.1.smithi179.stdout:0/93: creat d1/da/f17 x:0 0 0 2024-04-02T04:47:37.843 INFO:tasks.workunit.client.0.smithi070.stdout:9/47: creat d4/fc x:0 0 0 2024-04-02T04:47:37.844 INFO:tasks.workunit.client.0.smithi070.stdout:9/48: chown d4/ca 17890 1 2024-04-02T04:47:37.845 INFO:tasks.workunit.client.0.smithi070.stdout:2/99: dread d2/f4 [0,4194304] 0 2024-04-02T04:47:37.846 INFO:tasks.workunit.client.0.smithi070.stdout:9/49: write d4/f7 [2212587,110678] 0 2024-04-02T04:47:37.847 INFO:tasks.workunit.client.0.smithi070.stdout:9/50: write d4/f7 [3408923,90460] 0 2024-04-02T04:47:37.849 INFO:tasks.workunit.client.0.smithi070.stdout:2/100: mknod d2/d12/c18 0 2024-04-02T04:47:37.850 INFO:tasks.workunit.client.0.smithi070.stdout:2/101: dread - d2/d12/f13 zero size 2024-04-02T04:47:37.851 INFO:tasks.workunit.client.0.smithi070.stdout:2/102: write d2/fb [1987522,10734] 0 2024-04-02T04:47:37.852 INFO:tasks.workunit.client.0.smithi070.stdout:9/51: symlink d4/ld 0 2024-04-02T04:47:37.855 INFO:tasks.workunit.client.0.smithi070.stdout:2/103: mkdir d2/d10/d19 0 2024-04-02T04:47:37.858 INFO:tasks.workunit.client.0.smithi070.stdout:2/104: creat d2/d12/f1a x:0 0 0 2024-04-02T04:47:37.861 INFO:tasks.workunit.client.0.smithi070.stdout:2/105: creat d2/f1b x:0 0 0 2024-04-02T04:47:37.861 INFO:tasks.workunit.client.0.smithi070.stdout:9/52: dwrite d4/f8 [0,4194304] 0 2024-04-02T04:47:37.864 INFO:tasks.workunit.client.0.smithi070.stdout:2/106: creat d2/f1c x:0 0 0 2024-04-02T04:47:37.866 INFO:tasks.workunit.client.0.smithi070.stdout:2/107: write d2/fc [5990023,554] 0 2024-04-02T04:47:37.869 INFO:tasks.workunit.client.0.smithi070.stdout:9/53: dwrite d4/f7 [0,4194304] 0 2024-04-02T04:47:37.873 INFO:tasks.workunit.client.0.smithi070.stdout:9/54: mknod d4/ce 0 2024-04-02T04:47:37.876 INFO:tasks.workunit.client.0.smithi070.stdout:9/55: rename d4/ca to d4/cf 0 2024-04-02T04:47:37.880 INFO:tasks.workunit.client.0.smithi070.stdout:2/108: truncate d2/fb 2474380 0 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.878+0000 7fa66e142700 1 -- 172.21.15.70:0/384891003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668101f40 msgr2=0x7fa6681023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.878+0000 7fa66e142700 1 --2- 172.21.15.70:0/384891003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668101f40 0x7fa6681023a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009910 tx=0x7fa65c009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.878+0000 7fa66e142700 1 -- 172.21.15.70:0/384891003 shutdown_connections 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.878+0000 7fa66e142700 1 --2- 172.21.15.70:0/384891003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668101f40 0x7fa6681023a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.878+0000 7fa66e142700 1 --2- 172.21.15.70:0/384891003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680fef60 0x7fa6680ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.878+0000 7fa66e142700 1 -- 172.21.15.70:0/384891003 >> 172.21.15.70:0/384891003 conn(0x7fa6680fa8c0 msgr2=0x7fa6680fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.878+0000 7fa66e142700 1 -- 172.21.15.70:0/384891003 shutdown_connections 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.878+0000 7fa66e142700 1 -- 172.21.15.70:0/384891003 wait complete. 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa66e142700 1 Processor -- start 2024-04-02T04:47:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa66e142700 1 -- start start 2024-04-02T04:47:37.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa66e142700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680fef60 0x7fa668196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:37.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa66e142700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668196c60 0x7fa66819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:37.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa66e142700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668197170 con 0x7fa668196c60 2024-04-02T04:47:37.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa66e142700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6681972e0 con 0x7fa6680fef60 2024-04-02T04:47:37.883 INFO:tasks.workunit.client.0.smithi070.stdout:9/56: chown d4/cf 28245975 1 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa6677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680fef60 0x7fa668196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa6677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680fef60 0x7fa668196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:39686/0 (socket says 172.21.15.70:39686) 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.879+0000 7fa6677fe700 1 -- 172.21.15.70:0/3288520310 learned_addr learned my addr 172.21.15.70:0/3288520310 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.880+0000 7fa666ffd700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668196c60 0x7fa66819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.880+0000 7fa666ffd700 1 -- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680fef60 msgr2=0x7fa668196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.880+0000 7fa666ffd700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680fef60 0x7fa668196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.880+0000 7fa666ffd700 1 -- 172.21.15.70:0/3288520310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0095f0 con 0x7fa668196c60 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.880+0000 7fa6677fe700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680fef60 0x7fa668196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.880+0000 7fa666ffd700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668196c60 0x7fa66819bcd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa65c008000 tx=0x7fa65c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.881+0000 7fa664ff9700 1 -- 172.21.15.70:0/3288520310 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa65c01d070 con 0x7fa668196c60 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.881+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6680ffd30 con 0x7fa668196c60 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.881+0000 7fa664ff9700 1 -- 172.21.15.70:0/3288520310 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa65c005640 con 0x7fa668196c60 2024-04-02T04:47:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.881+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6681001f0 con 0x7fa668196c60 2024-04-02T04:47:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.881+0000 7fa664ff9700 1 -- 172.21.15.70:0/3288520310 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa65c003e30 con 0x7fa668196c60 2024-04-02T04:47:37.886 INFO:tasks.workunit.client.0.smithi070.stdout:9/57: mkdir d4/d10 0 2024-04-02T04:47:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.882+0000 7fa664ff9700 1 -- 172.21.15.70:0/3288520310 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 23) v1 ==== 46257+0+0 (secure 0 0 0) 0x7fa65c02b440 con 0x7fa668196c60 2024-04-02T04:47:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.882+0000 7fa664ff9700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fa650039110 0x7fa65003b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.882+0000 7fa664ff9700 1 -- 172.21.15.70:0/3288520310 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fa65c04e690 con 0x7fa668196c60 2024-04-02T04:47:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.882+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa654005320 con 0x7fa668196c60 2024-04-02T04:47:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.882+0000 7fa6677fe700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fa650039110 0x7fa65003b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.883+0000 7fa6677fe700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fa650039110 0x7fa65003b5d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa658005950 tx=0x7fa658009260 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:37.887 INFO:tasks.workunit.client.0.smithi070.stdout:9/58: unlink d4/ld 0 2024-04-02T04:47:37.888 INFO:tasks.workunit.client.0.smithi070.stdout:2/109: write d2/fa [5225752,97407] 0 2024-04-02T04:47:37.890 INFO:tasks.workunit.client.0.smithi070.stdout:9/59: mkdir d4/d10/d11 0 2024-04-02T04:47:37.891 INFO:tasks.workunit.client.0.smithi070.stdout:2/110: mknod d2/d10/d19/c1d 0 2024-04-02T04:47:37.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.889+0000 7fa664ff9700 1 -- 172.21.15.70:0/3288520310 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fa65c017400 con 0x7fa668196c60 2024-04-02T04:47:37.892 INFO:tasks.workunit.client.0.smithi070.stdout:9/60: mkdir d4/d10/d12 0 2024-04-02T04:47:37.894 INFO:tasks.workunit.client.0.smithi070.stdout:2/111: creat d2/d10/d19/f1e x:0 0 0 2024-04-02T04:47:37.896 INFO:tasks.workunit.client.0.smithi070.stdout:2/112: mknod d2/d12/c1f 0 2024-04-02T04:47:37.896 INFO:tasks.workunit.client.0.smithi070.stdout:2/113: readlink d2/l15 0 2024-04-02T04:47:37.900 INFO:tasks.workunit.client.0.smithi070.stdout:2/114: rename d2/d12/f1a to d2/d10/f20 0 2024-04-02T04:47:37.903 INFO:tasks.workunit.client.0.smithi070.stdout:2/115: mkdir d2/d10/d21 0 2024-04-02T04:47:37.905 INFO:tasks.workunit.client.0.smithi070.stdout:2/116: write d2/d10/f20 [295470,83012] 0 2024-04-02T04:47:37.906 INFO:tasks.workunit.client.0.smithi070.stdout:2/117: fsync d2/fa 0 2024-04-02T04:47:37.906 INFO:tasks.workunit.client.0.smithi070.stdout:9/61: write f3 [50160,17909] 0 2024-04-02T04:47:37.910 INFO:tasks.workunit.client.0.smithi070.stdout:9/62: mkdir d4/d13 0 2024-04-02T04:47:37.910 INFO:tasks.workunit.client.0.smithi070.stdout:2/118: dread d2/fc [4194304,4194304] 0 2024-04-02T04:47:37.913 INFO:tasks.workunit.client.0.smithi070.stdout:2/119: symlink d2/d10/d19/l22 0 2024-04-02T04:47:37.916 INFO:tasks.workunit.client.0.smithi070.stdout:2/120: symlink d2/d10/d19/l23 0 2024-04-02T04:47:37.916 INFO:tasks.workunit.client.0.smithi070.stdout:9/63: dwrite d4/f9 [0,4194304] 0 2024-04-02T04:47:37.922 INFO:tasks.workunit.client.0.smithi070.stdout:9/64: write d4/f9 [5204444,64968] 0 2024-04-02T04:47:37.924 INFO:tasks.workunit.client.0.smithi070.stdout:9/65: unlink d4/cb 0 2024-04-02T04:47:37.927 INFO:tasks.workunit.client.0.smithi070.stdout:9/66: creat d4/d10/d11/f14 x:0 0 0 2024-04-02T04:47:37.930 INFO:tasks.workunit.client.0.smithi070.stdout:9/67: mkdir d4/d10/d12/d15 0 2024-04-02T04:47:37.931 INFO:tasks.workunit.client.0.smithi070.stdout:2/121: dwrite d2/fb [0,4194304] 0 2024-04-02T04:47:37.933 INFO:tasks.workunit.client.0.smithi070.stdout:2/122: readlink d2/d10/d19/l22 0 2024-04-02T04:47:37.933 INFO:tasks.workunit.client.0.smithi070.stdout:9/68: rename d4/d10/d12/d15 to d4/d13/d16 0 2024-04-02T04:47:37.934 INFO:tasks.workunit.client.0.smithi070.stdout:9/69: readlink - no filename 2024-04-02T04:47:37.934 INFO:tasks.workunit.client.0.smithi070.stdout:2/123: chown d2/f8 17169 1 2024-04-02T04:47:37.938 INFO:tasks.workunit.client.0.smithi070.stdout:2/124: mkdir d2/d24 0 2024-04-02T04:47:37.938 INFO:tasks.workunit.client.0.smithi070.stdout:2/125: readlink d2/d10/d19/l23 0 2024-04-02T04:47:37.940 INFO:tasks.workunit.client.0.smithi070.stdout:9/70: rename d4/c6 to d4/d10/c17 0 2024-04-02T04:47:37.941 INFO:tasks.workunit.client.0.smithi070.stdout:2/126: rmdir d2/d12 39 2024-04-02T04:47:37.942 INFO:tasks.workunit.client.0.smithi070.stdout:2/127: dread - d2/f1c zero size 2024-04-02T04:47:37.945 INFO:tasks.workunit.client.0.smithi070.stdout:2/128: dread d2/fc [4194304,4194304] 0 2024-04-02T04:47:37.945 INFO:tasks.workunit.client.0.smithi070.stdout:9/71: dwrite d4/f7 [0,4194304] 0 2024-04-02T04:47:37.947 INFO:tasks.workunit.client.0.smithi070.stdout:2/129: write d2/d10/f20 [1101676,120679] 0 2024-04-02T04:47:37.949 INFO:tasks.workunit.client.0.smithi070.stdout:9/72: creat d4/d10/f18 x:0 0 0 2024-04-02T04:47:37.955 INFO:tasks.workunit.client.0.smithi070.stdout:2/130: truncate d2/fa 1195456 0 2024-04-02T04:47:37.956 INFO:tasks.workunit.client.0.smithi070.stdout:9/73: mkdir d4/d13/d16/d19 0 2024-04-02T04:47:37.957 INFO:tasks.workunit.client.0.smithi070.stdout:9/74: dread - d4/d10/f18 zero size 2024-04-02T04:47:37.958 INFO:tasks.workunit.client.0.smithi070.stdout:2/131: write d2/d12/f13 [603293,69321] 0 2024-04-02T04:47:37.959 INFO:tasks.workunit.client.0.smithi070.stdout:9/75: write d4/d10/d11/f14 [448971,34984] 0 2024-04-02T04:47:37.961 INFO:tasks.workunit.client.0.smithi070.stdout:9/76: chown d4/d10/d12 11 1 2024-04-02T04:47:37.962 INFO:tasks.workunit.client.0.smithi070.stdout:9/77: mknod d4/c1a 0 2024-04-02T04:47:37.964 INFO:tasks.workunit.client.0.smithi070.stdout:9/78: truncate f3 565901 0 2024-04-02T04:47:37.965 INFO:tasks.workunit.client.0.smithi070.stdout:9/79: read - d4/fc zero size 2024-04-02T04:47:37.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:37.963+0000 7fa664ff9700 1 -- 172.21.15.70:0/3288520310 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7fa65c0163b0 con 0x7fa668196c60 2024-04-02T04:47:37.966 INFO:tasks.workunit.client.0.smithi070.stdout:2/132: dwrite d2/d12/f13 [0,4194304] 0 2024-04-02T04:47:37.968 INFO:tasks.workunit.client.0.smithi070.stdout:2/133: write d2/fb [2038257,85771] 0 2024-04-02T04:47:37.970 INFO:tasks.workunit.client.0.smithi070.stdout:9/80: mknod d4/d13/d16/d19/c1b 0 2024-04-02T04:47:37.971 INFO:tasks.workunit.client.0.smithi070.stdout:9/81: chown d4/f8 28 1 2024-04-02T04:47:37.973 INFO:tasks.workunit.client.0.smithi070.stdout:2/134: rename d2/l3 to d2/d24/l25 0 2024-04-02T04:47:37.976 INFO:tasks.workunit.client.0.smithi070.stdout:9/82: dread d4/d10/d11/f14 [0,4194304] 0 2024-04-02T04:47:37.979 INFO:tasks.workunit.client.0.smithi070.stdout:2/135: symlink d2/d24/l26 0 2024-04-02T04:47:37.999 INFO:tasks.workunit.client.0.smithi070.stdout:9/83: creat d4/f1c x:0 0 0 2024-04-02T04:47:38.002 INFO:tasks.workunit.client.1.smithi179.stdout:4/9: sync 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.1.smithi179.stdout:2/2: sync 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.1.smithi179.stdout:4/10: dwrite - no filename 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.1.smithi179.stdout:4/11: dread - no filename 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.1.smithi179.stdout:4/12: fdatasync - no filename 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.1.smithi179.stdout:7/0: sync 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.1.smithi179.stdout:4/13: rename - no filename 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.1.smithi179.stdout:4/14: unlink - no file 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.1.smithi179.stdout:4/15: dread - no filename 2024-04-02T04:47:38.003 INFO:tasks.workunit.client.0.smithi070.stdout:2/136: creat d2/d24/f27 x:0 0 0 2024-04-02T04:47:38.004 INFO:tasks.workunit.client.0.smithi070.stdout:9/84: creat d4/d13/d16/f1d x:0 0 0 2024-04-02T04:47:38.005 INFO:tasks.workunit.client.0.smithi070.stdout:2/137: rename d2/d10/c11 to d2/d24/c28 0 2024-04-02T04:47:38.005 INFO:tasks.workunit.client.0.smithi070.stdout:2/138: stat d2/d10/d19 0 2024-04-02T04:47:38.006 INFO:tasks.workunit.client.0.smithi070.stdout:9/85: mknod d4/d10/d11/c1e 0 2024-04-02T04:47:38.009 INFO:tasks.workunit.client.0.smithi070.stdout:9/86: write d4/f7 [455345,66378] 0 2024-04-02T04:47:38.010 INFO:tasks.workunit.client.0.smithi070.stdout:9/87: dread - d4/d13/d16/f1d zero size 2024-04-02T04:47:38.011 INFO:tasks.workunit.client.0.smithi070.stdout:2/139: symlink d2/d10/d21/l29 0 2024-04-02T04:47:38.013 INFO:tasks.workunit.client.0.smithi070.stdout:9/88: symlink d4/d13/d16/l1f 0 2024-04-02T04:47:38.015 INFO:tasks.workunit.client.0.smithi070.stdout:9/89: read d4/d10/d11/f14 [265744,89044] 0 2024-04-02T04:47:38.015 INFO:tasks.workunit.client.0.smithi070.stdout:9/90: dread - d4/d10/f18 zero size 2024-04-02T04:47:38.016 INFO:tasks.workunit.client.0.smithi070.stdout:9/91: read f3 [231664,124344] 0 2024-04-02T04:47:38.017 INFO:tasks.workunit.client.0.smithi070.stdout:9/92: dread - d4/d13/d16/f1d zero size 2024-04-02T04:47:38.025 INFO:tasks.workunit.client.0.smithi070.stdout:2/140: rename d2/d24/l25 to d2/d10/d21/l2a 0 2024-04-02T04:47:38.031 INFO:tasks.workunit.client.0.smithi070.stdout:2/141: write d2/f4 [4546188,94898] 0 2024-04-02T04:47:38.036 INFO:tasks.workunit.client.0.smithi070.stdout:9/93: getdents d4/d10/d12 0 2024-04-02T04:47:38.041 INFO:tasks.workunit.client.0.smithi070.stdout:9/94: creat d4/d10/f20 x:0 0 0 2024-04-02T04:47:38.059 INFO:tasks.workunit.client.0.smithi070.stdout:2/142: truncate d2/f8 1249419 0 2024-04-02T04:47:38.062 INFO:tasks.workunit.client.0.smithi070.stdout:2/143: mkdir d2/d10/d21/d2b 0 2024-04-02T04:47:38.062 INFO:tasks.workunit.client.0.smithi070.stdout:9/95: getdents d4/d10 0 2024-04-02T04:47:38.063 INFO:tasks.workunit.client.0.smithi070.stdout:9/96: stat d4/d10/d11 0 2024-04-02T04:47:38.065 INFO:tasks.workunit.client.0.smithi070.stdout:2/144: creat d2/d12/f2c x:0 0 0 2024-04-02T04:47:38.066 INFO:tasks.workunit.client.1.smithi179.stdout:9/37: sync 2024-04-02T04:47:38.066 INFO:tasks.workunit.client.1.smithi179.stdout:6/14: sync 2024-04-02T04:47:38.066 INFO:tasks.workunit.client.1.smithi179.stdout:8/4: sync 2024-04-02T04:47:38.066 INFO:tasks.workunit.client.1.smithi179.stdout:5/4: sync 2024-04-02T04:47:38.067 INFO:tasks.workunit.client.1.smithi179.stdout:1/23: sync 2024-04-02T04:47:38.067 INFO:tasks.workunit.client.1.smithi179.stdout:3/25: sync 2024-04-02T04:47:38.067 INFO:tasks.workunit.client.0.smithi070.stdout:9/97: rename d4/f1c to d4/d10/d11/f21 0 2024-04-02T04:47:38.067 INFO:tasks.workunit.client.0.smithi070.stdout:9/98: fsync d4/f7 0 2024-04-02T04:47:38.068 INFO:tasks.workunit.client.1.smithi179.stdout:3/26: write f4 [140578,10561] 0 2024-04-02T04:47:38.068 INFO:tasks.workunit.client.0.smithi070.stdout:2/145: creat d2/d24/f2d x:0 0 0 2024-04-02T04:47:38.071 INFO:tasks.workunit.client.1.smithi179.stdout:6/15: dread f2 [0,4194304] 0 2024-04-02T04:47:38.073 INFO:tasks.workunit.client.1.smithi179.stdout:1/24: dwrite d3/d5/f7 [0,4194304] 0 2024-04-02T04:47:38.073 INFO:tasks.workunit.client.0.smithi070.stdout:2/146: rename d2/d10/d19/l22 to d2/d10/d19/l2e 0 2024-04-02T04:47:38.073 INFO:tasks.workunit.client.1.smithi179.stdout:1/25: read d3/d5/f7 [230376,109423] 0 2024-04-02T04:47:38.075 INFO:tasks.workunit.client.0.smithi070.stdout:9/99: dwrite d4/d10/f18 [0,4194304] 0 2024-04-02T04:47:38.076 INFO:tasks.workunit.client.0.smithi070.stdout:9/100: write d4/f9 [459390,97211] 0 2024-04-02T04:47:38.077 INFO:tasks.workunit.client.0.smithi070.stdout:2/147: mknod d2/d10/d19/c2f 0 2024-04-02T04:47:38.077 INFO:tasks.workunit.client.0.smithi070.stdout:9/101: chown d4/d13/d16/l1f 38129 1 2024-04-02T04:47:38.078 INFO:tasks.workunit.client.0.smithi070.stdout:9/102: stat d4/d10/f18 0 2024-04-02T04:47:38.086 INFO:tasks.workunit.client.0.smithi070.stdout:9/103: rename d4/d10/f20 to d4/d13/d16/d19/f22 0 2024-04-02T04:47:38.087 INFO:tasks.workunit.client.0.smithi070.stdout:2/148: dwrite d2/f1b [0,4194304] 0 2024-04-02T04:47:38.088 INFO:tasks.workunit.client.0.smithi070.stdout:9/104: write d4/fc [158934,79530] 0 2024-04-02T04:47:38.095 INFO:tasks.workunit.client.0.smithi070.stdout:2/149: mkdir d2/d10/d19/d30 0 2024-04-02T04:47:38.096 INFO:tasks.workunit.client.0.smithi070.stdout:2/150: fdatasync d2/d12/f2c 0 2024-04-02T04:47:38.097 INFO:tasks.workunit.client.0.smithi070.stdout:2/151: chown d2/d10/d21/l29 1616 1 2024-04-02T04:47:38.097 INFO:tasks.workunit.client.0.smithi070.stdout:9/105: rmdir d4/d10 39 2024-04-02T04:47:38.098 INFO:tasks.workunit.client.0.smithi070.stdout:2/152: write d2/f1b [1259241,109392] 0 2024-04-02T04:47:38.134 INFO:tasks.workunit.client.0.smithi070.stdout:2/153: rename d2/fc to d2/d10/d21/f31 0 2024-04-02T04:47:38.135 INFO:tasks.workunit.client.0.smithi070.stdout:2/154: readlink d2/l15 0 2024-04-02T04:47:38.135 INFO:tasks.workunit.client.0.smithi070.stdout:2/155: read - d2/d24/f27 zero size 2024-04-02T04:47:38.135 INFO:tasks.workunit.client.1.smithi179.stdout:5/5: fsync f1 0 2024-04-02T04:47:38.137 INFO:tasks.workunit.client.1.smithi179.stdout:9/38: rmdir d0 39 2024-04-02T04:47:38.139 INFO:tasks.workunit.client.1.smithi179.stdout:3/27: rename f3 to f5 0 2024-04-02T04:47:38.141 INFO:tasks.workunit.client.1.smithi179.stdout:6/16: rename f0 to d3/fb 0 2024-04-02T04:47:38.144 INFO:tasks.workunit.client.1.smithi179.stdout:1/26: symlink d3/l9 0 2024-04-02T04:47:38.145 INFO:tasks.workunit.client.0.smithi070.stdout:2/156: dwrite d2/f1c [0,4194304] 0 2024-04-02T04:47:38.146 INFO:tasks.workunit.client.1.smithi179.stdout:2/3: symlink l0 0 2024-04-02T04:47:38.146 INFO:tasks.workunit.client.1.smithi179.stdout:2/4: chown l0 4152 1 2024-04-02T04:47:38.148 INFO:tasks.workunit.client.1.smithi179.stdout:0/94: sync 2024-04-02T04:47:38.148 INFO:tasks.workunit.client.1.smithi179.stdout:7/1: symlink l0 0 2024-04-02T04:47:38.148 INFO:tasks.workunit.client.1.smithi179.stdout:7/2: write - no filename 2024-04-02T04:47:38.150 INFO:tasks.workunit.client.1.smithi179.stdout:4/16: creat f0 x:0 0 0 2024-04-02T04:47:38.150 INFO:tasks.workunit.client.1.smithi179.stdout:4/17: dread - f0 zero size 2024-04-02T04:47:38.153 INFO:tasks.workunit.client.1.smithi179.stdout:5/6: creat f2 x:0 0 0 2024-04-02T04:47:38.153 INFO:tasks.workunit.client.1.smithi179.stdout:5/7: readlink - no filename 2024-04-02T04:47:38.156 INFO:tasks.workunit.client.1.smithi179.stdout:8/5: symlink l0 0 2024-04-02T04:47:38.159 INFO:tasks.workunit.client.0.smithi070.stdout:2/157: rename d2/d12/f2c to d2/f32 0 2024-04-02T04:47:38.161 INFO:tasks.workunit.client.1.smithi179.stdout:3/28: write f5 [457495,96568] 0 2024-04-02T04:47:38.165 INFO:tasks.workunit.client.1.smithi179.stdout:1/27: fsync d3/d5/f7 0 2024-04-02T04:47:38.166 INFO:tasks.workunit.client.1.smithi179.stdout:1/28: write d3/d5/f7 [1975230,3784] 0 2024-04-02T04:47:38.166 INFO:tasks.workunit.client.1.smithi179.stdout:2/5: symlink l1 0 2024-04-02T04:47:38.166 INFO:tasks.workunit.client.1.smithi179.stdout:2/6: dread - no filename 2024-04-02T04:47:38.167 INFO:tasks.workunit.client.1.smithi179.stdout:1/29: write d3/d5/f7 [201288,47199] 0 2024-04-02T04:47:38.169 INFO:tasks.workunit.client.1.smithi179.stdout:7/3: mkdir d1 0 2024-04-02T04:47:38.170 INFO:tasks.workunit.client.1.smithi179.stdout:4/18: unlink f0 0 2024-04-02T04:47:38.170 INFO:tasks.workunit.client.1.smithi179.stdout:4/19: unlink - no file 2024-04-02T04:47:38.170 INFO:tasks.workunit.client.1.smithi179.stdout:4/20: rmdir - no directory 2024-04-02T04:47:38.170 INFO:tasks.workunit.client.1.smithi179.stdout:4/21: dread - no filename 2024-04-02T04:47:38.171 INFO:tasks.workunit.client.1.smithi179.stdout:5/8: symlink l3 0 2024-04-02T04:47:38.174 INFO:tasks.workunit.client.1.smithi179.stdout:3/29: symlink l6 0 2024-04-02T04:47:38.175 INFO:tasks.workunit.client.1.smithi179.stdout:3/30: write f4 [838711,90844] 0 2024-04-02T04:47:38.175 INFO:tasks.workunit.client.1.smithi179.stdout:6/17: link d3/fb d3/fc 0 2024-04-02T04:47:38.181 INFO:tasks.workunit.client.1.smithi179.stdout:4/22: mkdir d1 0 2024-04-02T04:47:38.181 INFO:tasks.workunit.client.1.smithi179.stdout:4/23: dwrite - no filename 2024-04-02T04:47:38.182 INFO:tasks.workunit.client.1.smithi179.stdout:4/24: rename d1 to d1/d2 22 2024-04-02T04:47:38.182 INFO:tasks.workunit.client.1.smithi179.stdout:4/25: write - no filename 2024-04-02T04:47:38.187 INFO:tasks.workunit.client.1.smithi179.stdout:7/4: symlink d1/l2 0 2024-04-02T04:47:38.187 INFO:tasks.workunit.client.1.smithi179.stdout:7/5: dwrite - no filename 2024-04-02T04:47:38.187 INFO:tasks.workunit.client.1.smithi179.stdout:7/6: readlink l0 0 2024-04-02T04:47:38.189 INFO:tasks.workunit.client.1.smithi179.stdout:0/95: getdents d1/db 0 2024-04-02T04:47:38.191 INFO:tasks.workunit.client.1.smithi179.stdout:4/26: mkdir d1/d3 0 2024-04-02T04:47:38.191 INFO:tasks.workunit.client.1.smithi179.stdout:4/27: read - no filename 2024-04-02T04:47:38.191 INFO:tasks.workunit.client.1.smithi179.stdout:4/28: chown d1 2528 1 2024-04-02T04:47:38.191 INFO:tasks.workunit.client.1.smithi179.stdout:4/29: write - no filename 2024-04-02T04:47:38.192 INFO:tasks.workunit.client.1.smithi179.stdout:8/6: getdents . 0 2024-04-02T04:47:38.192 INFO:tasks.workunit.client.1.smithi179.stdout:8/7: write - no filename 2024-04-02T04:47:38.192 INFO:tasks.workunit.client.1.smithi179.stdout:8/8: truncate - no filename 2024-04-02T04:47:38.192 INFO:tasks.workunit.client.1.smithi179.stdout:8/9: write - no filename 2024-04-02T04:47:38.192 INFO:tasks.workunit.client.1.smithi179.stdout:8/10: dwrite - no filename 2024-04-02T04:47:38.192 INFO:tasks.workunit.client.0.smithi070.stdout:2/158: fsync d2/f32 0 2024-04-02T04:47:38.197 INFO:tasks.workunit.client.1.smithi179.stdout:1/30: getdents d3 0 2024-04-02T04:47:38.198 INFO:tasks.workunit.client.0.smithi070.stdout:2/159: dread d2/f8 [0,4194304] 0 2024-04-02T04:47:38.198 INFO:tasks.workunit.client.1.smithi179.stdout:4/30: creat d1/f4 x:0 0 0 2024-04-02T04:47:38.200 INFO:tasks.workunit.client.1.smithi179.stdout:8/11: creat f1 x:0 0 0 2024-04-02T04:47:38.200 INFO:tasks.workunit.client.1.smithi179.stdout:1/31: dread d3/d5/f7 [0,4194304] 0 2024-04-02T04:47:38.200 INFO:tasks.workunit.client.1.smithi179.stdout:1/32: chown d3/l8 31541381 1 2024-04-02T04:47:38.201 INFO:tasks.workunit.client.0.smithi070.stdout:2/160: rmdir d2/d12 39 2024-04-02T04:47:38.202 INFO:tasks.workunit.client.1.smithi179.stdout:9/39: write d0/f5 [841804,46903] 0 2024-04-02T04:47:38.203 INFO:tasks.workunit.client.1.smithi179.stdout:3/31: getdents . 0 2024-04-02T04:47:38.203 INFO:tasks.workunit.client.1.smithi179.stdout:3/32: chown l6 3783869 1 2024-04-02T04:47:38.204 INFO:tasks.workunit.client.0.smithi070.stdout:2/161: rmdir d2/d10 39 2024-04-02T04:47:38.205 INFO:tasks.workunit.client.1.smithi179.stdout:1/33: dread d3/d5/f7 [0,4194304] 0 2024-04-02T04:47:38.206 INFO:tasks.workunit.client.1.smithi179.stdout:4/31: creat d1/f5 x:0 0 0 2024-04-02T04:47:38.206 INFO:tasks.workunit.client.1.smithi179.stdout:9/40: dread d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:38.207 INFO:tasks.workunit.client.1.smithi179.stdout:4/32: dread - d1/f4 zero size 2024-04-02T04:47:38.208 INFO:tasks.workunit.client.1.smithi179.stdout:4/33: write d1/f5 [974881,65035] 0 2024-04-02T04:47:38.208 INFO:tasks.workunit.client.1.smithi179.stdout:4/34: chown d1/f5 195223 1 2024-04-02T04:47:38.209 INFO:tasks.workunit.client.1.smithi179.stdout:4/35: chown d1/f4 73363725 1 2024-04-02T04:47:38.209 INFO:tasks.workunit.client.1.smithi179.stdout:4/36: chown d1/f4 2943681 1 2024-04-02T04:47:38.210 INFO:tasks.workunit.client.1.smithi179.stdout:9/41: dread d0/f5 [0,4194304] 0 2024-04-02T04:47:38.211 INFO:tasks.workunit.client.0.smithi070.stdout:2/162: creat d2/d10/d21/f33 x:0 0 0 2024-04-02T04:47:38.213 INFO:tasks.workunit.client.1.smithi179.stdout:9/42: dread d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:38.213 INFO:tasks.workunit.client.1.smithi179.stdout:3/33: mknod c7 0 2024-04-02T04:47:38.215 INFO:tasks.workunit.client.1.smithi179.stdout:1/34: creat d3/fa x:0 0 0 2024-04-02T04:47:38.217 INFO:tasks.workunit.client.0.smithi070.stdout:2/163: rename d2/d24/f27 to d2/d10/d21/f34 0 2024-04-02T04:47:38.222 INFO:tasks.workunit.client.1.smithi179.stdout:4/37: rmdir d1 39 2024-04-02T04:47:38.224 INFO:tasks.workunit.client.1.smithi179.stdout:9/43: write d0/d1/f7 [1893960,23031] 0 2024-04-02T04:47:38.225 INFO:tasks.workunit.client.1.smithi179.stdout:3/34: mkdir d8 0 2024-04-02T04:47:38.225 INFO:tasks.workunit.client.1.smithi179.stdout:9/44: write d0/d1/f7 [5140078,104272] 0 2024-04-02T04:47:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.223+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 --> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa654000bf0 con 0x7fa650039110 2024-04-02T04:47:38.226 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:37 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.70:0/3837605501' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.igidvr/crt"}]: dispatch 2024-04-02T04:47:38.226 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:37 smithi070 ceph-mon[29212]: Standby manager daemon smithi070.igidvr started 2024-04-02T04:47:38.226 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:37 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.70:0/3837605501' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:47:38.227 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:37 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.70:0/3837605501' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.igidvr/key"}]: dispatch 2024-04-02T04:47:38.227 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:37 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.70:0/3837605501' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:47:38.227 INFO:tasks.workunit.client.1.smithi179.stdout:1/35: unlink c1 0 2024-04-02T04:47:38.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.225+0000 7fa664ff9700 1 -- 172.21.15.70:0/3288520310 <== mgr.24359 v2:172.21.15.179:6828/1862820759 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fa654000bf0 con 0x7fa650039110 2024-04-02T04:47:38.228 INFO:tasks.workunit.client.1.smithi179.stdout:8/12: link l0 l2 0 2024-04-02T04:47:38.228 INFO:tasks.workunit.client.1.smithi179.stdout:8/13: fsync f1 0 2024-04-02T04:47:38.229 INFO:tasks.workunit.client.1.smithi179.stdout:0/96: link d1/da/l16 d1/l18 0 2024-04-02T04:47:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.228+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fa650039110 msgr2=0x7fa65003b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.228+0000 7fa66e142700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fa650039110 0x7fa65003b5d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa658005950 tx=0x7fa658009260 comp rx=0 tx=0).stop 2024-04-02T04:47:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.228+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668196c60 msgr2=0x7fa66819bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.228+0000 7fa66e142700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668196c60 0x7fa66819bcd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa65c008000 tx=0x7fa65c00fa00 comp rx=0 tx=0).stop 2024-04-02T04:47:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.229+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 shutdown_connections 2024-04-02T04:47:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.229+0000 7fa66e142700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fa650039110 0x7fa65003b5d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.229+0000 7fa66e142700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680fef60 0x7fa668196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.229+0000 7fa66e142700 1 --2- 172.21.15.70:0/3288520310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668196c60 0x7fa66819bcd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.229+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 >> 172.21.15.70:0/3288520310 conn(0x7fa6680fa8c0 msgr2=0x7fa6680fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.229+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 shutdown_connections 2024-04-02T04:47:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.229+0000 7fa66e142700 1 -- 172.21.15.70:0/3288520310 wait complete. 2024-04-02T04:47:38.234 INFO:tasks.workunit.client.1.smithi179.stdout:1/36: write d3/d5/f7 [4893517,23157] 0 2024-04-02T04:47:38.234 INFO:tasks.workunit.client.0.smithi070.stdout:2/164: dwrite d2/f8 [0,4194304] 0 2024-04-02T04:47:38.235 INFO:tasks.workunit.client.1.smithi179.stdout:8/14: chown l2 75320428 1 2024-04-02T04:47:38.236 INFO:tasks.workunit.client.1.smithi179.stdout:8/15: dread - f1 zero size 2024-04-02T04:47:38.236 INFO:tasks.workunit.client.1.smithi179.stdout:3/35: symlink d8/l9 0 2024-04-02T04:47:38.237 INFO:tasks.workunit.client.1.smithi179.stdout:4/38: dwrite d1/f5 [0,4194304] 0 2024-04-02T04:47:38.237 INFO:tasks.workunit.client.1.smithi179.stdout:4/39: stat d1/f5 0 2024-04-02T04:47:38.237 INFO:tasks.workunit.client.1.smithi179.stdout:8/16: truncate f1 159274 0 2024-04-02T04:47:38.238 INFO:tasks.workunit.client.1.smithi179.stdout:9/45: creat d0/d1/db/fe x:0 0 0 2024-04-02T04:47:38.238 INFO:tasks.workunit.client.1.smithi179.stdout:3/36: read f4 [678154,110431] 0 2024-04-02T04:47:38.239 INFO:tasks.workunit.client.1.smithi179.stdout:3/37: read f5 [495904,94135] 0 2024-04-02T04:47:38.241 INFO:tasks.workunit.client.0.smithi070.stdout:2/165: rename d2/d10/d19/c2f to d2/d10/d19/c35 0 2024-04-02T04:47:38.242 INFO:tasks.workunit.client.0.smithi070.stdout:2/166: write d2/f32 [224161,96711] 0 2024-04-02T04:47:38.245 INFO:tasks.workunit.client.1.smithi179.stdout:1/37: unlink d3/d5/l6 0 2024-04-02T04:47:38.245 INFO:tasks.workunit.client.1.smithi179.stdout:8/17: dread f1 [0,4194304] 0 2024-04-02T04:47:38.247 INFO:tasks.workunit.client.1.smithi179.stdout:1/38: truncate d3/fa 116950 0 2024-04-02T04:47:38.247 INFO:tasks.workunit.client.1.smithi179.stdout:3/38: dread f4 [0,4194304] 0 2024-04-02T04:47:38.247 INFO:tasks.workunit.client.1.smithi179.stdout:3/39: readlink l6 0 2024-04-02T04:47:38.248 INFO:tasks.workunit.client.0.smithi070.stdout:2/167: rename d2/d10/d19/c1d to d2/d10/d19/d30/c36 0 2024-04-02T04:47:38.249 INFO:tasks.workunit.client.1.smithi179.stdout:4/40: rmdir d1 39 2024-04-02T04:47:38.250 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:47:38.251 INFO:tasks.workunit.client.0.smithi070.stdout:2/168: creat d2/d24/f37 x:0 0 0 2024-04-02T04:47:38.253 INFO:tasks.workunit.client.1.smithi179.stdout:8/18: mkdir d3 0 2024-04-02T04:47:38.254 INFO:tasks.workunit.client.0.smithi070.stdout:4/56: sync 2024-04-02T04:47:38.254 INFO:tasks.workunit.client.1.smithi179.stdout:0/97: dwrite d1/f6 [4194304,4194304] 0 2024-04-02T04:47:38.254 INFO:tasks.workunit.client.0.smithi070.stdout:4/57: readlink d1/l10 0 2024-04-02T04:47:38.255 INFO:tasks.workunit.client.1.smithi179.stdout:1/39: unlink d3/d5/f7 0 2024-04-02T04:47:38.256 INFO:tasks.workunit.client.1.smithi179.stdout:3/40: symlink d8/la 0 2024-04-02T04:47:38.257 INFO:tasks.workunit.client.1.smithi179.stdout:9/46: symlink d0/d1/lf 0 2024-04-02T04:47:38.258 INFO:tasks.workunit.client.0.smithi070.stdout:4/58: creat d1/d3/d9/f15 x:0 0 0 2024-04-02T04:47:38.259 INFO:tasks.workunit.client.0.smithi070.stdout:4/59: dread - d1/fd zero size 2024-04-02T04:47:38.260 INFO:tasks.workunit.client.0.smithi070.stdout:2/169: dwrite d2/f32 [0,4194304] 0 2024-04-02T04:47:38.261 INFO:tasks.workunit.client.0.smithi070.stdout:2/170: fsync d2/d10/d19/f1e 0 2024-04-02T04:47:38.262 INFO:tasks.workunit.client.0.smithi070.stdout:4/60: truncate d1/d3/d9/f11 631057 0 2024-04-02T04:47:38.310 INFO:tasks.workunit.client.1.smithi179.stdout:9/47: rmdir d0/d1/db 39 2024-04-02T04:47:38.311 INFO:tasks.workunit.client.1.smithi179.stdout:8/19: creat d3/f4 x:0 0 0 2024-04-02T04:47:38.314 INFO:tasks.workunit.client.0.smithi070.stdout:2/171: creat d2/d12/f38 x:0 0 0 2024-04-02T04:47:38.317 INFO:tasks.workunit.client.1.smithi179.stdout:4/41: mknod d1/d3/c6 0 2024-04-02T04:47:38.317 INFO:tasks.workunit.client.1.smithi179.stdout:4/42: readlink - no filename 2024-04-02T04:47:38.317 INFO:tasks.workunit.client.0.smithi070.stdout:2/172: creat d2/d10/d19/f39 x:0 0 0 2024-04-02T04:47:38.319 INFO:tasks.workunit.client.1.smithi179.stdout:8/20: fdatasync f1 0 2024-04-02T04:47:38.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:37 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.70:0/3837605501' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.igidvr/crt"}]: dispatch 2024-04-02T04:47:38.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:37 smithi179 ceph-mon[36603]: Standby manager daemon smithi070.igidvr started 2024-04-02T04:47:38.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:37 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.70:0/3837605501' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:47:38.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:37 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.70:0/3837605501' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.igidvr/key"}]: dispatch 2024-04-02T04:47:38.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:37 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.70:0/3837605501' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:47:38.322 INFO:tasks.workunit.client.1.smithi179.stdout:4/43: creat d1/d3/f7 x:0 0 0 2024-04-02T04:47:38.322 INFO:tasks.workunit.client.1.smithi179.stdout:4/44: fdatasync d1/f4 0 2024-04-02T04:47:38.323 INFO:tasks.workunit.client.0.smithi070.stdout:2/173: symlink d2/l3a 0 2024-04-02T04:47:38.323 INFO:tasks.workunit.client.1.smithi179.stdout:4/45: dread - d1/f4 zero size 2024-04-02T04:47:38.324 INFO:tasks.workunit.client.0.smithi070.stdout:2/174: write d2/f4 [1342595,118833] 0 2024-04-02T04:47:38.325 INFO:tasks.workunit.client.0.smithi070.stdout:4/61: write d1/fd [334613,14289] 0 2024-04-02T04:47:38.325 INFO:tasks.workunit.client.1.smithi179.stdout:8/21: mknod d3/c5 0 2024-04-02T04:47:38.327 INFO:tasks.workunit.client.0.smithi070.stdout:4/62: write d1/d3/d9/f15 [758263,82856] 0 2024-04-02T04:47:38.327 INFO:tasks.workunit.client.1.smithi179.stdout:1/40: getdents d3/d5 0 2024-04-02T04:47:38.328 INFO:tasks.workunit.client.1.smithi179.stdout:1/41: fsync d3/fa 0 2024-04-02T04:47:38.328 INFO:tasks.workunit.client.0.smithi070.stdout:2/175: write d2/f14 [404415,130645] 0 2024-04-02T04:47:38.330 INFO:tasks.workunit.client.1.smithi179.stdout:4/46: dwrite d1/f5 [0,4194304] 0 2024-04-02T04:47:38.330 INFO:tasks.workunit.client.0.smithi070.stdout:4/63: readlink d1/d3/l7 0 2024-04-02T04:47:38.331 INFO:tasks.workunit.client.0.smithi070.stdout:4/64: chown d1/c14 1 1 2024-04-02T04:47:38.332 INFO:tasks.workunit.client.0.smithi070.stdout:4/65: read d1/fb [2090303,6798] 0 2024-04-02T04:47:38.333 INFO:tasks.workunit.client.0.smithi070.stdout:4/66: write d1/fe [1106985,33445] 0 2024-04-02T04:47:38.334 INFO:tasks.workunit.client.1.smithi179.stdout:8/22: mknod d3/c6 0 2024-04-02T04:47:38.335 INFO:tasks.workunit.client.1.smithi179.stdout:0/98: dwrite d1/fe [0,4194304] 0 2024-04-02T04:47:38.335 INFO:tasks.workunit.client.1.smithi179.stdout:8/23: dread f1 [0,4194304] 0 2024-04-02T04:47:38.340 INFO:tasks.workunit.client.1.smithi179.stdout:1/42: creat d3/d5/fb x:0 0 0 2024-04-02T04:47:38.342 INFO:tasks.workunit.client.1.smithi179.stdout:4/47: mknod d1/c8 0 2024-04-02T04:47:38.343 INFO:tasks.workunit.client.1.smithi179.stdout:4/48: truncate d1/d3/f7 116470 0 2024-04-02T04:47:38.343 INFO:tasks.workunit.client.1.smithi179.stdout:4/49: stat d1/d3 0 2024-04-02T04:47:38.343 INFO:tasks.workunit.client.1.smithi179.stdout:1/43: dread d3/fa [0,4194304] 0 2024-04-02T04:47:38.345 INFO:tasks.workunit.client.1.smithi179.stdout:9/48: getdents d0/d1 0 2024-04-02T04:47:38.346 INFO:tasks.workunit.client.1.smithi179.stdout:9/49: read d0/d1/f7 [2950136,118780] 0 2024-04-02T04:47:38.347 INFO:tasks.workunit.client.0.smithi070.stdout:2/176: truncate d2/f1c 2376908 0 2024-04-02T04:47:38.352 INFO:tasks.workunit.client.0.smithi070.stdout:4/67: dwrite d1/fc [0,4194304] 0 2024-04-02T04:47:38.352 INFO:tasks.workunit.client.1.smithi179.stdout:8/24: creat d3/f7 x:0 0 0 2024-04-02T04:47:38.354 INFO:tasks.workunit.client.0.smithi070.stdout:4/68: write d1/d3/d9/f11 [1435942,8634] 0 2024-04-02T04:47:38.355 INFO:tasks.workunit.client.1.smithi179.stdout:4/50: rmdir d1 39 2024-04-02T04:47:38.356 INFO:tasks.workunit.client.0.smithi070.stdout:4/69: truncate d1/d3/d9/f15 1838889 0 2024-04-02T04:47:38.356 INFO:tasks.workunit.client.1.smithi179.stdout:1/44: symlink d3/lc 0 2024-04-02T04:47:38.357 INFO:tasks.workunit.client.1.smithi179.stdout:1/45: write d3/fa [1035179,86853] 0 2024-04-02T04:47:38.357 INFO:tasks.workunit.client.1.smithi179.stdout:0/99: write d1/f14 [881875,25055] 0 2024-04-02T04:47:38.359 INFO:tasks.workunit.client.1.smithi179.stdout:1/46: write d3/d5/fb [1026206,300] 0 2024-04-02T04:47:38.361 INFO:tasks.workunit.client.1.smithi179.stdout:9/50: mkdir d0/d1/d10 0 2024-04-02T04:47:38.365 INFO:tasks.workunit.client.1.smithi179.stdout:8/25: mknod d3/c8 0 2024-04-02T04:47:38.365 INFO:tasks.workunit.client.1.smithi179.stdout:8/26: chown f1 337137600 1 2024-04-02T04:47:38.365 INFO:tasks.workunit.client.0.smithi070.stdout:4/70: mknod d1/d3/c16 0 2024-04-02T04:47:38.366 INFO:tasks.workunit.client.0.smithi070.stdout:4/71: chown d1 3415 1 2024-04-02T04:47:38.366 INFO:tasks.workunit.client.1.smithi179.stdout:4/51: write d1/f4 [220982,117573] 0 2024-04-02T04:47:38.367 INFO:tasks.workunit.client.1.smithi179.stdout:0/100: symlink d1/da/l19 0 2024-04-02T04:47:38.368 INFO:tasks.workunit.client.0.smithi070.stdout:2/177: link d2/d10/d19/c35 d2/d10/c3b 0 2024-04-02T04:47:38.370 INFO:tasks.workunit.client.0.smithi070.stdout:4/72: creat d1/d3/d9/f17 x:0 0 0 2024-04-02T04:47:38.370 INFO:tasks.workunit.client.0.smithi070.stdout:4/73: readlink d1/l10 0 2024-04-02T04:47:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.368+0000 7f28999ee700 1 -- 172.21.15.70:0/1232757139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ffc10 msgr2=0x7f2894100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.368+0000 7f28999ee700 1 --2- 172.21.15.70:0/1232757139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ffc10 0x7f2894100030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2888009910 tx=0x7f2888009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.368+0000 7f28999ee700 1 -- 172.21.15.70:0/1232757139 shutdown_connections 2024-04-02T04:47:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.368+0000 7f28999ee700 1 --2- 172.21.15.70:0/1232757139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2894100e10 0x7f2894101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.368+0000 7f28999ee700 1 --2- 172.21.15.70:0/1232757139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ffc10 0x7f2894100030 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.368+0000 7f28999ee700 1 -- 172.21.15.70:0/1232757139 >> 172.21.15.70:0/1232757139 conn(0x7f28940fb190 msgr2=0x7f28940fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.368+0000 7f28999ee700 1 -- 172.21.15.70:0/1232757139 shutdown_connections 2024-04-02T04:47:38.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.368+0000 7f28999ee700 1 -- 172.21.15.70:0/1232757139 wait complete. 2024-04-02T04:47:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28999ee700 1 Processor -- start 2024-04-02T04:47:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28999ee700 1 -- start start 2024-04-02T04:47:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28999ee700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28940ffc10 0x7f28941942a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:38.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28999ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894100e10 0x7f28941947e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28999ee700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2894194e00 con 0x7f28940ffc10 2024-04-02T04:47:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28999ee700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2894194f40 con 0x7f2894100e10 2024-04-02T04:47:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28927fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894100e10 0x7f28941947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28940ffc10 0x7f28941942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28927fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894100e10 0x7f28941947e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:39698/0 (socket says 172.21.15.70:39698) 2024-04-02T04:47:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.369+0000 7f28927fc700 1 -- 172.21.15.70:0/3487186718 learned_addr learned my addr 172.21.15.70:0/3487186718 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.370+0000 7f2892ffd700 1 -- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894100e10 msgr2=0x7f28941947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.370+0000 7f2892ffd700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894100e10 0x7f28941947e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.370+0000 7f2892ffd700 1 -- 172.21.15.70:0/3487186718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28880095f0 con 0x7f28940ffc10 2024-04-02T04:47:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.370+0000 7f2892ffd700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28940ffc10 0x7f28941942a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2888009910 tx=0x7f288800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:38.374 INFO:tasks.workunit.client.0.smithi070.stdout:4/74: dread d1/fa [0,4194304] 0 2024-04-02T04:47:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.371+0000 7f28989ec700 1 -- 172.21.15.70:0/3487186718 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f288801d070 con 0x7f28940ffc10 2024-04-02T04:47:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.371+0000 7f28989ec700 1 -- 172.21.15.70:0/3487186718 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2888005dc0 con 0x7f28940ffc10 2024-04-02T04:47:38.375 INFO:tasks.workunit.client.0.smithi070.stdout:4/75: readlink d1/l10 0 2024-04-02T04:47:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.371+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2894199990 con 0x7f28940ffc10 2024-04-02T04:47:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.371+0000 7f28989ec700 1 -- 172.21.15.70:0/3487186718 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2888017630 con 0x7f28940ffc10 2024-04-02T04:47:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.372+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2894199e80 con 0x7f28940ffc10 2024-04-02T04:47:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.373+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28941075f0 con 0x7f28940ffc10 2024-04-02T04:47:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.373+0000 7f28989ec700 1 -- 172.21.15.70:0/3487186718 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f2888017790 con 0x7f28940ffc10 2024-04-02T04:47:38.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.374+0000 7f28989ec700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f287c06d0a0 0x7f287c06f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:38.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.374+0000 7f28989ec700 1 -- 172.21.15.70:0/3487186718 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f288808db70 con 0x7f28940ffc10 2024-04-02T04:47:38.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.374+0000 7f28927fc700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f287c06d0a0 0x7f287c06f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:38.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.374+0000 7f28927fc700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f287c06d0a0 0x7f287c06f560 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f288400bf30 tx=0x7f288400d040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:38.378 INFO:tasks.workunit.client.0.smithi070.stdout:4/76: rename d1/d3/d9/f15 to d1/f18 0 2024-04-02T04:47:38.382 INFO:tasks.workunit.client.0.smithi070.stdout:4/77: mkdir d1/d3/d19 0 2024-04-02T04:47:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.380+0000 7f28989ec700 1 -- 172.21.15.70:0/3487186718 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f288805b190 con 0x7f28940ffc10 2024-04-02T04:47:38.424 INFO:tasks.workunit.client.1.smithi179.stdout:1/47: creat d3/fd x:0 0 0 2024-04-02T04:47:38.424 INFO:tasks.workunit.client.1.smithi179.stdout:4/52: mknod d1/c9 0 2024-04-02T04:47:38.424 INFO:tasks.workunit.client.1.smithi179.stdout:8/27: creat d3/f9 x:0 0 0 2024-04-02T04:47:38.425 INFO:tasks.workunit.client.1.smithi179.stdout:1/48: fdatasync d3/fa 0 2024-04-02T04:47:38.427 INFO:tasks.workunit.client.1.smithi179.stdout:4/53: creat d1/d3/fa x:0 0 0 2024-04-02T04:47:38.428 INFO:tasks.workunit.client.1.smithi179.stdout:9/51: mkdir d0/d1/d11 0 2024-04-02T04:47:38.429 INFO:tasks.workunit.client.1.smithi179.stdout:0/101: dwrite d1/f6 [4194304,4194304] 0 2024-04-02T04:47:38.429 INFO:tasks.workunit.client.1.smithi179.stdout:1/49: mkdir d3/d5/de 0 2024-04-02T04:47:38.429 INFO:tasks.workunit.client.1.smithi179.stdout:8/28: write f1 [961850,70769] 0 2024-04-02T04:47:38.430 INFO:tasks.workunit.client.1.smithi179.stdout:0/102: write d1/f6 [6733385,9791] 0 2024-04-02T04:47:38.430 INFO:tasks.workunit.client.1.smithi179.stdout:4/54: creat d1/fb x:0 0 0 2024-04-02T04:47:38.431 INFO:tasks.workunit.client.1.smithi179.stdout:8/29: chown d3/f7 0 1 2024-04-02T04:47:38.431 INFO:tasks.workunit.client.1.smithi179.stdout:9/52: fsync d0/fd 0 2024-04-02T04:47:38.434 INFO:tasks.workunit.client.1.smithi179.stdout:4/55: chown d1/d3/c6 18604 1 2024-04-02T04:47:38.435 INFO:tasks.workunit.client.1.smithi179.stdout:8/30: mknod d3/ca 0 2024-04-02T04:47:38.436 INFO:tasks.workunit.client.1.smithi179.stdout:4/56: fsync d1/f5 0 2024-04-02T04:47:38.436 INFO:tasks.workunit.client.1.smithi179.stdout:8/31: readlink l2 0 2024-04-02T04:47:38.439 INFO:tasks.workunit.client.1.smithi179.stdout:0/103: rmdir d1/d15 0 2024-04-02T04:47:38.440 INFO:tasks.workunit.client.1.smithi179.stdout:8/32: creat d3/fb x:0 0 0 2024-04-02T04:47:38.440 INFO:tasks.workunit.client.1.smithi179.stdout:1/50: dwrite d3/fd [0,4194304] 0 2024-04-02T04:47:38.441 INFO:tasks.workunit.client.1.smithi179.stdout:4/57: mkdir d1/d3/dc 0 2024-04-02T04:47:38.441 INFO:tasks.workunit.client.1.smithi179.stdout:0/104: write d1/db/fc [5175411,7161] 0 2024-04-02T04:47:38.442 INFO:tasks.workunit.client.1.smithi179.stdout:8/33: dread - d3/f7 zero size 2024-04-02T04:47:38.443 INFO:tasks.workunit.client.1.smithi179.stdout:9/53: getdents d0/d1/db 0 2024-04-02T04:47:38.443 INFO:tasks.workunit.client.1.smithi179.stdout:0/105: dread - d1/da/f17 zero size 2024-04-02T04:47:38.443 INFO:tasks.workunit.client.1.smithi179.stdout:8/34: read f1 [368782,66740] 0 2024-04-02T04:47:38.444 INFO:tasks.workunit.client.1.smithi179.stdout:1/51: write d3/fd [2612817,97190] 0 2024-04-02T04:47:38.447 INFO:tasks.workunit.client.1.smithi179.stdout:0/106: mknod d1/da/c1a 0 2024-04-02T04:47:38.450 INFO:tasks.workunit.client.1.smithi179.stdout:9/54: truncate d0/fd 3483310 0 2024-04-02T04:47:38.450 INFO:tasks.workunit.client.1.smithi179.stdout:4/58: creat d1/d3/dc/fd x:0 0 0 2024-04-02T04:47:38.451 INFO:tasks.workunit.client.1.smithi179.stdout:1/52: symlink d3/d5/lf 0 2024-04-02T04:47:38.452 INFO:tasks.workunit.client.1.smithi179.stdout:4/59: write d1/f5 [3961912,5949] 0 2024-04-02T04:47:38.456 INFO:tasks.workunit.client.1.smithi179.stdout:8/35: dwrite d3/f9 [0,4194304] 0 2024-04-02T04:47:38.457 INFO:tasks.workunit.client.1.smithi179.stdout:9/55: unlink d0/d1/lf 0 2024-04-02T04:47:38.457 INFO:tasks.workunit.client.1.smithi179.stdout:0/107: dread d1/da/df/f12 [0,4194304] 0 2024-04-02T04:47:38.458 INFO:tasks.workunit.client.1.smithi179.stdout:9/56: stat d0/c9 0 2024-04-02T04:47:38.463 INFO:tasks.workunit.client.1.smithi179.stdout:0/108: dread d1/fe [0,4194304] 0 2024-04-02T04:47:38.465 INFO:tasks.workunit.client.1.smithi179.stdout:1/53: link d3/c4 d3/d5/de/c10 0 2024-04-02T04:47:38.468 INFO:tasks.workunit.client.1.smithi179.stdout:8/36: write f1 [228603,41730] 0 2024-04-02T04:47:38.468 INFO:tasks.workunit.client.1.smithi179.stdout:8/37: rename d3 to d3/dc 22 2024-04-02T04:47:38.468 INFO:tasks.workunit.client.1.smithi179.stdout:0/109: creat d1/da/f1b x:0 0 0 2024-04-02T04:47:38.469 INFO:tasks.workunit.client.1.smithi179.stdout:0/110: stat d1 0 2024-04-02T04:47:38.469 INFO:tasks.workunit.client.1.smithi179.stdout:8/38: write f1 [1964823,97828] 0 2024-04-02T04:47:38.469 INFO:tasks.workunit.client.1.smithi179.stdout:2/7: sync 2024-04-02T04:47:38.470 INFO:tasks.workunit.client.1.smithi179.stdout:2/8: truncate - no filename 2024-04-02T04:47:38.471 INFO:tasks.workunit.client.1.smithi179.stdout:9/57: rename d0/d1/db to d0/d1/d12 0 2024-04-02T04:47:38.472 INFO:tasks.workunit.client.1.smithi179.stdout:1/54: unlink c2 0 2024-04-02T04:47:38.473 INFO:tasks.workunit.client.1.smithi179.stdout:1/55: write d3/fa [114875,40653] 0 2024-04-02T04:47:38.473 INFO:tasks.workunit.client.1.smithi179.stdout:8/39: creat d3/fd x:0 0 0 2024-04-02T04:47:38.474 INFO:tasks.workunit.client.1.smithi179.stdout:8/40: dread - d3/f4 zero size 2024-04-02T04:47:38.474 INFO:tasks.workunit.client.1.smithi179.stdout:0/111: write d1/da/df/f11 [966901,65532] 0 2024-04-02T04:47:38.475 INFO:tasks.workunit.client.1.smithi179.stdout:8/41: chown d3/fb 2123192 1 2024-04-02T04:47:38.475 INFO:tasks.workunit.client.1.smithi179.stdout:8/42: stat l0 0 2024-04-02T04:47:38.475 INFO:tasks.workunit.client.1.smithi179.stdout:8/43: fsync f1 0 2024-04-02T04:47:38.476 INFO:tasks.workunit.client.1.smithi179.stdout:8/44: chown d3/fb 23804230 1 2024-04-02T04:47:38.476 INFO:tasks.workunit.client.1.smithi179.stdout:2/9: unlink l1 0 2024-04-02T04:47:38.476 INFO:tasks.workunit.client.1.smithi179.stdout:2/10: dread - no filename 2024-04-02T04:47:38.476 INFO:tasks.workunit.client.1.smithi179.stdout:2/11: dwrite - no filename 2024-04-02T04:47:38.478 INFO:tasks.workunit.client.1.smithi179.stdout:0/112: mknod d1/da/df/c1c 0 2024-04-02T04:47:38.479 INFO:tasks.workunit.client.1.smithi179.stdout:8/45: mknod d3/ce 0 2024-04-02T04:47:38.480 INFO:tasks.workunit.client.1.smithi179.stdout:2/12: creat f2 x:0 0 0 2024-04-02T04:47:38.480 INFO:tasks.workunit.client.1.smithi179.stdout:9/58: getdents d0/d1/d10 0 2024-04-02T04:47:38.481 INFO:tasks.workunit.client.1.smithi179.stdout:0/113: creat d1/da/f1d x:0 0 0 2024-04-02T04:47:38.481 INFO:tasks.workunit.client.1.smithi179.stdout:8/46: mknod d3/cf 0 2024-04-02T04:47:38.483 INFO:tasks.workunit.client.1.smithi179.stdout:0/114: read d1/fe [3788577,29896] 0 2024-04-02T04:47:38.483 INFO:tasks.workunit.client.1.smithi179.stdout:8/47: chown d3/fd 68 1 2024-04-02T04:47:38.483 INFO:tasks.workunit.client.1.smithi179.stdout:1/56: getdents d3/d5 0 2024-04-02T04:47:38.486 INFO:tasks.workunit.client.1.smithi179.stdout:8/48: symlink d3/l10 0 2024-04-02T04:47:38.488 INFO:tasks.workunit.client.1.smithi179.stdout:8/49: truncate f1 2267529 0 2024-04-02T04:47:38.489 INFO:tasks.workunit.client.1.smithi179.stdout:2/13: dwrite f2 [0,4194304] 0 2024-04-02T04:47:38.489 INFO:tasks.workunit.client.1.smithi179.stdout:8/50: chown d3/c6 3 1 2024-04-02T04:47:38.490 INFO:tasks.workunit.client.1.smithi179.stdout:1/57: dread d3/d5/fb [0,4194304] 0 2024-04-02T04:47:38.490 INFO:tasks.workunit.client.1.smithi179.stdout:2/14: write f2 [3510513,90844] 0 2024-04-02T04:47:38.491 INFO:tasks.workunit.client.1.smithi179.stdout:1/58: chown d3/d5/fb 2 1 2024-04-02T04:47:38.491 INFO:tasks.workunit.client.1.smithi179.stdout:1/59: readlink d3/l9 0 2024-04-02T04:47:38.491 INFO:tasks.workunit.client.1.smithi179.stdout:0/115: creat d1/f1e x:0 0 0 2024-04-02T04:47:38.492 INFO:tasks.workunit.client.1.smithi179.stdout:1/60: truncate d3/d5/fb 1399704 0 2024-04-02T04:47:38.493 INFO:tasks.workunit.client.1.smithi179.stdout:1/61: stat d3/fa 0 2024-04-02T04:47:38.495 INFO:tasks.workunit.client.1.smithi179.stdout:2/15: rename l0 to l3 0 2024-04-02T04:47:38.496 INFO:tasks.workunit.client.1.smithi179.stdout:0/116: symlink d1/da/df/l1f 0 2024-04-02T04:47:38.498 INFO:tasks.workunit.client.1.smithi179.stdout:1/62: rmdir d3 39 2024-04-02T04:47:38.500 INFO:tasks.workunit.client.1.smithi179.stdout:0/117: creat d1/da/df/f20 x:0 0 0 2024-04-02T04:47:38.502 INFO:tasks.workunit.client.1.smithi179.stdout:2/16: link f2 f4 0 2024-04-02T04:47:38.502 INFO:tasks.workunit.client.1.smithi179.stdout:1/63: mknod d3/c11 0 2024-04-02T04:47:38.503 INFO:tasks.workunit.client.1.smithi179.stdout:0/118: creat d1/da/df/f21 x:0 0 0 2024-04-02T04:47:38.505 INFO:tasks.workunit.client.1.smithi179.stdout:9/59: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:47:38.506 INFO:tasks.workunit.client.1.smithi179.stdout:0/119: write d1/da/f1d [83651,125798] 0 2024-04-02T04:47:38.509 INFO:tasks.workunit.client.1.smithi179.stdout:9/60: fdatasync d0/d1/d12/fe 0 2024-04-02T04:47:38.510 INFO:tasks.workunit.client.1.smithi179.stdout:8/51: truncate d3/f9 3287097 0 2024-04-02T04:47:38.512 INFO:tasks.workunit.client.1.smithi179.stdout:2/17: getdents . 0 2024-04-02T04:47:38.513 INFO:tasks.workunit.client.1.smithi179.stdout:8/52: read - d3/f7 zero size 2024-04-02T04:47:38.513 INFO:tasks.workunit.client.1.smithi179.stdout:0/120: creat d1/f22 x:0 0 0 2024-04-02T04:47:38.514 INFO:tasks.workunit.client.1.smithi179.stdout:9/61: creat d0/d1/d10/f13 x:0 0 0 2024-04-02T04:47:38.515 INFO:tasks.workunit.client.1.smithi179.stdout:1/64: dwrite d3/fa [0,4194304] 0 2024-04-02T04:47:38.516 INFO:tasks.workunit.client.1.smithi179.stdout:2/18: write f2 [613386,30663] 0 2024-04-02T04:47:38.516 INFO:tasks.workunit.client.1.smithi179.stdout:0/121: write d1/da/f17 [788490,80297] 0 2024-04-02T04:47:38.519 INFO:tasks.workunit.client.1.smithi179.stdout:1/65: mknod d3/d5/c12 0 2024-04-02T04:47:38.521 INFO:tasks.workunit.client.1.smithi179.stdout:9/62: symlink d0/d1/l14 0 2024-04-02T04:47:38.523 INFO:tasks.workunit.client.1.smithi179.stdout:1/66: creat d3/d5/f13 x:0 0 0 2024-04-02T04:47:38.525 INFO:tasks.workunit.client.1.smithi179.stdout:8/53: dwrite d3/f4 [0,4194304] 0 2024-04-02T04:47:38.528 INFO:tasks.workunit.client.1.smithi179.stdout:0/122: creat d1/f23 x:0 0 0 2024-04-02T04:47:38.532 INFO:tasks.workunit.client.1.smithi179.stdout:0/123: symlink d1/db/l24 0 2024-04-02T04:47:38.533 INFO:tasks.workunit.client.1.smithi179.stdout:8/54: link d3/c8 d3/c11 0 2024-04-02T04:47:38.534 INFO:tasks.workunit.client.1.smithi179.stdout:9/63: dwrite d0/d1/d10/f13 [0,4194304] 0 2024-04-02T04:47:38.535 INFO:tasks.workunit.client.1.smithi179.stdout:0/124: chown d1/l10 93370 1 2024-04-02T04:47:38.536 INFO:tasks.workunit.client.1.smithi179.stdout:8/55: chown d3/c6 341329 1 2024-04-02T04:47:38.538 INFO:tasks.workunit.client.1.smithi179.stdout:1/67: dwrite d3/d5/fb [0,4194304] 0 2024-04-02T04:47:38.539 INFO:tasks.workunit.client.1.smithi179.stdout:0/125: mknod d1/db/c25 0 2024-04-02T04:47:38.540 INFO:tasks.workunit.client.1.smithi179.stdout:8/56: mkdir d3/d12 0 2024-04-02T04:47:38.544 INFO:tasks.workunit.client.1.smithi179.stdout:1/68: symlink d3/l14 0 2024-04-02T04:47:38.546 INFO:tasks.workunit.client.1.smithi179.stdout:8/57: rename d3/ce to d3/c13 0 2024-04-02T04:47:38.547 INFO:tasks.workunit.client.1.smithi179.stdout:8/58: dread - d3/f7 zero size 2024-04-02T04:47:38.548 INFO:tasks.workunit.client.1.smithi179.stdout:8/59: readlink l2 0 2024-04-02T04:47:38.551 INFO:tasks.workunit.client.1.smithi179.stdout:8/60: rmdir d3 39 2024-04-02T04:47:38.552 INFO:tasks.workunit.client.1.smithi179.stdout:1/69: dwrite d3/d5/f13 [0,4194304] 0 2024-04-02T04:47:38.555 INFO:tasks.workunit.client.1.smithi179.stdout:8/61: symlink d3/l14 0 2024-04-02T04:47:38.560 INFO:tasks.workunit.client.1.smithi179.stdout:8/62: rename d3/l14 to d3/l15 0 2024-04-02T04:47:38.562 INFO:tasks.workunit.client.1.smithi179.stdout:9/64: dwrite d0/d1/f4 [4194304,4194304] 0 2024-04-02T04:47:38.563 INFO:tasks.workunit.client.1.smithi179.stdout:8/63: write d3/fd [271774,112684] 0 2024-04-02T04:47:38.564 INFO:tasks.workunit.client.1.smithi179.stdout:0/126: dwrite d1/f2 [0,4194304] 0 2024-04-02T04:47:38.564 INFO:tasks.workunit.client.1.smithi179.stdout:9/65: dread - d0/d1/d12/fe zero size 2024-04-02T04:47:38.566 INFO:tasks.workunit.client.1.smithi179.stdout:8/64: unlink d3/fb 0 2024-04-02T04:47:38.567 INFO:tasks.workunit.client.1.smithi179.stdout:8/65: chown d3/c6 1861 1 2024-04-02T04:47:38.568 INFO:tasks.workunit.client.1.smithi179.stdout:0/127: mknod d1/c26 0 2024-04-02T04:47:38.568 INFO:tasks.workunit.client.1.smithi179.stdout:8/66: creat d3/f16 x:0 0 0 2024-04-02T04:47:38.569 INFO:tasks.workunit.client.1.smithi179.stdout:9/66: fsync d0/d1/f7 0 2024-04-02T04:47:38.570 INFO:tasks.workunit.client.1.smithi179.stdout:0/128: creat d1/da/f27 x:0 0 0 2024-04-02T04:47:38.571 INFO:tasks.workunit.client.1.smithi179.stdout:0/129: write d1/f23 [96516,92483] 0 2024-04-02T04:47:38.571 INFO:tasks.workunit.client.1.smithi179.stdout:0/130: fsync d1/f1e 0 2024-04-02T04:47:38.572 INFO:tasks.workunit.client.1.smithi179.stdout:0/131: chown d1/f1e 7764 1 2024-04-02T04:47:38.572 INFO:tasks.workunit.client.1.smithi179.stdout:8/67: rename d3/c5 to d3/d12/c17 0 2024-04-02T04:47:38.573 INFO:tasks.workunit.client.1.smithi179.stdout:8/68: chown d3/ca 1734 1 2024-04-02T04:47:38.574 INFO:tasks.workunit.client.1.smithi179.stdout:1/70: truncate d3/fa 2789159 0 2024-04-02T04:47:38.574 INFO:tasks.workunit.client.1.smithi179.stdout:1/71: chown d3/d5/de 3 1 2024-04-02T04:47:38.575 INFO:tasks.workunit.client.1.smithi179.stdout:9/67: symlink d0/d1/l15 0 2024-04-02T04:47:38.576 INFO:tasks.workunit.client.1.smithi179.stdout:8/69: creat d3/f18 x:0 0 0 2024-04-02T04:47:38.580 INFO:tasks.workunit.client.1.smithi179.stdout:1/72: mknod d3/d5/de/c15 0 2024-04-02T04:47:38.581 INFO:tasks.workunit.client.1.smithi179.stdout:9/68: write d0/fd [3861601,60598] 0 2024-04-02T04:47:38.584 INFO:tasks.workunit.client.1.smithi179.stdout:8/70: dwrite f1 [0,4194304] 0 2024-04-02T04:47:38.586 INFO:tasks.workunit.client.1.smithi179.stdout:9/69: mknod d0/d1/d10/c16 0 2024-04-02T04:47:38.587 INFO:tasks.workunit.client.1.smithi179.stdout:9/70: write d0/f5 [2206136,86339] 0 2024-04-02T04:47:38.587 INFO:tasks.workunit.client.1.smithi179.stdout:1/73: dread d3/d5/fb [0,4194304] 0 2024-04-02T04:47:38.588 INFO:tasks.workunit.client.1.smithi179.stdout:9/71: fsync d0/d1/d12/fe 0 2024-04-02T04:47:38.592 INFO:tasks.workunit.client.1.smithi179.stdout:1/74: creat d3/d5/de/f16 x:0 0 0 2024-04-02T04:47:38.592 INFO:tasks.workunit.client.1.smithi179.stdout:9/72: creat d0/d1/d12/f17 x:0 0 0 2024-04-02T04:47:38.594 INFO:tasks.workunit.client.1.smithi179.stdout:9/73: creat d0/d1/d10/f18 x:0 0 0 2024-04-02T04:47:38.595 INFO:tasks.workunit.client.1.smithi179.stdout:5/9: sync 2024-04-02T04:47:38.596 INFO:tasks.workunit.client.1.smithi179.stdout:5/10: readlink l3 0 2024-04-02T04:47:38.597 INFO:tasks.workunit.client.1.smithi179.stdout:5/11: chown l3 9 1 2024-04-02T04:47:38.597 INFO:tasks.workunit.client.1.smithi179.stdout:1/75: dread d3/d5/fb [0,4194304] 0 2024-04-02T04:47:38.598 INFO:tasks.workunit.client.1.smithi179.stdout:5/12: symlink l4 0 2024-04-02T04:47:38.599 INFO:tasks.workunit.client.1.smithi179.stdout:5/13: dread - f2 zero size 2024-04-02T04:47:38.600 INFO:tasks.workunit.client.1.smithi179.stdout:1/76: mknod d3/d5/c17 0 2024-04-02T04:47:38.600 INFO:tasks.workunit.client.1.smithi179.stdout:8/71: dwrite d3/f9 [0,4194304] 0 2024-04-02T04:47:38.604 INFO:tasks.workunit.client.1.smithi179.stdout:1/77: rename d3/l14 to d3/d5/l18 0 2024-04-02T04:47:38.607 INFO:tasks.workunit.client.1.smithi179.stdout:9/74: fsync d0/fd 0 2024-04-02T04:47:38.610 INFO:tasks.workunit.client.1.smithi179.stdout:8/72: rename f1 to d3/d12/f19 0 2024-04-02T04:47:38.612 INFO:tasks.workunit.client.1.smithi179.stdout:5/14: dwrite f2 [0,4194304] 0 2024-04-02T04:47:38.614 INFO:tasks.workunit.client.1.smithi179.stdout:9/75: unlink d0/d1/d12/cc 0 2024-04-02T04:47:38.617 INFO:tasks.workunit.client.1.smithi179.stdout:1/78: rename d3/d5/lf to d3/d5/de/l19 0 2024-04-02T04:47:38.618 INFO:tasks.workunit.client.1.smithi179.stdout:8/73: dread d3/d12/f19 [0,4194304] 0 2024-04-02T04:47:38.619 INFO:tasks.workunit.client.1.smithi179.stdout:5/15: creat f5 x:0 0 0 2024-04-02T04:47:38.623 INFO:tasks.workunit.client.1.smithi179.stdout:1/79: dread d3/d5/f13 [0,4194304] 0 2024-04-02T04:47:38.624 INFO:tasks.workunit.client.1.smithi179.stdout:1/80: write d3/fd [3199569,57915] 0 2024-04-02T04:47:38.624 INFO:tasks.workunit.client.1.smithi179.stdout:8/74: symlink d3/d12/l1a 0 2024-04-02T04:47:38.625 INFO:tasks.workunit.client.1.smithi179.stdout:1/81: write d3/d5/de/f16 [270516,113796] 0 2024-04-02T04:47:38.632 INFO:tasks.workunit.client.1.smithi179.stdout:8/75: mkdir d3/d1b 0 2024-04-02T04:47:38.634 INFO:tasks.workunit.client.1.smithi179.stdout:8/76: write d3/fd [833033,24570] 0 2024-04-02T04:47:38.635 INFO:tasks.workunit.client.1.smithi179.stdout:1/82: readlink d3/d5/l18 0 2024-04-02T04:47:38.636 INFO:tasks.workunit.client.1.smithi179.stdout:9/76: getdents d0/d1/d12 0 2024-04-02T04:47:38.636 INFO:tasks.workunit.client.1.smithi179.stdout:1/83: stat d3/d5/fb 0 2024-04-02T04:47:38.638 INFO:tasks.workunit.client.1.smithi179.stdout:9/77: chown d0/d1/d12/f17 29253854 1 2024-04-02T04:47:38.639 INFO:tasks.workunit.client.1.smithi179.stdout:1/84: creat d3/d5/de/f1a x:0 0 0 2024-04-02T04:47:38.639 INFO:tasks.workunit.client.1.smithi179.stdout:9/78: chown d0/c9 11362 1 2024-04-02T04:47:38.640 INFO:tasks.workunit.client.1.smithi179.stdout:5/16: dwrite f2 [4194304,4194304] 0 2024-04-02T04:47:38.642 INFO:tasks.workunit.client.1.smithi179.stdout:1/85: mknod d3/d5/de/c1b 0 2024-04-02T04:47:38.644 INFO:tasks.workunit.client.1.smithi179.stdout:5/17: rename f5 to f6 0 2024-04-02T04:47:38.645 INFO:tasks.workunit.client.1.smithi179.stdout:1/86: link d3/d5/de/c1b d3/d5/de/c1c 0 2024-04-02T04:47:38.649 INFO:tasks.workunit.client.1.smithi179.stdout:1/87: rename d3/fd to d3/d5/f1d 0 2024-04-02T04:47:38.656 INFO:tasks.workunit.client.1.smithi179.stdout:1/88: dwrite d3/d5/de/f16 [0,4194304] 0 2024-04-02T04:47:38.661 INFO:tasks.workunit.client.1.smithi179.stdout:5/18: getdents . 0 2024-04-02T04:47:38.662 INFO:tasks.workunit.client.1.smithi179.stdout:5/19: fsync f2 0 2024-04-02T04:47:38.663 INFO:tasks.workunit.client.1.smithi179.stdout:1/89: dwrite d3/fa [0,4194304] 0 2024-04-02T04:47:38.664 INFO:tasks.workunit.client.1.smithi179.stdout:5/20: chown f6 0 1 2024-04-02T04:47:38.664 INFO:tasks.workunit.client.1.smithi179.stdout:5/21: chown l4 198 1 2024-04-02T04:47:38.677 INFO:tasks.workunit.client.1.smithi179.stdout:1/90: fsync d3/d5/fb 0 2024-04-02T04:47:38.677 INFO:tasks.workunit.client.1.smithi179.stdout:5/22: mknod c7 0 2024-04-02T04:47:38.679 INFO:tasks.workunit.client.1.smithi179.stdout:5/23: mkdir d8 0 2024-04-02T04:47:38.679 INFO:tasks.workunit.client.1.smithi179.stdout:1/91: dread d3/d5/fb [0,4194304] 0 2024-04-02T04:47:38.682 INFO:tasks.workunit.client.1.smithi179.stdout:5/24: symlink d8/l9 0 2024-04-02T04:47:38.684 INFO:tasks.workunit.client.1.smithi179.stdout:1/92: rename d3/d5/de/c1b to d3/d5/de/c1e 0 2024-04-02T04:47:38.686 INFO:tasks.workunit.client.1.smithi179.stdout:1/93: creat d3/f1f x:0 0 0 2024-04-02T04:47:38.687 INFO:tasks.workunit.client.1.smithi179.stdout:1/94: mknod d3/d5/c20 0 2024-04-02T04:47:38.688 INFO:tasks.workunit.client.1.smithi179.stdout:1/95: chown d3/d5/f13 118641191 1 2024-04-02T04:47:38.689 INFO:tasks.workunit.client.1.smithi179.stdout:1/96: fdatasync d3/d5/f1d 0 2024-04-02T04:47:38.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.686+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 --> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2894105760 con 0x7f287c06d0a0 2024-04-02T04:47:38.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.688+0000 7f28989ec700 1 -- 172.21.15.70:0/3487186718 <== mgr.24359 v2:172.21.15.179:6828/1862820759 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f2894105760 con 0x7f287c06d0a0 2024-04-02T04:47:38.691 INFO:tasks.workunit.client.1.smithi179.stdout:1/97: creat d3/d5/f21 x:0 0 0 2024-04-02T04:47:38.693 INFO:tasks.workunit.client.1.smithi179.stdout:1/98: mknod d3/c22 0 2024-04-02T04:47:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f287c06d0a0 msgr2=0x7f287c06f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f287c06d0a0 0x7f287c06f560 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f288400bf30 tx=0x7f288400d040 comp rx=0 tx=0).stop 2024-04-02T04:47:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28940ffc10 msgr2=0x7f28941942a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28940ffc10 0x7f28941942a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2888009910 tx=0x7f288800f690 comp rx=0 tx=0).stop 2024-04-02T04:47:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 shutdown_connections 2024-04-02T04:47:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f287c06d0a0 0x7f287c06f560 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28940ffc10 0x7f28941942a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 --2- 172.21.15.70:0/3487186718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894100e10 0x7f28941947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 >> 172.21.15.70:0/3487186718 conn(0x7f28940fb190 msgr2=0x7f2894104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:38.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 shutdown_connections 2024-04-02T04:47:38.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.691+0000 7f28999ee700 1 -- 172.21.15.70:0/3487186718 wait complete. 2024-04-02T04:47:38.701 INFO:tasks.workunit.client.1.smithi179.stdout:1/99: dwrite d3/d5/f13 [4194304,4194304] 0 2024-04-02T04:47:38.701 INFO:tasks.workunit.client.1.smithi179.stdout:1/100: stat d3/c22 0 2024-04-02T04:47:38.709 INFO:tasks.workunit.client.1.smithi179.stdout:1/101: link d3/l9 d3/d5/de/l23 0 2024-04-02T04:47:38.712 INFO:tasks.workunit.client.1.smithi179.stdout:1/102: creat d3/d5/de/f24 x:0 0 0 2024-04-02T04:47:38.751 INFO:tasks.workunit.client.1.smithi179.stdout:1/103: truncate d3/d5/f13 3919449 0 2024-04-02T04:47:38.753 INFO:tasks.workunit.client.1.smithi179.stdout:1/104: creat d3/d5/f25 x:0 0 0 2024-04-02T04:47:38.756 INFO:tasks.workunit.client.1.smithi179.stdout:1/105: dread d3/d5/fb [0,4194304] 0 2024-04-02T04:47:38.756 INFO:tasks.workunit.client.1.smithi179.stdout:1/106: write d3/d5/f21 [146539,17627] 0 2024-04-02T04:47:38.757 INFO:tasks.workunit.client.1.smithi179.stdout:1/107: chown d3/d5/c20 268 1 2024-04-02T04:47:38.760 INFO:tasks.workunit.client.1.smithi179.stdout:1/108: dread d3/d5/fb [0,4194304] 0 2024-04-02T04:47:38.762 INFO:tasks.workunit.client.1.smithi179.stdout:1/109: creat d3/d5/de/f26 x:0 0 0 2024-04-02T04:47:38.763 INFO:tasks.workunit.client.1.smithi179.stdout:1/110: creat d3/d5/f27 x:0 0 0 2024-04-02T04:47:38.764 INFO:tasks.workunit.client.1.smithi179.stdout:1/111: read - d3/d5/de/f24 zero size 2024-04-02T04:47:38.764 INFO:tasks.workunit.client.1.smithi179.stdout:1/112: stat d3/d5/c12 0 2024-04-02T04:47:38.768 INFO:tasks.workunit.client.1.smithi179.stdout:1/113: link d3/d5/de/f1a d3/f28 0 2024-04-02T04:47:38.769 INFO:tasks.workunit.client.1.smithi179.stdout:1/114: truncate d3/d5/f25 316753 0 2024-04-02T04:47:38.769 INFO:tasks.workunit.client.1.smithi179.stdout:1/115: write d3/d5/f25 [674926,93733] 0 2024-04-02T04:47:38.770 INFO:tasks.workunit.client.1.smithi179.stdout:1/116: truncate d3/d5/f27 313284 0 2024-04-02T04:47:38.773 INFO:tasks.workunit.client.1.smithi179.stdout:1/117: mknod d3/d5/de/c29 0 2024-04-02T04:47:38.775 INFO:tasks.workunit.client.1.smithi179.stdout:1/118: symlink d3/l2a 0 2024-04-02T04:47:38.777 INFO:tasks.workunit.client.1.smithi179.stdout:1/119: chown d3/d5/de/l19 14418512 1 2024-04-02T04:47:38.781 INFO:tasks.workunit.client.1.smithi179.stdout:1/120: dwrite d3/d5/f27 [0,4194304] 0 2024-04-02T04:47:38.783 INFO:tasks.workunit.client.1.smithi179.stdout:1/121: write d3/d5/de/f16 [1650004,119375] 0 2024-04-02T04:47:38.786 INFO:tasks.workunit.client.1.smithi179.stdout:1/122: symlink d3/d5/l2b 0 2024-04-02T04:47:38.791 INFO:tasks.workunit.client.1.smithi179.stdout:1/123: rename d3/d5/l18 to d3/d5/l2c 0 2024-04-02T04:47:38.793 INFO:tasks.workunit.client.1.smithi179.stdout:1/124: write d3/d5/de/f16 [1953749,99976] 0 2024-04-02T04:47:38.793 INFO:tasks.workunit.client.1.smithi179.stdout:1/125: dread - d3/f1f zero size 2024-04-02T04:47:38.794 INFO:tasks.workunit.client.1.smithi179.stdout:1/126: write d3/d5/de/f26 [345418,82632] 0 2024-04-02T04:47:38.796 INFO:tasks.workunit.client.1.smithi179.stdout:6/18: sync 2024-04-02T04:47:38.797 INFO:tasks.workunit.client.1.smithi179.stdout:1/127: creat d3/f2d x:0 0 0 2024-04-02T04:47:38.798 INFO:tasks.workunit.client.1.smithi179.stdout:1/128: write d3/f2d [945339,20829] 0 2024-04-02T04:47:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 -- 172.21.15.70:0/651217820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8101800 msgr2=0x7f65c8103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 --2- 172.21.15.70:0/651217820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8101800 0x7f65c8103bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009910 tx=0x7f65c4009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 -- 172.21.15.70:0/651217820 shutdown_connections 2024-04-02T04:47:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 --2- 172.21.15.70:0/651217820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65c8104130 0x7f65c8106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 --2- 172.21.15.70:0/651217820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8101800 0x7f65c8103bf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 -- 172.21.15.70:0/651217820 >> 172.21.15.70:0/651217820 conn(0x7f65c80fb190 msgr2=0x7f65c80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 -- 172.21.15.70:0/651217820 shutdown_connections 2024-04-02T04:47:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 -- 172.21.15.70:0/651217820 wait complete. 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.813+0000 7f65cfdc4700 1 Processor -- start 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cfdc4700 1 -- start start 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cfdc4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65c8101800 0x7f65c8196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cfdc4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8104130 0x7f65c81969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cfdc4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8196fe0 con 0x7f65c8104130 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cfdc4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8197120 con 0x7f65c8101800 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cd35f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8104130 0x7f65c81969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cd35f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8104130 0x7f65c81969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41596/0 (socket says 172.21.15.70:41596) 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cd35f700 1 -- 172.21.15.70:0/786338337 learned_addr learned my addr 172.21.15.70:0/786338337 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cdb60700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65c8101800 0x7f65c8196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cd35f700 1 -- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65c8101800 msgr2=0x7f65c8196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cd35f700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65c8101800 0x7f65c8196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cd35f700 1 -- 172.21.15.70:0/786338337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c40095f0 con 0x7f65c8104130 2024-04-02T04:47:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.814+0000 7f65cd35f700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8104130 0x7f65c81969c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d8d0 tx=0x7f65b800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.815+0000 7f65beffd700 1 -- 172.21.15.70:0/786338337 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65b80098a0 con 0x7f65c8104130 2024-04-02T04:47:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.815+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c819bbd0 con 0x7f65c8104130 2024-04-02T04:47:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.815+0000 7f65beffd700 1 -- 172.21.15.70:0/786338337 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f65b8010460 con 0x7f65c8104130 2024-04-02T04:47:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.815+0000 7f65beffd700 1 -- 172.21.15.70:0/786338337 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65b800f5f0 con 0x7f65c8104130 2024-04-02T04:47:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.815+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c819c060 con 0x7f65c8104130 2024-04-02T04:47:38.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.816+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c8190610 con 0x7f65c8104130 2024-04-02T04:47:38.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.816+0000 7f65beffd700 1 -- 172.21.15.70:0/786338337 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f65b800f750 con 0x7f65c8104130 2024-04-02T04:47:38.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.817+0000 7f65beffd700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f65b406d0a0 0x7f65b406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:38.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.817+0000 7f65beffd700 1 -- 172.21.15.70:0/786338337 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f65b808c8a0 con 0x7f65c8104130 2024-04-02T04:47:38.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.817+0000 7f65cdb60700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f65b406d0a0 0x7f65b406f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:38.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.817+0000 7f65cdb60700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f65b406d0a0 0x7f65b406f560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006010 tx=0x7f65c400b540 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:38.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:38.821+0000 7f65beffd700 1 -- 172.21.15.70:0/786338337 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f65b805ada0 con 0x7f65c8104130 2024-04-02T04:47:38.877 INFO:tasks.workunit.client.1.smithi179.stdout:7/7: sync 2024-04-02T04:47:38.877 INFO:tasks.workunit.client.1.smithi179.stdout:3/41: sync 2024-04-02T04:47:38.877 INFO:tasks.workunit.client.1.smithi179.stdout:7/8: dread - no filename 2024-04-02T04:47:38.878 INFO:tasks.workunit.client.1.smithi179.stdout:7/9: chown d1/l2 2071212420 1 2024-04-02T04:47:38.879 INFO:tasks.workunit.client.1.smithi179.stdout:3/42: mkdir d8/db 0 2024-04-02T04:47:38.882 INFO:tasks.workunit.client.1.smithi179.stdout:3/43: truncate f5 1432661 0 2024-04-02T04:47:38.890 INFO:tasks.workunit.client.1.smithi179.stdout:7/10: getdents d1 0 2024-04-02T04:47:38.894 INFO:tasks.workunit.client.1.smithi179.stdout:7/11: symlink d1/l3 0 2024-04-02T04:47:38.896 INFO:tasks.workunit.client.1.smithi179.stdout:1/129: rmdir d3 39 2024-04-02T04:47:38.898 INFO:tasks.workunit.client.1.smithi179.stdout:7/12: mkdir d1/d4 0 2024-04-02T04:47:38.898 INFO:tasks.workunit.client.1.smithi179.stdout:7/13: dread - no filename 2024-04-02T04:47:38.902 INFO:tasks.workunit.client.1.smithi179.stdout:1/130: read d3/d5/f1d [96064,2607] 0 2024-04-02T04:47:38.905 INFO:tasks.workunit.client.1.smithi179.stdout:1/131: readlink d3/l9 0 2024-04-02T04:47:38.905 INFO:tasks.workunit.client.1.smithi179.stdout:6/19: dwrite d3/fc [0,4194304] 0 2024-04-02T04:47:38.905 INFO:tasks.workunit.client.1.smithi179.stdout:1/132: readlink d3/l8 0 2024-04-02T04:47:38.914 INFO:tasks.workunit.client.1.smithi179.stdout:1/133: dwrite d3/d5/de/f26 [0,4194304] 0 2024-04-02T04:47:38.919 INFO:tasks.workunit.client.1.smithi179.stdout:8/77: sync 2024-04-02T04:47:38.919 INFO:tasks.workunit.client.1.smithi179.stdout:9/79: sync 2024-04-02T04:47:38.919 INFO:tasks.workunit.client.1.smithi179.stdout:4/60: sync 2024-04-02T04:47:38.919 INFO:tasks.workunit.client.1.smithi179.stdout:2/19: sync 2024-04-02T04:47:38.919 INFO:tasks.workunit.client.1.smithi179.stdout:7/14: rmdir d1/d4 0 2024-04-02T04:47:38.920 INFO:tasks.workunit.client.1.smithi179.stdout:7/15: dread - no filename 2024-04-02T04:47:38.920 INFO:tasks.workunit.client.1.smithi179.stdout:0/132: sync 2024-04-02T04:47:38.920 INFO:tasks.workunit.client.1.smithi179.stdout:5/25: sync 2024-04-02T04:47:38.921 INFO:tasks.workunit.client.1.smithi179.stdout:2/20: write f2 [2669561,17236] 0 2024-04-02T04:47:38.921 INFO:tasks.workunit.client.1.smithi179.stdout:5/26: write f2 [872006,37595] 0 2024-04-02T04:47:38.921 INFO:tasks.workunit.client.1.smithi179.stdout:4/61: write d1/d3/dc/fd [946896,77465] 0 2024-04-02T04:47:38.923 INFO:tasks.workunit.client.1.smithi179.stdout:5/27: write f1 [664306,70439] 0 2024-04-02T04:47:38.923 INFO:tasks.workunit.client.1.smithi179.stdout:4/62: fsync d1/d3/f7 0 2024-04-02T04:47:38.923 INFO:tasks.workunit.client.1.smithi179.stdout:5/28: readlink l3 0 2024-04-02T04:47:38.924 INFO:tasks.workunit.client.1.smithi179.stdout:5/29: chown f2 841450 1 2024-04-02T04:47:38.925 INFO:tasks.workunit.client.1.smithi179.stdout:5/30: dread - f6 zero size 2024-04-02T04:47:38.925 INFO:tasks.workunit.client.1.smithi179.stdout:4/63: dread d1/d3/f7 [0,4194304] 0 2024-04-02T04:47:38.927 INFO:tasks.workunit.client.1.smithi179.stdout:4/64: fsync d1/f4 0 2024-04-02T04:47:38.928 INFO:tasks.workunit.client.1.smithi179.stdout:9/80: mkdir d0/d19 0 2024-04-02T04:47:38.930 INFO:tasks.workunit.client.1.smithi179.stdout:7/16: mknod d1/c5 0 2024-04-02T04:47:38.932 INFO:tasks.workunit.client.1.smithi179.stdout:0/133: fsync d1/db/fc 0 2024-04-02T04:47:38.934 INFO:tasks.workunit.client.1.smithi179.stdout:0/134: chown d1/db/l24 9 1 2024-04-02T04:47:38.935 INFO:tasks.workunit.client.1.smithi179.stdout:4/65: dwrite d1/d3/f7 [0,4194304] 0 2024-04-02T04:47:38.936 INFO:tasks.workunit.client.1.smithi179.stdout:9/81: dread d0/d1/d10/f13 [0,4194304] 0 2024-04-02T04:47:38.937 INFO:tasks.workunit.client.1.smithi179.stdout:9/82: fdatasync d0/d1/d12/fe 0 2024-04-02T04:47:38.937 INFO:tasks.workunit.client.1.smithi179.stdout:9/83: fdatasync d0/d1/d10/f13 0 2024-04-02T04:47:38.938 INFO:tasks.workunit.client.1.smithi179.stdout:5/31: unlink d8/l9 0 2024-04-02T04:47:38.940 INFO:tasks.workunit.client.1.smithi179.stdout:1/134: rename d3/d5/de/c1e to d3/d5/de/c2e 0 2024-04-02T04:47:38.947 INFO:tasks.workunit.client.1.smithi179.stdout:2/21: link f4 f5 0 2024-04-02T04:47:38.948 INFO:tasks.workunit.client.1.smithi179.stdout:1/135: dwrite d3/d5/fb [0,4194304] 0 2024-04-02T04:47:38.948 INFO:tasks.workunit.client.1.smithi179.stdout:4/66: creat d1/d3/dc/fe x:0 0 0 2024-04-02T04:47:38.949 INFO:tasks.workunit.client.1.smithi179.stdout:4/67: fdatasync d1/f4 0 2024-04-02T04:47:38.949 INFO:tasks.workunit.client.1.smithi179.stdout:9/84: mknod d0/d1/d12/c1a 0 2024-04-02T04:47:38.954 INFO:tasks.workunit.client.1.smithi179.stdout:2/22: write f4 [1735130,35489] 0 2024-04-02T04:47:38.957 INFO:tasks.workunit.client.1.smithi179.stdout:4/68: mkdir d1/d3/df 0 2024-04-02T04:47:38.960 INFO:tasks.workunit.client.1.smithi179.stdout:2/23: dread f2 [0,4194304] 0 2024-04-02T04:47:38.960 INFO:tasks.workunit.client.1.smithi179.stdout:9/85: symlink d0/d1/d12/l1b 0 2024-04-02T04:47:38.960 INFO:tasks.workunit.client.1.smithi179.stdout:2/24: write f2 [3723322,24103] 0 2024-04-02T04:47:38.962 INFO:tasks.workunit.client.1.smithi179.stdout:8/78: getdents d3 0 2024-04-02T04:47:38.970 INFO:tasks.workunit.client.1.smithi179.stdout:8/79: dwrite d3/f16 [0,4194304] 0 2024-04-02T04:47:38.970 INFO:tasks.workunit.client.1.smithi179.stdout:9/86: write d0/d1/d10/f13 [3356625,127205] 0 2024-04-02T04:47:38.971 INFO:tasks.workunit.client.1.smithi179.stdout:0/135: getdents d1 0 2024-04-02T04:47:38.973 INFO:tasks.workunit.client.1.smithi179.stdout:4/69: mkdir d1/d3/df/d10 0 2024-04-02T04:47:38.975 INFO:tasks.workunit.client.1.smithi179.stdout:2/25: rename f5 to f6 0 2024-04-02T04:47:38.979 INFO:tasks.workunit.client.1.smithi179.stdout:8/80: unlink d3/fd 0 2024-04-02T04:47:38.980 INFO:tasks.workunit.client.1.smithi179.stdout:0/136: mkdir d1/da/d28 0 2024-04-02T04:47:38.983 INFO:tasks.workunit.client.1.smithi179.stdout:2/26: mknod c7 0 2024-04-02T04:47:38.985 INFO:tasks.workunit.client.1.smithi179.stdout:9/87: mkdir d0/d19/d1c 0 2024-04-02T04:47:38.986 INFO:tasks.workunit.client.1.smithi179.stdout:2/27: dread f2 [0,4194304] 0 2024-04-02T04:47:38.987 INFO:tasks.workunit.client.1.smithi179.stdout:4/70: creat d1/d3/df/d10/f11 x:0 0 0 2024-04-02T04:47:38.988 INFO:tasks.workunit.client.1.smithi179.stdout:4/71: read d1/f5 [518231,128504] 0 2024-04-02T04:47:38.988 INFO:tasks.workunit.client.1.smithi179.stdout:8/81: creat d3/d1b/f1c x:0 0 0 2024-04-02T04:47:38.989 INFO:tasks.workunit.client.1.smithi179.stdout:8/82: chown d3/d12/f19 1559494 1 2024-04-02T04:47:38.989 INFO:tasks.workunit.client.1.smithi179.stdout:9/88: mknod d0/d1/d12/c1d 0 2024-04-02T04:47:38.989 INFO:tasks.workunit.client.1.smithi179.stdout:8/83: dread - d3/f7 zero size 2024-04-02T04:47:38.990 INFO:tasks.workunit.client.1.smithi179.stdout:0/137: link d1/f14 d1/da/d28/f29 0 2024-04-02T04:47:38.991 INFO:tasks.workunit.client.1.smithi179.stdout:2/28: mknod c8 0 2024-04-02T04:47:38.994 INFO:tasks.workunit.client.1.smithi179.stdout:8/84: mknod d3/c1d 0 2024-04-02T04:47:38.995 INFO:tasks.workunit.client.1.smithi179.stdout:0/138: dwrite d1/fe [0,4194304] 0 2024-04-02T04:47:38.996 INFO:tasks.workunit.client.1.smithi179.stdout:0/139: truncate d1/f1e 524647 0 2024-04-02T04:47:38.999 INFO:tasks.workunit.client.1.smithi179.stdout:0/140: rmdir d1 39 2024-04-02T04:47:39.000 INFO:tasks.workunit.client.1.smithi179.stdout:4/72: link d1/c9 d1/d3/c12 0 2024-04-02T04:47:39.003 INFO:tasks.workunit.client.1.smithi179.stdout:0/141: truncate d1/f14 1700637 0 2024-04-02T04:47:39.003 INFO:tasks.workunit.client.1.smithi179.stdout:3/44: truncate f5 996928 0 2024-04-02T04:47:39.004 INFO:tasks.workunit.client.1.smithi179.stdout:4/73: creat d1/d3/f13 x:0 0 0 2024-04-02T04:47:39.007 INFO:tasks.workunit.client.0.smithi070.stdout:6/54: sync 2024-04-02T04:47:39.007 INFO:tasks.workunit.client.0.smithi070.stdout:5/74: sync 2024-04-02T04:47:39.007 INFO:tasks.workunit.client.0.smithi070.stdout:0/234: sync 2024-04-02T04:47:39.007 INFO:tasks.workunit.client.1.smithi179.stdout:0/142: creat d1/f2a x:0 0 0 2024-04-02T04:47:39.008 INFO:tasks.workunit.client.1.smithi179.stdout:4/74: creat d1/d3/df/f14 x:0 0 0 2024-04-02T04:47:39.010 INFO:tasks.workunit.client.0.smithi070.stdout:0/235: mknod d4/d5/c3b 0 2024-04-02T04:47:39.010 INFO:tasks.workunit.client.0.smithi070.stdout:0/236: stat d4/d5/d13/f19 0 2024-04-02T04:47:39.011 INFO:tasks.workunit.client.0.smithi070.stdout:0/237: dread f0 [0,4194304] 0 2024-04-02T04:47:39.012 INFO:tasks.workunit.client.0.smithi070.stdout:0/238: dread f0 [0,4194304] 0 2024-04-02T04:47:39.014 INFO:tasks.workunit.client.0.smithi070.stdout:5/75: creat d5/d14/f16 x:0 0 0 2024-04-02T04:47:39.015 INFO:tasks.workunit.client.0.smithi070.stdout:5/76: write d5/f6 [549175,17036] 0 2024-04-02T04:47:39.017 INFO:tasks.workunit.client.1.smithi179.stdout:4/75: mkdir d1/d15 0 2024-04-02T04:47:39.017 INFO:tasks.workunit.client.0.smithi070.stdout:5/77: write d5/fa [1631796,81876] 0 2024-04-02T04:47:39.019 INFO:tasks.workunit.client.1.smithi179.stdout:3/45: rmdir d8/db 0 2024-04-02T04:47:39.020 INFO:tasks.workunit.client.0.smithi070.stdout:0/239: dwrite d4/d5/d13/f32 [0,4194304] 0 2024-04-02T04:47:39.022 INFO:tasks.workunit.client.1.smithi179.stdout:4/76: mknod d1/d3/dc/c16 0 2024-04-02T04:47:39.022 INFO:tasks.workunit.client.1.smithi179.stdout:4/77: stat d1/c8 0 2024-04-02T04:47:39.023 INFO:tasks.workunit.client.1.smithi179.stdout:3/46: rmdir d8 39 2024-04-02T04:47:39.024 INFO:tasks.workunit.client.0.smithi070.stdout:5/78: mkdir d5/d17 0 2024-04-02T04:47:39.025 INFO:tasks.workunit.client.1.smithi179.stdout:4/78: symlink d1/d3/df/d10/l17 0 2024-04-02T04:47:39.027 INFO:tasks.workunit.client.1.smithi179.stdout:4/79: truncate d1/d3/fa 400633 0 2024-04-02T04:47:39.029 INFO:tasks.workunit.client.1.smithi179.stdout:4/80: creat d1/d3/dc/f18 x:0 0 0 2024-04-02T04:47:39.030 INFO:tasks.workunit.client.1.smithi179.stdout:4/81: chown d1/d3/dc 882 1 2024-04-02T04:47:39.032 INFO:tasks.workunit.client.0.smithi070.stdout:5/79: dwrite d5/f6 [0,4194304] 0 2024-04-02T04:47:39.036 INFO:tasks.workunit.client.0.smithi070.stdout:5/80: symlink d5/l18 0 2024-04-02T04:47:39.037 INFO:tasks.workunit.client.0.smithi070.stdout:0/240: getdents d4/d5/d13/d2a 0 2024-04-02T04:47:39.038 INFO:tasks.workunit.client.0.smithi070.stdout:0/241: readlink d4/d5/d21/l33 0 2024-04-02T04:47:39.039 INFO:tasks.workunit.client.0.smithi070.stdout:5/81: creat d5/d9/d10/f19 x:0 0 0 2024-04-02T04:47:39.040 INFO:tasks.workunit.client.0.smithi070.stdout:0/242: rmdir d4/d5/d13/d2a 39 2024-04-02T04:47:39.043 INFO:tasks.workunit.client.0.smithi070.stdout:0/243: stat d4/d5/d13/d2a/d29 0 2024-04-02T04:47:39.048 INFO:tasks.workunit.client.0.smithi070.stdout:0/244: truncate d4/f9 1225880 0 2024-04-02T04:47:39.052 INFO:tasks.workunit.client.0.smithi070.stdout:0/245: rename d4/d5/da/d2d/d27 to d4/d3c 0 2024-04-02T04:47:39.054 INFO:tasks.workunit.client.0.smithi070.stdout:0/246: unlink d4/d5/da/c30 0 2024-04-02T04:47:39.056 INFO:tasks.workunit.client.0.smithi070.stdout:0/247: unlink d4/d5/da/f8 0 2024-04-02T04:47:39.057 INFO:tasks.workunit.client.0.smithi070.stdout:0/248: chown d4/l1d 7 1 2024-04-02T04:47:39.115 INFO:tasks.workunit.client.1.smithi179.stdout:3/47: rename l6 to d8/lc 0 2024-04-02T04:47:39.115 INFO:tasks.workunit.client.1.smithi179.stdout:6/20: truncate d3/fb 2452400 0 2024-04-02T04:47:39.116 INFO:tasks.workunit.client.1.smithi179.stdout:7/17: getdents d1 0 2024-04-02T04:47:39.116 INFO:tasks.workunit.client.1.smithi179.stdout:7/18: dread - no filename 2024-04-02T04:47:39.116 INFO:tasks.workunit.client.1.smithi179.stdout:7/19: fdatasync - no filename 2024-04-02T04:47:39.116 INFO:tasks.workunit.client.1.smithi179.stdout:7/20: fdatasync - no filename 2024-04-02T04:47:39.116 INFO:tasks.workunit.client.1.smithi179.stdout:7/21: write - no filename 2024-04-02T04:47:39.116 INFO:tasks.workunit.client.1.smithi179.stdout:5/32: getdents d8 0 2024-04-02T04:47:39.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.113+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 --> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f65c80611d0 con 0x7f65b406d0a0 2024-04-02T04:47:39.117 INFO:tasks.workunit.client.1.smithi179.stdout:1/136: write d3/d5/de/f1a [754238,37114] 0 2024-04-02T04:47:39.117 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:38 smithi070 ceph-mon[29212]: pgmap v8: 65 pgs: 65 active+clean; 222 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-04-02T04:47:39.117 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:38 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:47:39.117 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:38 smithi070 ceph-mon[29212]: mgrmap e24: smithi179.labmvs(active, since 10s), standbys: smithi070.igidvr 2024-04-02T04:47:39.117 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:38 smithi070 ceph-mon[29212]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:39.118 INFO:tasks.workunit.client.1.smithi179.stdout:9/89: getdents d0/d19 0 2024-04-02T04:47:39.118 INFO:tasks.workunit.client.1.smithi179.stdout:8/85: getdents d3/d1b 0 2024-04-02T04:47:39.123 INFO:tasks.workunit.client.1.smithi179.stdout:1/137: dwrite d3/d5/de/f16 [0,4194304] 0 2024-04-02T04:47:39.123 INFO:tasks.workunit.client.1.smithi179.stdout:9/90: dread d0/d1/d10/f13 [0,4194304] 0 2024-04-02T04:47:39.128 INFO:tasks.workunit.client.1.smithi179.stdout:1/138: dwrite d3/d5/de/f26 [0,4194304] 0 2024-04-02T04:47:39.131 INFO:tasks.workunit.client.0.smithi070.stdout:8/85: sync 2024-04-02T04:47:39.131 INFO:tasks.workunit.client.0.smithi070.stdout:3/34: sync 2024-04-02T04:47:39.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.129+0000 7f65beffd700 1 -- 172.21.15.70:0/786338337 <== mgr.24359 v2:172.21.15.179:6828/1862820759 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f65c80611d0 con 0x7f65b406d0a0 2024-04-02T04:47:39.132 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:47:39.132 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (6m) 2m ago 8m 24.5M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:47:39.132 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (8m) 2m ago 8m 7457k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:47:39.132 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (7m) 20s ago 7m 7340k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:47:39.132 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (8m) 2m ago 8m 6665k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (7m) 20s ago 7m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (6m) 2m ago 7m 85.6M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (2m) 2m ago 2m 16.5M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (2m) 2m ago 2m 14.6M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (2m) 20s ago 2m 14.1M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (2m) 20s ago 2m 15.6M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:9283,8765,8443 running (9m) 2m ago 9m 489M - 18.2.1 5be31c24972a 3299b9e265dd 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (26s) 20s ago 7m 51.6M - 19.0.0-1979-g6cec793d 78f830190839 a0c5bed955ba 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (9m) 2m ago 9m 44.4M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (7m) 20s ago 7m 40.5M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 2m ago 8m 14.3M - 1.5.0 0da6a335fe13 bc06c0b59df5 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (7m) 20s ago 7m 17.0M - 1.5.0 0da6a335fe13 6fe44da459b3 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (6m) 2m ago 6m 47.9M 3946M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (5m) 2m ago 5m 48.2M 3946M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:47:39.133 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (5m) 2m ago 5m 46.5M 3946M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:47:39.134 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (5m) 20s ago 5m 109M 4970M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:47:39.134 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (4m) 20s ago 4m 101M 4970M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:47:39.134 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (4m) 20s ago 4m 94.6M 4970M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:47:39.134 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (6m) 2m ago 7m 56.3M - 2.43.0 a07b618ecd1d b1f1f05601f5 2024-04-02T04:47:39.134 INFO:tasks.workunit.client.0.smithi070.stdout:8/86: dread - d2/d8/d14/db/f18 zero size 2024-04-02T04:47:39.135 INFO:tasks.workunit.client.1.smithi179.stdout:4/82: mkdir d1/d3/d19 0 2024-04-02T04:47:39.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f65b406d0a0 msgr2=0x7f65b406f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:39.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f65b406d0a0 0x7f65b406f560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006010 tx=0x7f65c400b540 comp rx=0 tx=0).stop 2024-04-02T04:47:39.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8104130 msgr2=0x7f65c81969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8104130 0x7f65c81969c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d8d0 tx=0x7f65b800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 shutdown_connections 2024-04-02T04:47:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f65b406d0a0 0x7f65b406f560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65c8101800 0x7f65c8196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 --2- 172.21.15.70:0/786338337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c8104130 0x7f65c81969c0 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d8d0 tx=0x7f65b800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.132+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 >> 172.21.15.70:0/786338337 conn(0x7f65c80fb190 msgr2=0x7f65c80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.133+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 shutdown_connections 2024-04-02T04:47:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.133+0000 7f65cfdc4700 1 -- 172.21.15.70:0/786338337 wait complete. 2024-04-02T04:47:39.138 INFO:tasks.workunit.client.1.smithi179.stdout:6/21: symlink d3/ld 0 2024-04-02T04:47:39.139 INFO:tasks.workunit.client.1.smithi179.stdout:6/22: dread - d3/f4 zero size 2024-04-02T04:47:39.141 INFO:tasks.workunit.client.1.smithi179.stdout:7/22: creat d1/f6 x:0 0 0 2024-04-02T04:47:39.141 INFO:tasks.workunit.client.1.smithi179.stdout:7/23: chown d1/c5 350225 1 2024-04-02T04:47:39.142 INFO:tasks.workunit.client.0.smithi070.stdout:3/35: link fd ff 0 2024-04-02T04:47:39.143 INFO:tasks.workunit.client.1.smithi179.stdout:5/33: creat d8/fa x:0 0 0 2024-04-02T04:47:39.144 INFO:tasks.workunit.client.1.smithi179.stdout:5/34: readlink l3 0 2024-04-02T04:47:39.144 INFO:tasks.workunit.client.0.smithi070.stdout:8/87: symlink d2/l1a 0 2024-04-02T04:47:39.145 INFO:tasks.workunit.client.0.smithi070.stdout:8/88: truncate d2/d3/d6/f16 397307 0 2024-04-02T04:47:39.147 INFO:tasks.workunit.client.0.smithi070.stdout:3/36: mkdir d10 0 2024-04-02T04:47:39.150 INFO:tasks.workunit.client.1.smithi179.stdout:8/86: mkdir d3/d12/d1e 0 2024-04-02T04:47:39.152 INFO:tasks.workunit.client.0.smithi070.stdout:8/89: dwrite d2/d3/f7 [0,4194304] 0 2024-04-02T04:47:39.153 INFO:tasks.workunit.client.0.smithi070.stdout:8/90: chown d2/d3/f9 14 1 2024-04-02T04:47:39.153 INFO:tasks.workunit.client.1.smithi179.stdout:9/91: symlink d0/d1/d10/l1e 0 2024-04-02T04:47:39.154 INFO:tasks.workunit.client.1.smithi179.stdout:9/92: stat d0/d1/d10/f18 0 2024-04-02T04:47:39.155 INFO:tasks.workunit.client.0.smithi070.stdout:3/37: symlink d10/l11 0 2024-04-02T04:47:39.155 INFO:tasks.workunit.client.1.smithi179.stdout:9/93: chown d0/d1/d10/l1e 1855 1 2024-04-02T04:47:39.156 INFO:tasks.workunit.client.1.smithi179.stdout:1/139: rmdir d3/d5/de 39 2024-04-02T04:47:39.157 INFO:tasks.workunit.client.1.smithi179.stdout:1/140: write d3/d5/f27 [2838535,96258] 0 2024-04-02T04:47:39.157 INFO:tasks.workunit.client.1.smithi179.stdout:1/141: chown d3 183 1 2024-04-02T04:47:39.158 INFO:tasks.workunit.client.0.smithi070.stdout:5/82: truncate d5/ff 3885817 0 2024-04-02T04:47:39.158 INFO:tasks.workunit.client.1.smithi179.stdout:1/142: write d3/f28 [1048515,104670] 0 2024-04-02T04:47:39.160 INFO:tasks.workunit.client.0.smithi070.stdout:3/38: rmdir d10 39 2024-04-02T04:47:39.163 INFO:tasks.workunit.client.0.smithi070.stdout:8/91: dwrite d2/d8/d14/db/f13 [0,4194304] 0 2024-04-02T04:47:39.163 INFO:tasks.workunit.client.0.smithi070.stdout:3/39: write f0 [4376937,125485] 0 2024-04-02T04:47:39.164 INFO:tasks.workunit.client.0.smithi070.stdout:8/92: chown d2/l1a 1997392538 1 2024-04-02T04:47:39.164 INFO:tasks.workunit.client.1.smithi179.stdout:2/29: sync 2024-04-02T04:47:39.166 INFO:tasks.workunit.client.0.smithi070.stdout:5/83: dread d5/fa [0,4194304] 0 2024-04-02T04:47:39.174 INFO:tasks.workunit.client.0.smithi070.stdout:0/249: write d4/d5/d13/f2b [369609,37357] 0 2024-04-02T04:47:39.176 INFO:tasks.workunit.client.1.smithi179.stdout:0/143: dread d1/f1e [0,4194304] 0 2024-04-02T04:47:39.177 INFO:tasks.workunit.client.1.smithi179.stdout:6/23: creat d3/fe x:0 0 0 2024-04-02T04:47:39.179 INFO:tasks.workunit.client.1.smithi179.stdout:5/35: creat d8/fb x:0 0 0 2024-04-02T04:47:39.180 INFO:tasks.workunit.client.1.smithi179.stdout:5/36: dread - d8/fb zero size 2024-04-02T04:47:39.180 INFO:tasks.workunit.client.0.smithi070.stdout:5/84: rename d5/d9/c12 to d5/d17/c1a 0 2024-04-02T04:47:39.181 INFO:tasks.workunit.client.0.smithi070.stdout:5/85: dread - d5/d14/f16 zero size 2024-04-02T04:47:39.183 INFO:tasks.workunit.client.1.smithi179.stdout:8/87: mknod d3/c1f 0 2024-04-02T04:47:39.188 INFO:tasks.workunit.client.0.smithi070.stdout:5/86: mknod d5/d9/d10/c1b 0 2024-04-02T04:47:39.189 INFO:tasks.workunit.client.0.smithi070.stdout:0/250: getdents d4/d5/d13/d2a/d3a 0 2024-04-02T04:47:39.192 INFO:tasks.workunit.client.1.smithi179.stdout:2/30: creat f9 x:0 0 0 2024-04-02T04:47:39.192 INFO:tasks.workunit.client.1.smithi179.stdout:2/31: rmdir - no directory 2024-04-02T04:47:39.192 INFO:tasks.workunit.client.1.smithi179.stdout:3/48: dwrite f5 [0,4194304] 0 2024-04-02T04:47:39.192 INFO:tasks.workunit.client.0.smithi070.stdout:3/40: symlink d10/l12 0 2024-04-02T04:47:39.193 INFO:tasks.workunit.client.1.smithi179.stdout:3/49: read f5 [4068365,131034] 0 2024-04-02T04:47:39.194 INFO:tasks.workunit.client.1.smithi179.stdout:4/83: mknod d1/d3/d19/c1a 0 2024-04-02T04:47:39.196 INFO:tasks.workunit.client.0.smithi070.stdout:5/87: symlink d5/d9/l1c 0 2024-04-02T04:47:39.199 INFO:tasks.workunit.client.0.smithi070.stdout:0/251: rmdir d4/d5/da 39 2024-04-02T04:47:39.203 INFO:tasks.workunit.client.1.smithi179.stdout:6/24: creat d3/ff x:0 0 0 2024-04-02T04:47:39.203 INFO:tasks.workunit.client.1.smithi179.stdout:6/25: fsync d3/f4 0 2024-04-02T04:47:39.206 INFO:tasks.workunit.client.0.smithi070.stdout:8/93: link d2/c12 d2/d3/c1b 0 2024-04-02T04:47:39.208 INFO:tasks.workunit.client.0.smithi070.stdout:8/94: truncate d2/d8/d14/f15 401852 0 2024-04-02T04:47:39.208 INFO:tasks.workunit.client.1.smithi179.stdout:8/88: symlink d3/d12/l20 0 2024-04-02T04:47:39.208 INFO:tasks.workunit.client.1.smithi179.stdout:8/89: truncate d3/f16 5194473 0 2024-04-02T04:47:39.210 INFO:tasks.workunit.client.1.smithi179.stdout:1/143: mkdir d3/d5/d2f 0 2024-04-02T04:47:39.216 INFO:tasks.workunit.client.0.smithi070.stdout:3/41: unlink f5 0 2024-04-02T04:47:39.218 INFO:tasks.workunit.client.0.smithi070.stdout:5/88: dread d5/fa [0,4194304] 0 2024-04-02T04:47:39.218 INFO:tasks.workunit.client.1.smithi179.stdout:2/32: rename c8 to ca 0 2024-04-02T04:47:39.219 INFO:tasks.workunit.client.0.smithi070.stdout:8/95: dwrite d2/d3/fd [0,4194304] 0 2024-04-02T04:47:39.220 INFO:tasks.workunit.client.1.smithi179.stdout:3/50: mknod d8/cd 0 2024-04-02T04:47:39.221 INFO:tasks.workunit.client.0.smithi070.stdout:0/252: dwrite d4/d5/d13/f1f [0,4194304] 0 2024-04-02T04:47:39.223 INFO:tasks.workunit.client.1.smithi179.stdout:4/84: unlink d1/d3/df/f14 0 2024-04-02T04:47:39.224 INFO:tasks.workunit.client.1.smithi179.stdout:4/85: truncate d1/d3/f13 991001 0 2024-04-02T04:47:39.227 INFO:tasks.workunit.client.0.smithi070.stdout:5/89: creat d5/f1d x:0 0 0 2024-04-02T04:47:39.228 INFO:tasks.workunit.client.0.smithi070.stdout:5/90: write f4 [1208930,56920] 0 2024-04-02T04:47:39.229 INFO:tasks.workunit.client.1.smithi179.stdout:0/144: creat d1/db/f2b x:0 0 0 2024-04-02T04:47:39.234 INFO:tasks.workunit.client.1.smithi179.stdout:6/26: unlink d3/f4 0 2024-04-02T04:47:39.237 INFO:tasks.workunit.client.1.smithi179.stdout:5/37: rename d8/fa to d8/fc 0 2024-04-02T04:47:39.238 INFO:tasks.workunit.client.1.smithi179.stdout:5/38: chown f2 313 1 2024-04-02T04:47:39.238 INFO:tasks.workunit.client.1.smithi179.stdout:5/39: truncate f6 911650 0 2024-04-02T04:47:39.239 INFO:tasks.workunit.client.0.smithi070.stdout:5/91: mknod d5/d9/c1e 0 2024-04-02T04:47:39.239 INFO:tasks.workunit.client.1.smithi179.stdout:5/40: chown f1 46889 1 2024-04-02T04:47:39.240 INFO:tasks.workunit.client.1.smithi179.stdout:1/144: chown d3/c4 2 1 2024-04-02T04:47:39.242 INFO:tasks.workunit.client.0.smithi070.stdout:0/253: link d4/d5/f14 d4/d3c/f3d 0 2024-04-02T04:47:39.245 INFO:tasks.workunit.client.1.smithi179.stdout:1/145: dread d3/f28 [0,4194304] 0 2024-04-02T04:47:39.246 INFO:tasks.workunit.client.0.smithi070.stdout:3/42: dwrite ff [0,4194304] 0 2024-04-02T04:47:39.247 INFO:tasks.workunit.client.0.smithi070.stdout:8/96: link d2/d3/c10 d2/d8/d14/c1c 0 2024-04-02T04:47:39.248 INFO:tasks.workunit.client.0.smithi070.stdout:5/92: rename d5/c8 to d5/d9/c1f 0 2024-04-02T04:47:39.249 INFO:tasks.workunit.client.0.smithi070.stdout:8/97: rename d2 to d2/d8/d14/db/d1d 22 2024-04-02T04:47:39.249 INFO:tasks.workunit.client.0.smithi070.stdout:4/78: sync 2024-04-02T04:47:39.249 INFO:tasks.workunit.client.0.smithi070.stdout:9/106: sync 2024-04-02T04:47:39.249 INFO:tasks.workunit.client.0.smithi070.stdout:7/82: sync 2024-04-02T04:47:39.249 INFO:tasks.workunit.client.0.smithi070.stdout:1/113: sync 2024-04-02T04:47:39.249 INFO:tasks.workunit.client.0.smithi070.stdout:2/178: sync 2024-04-02T04:47:39.250 INFO:tasks.workunit.client.1.smithi179.stdout:4/86: creat d1/d3/f1b x:0 0 0 2024-04-02T04:47:39.251 INFO:tasks.workunit.client.0.smithi070.stdout:1/114: rename d0 to d0/d9/d1d/d27 22 2024-04-02T04:47:39.251 INFO:tasks.workunit.client.0.smithi070.stdout:4/79: readlink d1/d3/l7 0 2024-04-02T04:47:39.253 INFO:tasks.workunit.client.0.smithi070.stdout:2/179: truncate d2/d12/f38 1011001 0 2024-04-02T04:47:39.253 INFO:tasks.workunit.client.0.smithi070.stdout:3/43: symlink d10/l13 0 2024-04-02T04:47:39.253 INFO:tasks.workunit.client.1.smithi179.stdout:9/94: sync 2024-04-02T04:47:39.254 INFO:tasks.workunit.client.1.smithi179.stdout:0/145: write d1/da/f27 [843782,102808] 0 2024-04-02T04:47:39.254 INFO:tasks.workunit.client.0.smithi070.stdout:1/115: read - d0/d9/d1d/f26 zero size 2024-04-02T04:47:39.255 INFO:tasks.workunit.client.0.smithi070.stdout:2/180: chown d2/d24/f2d 16196371 1 2024-04-02T04:47:39.255 INFO:tasks.workunit.client.1.smithi179.stdout:7/24: getdents d1 0 2024-04-02T04:47:39.255 INFO:tasks.workunit.client.1.smithi179.stdout:9/95: write d0/d1/f7 [1151828,4675] 0 2024-04-02T04:47:39.256 INFO:tasks.workunit.client.1.smithi179.stdout:7/25: chown l0 33 1 2024-04-02T04:47:39.256 INFO:tasks.workunit.client.0.smithi070.stdout:2/181: fsync d2/d10/d21/f31 0 2024-04-02T04:47:39.257 INFO:tasks.workunit.client.0.smithi070.stdout:1/116: chown d0/d9/f14 127473643 1 2024-04-02T04:47:39.257 INFO:tasks.workunit.client.0.smithi070.stdout:7/83: dread d7/dd/df/f10 [0,4194304] 0 2024-04-02T04:47:39.257 INFO:tasks.workunit.client.1.smithi179.stdout:7/26: read - d1/f6 zero size 2024-04-02T04:47:39.257 INFO:tasks.workunit.client.0.smithi070.stdout:5/93: symlink d5/d17/l20 0 2024-04-02T04:47:39.258 INFO:tasks.workunit.client.0.smithi070.stdout:1/117: readlink d0/d9/d1d/l12 0 2024-04-02T04:47:39.258 INFO:tasks.workunit.client.1.smithi179.stdout:7/27: write d1/f6 [681809,61195] 0 2024-04-02T04:47:39.259 INFO:tasks.workunit.client.0.smithi070.stdout:2/182: write d2/d10/d19/f1e [580384,41242] 0 2024-04-02T04:47:39.259 INFO:tasks.workunit.client.0.smithi070.stdout:0/254: mknod d4/d5/d13/d2a/d3a/c3e 0 2024-04-02T04:47:39.260 INFO:tasks.workunit.client.0.smithi070.stdout:2/183: write d2/d24/f2d [181158,125571] 0 2024-04-02T04:47:39.261 INFO:tasks.workunit.client.1.smithi179.stdout:6/27: creat d3/f10 x:0 0 0 2024-04-02T04:47:39.262 INFO:tasks.workunit.client.1.smithi179.stdout:9/96: dwrite d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:39.263 INFO:tasks.workunit.client.0.smithi070.stdout:0/255: write d4/d5/d13/d2a/f20 [929130,121278] 0 2024-04-02T04:47:39.265 INFO:tasks.workunit.client.1.smithi179.stdout:9/97: chown d0/d1/d10/l1e 0 1 2024-04-02T04:47:39.266 INFO:tasks.workunit.client.0.smithi070.stdout:7/84: dread d7/fc [4194304,4194304] 0 2024-04-02T04:47:39.267 INFO:tasks.workunit.client.0.smithi070.stdout:0/256: write d4/d5/d13/d23/f36 [1580216,24653] 0 2024-04-02T04:47:39.267 INFO:tasks.workunit.client.0.smithi070.stdout:8/98: rename d2/d3/ce to d2/d8/d14/db/c1e 0 2024-04-02T04:47:39.269 INFO:tasks.workunit.client.0.smithi070.stdout:8/99: write d2/d8/d14/f15 [740716,85926] 0 2024-04-02T04:47:39.270 INFO:tasks.workunit.client.0.smithi070.stdout:4/80: creat d1/d3/d9/f1a x:0 0 0 2024-04-02T04:47:39.271 INFO:tasks.workunit.client.1.smithi179.stdout:2/33: rename ca to cb 0 2024-04-02T04:47:39.273 INFO:tasks.workunit.client.1.smithi179.stdout:9/98: dwrite d0/fd [0,4194304] 0 2024-04-02T04:47:39.274 INFO:tasks.workunit.client.0.smithi070.stdout:8/100: truncate d2/d3/f17 429697 0 2024-04-02T04:47:39.274 INFO:tasks.workunit.client.0.smithi070.stdout:4/81: chown d1/d3/d19 1612 1 2024-04-02T04:47:39.274 INFO:tasks.workunit.client.0.smithi070.stdout:0/257: dread d4/d3c/f3d [0,4194304] 0 2024-04-02T04:47:39.274 INFO:tasks.workunit.client.0.smithi070.stdout:8/101: write d2/d3/f17 [296344,99561] 0 2024-04-02T04:47:39.275 INFO:tasks.workunit.client.1.smithi179.stdout:9/99: chown d0/d1/f7 30403 1 2024-04-02T04:47:39.277 INFO:tasks.workunit.client.1.smithi179.stdout:9/100: chown d0/d1/d12/c1d 3768 1 2024-04-02T04:47:39.277 INFO:tasks.workunit.client.0.smithi070.stdout:0/258: dread d4/f25 [0,4194304] 0 2024-04-02T04:47:39.280 INFO:tasks.workunit.client.1.smithi179.stdout:2/34: dread f4 [0,4194304] 0 2024-04-02T04:47:39.280 INFO:tasks.workunit.client.1.smithi179.stdout:9/101: dread - d0/d1/d12/f17 zero size 2024-04-02T04:47:39.281 INFO:tasks.workunit.client.1.smithi179.stdout:1/146: dread d3/d5/de/f16 [0,4194304] 0 2024-04-02T04:47:39.281 INFO:tasks.workunit.client.1.smithi179.stdout:4/87: creat d1/d3/dc/f1c x:0 0 0 2024-04-02T04:47:39.281 INFO:tasks.workunit.client.1.smithi179.stdout:9/102: fsync d0/f5 0 2024-04-02T04:47:39.282 INFO:tasks.workunit.client.0.smithi070.stdout:5/94: symlink d5/l21 0 2024-04-02T04:47:39.285 INFO:tasks.workunit.client.1.smithi179.stdout:0/146: unlink d1/db/l24 0 2024-04-02T04:47:39.289 INFO:tasks.workunit.client.1.smithi179.stdout:0/147: write d1/da/df/f20 [534474,40202] 0 2024-04-02T04:47:39.290 INFO:tasks.workunit.client.1.smithi179.stdout:7/28: creat d1/f7 x:0 0 0 2024-04-02T04:47:39.291 INFO:tasks.workunit.client.1.smithi179.stdout:3/51: truncate f5 1197907 0 2024-04-02T04:47:39.291 INFO:tasks.workunit.client.0.smithi070.stdout:1/118: dwrite d0/d7/d1a/f1b [0,4194304] 0 2024-04-02T04:47:39.292 INFO:tasks.workunit.client.1.smithi179.stdout:2/35: dwrite f9 [0,4194304] 0 2024-04-02T04:47:39.292 INFO:tasks.workunit.client.0.smithi070.stdout:2/184: mkdir d2/d12/d3c 0 2024-04-02T04:47:39.292 INFO:tasks.workunit.client.1.smithi179.stdout:3/52: rename d8 to d8/de 22 2024-04-02T04:47:39.292 INFO:tasks.workunit.client.1.smithi179.stdout:9/103: dwrite d0/fd [0,4194304] 0 2024-04-02T04:47:39.293 INFO:tasks.workunit.client.0.smithi070.stdout:1/119: readlink d0/d9/d1d/l1f 0 2024-04-02T04:47:39.293 INFO:tasks.workunit.client.1.smithi179.stdout:6/28: symlink d3/l11 0 2024-04-02T04:47:39.293 INFO:tasks.workunit.client.1.smithi179.stdout:2/36: read f6 [481451,46823] 0 2024-04-02T04:47:39.294 INFO:tasks.workunit.client.0.smithi070.stdout:9/107: mknod d4/d10/d12/c23 0 2024-04-02T04:47:39.296 INFO:tasks.workunit.client.0.smithi070.stdout:9/108: chown d4/ce 2347 1 2024-04-02T04:47:39.297 INFO:tasks.workunit.client.0.smithi070.stdout:9/109: chown d4/fc 1788909 1 2024-04-02T04:47:39.299 INFO:tasks.workunit.client.0.smithi070.stdout:7/85: symlink d7/dd/l13 0 2024-04-02T04:47:39.301 INFO:tasks.workunit.client.0.smithi070.stdout:7/86: readlink d7/dd/l13 0 2024-04-02T04:47:39.302 INFO:tasks.workunit.client.0.smithi070.stdout:2/185: dwrite d2/d10/d19/f39 [0,4194304] 0 2024-04-02T04:47:39.307 INFO:tasks.workunit.client.1.smithi179.stdout:4/88: mknod d1/d3/df/d10/c1d 0 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.305+0000 7fc104fb9700 1 -- 172.21.15.70:0/359637528 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0f0004a00 con 0x7fc10010a700 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.306+0000 7fc10821f700 1 -- 172.21.15.70:0/359637528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10010a700 msgr2=0x7fc10010cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.306+0000 7fc10821f700 1 --2- 172.21.15.70:0/359637528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10010a700 0x7fc10010cb90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009910 tx=0x7fc0f0009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.307+0000 7fc10821f700 1 -- 172.21.15.70:0/359637528 shutdown_connections 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.307+0000 7fc10821f700 1 --2- 172.21.15.70:0/359637528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10010a700 0x7fc10010cb90 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009910 tx=0x7fc0f0009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.307+0000 7fc10821f700 1 --2- 172.21.15.70:0/359637528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc100107d90 0x7fc10010a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.307+0000 7fc10821f700 1 -- 172.21.15.70:0/359637528 >> 172.21.15.70:0/359637528 conn(0x7fc10006da40 msgr2=0x7fc10006fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.307+0000 7fc10821f700 1 -- 172.21.15.70:0/359637528 shutdown_connections 2024-04-02T04:47:39.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.307+0000 7fc10821f700 1 -- 172.21.15.70:0/359637528 wait complete. 2024-04-02T04:47:39.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc10821f700 1 Processor -- start 2024-04-02T04:47:39.311 INFO:tasks.workunit.client.1.smithi179.stdout:1/147: symlink d3/l30 0 2024-04-02T04:47:39.311 INFO:tasks.workunit.client.1.smithi179.stdout:4/89: dread d1/d3/fa [0,4194304] 0 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc10821f700 1 -- start start 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc10821f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc100107d90 0x7fc100116d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc10821f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc100117270 0x7fc1001b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc10821f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc100117780 con 0x7fc100107d90 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc10821f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1001178c0 con 0x7fc100117270 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc105fbb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc100107d90 0x7fc100116d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc105fbb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc100107d90 0x7fc100116d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41614/0 (socket says 172.21.15.70:41614) 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.308+0000 7fc105fbb700 1 -- 172.21.15.70:0/2570037957 learned_addr learned my addr 172.21.15.70:0/2570037957 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:39.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.309+0000 7fc105fbb700 1 -- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc100117270 msgr2=0x7fc1001b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:39.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.309+0000 7fc105fbb700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc100117270 0x7fc1001b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.309+0000 7fc105fbb700 1 -- 172.21.15.70:0/2570037957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f00095f0 con 0x7fc100107d90 2024-04-02T04:47:39.313 INFO:tasks.workunit.client.0.smithi070.stdout:9/110: dwrite d4/f9 [4194304,4194304] 0 2024-04-02T04:47:39.313 INFO:tasks.workunit.client.0.smithi070.stdout:2/186: dread d2/f14 [0,4194304] 0 2024-04-02T04:47:39.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.309+0000 7fc105fbb700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc100107d90 0x7fc100116d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00b4b0 tx=0x7fc0fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:39.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.310+0000 7fc0f6ffd700 1 -- 172.21.15.70:0/2570037957 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0fc011800 con 0x7fc100107d90 2024-04-02T04:47:39.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.310+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1001b38a0 con 0x7fc100107d90 2024-04-02T04:47:39.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.310+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1001b3dc0 con 0x7fc100107d90 2024-04-02T04:47:39.314 INFO:tasks.workunit.client.0.smithi070.stdout:2/187: chown d2/f14 84184303 1 2024-04-02T04:47:39.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.311+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc100110c60 con 0x7fc100107d90 2024-04-02T04:47:39.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.311+0000 7fc0f6ffd700 1 -- 172.21.15.70:0/2570037957 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0fc011e10 con 0x7fc100107d90 2024-04-02T04:47:39.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.311+0000 7fc0f6ffd700 1 -- 172.21.15.70:0/2570037957 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0fc0193b0 con 0x7fc100107d90 2024-04-02T04:47:39.315 INFO:tasks.workunit.client.0.smithi070.stdout:6/55: sync 2024-04-02T04:47:39.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.313+0000 7fc0f6ffd700 1 -- 172.21.15.70:0/2570037957 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7fc0fc019550 con 0x7fc100107d90 2024-04-02T04:47:39.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.313+0000 7fc0f6ffd700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fc0ec06d0a0 0x7fc0ec06f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:39.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.313+0000 7fc0f6ffd700 1 -- 172.21.15.70:0/2570037957 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fc0fc011c10 con 0x7fc100107d90 2024-04-02T04:47:39.317 INFO:tasks.workunit.client.0.smithi070.stdout:6/56: chown d0/d3/f13 6543593 1 2024-04-02T04:47:39.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.316+0000 7fc1057ba700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fc0ec06d0a0 0x7fc0ec06f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:39.320 INFO:tasks.workunit.client.1.smithi179.stdout:7/29: symlink d1/l8 0 2024-04-02T04:47:39.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.318+0000 7fc1057ba700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fc0ec06d0a0 0x7fc0ec06f560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0000c00 tx=0x7fc0f000b540 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:39.321 INFO:tasks.workunit.client.0.smithi070.stdout:5/95: creat d5/f22 x:0 0 0 2024-04-02T04:47:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:38 smithi179 ceph-mon[36603]: pgmap v8: 65 pgs: 65 active+clean; 222 MiB data, 985 MiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-04-02T04:47:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:38 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:47:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:38 smithi179 ceph-mon[36603]: mgrmap e24: smithi179.labmvs(active, since 10s), standbys: smithi070.igidvr 2024-04-02T04:47:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:38 smithi179 ceph-mon[36603]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:39.323 INFO:tasks.workunit.client.1.smithi179.stdout:2/37: mkdir dc 0 2024-04-02T04:47:39.325 INFO:tasks.workunit.client.0.smithi070.stdout:1/120: mkdir d0/d7/d28 0 2024-04-02T04:47:39.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.322+0000 7fc0f6ffd700 1 -- 172.21.15.70:0/2570037957 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fc0fc05aa10 con 0x7fc100107d90 2024-04-02T04:47:39.325 INFO:tasks.workunit.client.1.smithi179.stdout:6/29: mkdir d3/d12 0 2024-04-02T04:47:39.326 INFO:tasks.workunit.client.0.smithi070.stdout:1/121: write d0/d9/d1d/f2 [268954,2930] 0 2024-04-02T04:47:39.327 INFO:tasks.workunit.client.0.smithi070.stdout:1/122: write d0/d7/d1a/f24 [874176,81215] 0 2024-04-02T04:47:39.328 INFO:tasks.workunit.client.1.smithi179.stdout:9/104: fsync d0/d1/d10/f13 0 2024-04-02T04:47:39.328 INFO:tasks.workunit.client.0.smithi070.stdout:1/123: chown d0/d9/d1d/l12 499724 1 2024-04-02T04:47:39.329 INFO:tasks.workunit.client.0.smithi070.stdout:1/124: fsync d0/d9/d1d/f26 0 2024-04-02T04:47:39.330 INFO:tasks.workunit.client.0.smithi070.stdout:1/125: write d0/d9/d1d/f26 [144310,49448] 0 2024-04-02T04:47:39.331 INFO:tasks.workunit.client.1.smithi179.stdout:8/90: getdents d3 0 2024-04-02T04:47:39.331 INFO:tasks.workunit.client.1.smithi179.stdout:4/90: dwrite d1/f4 [0,4194304] 0 2024-04-02T04:47:39.331 INFO:tasks.workunit.client.1.smithi179.stdout:6/30: dread - d3/ff zero size 2024-04-02T04:47:39.332 INFO:tasks.workunit.client.0.smithi070.stdout:1/126: rename d0/d9/d1d to d0/d9/d1d/d29 22 2024-04-02T04:47:39.332 INFO:tasks.workunit.client.1.smithi179.stdout:1/148: dwrite d3/fa [4194304,4194304] 0 2024-04-02T04:47:39.334 INFO:tasks.workunit.client.1.smithi179.stdout:2/38: fdatasync f9 0 2024-04-02T04:47:39.335 INFO:tasks.workunit.client.1.smithi179.stdout:6/31: dread - d3/f10 zero size 2024-04-02T04:47:39.336 INFO:tasks.workunit.client.0.smithi070.stdout:7/87: dread d7/fc [0,4194304] 0 2024-04-02T04:47:39.338 INFO:tasks.workunit.client.1.smithi179.stdout:1/149: fdatasync d3/f1f 0 2024-04-02T04:47:39.338 INFO:tasks.workunit.client.0.smithi070.stdout:9/111: creat d4/d13/d16/d19/f24 x:0 0 0 2024-04-02T04:47:39.339 INFO:tasks.workunit.client.0.smithi070.stdout:2/188: mkdir d2/d10/d21/d3d 0 2024-04-02T04:47:39.340 INFO:tasks.workunit.client.1.smithi179.stdout:0/148: dwrite d1/f2 [4194304,4194304] 0 2024-04-02T04:47:39.341 INFO:tasks.workunit.client.1.smithi179.stdout:7/30: mkdir d1/d9 0 2024-04-02T04:47:39.345 INFO:tasks.workunit.client.0.smithi070.stdout:7/88: dread d7/d9/fa [0,4194304] 0 2024-04-02T04:47:39.346 INFO:tasks.workunit.client.0.smithi070.stdout:6/57: fsync d0/d3/fb 0 2024-04-02T04:47:39.347 INFO:tasks.workunit.client.1.smithi179.stdout:5/41: getdents d8 0 2024-04-02T04:47:39.349 INFO:tasks.workunit.client.0.smithi070.stdout:5/96: mkdir d5/d9/d23 0 2024-04-02T04:47:39.351 INFO:tasks.workunit.client.0.smithi070.stdout:5/97: write d5/f22 [274471,33571] 0 2024-04-02T04:47:39.351 INFO:tasks.workunit.client.1.smithi179.stdout:9/105: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:47:39.351 INFO:tasks.workunit.client.0.smithi070.stdout:5/98: read f4 [1474290,71196] 0 2024-04-02T04:47:39.352 INFO:tasks.workunit.client.0.smithi070.stdout:7/89: dread d7/fc [0,4194304] 0 2024-04-02T04:47:39.353 INFO:tasks.workunit.client.1.smithi179.stdout:8/91: mkdir d3/d12/d21 0 2024-04-02T04:47:39.353 INFO:tasks.workunit.client.0.smithi070.stdout:7/90: readlink d7/dd/l13 0 2024-04-02T04:47:39.355 INFO:tasks.workunit.client.1.smithi179.stdout:1/150: dwrite d3/d5/f21 [0,4194304] 0 2024-04-02T04:47:39.360 INFO:tasks.workunit.client.0.smithi070.stdout:4/82: rmdir d1/d3/d19 0 2024-04-02T04:47:39.363 INFO:tasks.workunit.client.1.smithi179.stdout:4/91: dread d1/d3/f7 [0,4194304] 0 2024-04-02T04:47:39.363 INFO:tasks.workunit.client.1.smithi179.stdout:6/32: mknod d3/c13 0 2024-04-02T04:47:39.366 INFO:tasks.workunit.client.0.smithi070.stdout:5/99: dwrite d5/f22 [0,4194304] 0 2024-04-02T04:47:39.367 INFO:tasks.workunit.client.1.smithi179.stdout:7/31: dwrite d1/f7 [0,4194304] 0 2024-04-02T04:47:39.367 INFO:tasks.workunit.client.0.smithi070.stdout:3/44: dwrite f0 [0,4194304] 0 2024-04-02T04:47:39.367 INFO:tasks.workunit.client.1.smithi179.stdout:4/92: stat d1/d3/f7 0 2024-04-02T04:47:39.370 INFO:tasks.workunit.client.0.smithi070.stdout:9/112: creat d4/d10/f25 x:0 0 0 2024-04-02T04:47:39.371 INFO:tasks.workunit.client.0.smithi070.stdout:9/113: dread - d4/d13/d16/d19/f22 zero size 2024-04-02T04:47:39.371 INFO:tasks.workunit.client.1.smithi179.stdout:5/42: mknod d8/cd 0 2024-04-02T04:47:39.372 INFO:tasks.workunit.client.0.smithi070.stdout:9/114: dread - d4/d10/f25 zero size 2024-04-02T04:47:39.374 INFO:tasks.workunit.client.0.smithi070.stdout:2/189: read d2/f14 [116331,35040] 0 2024-04-02T04:47:39.374 INFO:tasks.workunit.client.1.smithi179.stdout:0/149: read d1/f23 [54096,55107] 0 2024-04-02T04:47:39.375 INFO:tasks.workunit.client.0.smithi070.stdout:8/102: creat d2/f1f x:0 0 0 2024-04-02T04:47:39.379 INFO:tasks.workunit.client.0.smithi070.stdout:0/259: creat d4/d5/f3f x:0 0 0 2024-04-02T04:47:39.383 INFO:tasks.workunit.client.1.smithi179.stdout:1/151: mkdir d3/d31 0 2024-04-02T04:47:39.384 INFO:tasks.workunit.client.1.smithi179.stdout:9/106: dwrite d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:39.384 INFO:tasks.workunit.client.0.smithi070.stdout:0/260: write d4/d5/f15 [571069,38017] 0 2024-04-02T04:47:39.385 INFO:tasks.workunit.client.0.smithi070.stdout:8/103: dread d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:39.386 INFO:tasks.workunit.client.1.smithi179.stdout:2/39: dwrite f6 [0,4194304] 0 2024-04-02T04:47:39.387 INFO:tasks.workunit.client.1.smithi179.stdout:0/150: truncate d1/da/d28/f29 2641442 0 2024-04-02T04:47:39.387 INFO:tasks.workunit.client.0.smithi070.stdout:8/104: readlink d2/l1a 0 2024-04-02T04:47:39.389 INFO:tasks.workunit.client.0.smithi070.stdout:7/91: symlink d7/l14 0 2024-04-02T04:47:39.391 INFO:tasks.workunit.client.0.smithi070.stdout:8/105: fdatasync d2/d3/f17 0 2024-04-02T04:47:39.392 INFO:tasks.workunit.client.1.smithi179.stdout:4/93: dwrite d1/d3/dc/fd [0,4194304] 0 2024-04-02T04:47:39.394 INFO:tasks.workunit.client.0.smithi070.stdout:6/58: dwrite d0/f15 [0,4194304] 0 2024-04-02T04:47:39.395 INFO:tasks.workunit.client.1.smithi179.stdout:7/32: dwrite d1/f6 [0,4194304] 0 2024-04-02T04:47:39.395 INFO:tasks.workunit.client.0.smithi070.stdout:4/83: mknod d1/c1b 0 2024-04-02T04:47:39.397 INFO:tasks.workunit.client.0.smithi070.stdout:8/106: write d2/d8/d14/f15 [703131,16304] 0 2024-04-02T04:47:39.398 INFO:tasks.workunit.client.0.smithi070.stdout:3/45: rename c2 to d10/c14 0 2024-04-02T04:47:39.398 INFO:tasks.workunit.client.1.smithi179.stdout:7/33: chown l0 192 1 2024-04-02T04:47:39.399 INFO:tasks.workunit.client.1.smithi179.stdout:6/33: truncate f2 2611315 0 2024-04-02T04:47:39.400 INFO:tasks.workunit.client.0.smithi070.stdout:5/100: symlink d5/d14/l24 0 2024-04-02T04:47:39.400 INFO:tasks.workunit.client.0.smithi070.stdout:4/84: chown d1/f18 1881495 1 2024-04-02T04:47:39.400 INFO:tasks.workunit.client.0.smithi070.stdout:0/261: dwrite d4/d5/f3f [0,4194304] 0 2024-04-02T04:47:39.402 INFO:tasks.workunit.client.1.smithi179.stdout:6/34: chown d3/c13 1334571 1 2024-04-02T04:47:39.403 INFO:tasks.workunit.client.0.smithi070.stdout:8/107: write d2/d3/f7 [3952887,78479] 0 2024-04-02T04:47:39.403 INFO:tasks.workunit.client.1.smithi179.stdout:9/107: dwrite d0/fd [0,4194304] 0 2024-04-02T04:47:39.404 INFO:tasks.workunit.client.1.smithi179.stdout:1/152: dread d3/d5/f25 [0,4194304] 0 2024-04-02T04:47:39.405 INFO:tasks.workunit.client.0.smithi070.stdout:9/115: creat d4/d13/f26 x:0 0 0 2024-04-02T04:47:39.411 INFO:tasks.workunit.client.0.smithi070.stdout:2/190: rename d2/d12/d3c to d2/d12/d3e 0 2024-04-02T04:47:39.412 INFO:tasks.workunit.client.0.smithi070.stdout:5/101: dread d5/f6 [0,4194304] 0 2024-04-02T04:47:39.416 INFO:tasks.workunit.client.0.smithi070.stdout:9/116: dwrite d4/d13/d16/d19/f24 [0,4194304] 0 2024-04-02T04:47:39.421 INFO:tasks.workunit.client.0.smithi070.stdout:1/127: creat d0/f2a x:0 0 0 2024-04-02T04:47:39.423 INFO:tasks.workunit.client.0.smithi070.stdout:2/191: dwrite d2/d24/f37 [0,4194304] 0 2024-04-02T04:47:39.426 INFO:tasks.workunit.client.0.smithi070.stdout:6/59: creat d0/f16 x:0 0 0 2024-04-02T04:47:39.426 INFO:tasks.workunit.client.0.smithi070.stdout:1/128: write d0/d9/d1d/f16 [430934,60996] 0 2024-04-02T04:47:39.427 INFO:tasks.workunit.client.0.smithi070.stdout:3/46: creat d10/f15 x:0 0 0 2024-04-02T04:47:39.429 INFO:tasks.workunit.client.0.smithi070.stdout:6/60: write d0/d3/fb [2365809,65027] 0 2024-04-02T04:47:39.429 INFO:tasks.workunit.client.0.smithi070.stdout:3/47: stat f1 0 2024-04-02T04:47:39.431 INFO:tasks.workunit.client.0.smithi070.stdout:4/85: mknod d1/d3/d9/c1c 0 2024-04-02T04:47:39.431 INFO:tasks.workunit.client.0.smithi070.stdout:0/262: symlink d4/d3c/l40 0 2024-04-02T04:47:39.433 INFO:tasks.workunit.client.1.smithi179.stdout:4/94: dwrite d1/d3/dc/fd [0,4194304] 0 2024-04-02T04:47:39.433 INFO:tasks.workunit.client.1.smithi179.stdout:4/95: stat d1/d15 0 2024-04-02T04:47:39.435 INFO:tasks.workunit.client.1.smithi179.stdout:8/92: mkdir d3/d12/d21/d22 0 2024-04-02T04:47:39.435 INFO:tasks.workunit.client.0.smithi070.stdout:6/61: read - d0/d3/f13 zero size 2024-04-02T04:47:39.437 INFO:tasks.workunit.client.1.smithi179.stdout:9/108: symlink d0/d1/d12/l1f 0 2024-04-02T04:47:39.437 INFO:tasks.workunit.client.1.smithi179.stdout:1/153: creat d3/d5/de/f32 x:0 0 0 2024-04-02T04:47:39.437 INFO:tasks.workunit.client.1.smithi179.stdout:6/35: write d3/fb [816059,99190] 0 2024-04-02T04:47:39.437 INFO:tasks.workunit.client.0.smithi070.stdout:5/102: unlink d5/d9/c1e 0 2024-04-02T04:47:39.438 INFO:tasks.workunit.client.0.smithi070.stdout:7/92: dwrite d7/d9/fa [0,4194304] 0 2024-04-02T04:47:39.439 INFO:tasks.workunit.client.0.smithi070.stdout:9/117: mkdir d4/d10/d12/d27 0 2024-04-02T04:47:39.439 INFO:tasks.workunit.client.1.smithi179.stdout:9/109: write d0/d1/d10/f18 [970763,27482] 0 2024-04-02T04:47:39.441 INFO:tasks.workunit.client.1.smithi179.stdout:9/110: chown d0 12 1 2024-04-02T04:47:39.441 INFO:tasks.workunit.client.0.smithi070.stdout:5/103: dread - d5/d9/d10/f11 zero size 2024-04-02T04:47:39.450 INFO:tasks.workunit.client.1.smithi179.stdout:7/34: mknod d1/d9/ca 0 2024-04-02T04:47:39.452 INFO:tasks.workunit.client.0.smithi070.stdout:1/129: rmdir d0/d7/d1a 39 2024-04-02T04:47:39.453 INFO:tasks.workunit.client.0.smithi070.stdout:5/104: dwrite d5/d14/f16 [0,4194304] 0 2024-04-02T04:47:39.455 INFO:tasks.workunit.client.0.smithi070.stdout:1/130: truncate d0/d9/f14 1620717 0 2024-04-02T04:47:39.458 INFO:tasks.workunit.client.0.smithi070.stdout:1/131: chown d0/d9/d1d/f2 17036799 1 2024-04-02T04:47:39.458 INFO:tasks.workunit.client.1.smithi179.stdout:8/93: rmdir d3/d12/d21 39 2024-04-02T04:47:39.458 INFO:tasks.workunit.client.1.smithi179.stdout:9/111: dwrite d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:39.460 INFO:tasks.workunit.client.0.smithi070.stdout:3/48: symlink d10/l16 0 2024-04-02T04:47:39.463 INFO:tasks.workunit.client.0.smithi070.stdout:5/105: dwrite d5/f22 [0,4194304] 0 2024-04-02T04:47:39.463 INFO:tasks.workunit.client.1.smithi179.stdout:7/35: dwrite d1/f7 [0,4194304] 0 2024-04-02T04:47:39.463 INFO:tasks.workunit.client.1.smithi179.stdout:6/36: mknod d3/c14 0 2024-04-02T04:47:39.465 INFO:tasks.workunit.client.1.smithi179.stdout:6/37: write d3/f10 [44458,19544] 0 2024-04-02T04:47:39.468 INFO:tasks.workunit.client.0.smithi070.stdout:0/263: mknod d4/d5/d13/d2a/d3a/c41 0 2024-04-02T04:47:39.469 INFO:tasks.workunit.client.1.smithi179.stdout:1/154: mkdir d3/d5/de/d33 0 2024-04-02T04:47:39.470 INFO:tasks.workunit.client.0.smithi070.stdout:8/108: mknod d2/c20 0 2024-04-02T04:47:39.475 INFO:tasks.workunit.client.1.smithi179.stdout:8/94: dread d3/d12/f19 [0,4194304] 0 2024-04-02T04:47:39.476 INFO:tasks.workunit.client.0.smithi070.stdout:9/118: fdatasync d4/d13/d16/d19/f22 0 2024-04-02T04:47:39.477 INFO:tasks.workunit.client.1.smithi179.stdout:7/36: dwrite d1/f6 [0,4194304] 0 2024-04-02T04:47:39.477 INFO:tasks.workunit.client.0.smithi070.stdout:9/119: fdatasync d4/d10/f18 0 2024-04-02T04:47:39.478 INFO:tasks.workunit.client.1.smithi179.stdout:8/95: write d3/d1b/f1c [697774,107962] 0 2024-04-02T04:47:39.479 INFO:tasks.workunit.client.0.smithi070.stdout:1/132: dread d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:39.480 INFO:tasks.workunit.client.1.smithi179.stdout:8/96: read d3/f16 [2325214,73130] 0 2024-04-02T04:47:39.481 INFO:tasks.workunit.client.0.smithi070.stdout:2/192: truncate d2/f4 2793452 0 2024-04-02T04:47:39.481 INFO:tasks.workunit.client.1.smithi179.stdout:6/38: mknod d3/c15 0 2024-04-02T04:47:39.483 INFO:tasks.workunit.client.0.smithi070.stdout:9/120: dread d4/f9 [4194304,4194304] 0 2024-04-02T04:47:39.484 INFO:tasks.workunit.client.1.smithi179.stdout:1/155: getdents d3/d31 0 2024-04-02T04:47:39.487 INFO:tasks.workunit.client.1.smithi179.stdout:7/37: rename d1/f6 to d1/d9/fb 0 2024-04-02T04:47:39.489 INFO:tasks.workunit.client.0.smithi070.stdout:3/49: mkdir d10/d17 0 2024-04-02T04:47:39.492 INFO:tasks.workunit.client.1.smithi179.stdout:8/97: creat d3/d1b/f23 x:0 0 0 2024-04-02T04:47:39.492 INFO:tasks.workunit.client.0.smithi070.stdout:3/50: dread f1 [0,4194304] 0 2024-04-02T04:47:39.493 INFO:tasks.workunit.client.1.smithi179.stdout:8/98: write d3/d1b/f1c [523220,107314] 0 2024-04-02T04:47:39.500 INFO:tasks.workunit.client.1.smithi179.stdout:7/38: unlink d1/f7 0 2024-04-02T04:47:39.500 INFO:tasks.workunit.client.0.smithi070.stdout:3/51: dwrite d10/f15 [0,4194304] 0 2024-04-02T04:47:39.500 INFO:tasks.workunit.client.0.smithi070.stdout:3/52: chown l3 43378 1 2024-04-02T04:47:39.501 INFO:tasks.workunit.client.1.smithi179.stdout:7/39: chown d1/l2 5278051 1 2024-04-02T04:47:39.501 INFO:tasks.workunit.client.1.smithi179.stdout:7/40: truncate d1/d9/fb 4316946 0 2024-04-02T04:47:39.502 INFO:tasks.workunit.client.0.smithi070.stdout:7/93: rename c4 to d7/c15 0 2024-04-02T04:47:39.505 INFO:tasks.workunit.client.1.smithi179.stdout:6/39: creat d3/d12/f16 x:0 0 0 2024-04-02T04:47:39.509 INFO:tasks.workunit.client.1.smithi179.stdout:8/99: write d3/f16 [5174316,110655] 0 2024-04-02T04:47:39.510 INFO:tasks.workunit.client.0.smithi070.stdout:2/193: creat d2/d24/f3f x:0 0 0 2024-04-02T04:47:39.511 INFO:tasks.workunit.client.0.smithi070.stdout:2/194: dread - d2/d10/d21/f34 zero size 2024-04-02T04:47:39.511 INFO:tasks.workunit.client.1.smithi179.stdout:1/156: creat d3/d5/d2f/f34 x:0 0 0 2024-04-02T04:47:39.512 INFO:tasks.workunit.client.0.smithi070.stdout:2/195: readlink d2/d24/l26 0 2024-04-02T04:47:39.516 INFO:tasks.workunit.client.1.smithi179.stdout:7/41: write d1/d9/fb [1165173,13619] 0 2024-04-02T04:47:39.518 INFO:tasks.workunit.client.1.smithi179.stdout:6/40: symlink d3/d12/l17 0 2024-04-02T04:47:39.518 INFO:tasks.workunit.client.0.smithi070.stdout:0/264: fdatasync f0 0 2024-04-02T04:47:39.519 INFO:tasks.workunit.client.1.smithi179.stdout:3/53: dread f5 [0,4194304] 0 2024-04-02T04:47:39.523 INFO:tasks.workunit.client.0.smithi070.stdout:3/53: symlink d10/l18 0 2024-04-02T04:47:39.526 INFO:tasks.workunit.client.1.smithi179.stdout:7/42: creat d1/fc x:0 0 0 2024-04-02T04:47:39.527 INFO:tasks.workunit.client.1.smithi179.stdout:7/43: chown d1/d9 396 1 2024-04-02T04:47:39.527 INFO:tasks.workunit.client.0.smithi070.stdout:7/94: symlink d7/dd/df/l16 0 2024-04-02T04:47:39.527 INFO:tasks.workunit.client.1.smithi179.stdout:7/44: chown d1/l8 6 1 2024-04-02T04:47:39.531 INFO:tasks.workunit.client.1.smithi179.stdout:6/41: rename d3/d12/f16 to d3/f18 0 2024-04-02T04:47:39.531 INFO:tasks.workunit.client.1.smithi179.stdout:8/100: mknod d3/d12/d1e/c24 0 2024-04-02T04:47:39.531 INFO:tasks.workunit.client.1.smithi179.stdout:5/43: write d8/fc [403521,19876] 0 2024-04-02T04:47:39.532 INFO:tasks.workunit.client.0.smithi070.stdout:1/133: symlink d0/d7/d1a/l2b 0 2024-04-02T04:47:39.535 INFO:tasks.workunit.client.1.smithi179.stdout:3/54: creat d8/ff x:0 0 0 2024-04-02T04:47:39.536 INFO:tasks.workunit.client.1.smithi179.stdout:3/55: chown f4 7568695 1 2024-04-02T04:47:39.536 INFO:tasks.workunit.client.1.smithi179.stdout:1/157: mkdir d3/d31/d35 0 2024-04-02T04:47:39.537 INFO:tasks.workunit.client.1.smithi179.stdout:7/45: dwrite d1/fc [0,4194304] 0 2024-04-02T04:47:39.537 INFO:tasks.workunit.client.1.smithi179.stdout:3/56: chown d8/ff 9 1 2024-04-02T04:47:39.538 INFO:tasks.workunit.client.1.smithi179.stdout:8/101: dread d3/d12/f19 [0,4194304] 0 2024-04-02T04:47:39.539 INFO:tasks.workunit.client.1.smithi179.stdout:3/57: truncate d8/ff 887968 0 2024-04-02T04:47:39.539 INFO:tasks.workunit.client.1.smithi179.stdout:6/42: symlink d3/l19 0 2024-04-02T04:47:39.540 INFO:tasks.workunit.client.1.smithi179.stdout:3/58: chown d8/l9 35961 1 2024-04-02T04:47:39.540 INFO:tasks.workunit.client.0.smithi070.stdout:0/265: mknod d4/d5/d13/d23/c42 0 2024-04-02T04:47:39.542 INFO:tasks.workunit.client.1.smithi179.stdout:5/44: symlink d8/le 0 2024-04-02T04:47:39.544 INFO:tasks.workunit.client.1.smithi179.stdout:1/158: dread d3/fa [0,4194304] 0 2024-04-02T04:47:39.544 INFO:tasks.workunit.client.1.smithi179.stdout:7/46: dread d1/fc [0,4194304] 0 2024-04-02T04:47:39.544 INFO:tasks.workunit.client.1.smithi179.stdout:5/45: write d8/fc [315657,91462] 0 2024-04-02T04:47:39.548 INFO:tasks.workunit.client.0.smithi070.stdout:3/54: write f1 [3379100,121922] 0 2024-04-02T04:47:39.548 INFO:tasks.workunit.client.0.smithi070.stdout:6/62: fsync d0/f16 0 2024-04-02T04:47:39.548 INFO:tasks.workunit.client.0.smithi070.stdout:3/55: fsync fd 0 2024-04-02T04:47:39.550 INFO:tasks.workunit.client.0.smithi070.stdout:1/134: mkdir d0/d9/d1d/d22/d2c 0 2024-04-02T04:47:39.552 INFO:tasks.workunit.client.0.smithi070.stdout:9/121: link d4/d10/d12/c23 d4/d10/c28 0 2024-04-02T04:47:39.554 INFO:tasks.workunit.client.1.smithi179.stdout:6/43: symlink d3/l1a 0 2024-04-02T04:47:39.554 INFO:tasks.workunit.client.1.smithi179.stdout:6/44: dread - d3/fa zero size 2024-04-02T04:47:39.554 INFO:tasks.workunit.client.0.smithi070.stdout:2/196: mknod d2/d10/d21/d2b/c40 0 2024-04-02T04:47:39.555 INFO:tasks.workunit.client.0.smithi070.stdout:3/56: dwrite f1 [0,4194304] 0 2024-04-02T04:47:39.560 INFO:tasks.workunit.client.0.smithi070.stdout:8/109: getdents d2/d3/d6 0 2024-04-02T04:47:39.560 INFO:tasks.workunit.client.0.smithi070.stdout:5/106: rmdir d5/d9 39 2024-04-02T04:47:39.562 INFO:tasks.workunit.client.1.smithi179.stdout:7/47: mkdir d1/dd 0 2024-04-02T04:47:39.563 INFO:tasks.workunit.client.0.smithi070.stdout:3/57: dwrite d10/f15 [0,4194304] 0 2024-04-02T04:47:39.566 INFO:tasks.workunit.client.1.smithi179.stdout:0/151: write d1/f14 [2118478,30287] 0 2024-04-02T04:47:39.567 INFO:tasks.workunit.client.0.smithi070.stdout:6/63: creat d0/f17 x:0 0 0 2024-04-02T04:47:39.568 INFO:tasks.workunit.client.0.smithi070.stdout:6/64: truncate d0/d3/f13 174689 0 2024-04-02T04:47:39.569 INFO:tasks.workunit.client.1.smithi179.stdout:1/159: mkdir d3/d5/d2f/d36 0 2024-04-02T04:47:39.572 INFO:tasks.workunit.client.0.smithi070.stdout:1/135: write d0/d9/f14 [606039,1080] 0 2024-04-02T04:47:39.573 INFO:tasks.workunit.client.1.smithi179.stdout:6/45: fdatasync d3/fa 0 2024-04-02T04:47:39.578 INFO:tasks.workunit.client.0.smithi070.stdout:2/197: mknod d2/d10/c41 0 2024-04-02T04:47:39.580 INFO:tasks.workunit.client.0.smithi070.stdout:1/136: dwrite d0/d9/d1d/f26 [0,4194304] 0 2024-04-02T04:47:39.583 INFO:tasks.workunit.client.0.smithi070.stdout:0/266: mkdir d4/d5/d13/d43 0 2024-04-02T04:47:39.585 INFO:tasks.workunit.client.0.smithi070.stdout:3/58: fsync d10/f15 0 2024-04-02T04:47:39.586 INFO:tasks.workunit.client.0.smithi070.stdout:1/137: dread d0/d7/d1a/f1b [0,4194304] 0 2024-04-02T04:47:39.587 INFO:tasks.workunit.client.0.smithi070.stdout:3/59: chown d10/l12 1 1 2024-04-02T04:47:39.587 INFO:tasks.workunit.client.1.smithi179.stdout:9/112: truncate d0/d1/f7 796404 0 2024-04-02T04:47:39.587 INFO:tasks.workunit.client.1.smithi179.stdout:7/48: dwrite d1/fc [0,4194304] 0 2024-04-02T04:47:39.588 INFO:tasks.workunit.client.0.smithi070.stdout:1/138: chown d0/d9/d1d/d22/d2c 3 1 2024-04-02T04:47:39.590 INFO:tasks.workunit.client.0.smithi070.stdout:8/110: creat d2/d8/d14/db/f21 x:0 0 0 2024-04-02T04:47:39.592 INFO:tasks.workunit.client.1.smithi179.stdout:1/160: creat d3/d31/f37 x:0 0 0 2024-04-02T04:47:39.593 INFO:tasks.workunit.client.1.smithi179.stdout:1/161: chown d3/d5/fb 0 1 2024-04-02T04:47:39.593 INFO:tasks.workunit.client.1.smithi179.stdout:0/152: dread d1/f6 [4194304,4194304] 0 2024-04-02T04:47:39.594 INFO:tasks.workunit.client.1.smithi179.stdout:0/153: stat d1/db 0 2024-04-02T04:47:39.595 INFO:tasks.workunit.client.1.smithi179.stdout:6/46: creat d3/d12/f1b x:0 0 0 2024-04-02T04:47:39.596 INFO:tasks.workunit.client.1.smithi179.stdout:0/154: write d1/da/df/f11 [1657248,85458] 0 2024-04-02T04:47:39.596 INFO:tasks.workunit.client.0.smithi070.stdout:6/65: symlink d0/d3/l18 0 2024-04-02T04:47:39.598 INFO:tasks.workunit.client.0.smithi070.stdout:4/86: truncate d1/fc 3534446 0 2024-04-02T04:47:39.600 INFO:tasks.workunit.client.1.smithi179.stdout:0/155: dread d1/da/df/f20 [0,4194304] 0 2024-04-02T04:47:39.605 INFO:tasks.workunit.client.1.smithi179.stdout:1/162: dread d3/d5/de/f16 [0,4194304] 0 2024-04-02T04:47:39.607 INFO:tasks.workunit.client.1.smithi179.stdout:6/47: creat d3/f1c x:0 0 0 2024-04-02T04:47:39.609 INFO:tasks.workunit.client.1.smithi179.stdout:6/48: read d3/f10 [25091,125063] 0 2024-04-02T04:47:39.610 INFO:tasks.workunit.client.0.smithi070.stdout:0/267: mkdir d4/d5/d21/d44 0 2024-04-02T04:47:39.611 INFO:tasks.workunit.client.0.smithi070.stdout:0/268: readlink d4/d5/d13/d23/l34 0 2024-04-02T04:47:39.612 INFO:tasks.workunit.client.0.smithi070.stdout:5/107: mknod d5/d9/c25 0 2024-04-02T04:47:39.616 INFO:tasks.workunit.client.1.smithi179.stdout:0/156: write d1/f6 [7749914,44932] 0 2024-04-02T04:47:39.617 INFO:tasks.workunit.client.0.smithi070.stdout:1/139: creat d0/d9/d1d/d22/f2d x:0 0 0 2024-04-02T04:47:39.618 INFO:tasks.workunit.client.1.smithi179.stdout:0/157: write d1/da/d28/f29 [1150827,66828] 0 2024-04-02T04:47:39.620 INFO:tasks.workunit.client.0.smithi070.stdout:8/111: mkdir d2/d8/d14/d22 0 2024-04-02T04:47:39.621 INFO:tasks.workunit.client.0.smithi070.stdout:8/112: readlink d2/d3/d6/l11 0 2024-04-02T04:47:39.623 INFO:tasks.workunit.client.0.smithi070.stdout:6/66: mkdir d0/d3/d19 0 2024-04-02T04:47:39.626 INFO:tasks.workunit.client.0.smithi070.stdout:4/87: write d1/f18 [222400,93357] 0 2024-04-02T04:47:39.629 INFO:tasks.workunit.client.1.smithi179.stdout:6/49: unlink d3/f8 0 2024-04-02T04:47:39.629 INFO:tasks.workunit.client.1.smithi179.stdout:6/50: dread - d3/ff zero size 2024-04-02T04:47:39.632 INFO:tasks.workunit.client.1.smithi179.stdout:3/59: rmdir d8 39 2024-04-02T04:47:39.635 INFO:tasks.workunit.client.1.smithi179.stdout:6/51: dwrite d3/f1c [0,4194304] 0 2024-04-02T04:47:39.635 INFO:tasks.workunit.client.0.smithi070.stdout:4/88: dwrite d1/d3/d9/f13 [0,4194304] 0 2024-04-02T04:47:39.637 INFO:tasks.workunit.client.0.smithi070.stdout:4/89: chown d1/l10 0 1 2024-04-02T04:47:39.638 INFO:tasks.workunit.client.1.smithi179.stdout:0/158: creat d1/db/f2c x:0 0 0 2024-04-02T04:47:39.640 INFO:tasks.workunit.client.0.smithi070.stdout:0/269: mkdir d4/d5/d13/d2a/d3a/d45 0 2024-04-02T04:47:39.646 INFO:tasks.workunit.client.1.smithi179.stdout:0/159: dwrite d1/da/df/f11 [0,4194304] 0 2024-04-02T04:47:39.646 INFO:tasks.workunit.client.0.smithi070.stdout:5/108: rename l1 to d5/d17/l26 0 2024-04-02T04:47:39.649 INFO:tasks.workunit.client.0.smithi070.stdout:3/60: mkdir d10/d17/d19 0 2024-04-02T04:47:39.650 INFO:tasks.workunit.client.0.smithi070.stdout:7/95: write d7/fc [5140128,41881] 0 2024-04-02T04:47:39.651 INFO:tasks.workunit.client.0.smithi070.stdout:7/96: chown d7/dd/df 193991649 1 2024-04-02T04:47:39.655 INFO:tasks.workunit.client.0.smithi070.stdout:1/140: creat d0/d7/d1a/f2e x:0 0 0 2024-04-02T04:47:39.657 INFO:tasks.workunit.client.0.smithi070.stdout:8/113: write d2/d3/f7 [1177387,121143] 0 2024-04-02T04:47:39.658 INFO:tasks.workunit.client.0.smithi070.stdout:3/61: dread f1 [0,4194304] 0 2024-04-02T04:47:39.662 INFO:tasks.workunit.client.1.smithi179.stdout:7/49: getdents d1 0 2024-04-02T04:47:39.662 INFO:tasks.workunit.client.1.smithi179.stdout:7/50: chown d1/dd 169160209 1 2024-04-02T04:47:39.663 INFO:tasks.workunit.client.0.smithi070.stdout:3/62: dread f8 [0,4194304] 0 2024-04-02T04:47:39.663 INFO:tasks.workunit.client.1.smithi179.stdout:7/51: fdatasync d1/d9/fb 0 2024-04-02T04:47:39.665 INFO:tasks.workunit.client.1.smithi179.stdout:6/52: mknod d3/d12/c1d 0 2024-04-02T04:47:39.665 INFO:tasks.workunit.client.1.smithi179.stdout:1/163: truncate d3/fa 3362452 0 2024-04-02T04:47:39.666 INFO:tasks.workunit.client.0.smithi070.stdout:2/198: dread d2/f8 [0,4194304] 0 2024-04-02T04:47:39.667 INFO:tasks.workunit.client.1.smithi179.stdout:1/164: write d3/d5/d2f/f34 [642676,104823] 0 2024-04-02T04:47:39.669 INFO:tasks.workunit.client.0.smithi070.stdout:0/270: chown d4/d5/da 7613251 1 2024-04-02T04:47:39.670 INFO:tasks.workunit.client.1.smithi179.stdout:7/52: dwrite d1/d9/fb [0,4194304] 0 2024-04-02T04:47:39.670 INFO:tasks.workunit.client.1.smithi179.stdout:7/53: chown d1/d9/ca 28 1 2024-04-02T04:47:39.671 INFO:tasks.workunit.client.1.smithi179.stdout:0/160: rename d1/db/f2b to d1/da/f2d 0 2024-04-02T04:47:39.673 INFO:tasks.workunit.client.0.smithi070.stdout:5/109: stat d5/c7 0 2024-04-02T04:47:39.674 INFO:tasks.workunit.client.1.smithi179.stdout:8/102: truncate d3/f9 3574843 0 2024-04-02T04:47:39.674 INFO:tasks.workunit.client.1.smithi179.stdout:8/103: stat l0 0 2024-04-02T04:47:39.676 INFO:tasks.workunit.client.1.smithi179.stdout:3/60: creat d8/f10 x:0 0 0 2024-04-02T04:47:39.676 INFO:tasks.workunit.client.1.smithi179.stdout:8/104: dread - d3/f7 zero size 2024-04-02T04:47:39.678 INFO:tasks.workunit.client.1.smithi179.stdout:0/161: dwrite d1/db/fc [0,4194304] 0 2024-04-02T04:47:39.678 INFO:tasks.workunit.client.1.smithi179.stdout:5/46: fsync f6 0 2024-04-02T04:47:39.679 INFO:tasks.workunit.client.1.smithi179.stdout:2/40: sync 2024-04-02T04:47:39.681 INFO:tasks.workunit.client.0.smithi070.stdout:2/199: dwrite d2/d12/f38 [0,4194304] 0 2024-04-02T04:47:39.684 INFO:tasks.workunit.client.0.smithi070.stdout:7/97: chown d7/c15 4867 1 2024-04-02T04:47:39.686 INFO:tasks.workunit.client.0.smithi070.stdout:9/122: dwrite d4/d10/d11/f14 [0,4194304] 0 2024-04-02T04:47:39.687 INFO:tasks.workunit.client.0.smithi070.stdout:7/98: write d7/d9/fa [4993859,963] 0 2024-04-02T04:47:39.689 INFO:tasks.workunit.client.0.smithi070.stdout:3/63: fsync fb 0 2024-04-02T04:47:39.693 INFO:tasks.workunit.client.1.smithi179.stdout:7/54: mknod d1/d9/ce 0 2024-04-02T04:47:39.699 INFO:tasks.workunit.client.1.smithi179.stdout:3/61: creat d8/f11 x:0 0 0 2024-04-02T04:47:39.705 INFO:tasks.workunit.client.1.smithi179.stdout:1/165: symlink d3/d5/d2f/d36/l38 0 2024-04-02T04:47:39.706 INFO:tasks.workunit.client.1.smithi179.stdout:1/166: readlink d3/d5/d2f/d36/l38 0 2024-04-02T04:47:39.707 INFO:tasks.workunit.client.0.smithi070.stdout:8/114: mkdir d2/d8/d23 0 2024-04-02T04:47:39.707 INFO:tasks.workunit.client.1.smithi179.stdout:3/62: dwrite d8/ff [0,4194304] 0 2024-04-02T04:47:39.709 INFO:tasks.workunit.client.1.smithi179.stdout:7/55: creat d1/ff x:0 0 0 2024-04-02T04:47:39.715 INFO:tasks.workunit.client.0.smithi070.stdout:9/123: fsync d4/d10/d11/f14 0 2024-04-02T04:47:39.716 INFO:tasks.workunit.client.0.smithi070.stdout:9/124: chown d4/d13/d16/d19 530324629 1 2024-04-02T04:47:39.717 INFO:tasks.workunit.client.1.smithi179.stdout:0/162: symlink d1/l2e 0 2024-04-02T04:47:39.719 INFO:tasks.workunit.client.0.smithi070.stdout:9/125: dread d4/fc [0,4194304] 0 2024-04-02T04:47:39.719 INFO:tasks.workunit.client.0.smithi070.stdout:7/99: creat d7/d9/f17 x:0 0 0 2024-04-02T04:47:39.722 INFO:tasks.workunit.client.1.smithi179.stdout:2/41: creat dc/fd x:0 0 0 2024-04-02T04:47:39.723 INFO:tasks.workunit.client.1.smithi179.stdout:2/42: read f4 [3013491,53239] 0 2024-04-02T04:47:39.724 INFO:tasks.workunit.client.0.smithi070.stdout:5/110: truncate d5/ff 833887 0 2024-04-02T04:47:39.724 INFO:tasks.workunit.client.0.smithi070.stdout:5/111: dread - d5/d9/d10/f11 zero size 2024-04-02T04:47:39.727 INFO:tasks.workunit.client.1.smithi179.stdout:6/53: rmdir d3/d12 39 2024-04-02T04:47:39.733 INFO:tasks.workunit.client.0.smithi070.stdout:9/126: dread f3 [0,4194304] 0 2024-04-02T04:47:39.734 INFO:tasks.workunit.client.0.smithi070.stdout:7/100: creat d7/dd/df/f18 x:0 0 0 2024-04-02T04:47:39.737 INFO:tasks.workunit.client.0.smithi070.stdout:3/64: creat d10/d17/d19/f1a x:0 0 0 2024-04-02T04:47:39.738 INFO:tasks.workunit.client.0.smithi070.stdout:7/101: chown d7/d9/d11/c12 66042058 1 2024-04-02T04:47:39.739 INFO:tasks.workunit.client.0.smithi070.stdout:5/112: rmdir d5/d9 39 2024-04-02T04:47:39.742 INFO:tasks.workunit.client.0.smithi070.stdout:4/90: write d1/fc [615632,76966] 0 2024-04-02T04:47:39.742 INFO:tasks.workunit.client.0.smithi070.stdout:8/115: mkdir d2/d8/d23/d24 0 2024-04-02T04:47:39.743 INFO:tasks.workunit.client.0.smithi070.stdout:9/127: dwrite d4/d10/f18 [4194304,4194304] 0 2024-04-02T04:47:39.750 INFO:tasks.workunit.client.0.smithi070.stdout:7/102: mkdir d7/d9/d19 0 2024-04-02T04:47:39.750 INFO:tasks.workunit.client.0.smithi070.stdout:8/116: write d2/d3/f7 [725201,84271] 0 2024-04-02T04:47:39.751 INFO:tasks.workunit.client.1.smithi179.stdout:0/163: creat d1/db/f2f x:0 0 0 2024-04-02T04:47:39.752 INFO:tasks.workunit.client.0.smithi070.stdout:3/65: dwrite d10/f15 [0,4194304] 0 2024-04-02T04:47:39.753 INFO:tasks.workunit.client.0.smithi070.stdout:5/113: mkdir d5/d17/d27 0 2024-04-02T04:47:39.754 INFO:tasks.workunit.client.0.smithi070.stdout:3/66: dread - fe zero size 2024-04-02T04:47:39.755 INFO:tasks.workunit.client.1.smithi179.stdout:5/47: getdents d8 0 2024-04-02T04:47:39.758 INFO:tasks.workunit.client.0.smithi070.stdout:4/91: dwrite d1/f12 [0,4194304] 0 2024-04-02T04:47:39.762 INFO:tasks.workunit.client.1.smithi179.stdout:9/113: sync 2024-04-02T04:47:39.762 INFO:tasks.workunit.client.1.smithi179.stdout:4/96: sync 2024-04-02T04:47:39.763 INFO:tasks.workunit.client.1.smithi179.stdout:2/43: rename f4 to dc/fe 0 2024-04-02T04:47:39.766 INFO:tasks.workunit.client.0.smithi070.stdout:9/128: rename d4/cf to d4/d10/d11/c29 0 2024-04-02T04:47:39.766 INFO:tasks.workunit.client.0.smithi070.stdout:9/129: fsync d4/d13/d16/d19/f22 0 2024-04-02T04:47:39.767 INFO:tasks.workunit.client.1.smithi179.stdout:4/97: write d1/d3/dc/fd [2167407,94117] 0 2024-04-02T04:47:39.767 INFO:tasks.workunit.client.1.smithi179.stdout:1/167: link d3/lc d3/d5/d2f/l39 0 2024-04-02T04:47:39.772 INFO:tasks.workunit.client.1.smithi179.stdout:9/114: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:47:39.772 INFO:tasks.workunit.client.0.smithi070.stdout:3/67: read f8 [3775335,85212] 0 2024-04-02T04:47:39.773 INFO:tasks.workunit.client.0.smithi070.stdout:3/68: readlink d10/l16 0 2024-04-02T04:47:39.773 INFO:tasks.workunit.client.1.smithi179.stdout:5/48: rmdir d8 39 2024-04-02T04:47:39.776 INFO:tasks.workunit.client.0.smithi070.stdout:4/92: creat d1/d3/d9/f1d x:0 0 0 2024-04-02T04:47:39.778 INFO:tasks.workunit.client.1.smithi179.stdout:8/105: write d3/f9 [1912020,58259] 0 2024-04-02T04:47:39.780 INFO:tasks.workunit.client.1.smithi179.stdout:2/44: write f2 [1909565,120889] 0 2024-04-02T04:47:39.781 INFO:tasks.workunit.client.1.smithi179.stdout:3/63: dwrite f5 [0,4194304] 0 2024-04-02T04:47:39.782 INFO:tasks.workunit.client.1.smithi179.stdout:4/98: mknod d1/d3/d19/c1e 0 2024-04-02T04:47:39.783 INFO:tasks.workunit.client.0.smithi070.stdout:5/114: creat d5/d17/d27/f28 x:0 0 0 2024-04-02T04:47:39.788 INFO:tasks.workunit.client.1.smithi179.stdout:7/56: sync 2024-04-02T04:47:39.788 INFO:tasks.workunit.client.0.smithi070.stdout:3/69: symlink d10/d17/d19/l1b 0 2024-04-02T04:47:39.793 INFO:tasks.workunit.client.1.smithi179.stdout:6/54: dwrite d3/f10 [0,4194304] 0 2024-04-02T04:47:39.794 INFO:tasks.workunit.client.1.smithi179.stdout:9/115: creat d0/d1/d12/f20 x:0 0 0 2024-04-02T04:47:39.797 INFO:tasks.workunit.client.1.smithi179.stdout:5/49: dread - d8/fb zero size 2024-04-02T04:47:39.799 INFO:tasks.workunit.client.0.smithi070.stdout:4/93: creat d1/f1e x:0 0 0 2024-04-02T04:47:39.802 INFO:tasks.workunit.client.1.smithi179.stdout:6/55: dwrite d3/fa [0,4194304] 0 2024-04-02T04:47:39.803 INFO:tasks.workunit.client.0.smithi070.stdout:7/103: link d7/d9/d11/c12 d7/dd/c1a 0 2024-04-02T04:47:39.807 INFO:tasks.workunit.client.1.smithi179.stdout:2/45: rename c7 to dc/cf 0 2024-04-02T04:47:39.808 INFO:tasks.workunit.client.1.smithi179.stdout:7/57: rename d1 to d1/d9/d10 22 2024-04-02T04:47:39.808 INFO:tasks.workunit.client.1.smithi179.stdout:2/46: chown dc 63601 1 2024-04-02T04:47:39.808 INFO:tasks.workunit.client.1.smithi179.stdout:7/58: chown d1/dd 905012091 1 2024-04-02T04:47:39.809 INFO:tasks.workunit.client.1.smithi179.stdout:7/59: stat d1 0 2024-04-02T04:47:39.810 INFO:tasks.workunit.client.0.smithi070.stdout:3/70: symlink d10/d17/d19/l1c 0 2024-04-02T04:47:39.811 INFO:tasks.workunit.client.1.smithi179.stdout:3/64: unlink d8/cd 0 2024-04-02T04:47:39.814 INFO:tasks.workunit.client.1.smithi179.stdout:4/99: mkdir d1/d3/df/d1f 0 2024-04-02T04:47:39.819 INFO:tasks.workunit.client.0.smithi070.stdout:7/104: creat d7/d9/f1b x:0 0 0 2024-04-02T04:47:39.821 INFO:tasks.workunit.client.0.smithi070.stdout:5/115: unlink d5/d17/l26 0 2024-04-02T04:47:39.823 INFO:tasks.workunit.client.0.smithi070.stdout:8/117: truncate d2/d3/fd 491514 0 2024-04-02T04:47:39.824 INFO:tasks.workunit.client.0.smithi070.stdout:8/118: write d2/d3/d6/f16 [1303251,114077] 0 2024-04-02T04:47:39.825 INFO:tasks.workunit.client.1.smithi179.stdout:5/50: rmdir d8 39 2024-04-02T04:47:39.826 INFO:tasks.workunit.client.0.smithi070.stdout:3/71: mkdir d10/d17/d19/d1d 0 2024-04-02T04:47:39.827 INFO:tasks.workunit.client.0.smithi070.stdout:3/72: chown d10/l16 779 1 2024-04-02T04:47:39.827 INFO:tasks.workunit.client.0.smithi070.stdout:3/73: stat d10/l13 0 2024-04-02T04:47:39.829 INFO:tasks.workunit.client.1.smithi179.stdout:6/56: fdatasync d3/f18 0 2024-04-02T04:47:39.830 INFO:tasks.workunit.client.1.smithi179.stdout:0/164: dwrite d1/f1e [0,4194304] 0 2024-04-02T04:47:39.830 INFO:tasks.workunit.client.0.smithi070.stdout:9/130: getdents d4/d13 0 2024-04-02T04:47:39.832 INFO:tasks.workunit.client.1.smithi179.stdout:8/106: creat d3/d12/d21/f25 x:0 0 0 2024-04-02T04:47:39.833 INFO:tasks.workunit.client.1.smithi179.stdout:6/57: dread d3/fb [0,4194304] 0 2024-04-02T04:47:39.834 INFO:tasks.workunit.client.1.smithi179.stdout:6/58: stat d3/fb 0 2024-04-02T04:47:39.834 INFO:tasks.workunit.client.0.smithi070.stdout:7/105: mkdir d7/d1c 0 2024-04-02T04:47:39.835 INFO:tasks.workunit.client.0.smithi070.stdout:7/106: write d7/d9/f1b [7269,44498] 0 2024-04-02T04:47:39.836 INFO:tasks.workunit.client.1.smithi179.stdout:2/47: mknod dc/c10 0 2024-04-02T04:47:39.838 INFO:tasks.workunit.client.1.smithi179.stdout:6/59: dread d3/fc [0,4194304] 0 2024-04-02T04:47:39.838 INFO:tasks.workunit.client.1.smithi179.stdout:8/107: dread d3/f4 [0,4194304] 0 2024-04-02T04:47:39.841 INFO:tasks.workunit.client.1.smithi179.stdout:7/60: creat d1/d9/f11 x:0 0 0 2024-04-02T04:47:39.841 INFO:tasks.workunit.client.1.smithi179.stdout:7/61: fdatasync d1/d9/fb 0 2024-04-02T04:47:39.842 INFO:tasks.workunit.client.1.smithi179.stdout:7/62: write d1/d9/fb [166461,60582] 0 2024-04-02T04:47:39.842 INFO:tasks.workunit.client.0.smithi070.stdout:8/119: creat d2/d3/d6/f25 x:0 0 0 2024-04-02T04:47:39.843 INFO:tasks.workunit.client.1.smithi179.stdout:7/63: chown d1/l3 2980 1 2024-04-02T04:47:39.843 INFO:tasks.workunit.client.1.smithi179.stdout:4/100: mknod d1/d3/c20 0 2024-04-02T04:47:39.845 INFO:tasks.workunit.client.0.smithi070.stdout:4/94: rename d1/l10 to d1/d3/l1f 0 2024-04-02T04:47:39.847 INFO:tasks.workunit.client.1.smithi179.stdout:1/168: link d3/c22 d3/d5/de/c3a 0 2024-04-02T04:47:39.849 INFO:tasks.workunit.client.1.smithi179.stdout:7/64: dread d1/d9/fb [0,4194304] 0 2024-04-02T04:47:39.852 INFO:tasks.workunit.client.1.smithi179.stdout:4/101: dwrite d1/d3/f1b [0,4194304] 0 2024-04-02T04:47:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.854+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc10002ceb0 con 0x7fc100107d90 2024-04-02T04:47:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.854+0000 7fc0f6ffd700 1 -- 172.21.15.70:0/2570037957 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fc0fc05a5a0 con 0x7fc100107d90 2024-04-02T04:47:39.857 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:47:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:47:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:47:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-04-02T04:47:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:47:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:47:39.859 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:47:39.860 INFO:tasks.workunit.client.0.smithi070.stdout:3/74: rename cc to d10/d17/c1e 0 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.859+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fc0ec06d0a0 msgr2=0x7fc0ec06f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.859+0000 7fc10821f700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fc0ec06d0a0 0x7fc0ec06f560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0000c00 tx=0x7fc0f000b540 comp rx=0 tx=0).stop 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.859+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc100107d90 msgr2=0x7fc100116d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.859+0000 7fc10821f700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc100107d90 0x7fc100116d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00b4b0 tx=0x7fc0fc00b870 comp rx=0 tx=0).stop 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.860+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 shutdown_connections 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.860+0000 7fc10821f700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fc0ec06d0a0 0x7fc0ec06f560 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.860+0000 7fc10821f700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc100107d90 0x7fc100116d30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.860+0000 7fc10821f700 1 --2- 172.21.15.70:0/2570037957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc100117270 0x7fc1001b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.860+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 >> 172.21.15.70:0/2570037957 conn(0x7fc10006da40 msgr2=0x7fc10006fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.860+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 shutdown_connections 2024-04-02T04:47:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:39.860+0000 7fc10821f700 1 -- 172.21.15.70:0/2570037957 wait complete. 2024-04-02T04:47:39.866 INFO:tasks.workunit.client.0.smithi070.stdout:9/131: write d4/f9 [196364,77865] 0 2024-04-02T04:47:39.870 INFO:tasks.workunit.client.0.smithi070.stdout:7/107: unlink d7/dd/c1a 0 2024-04-02T04:47:39.872 INFO:tasks.workunit.client.0.smithi070.stdout:9/132: dread d4/d10/d11/f14 [0,4194304] 0 2024-04-02T04:47:39.873 INFO:tasks.workunit.client.0.smithi070.stdout:8/120: mknod d2/c26 0 2024-04-02T04:47:39.876 INFO:tasks.workunit.client.0.smithi070.stdout:8/121: write d2/d3/f17 [671816,110981] 0 2024-04-02T04:47:39.876 INFO:tasks.workunit.client.1.smithi179.stdout:1/169: mknod d3/d5/d2f/c3b 0 2024-04-02T04:47:39.878 INFO:tasks.workunit.client.1.smithi179.stdout:7/65: creat d1/d9/f12 x:0 0 0 2024-04-02T04:47:39.881 INFO:tasks.workunit.client.1.smithi179.stdout:7/66: write d1/d9/f11 [636637,122746] 0 2024-04-02T04:47:39.881 INFO:tasks.workunit.client.1.smithi179.stdout:5/51: creat d8/ff x:0 0 0 2024-04-02T04:47:39.882 INFO:tasks.workunit.client.0.smithi070.stdout:7/108: write d7/dd/df/f10 [4351408,34479] 0 2024-04-02T04:47:39.885 INFO:tasks.workunit.client.1.smithi179.stdout:5/52: dread f1 [0,4194304] 0 2024-04-02T04:47:39.886 INFO:tasks.workunit.client.0.smithi070.stdout:4/95: dwrite d1/d3/d9/f13 [4194304,4194304] 0 2024-04-02T04:47:39.888 INFO:tasks.workunit.client.0.smithi070.stdout:3/75: rmdir d10/d17/d19/d1d 0 2024-04-02T04:47:39.888 INFO:tasks.workunit.client.0.smithi070.stdout:8/122: dread d2/d3/fd [0,4194304] 0 2024-04-02T04:47:39.891 INFO:tasks.workunit.client.0.smithi070.stdout:8/123: chown d2/d8/d14 378367006 1 2024-04-02T04:47:39.893 INFO:tasks.workunit.client.0.smithi070.stdout:5/116: dwrite d5/ff [0,4194304] 0 2024-04-02T04:47:39.894 INFO:tasks.workunit.client.0.smithi070.stdout:9/133: truncate d4/d10/d11/f14 2854675 0 2024-04-02T04:47:39.897 INFO:tasks.workunit.client.0.smithi070.stdout:5/117: write d5/ff [23360,30288] 0 2024-04-02T04:47:39.903 INFO:tasks.workunit.client.0.smithi070.stdout:3/76: dwrite fa [0,4194304] 0 2024-04-02T04:47:39.903 INFO:tasks.workunit.client.0.smithi070.stdout:7/109: unlink d7/dd/df/f18 0 2024-04-02T04:47:39.904 INFO:tasks.workunit.client.0.smithi070.stdout:3/77: chown d10/l11 19277 1 2024-04-02T04:47:39.910 INFO:tasks.workunit.client.0.smithi070.stdout:7/110: write d7/d9/f17 [736229,24152] 0 2024-04-02T04:47:39.911 INFO:tasks.workunit.client.1.smithi179.stdout:9/116: rename d0/d1/d12/c1d to d0/d1/c21 0 2024-04-02T04:47:39.912 INFO:tasks.workunit.client.1.smithi179.stdout:6/60: mknod d3/c1e 0 2024-04-02T04:47:39.913 INFO:tasks.workunit.client.1.smithi179.stdout:2/48: truncate f6 620205 0 2024-04-02T04:47:39.914 INFO:tasks.workunit.client.1.smithi179.stdout:1/170: creat d3/d5/de/f3c x:0 0 0 2024-04-02T04:47:39.915 INFO:tasks.workunit.client.0.smithi070.stdout:3/78: dread d10/f15 [0,4194304] 0 2024-04-02T04:47:39.915 INFO:tasks.workunit.client.0.smithi070.stdout:4/96: write d1/fa [2183899,115725] 0 2024-04-02T04:47:39.915 INFO:tasks.workunit.client.1.smithi179.stdout:2/49: read - dc/fd zero size 2024-04-02T04:47:39.916 INFO:tasks.workunit.client.0.smithi070.stdout:7/111: write d7/d9/f17 [1571910,23596] 0 2024-04-02T04:47:39.916 INFO:tasks.workunit.client.0.smithi070.stdout:5/118: dwrite f2 [0,4194304] 0 2024-04-02T04:47:39.916 INFO:tasks.workunit.client.1.smithi179.stdout:5/53: dwrite d8/ff [0,4194304] 0 2024-04-02T04:47:39.918 INFO:tasks.workunit.client.1.smithi179.stdout:0/165: creat d1/f30 x:0 0 0 2024-04-02T04:47:39.920 INFO:tasks.workunit.client.0.smithi070.stdout:4/97: truncate d1/d3/d9/f1a 335830 0 2024-04-02T04:47:39.924 INFO:tasks.workunit.client.1.smithi179.stdout:8/108: rename d3/c1d to d3/d12/d21/d22/c26 0 2024-04-02T04:47:39.924 INFO:tasks.workunit.client.1.smithi179.stdout:7/67: rename d1 to d1/d13 22 2024-04-02T04:47:39.932 INFO:tasks.workunit.client.0.smithi070.stdout:9/134: symlink d4/d13/l2a 0 2024-04-02T04:47:39.933 INFO:tasks.workunit.client.0.smithi070.stdout:7/112: dread d7/d9/f17 [0,4194304] 0 2024-04-02T04:47:39.933 INFO:tasks.workunit.client.1.smithi179.stdout:9/117: unlink d0/d1/d12/f17 0 2024-04-02T04:47:39.934 INFO:tasks.workunit.client.0.smithi070.stdout:7/113: write d7/d9/f1b [537529,53528] 0 2024-04-02T04:47:39.935 INFO:tasks.workunit.client.0.smithi070.stdout:9/135: chown d4/d10/d12 8503 1 2024-04-02T04:47:39.937 INFO:tasks.workunit.client.0.smithi070.stdout:9/136: chown d4/d10 3235 1 2024-04-02T04:47:39.941 INFO:tasks.workunit.client.1.smithi179.stdout:2/50: creat dc/f11 x:0 0 0 2024-04-02T04:47:39.942 INFO:tasks.workunit.client.1.smithi179.stdout:0/166: readlink d1/da/l19 0 2024-04-02T04:47:39.944 INFO:tasks.workunit.client.0.smithi070.stdout:8/124: dwrite d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:39.948 INFO:tasks.workunit.client.1.smithi179.stdout:8/109: creat d3/f27 x:0 0 0 2024-04-02T04:47:39.948 INFO:tasks.workunit.client.0.smithi070.stdout:3/79: creat d10/d17/d19/f1f x:0 0 0 2024-04-02T04:47:39.949 INFO:tasks.workunit.client.0.smithi070.stdout:3/80: truncate d10/d17/d19/f1a 362870 0 2024-04-02T04:47:39.952 INFO:tasks.workunit.client.1.smithi179.stdout:6/61: mknod d3/c1f 0 2024-04-02T04:47:39.952 INFO:tasks.workunit.client.1.smithi179.stdout:2/51: dwrite dc/fd [0,4194304] 0 2024-04-02T04:47:39.954 INFO:tasks.workunit.client.1.smithi179.stdout:0/167: dwrite d1/da/d28/f29 [0,4194304] 0 2024-04-02T04:47:39.956 INFO:tasks.workunit.client.1.smithi179.stdout:1/171: mknod d3/d5/de/d33/c3d 0 2024-04-02T04:47:39.957 INFO:tasks.workunit.client.0.smithi070.stdout:7/114: symlink d7/l1d 0 2024-04-02T04:47:39.959 INFO:tasks.workunit.client.0.smithi070.stdout:5/119: dwrite d5/fa [0,4194304] 0 2024-04-02T04:47:39.960 INFO:tasks.workunit.client.0.smithi070.stdout:9/137: creat d4/d13/d16/f2b x:0 0 0 2024-04-02T04:47:39.961 INFO:tasks.workunit.client.0.smithi070.stdout:9/138: truncate d4/d13/d16/f1d 1012386 0 2024-04-02T04:47:39.962 INFO:tasks.workunit.client.0.smithi070.stdout:9/139: chown d4/d10/d12/d27 448 1 2024-04-02T04:47:39.962 INFO:tasks.workunit.client.1.smithi179.stdout:7/68: creat d1/dd/f14 x:0 0 0 2024-04-02T04:47:39.966 INFO:tasks.workunit.client.0.smithi070.stdout:8/125: stat d2/d8/d14/db/c1e 0 2024-04-02T04:47:39.966 INFO:tasks.workunit.client.1.smithi179.stdout:6/62: dread - d3/d12/f1b zero size 2024-04-02T04:47:39.967 INFO:tasks.workunit.client.1.smithi179.stdout:2/52: dwrite f9 [4194304,4194304] 0 2024-04-02T04:47:39.970 INFO:tasks.workunit.client.0.smithi070.stdout:3/81: symlink d10/d17/l20 0 2024-04-02T04:47:39.971 INFO:tasks.workunit.client.1.smithi179.stdout:0/168: dwrite d1/db/f2f [0,4194304] 0 2024-04-02T04:47:39.972 INFO:tasks.workunit.client.0.smithi070.stdout:4/98: mknod d1/d3/c20 0 2024-04-02T04:47:39.973 INFO:tasks.workunit.client.1.smithi179.stdout:7/69: creat d1/f15 x:0 0 0 2024-04-02T04:47:39.974 INFO:tasks.workunit.client.1.smithi179.stdout:1/172: dread d3/fa [0,4194304] 0 2024-04-02T04:47:39.976 INFO:tasks.workunit.client.1.smithi179.stdout:7/70: write d1/d9/f12 [586559,30837] 0 2024-04-02T04:47:39.980 INFO:tasks.workunit.client.0.smithi070.stdout:5/120: unlink d5/d14/l24 0 2024-04-02T04:47:39.981 INFO:tasks.workunit.client.0.smithi070.stdout:4/99: dwrite d1/f1e [0,4194304] 0 2024-04-02T04:47:39.983 INFO:tasks.workunit.client.1.smithi179.stdout:7/71: dread d1/fc [0,4194304] 0 2024-04-02T04:47:39.984 INFO:tasks.workunit.client.1.smithi179.stdout:7/72: fdatasync d1/d9/f11 0 2024-04-02T04:47:39.986 INFO:tasks.workunit.client.1.smithi179.stdout:8/110: link d3/l15 d3/d12/d21/l28 0 2024-04-02T04:47:39.994 INFO:tasks.workunit.client.1.smithi179.stdout:2/53: rename dc/f11 to dc/f12 0 2024-04-02T04:47:39.996 INFO:tasks.workunit.client.1.smithi179.stdout:0/169: symlink d1/db/l31 0 2024-04-02T04:47:39.998 INFO:tasks.workunit.client.0.smithi070.stdout:3/82: symlink d10/l21 0 2024-04-02T04:47:40.001 INFO:tasks.workunit.client.0.smithi070.stdout:7/115: link d7/d9/f1b d7/f1e 0 2024-04-02T04:47:40.005 INFO:tasks.workunit.client.0.smithi070.stdout:5/121: chown d5/d9/ce 314114 1 2024-04-02T04:47:40.006 INFO:tasks.workunit.client.0.smithi070.stdout:5/122: truncate d5/f1d 937635 0 2024-04-02T04:47:40.007 INFO:tasks.workunit.client.0.smithi070.stdout:5/123: read - d5/d9/d10/f19 zero size 2024-04-02T04:47:40.009 INFO:tasks.workunit.client.1.smithi179.stdout:7/73: mkdir d1/dd/d16 0 2024-04-02T04:47:40.013 INFO:tasks.workunit.client.1.smithi179.stdout:8/111: rename d3/l10 to d3/d12/d1e/l29 0 2024-04-02T04:47:40.013 INFO:tasks.workunit.client.0.smithi070.stdout:9/140: dwrite f3 [0,4194304] 0 2024-04-02T04:47:40.014 INFO:tasks.workunit.client.1.smithi179.stdout:1/173: write d3/f28 [1170484,94582] 0 2024-04-02T04:47:40.015 INFO:tasks.workunit.client.0.smithi070.stdout:3/83: creat d10/f22 x:0 0 0 2024-04-02T04:47:40.018 INFO:tasks.workunit.client.0.smithi070.stdout:7/116: fsync d7/f1e 0 2024-04-02T04:47:40.019 INFO:tasks.workunit.client.1.smithi179.stdout:0/170: readlink d1/da/l16 0 2024-04-02T04:47:40.020 INFO:tasks.workunit.client.1.smithi179.stdout:1/174: dwrite d3/f2d [0,4194304] 0 2024-04-02T04:47:40.021 INFO:tasks.workunit.client.0.smithi070.stdout:5/124: symlink d5/d17/d27/l29 0 2024-04-02T04:47:40.027 INFO:tasks.workunit.client.0.smithi070.stdout:8/126: rename d2/d8/d14 to d2/d27 0 2024-04-02T04:47:40.028 INFO:tasks.workunit.client.0.smithi070.stdout:8/127: chown d2/d27/db/f21 53389 1 2024-04-02T04:47:40.031 INFO:tasks.workunit.client.1.smithi179.stdout:7/74: creat d1/dd/f17 x:0 0 0 2024-04-02T04:47:40.032 INFO:tasks.workunit.client.0.smithi070.stdout:8/128: dread d2/d3/f7 [0,4194304] 0 2024-04-02T04:47:40.032 INFO:tasks.workunit.client.0.smithi070.stdout:9/141: creat d4/d10/f2c x:0 0 0 2024-04-02T04:47:40.032 INFO:tasks.workunit.client.0.smithi070.stdout:8/129: chown d2/d8/d23/d24 16100 1 2024-04-02T04:47:40.033 INFO:tasks.workunit.client.1.smithi179.stdout:2/54: rename l3 to dc/l13 0 2024-04-02T04:47:40.034 INFO:tasks.workunit.client.1.smithi179.stdout:2/55: dread - dc/f12 zero size 2024-04-02T04:47:40.035 INFO:tasks.workunit.client.0.smithi070.stdout:3/84: unlink d10/l16 0 2024-04-02T04:47:40.036 INFO:tasks.workunit.client.0.smithi070.stdout:3/85: fdatasync ff 0 2024-04-02T04:47:40.036 INFO:tasks.workunit.client.0.smithi070.stdout:8/130: dread d2/d27/db/f13 [0,4194304] 0 2024-04-02T04:47:40.036 INFO:tasks.workunit.client.0.smithi070.stdout:8/131: chown d2/d8/d23 3681 1 2024-04-02T04:47:40.036 INFO:tasks.workunit.client.1.smithi179.stdout:8/112: mkdir d3/d1b/d2a 0 2024-04-02T04:47:40.037 INFO:tasks.workunit.client.0.smithi070.stdout:3/86: truncate d10/f22 55321 0 2024-04-02T04:47:40.040 INFO:tasks.workunit.client.0.smithi070.stdout:7/117: rmdir d7/d9/d11 39 2024-04-02T04:47:40.040 INFO:tasks.workunit.client.1.smithi179.stdout:8/113: dread d3/d1b/f1c [0,4194304] 0 2024-04-02T04:47:40.041 INFO:tasks.workunit.client.1.smithi179.stdout:8/114: chown d3/d12/f19 17 1 2024-04-02T04:47:40.042 INFO:tasks.workunit.client.0.smithi070.stdout:5/125: symlink d5/d9/d10/l2a 0 2024-04-02T04:47:40.043 INFO:tasks.workunit.client.1.smithi179.stdout:6/63: link d3/l11 d3/l20 0 2024-04-02T04:47:40.043 INFO:tasks.workunit.client.0.smithi070.stdout:8/132: dwrite d2/d27/db/f13 [0,4194304] 0 2024-04-02T04:47:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.042+0000 7f42de265700 1 -- 172.21.15.70:0/42436264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d80ffc10 msgr2=0x7f42d8100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.042+0000 7f42de265700 1 --2- 172.21.15.70:0/42436264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d80ffc10 0x7f42d8100030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009910 tx=0x7f42c8009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.042+0000 7f42de265700 1 -- 172.21.15.70:0/42436264 shutdown_connections 2024-04-02T04:47:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.042+0000 7f42de265700 1 --2- 172.21.15.70:0/42436264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42d8100e10 0x7f42d8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.042+0000 7f42de265700 1 --2- 172.21.15.70:0/42436264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d80ffc10 0x7f42d8100030 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:40.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.042+0000 7f42de265700 1 -- 172.21.15.70:0/42436264 >> 172.21.15.70:0/42436264 conn(0x7f42d80fb190 msgr2=0x7f42d80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:40.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.042+0000 7f42de265700 1 -- 172.21.15.70:0/42436264 shutdown_connections 2024-04-02T04:47:40.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.042+0000 7f42de265700 1 -- 172.21.15.70:0/42436264 wait complete. 2024-04-02T04:47:40.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.043+0000 7f42de265700 1 Processor -- start 2024-04-02T04:47:40.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.043+0000 7f42de265700 1 -- start start 2024-04-02T04:47:40.047 INFO:tasks.workunit.client.1.smithi179.stdout:7/75: symlink d1/d9/l18 0 2024-04-02T04:47:40.047 INFO:tasks.workunit.client.1.smithi179.stdout:8/115: dread d3/f9 [0,4194304] 0 2024-04-02T04:47:40.047 INFO:tasks.workunit.client.1.smithi179.stdout:8/116: rename d3 to d3/d12/d1e/d2b 22 2024-04-02T04:47:40.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.043+0000 7f42de265700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d80ffc10 0x7f42d8103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.044+0000 7f42de265700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42d8100e10 0x7f42d81021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.044+0000 7f42de265700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d819a120 con 0x7f42d80ffc10 2024-04-02T04:47:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.044+0000 7f42de265700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d819a260 con 0x7f42d8100e10 2024-04-02T04:47:40.048 INFO:tasks.workunit.client.0.smithi070.stdout:9/142: creat d4/d10/d11/f2d x:0 0 0 2024-04-02T04:47:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.045+0000 7f42d77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42d8100e10 0x7f42d81021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.045+0000 7f42d77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42d8100e10 0x7f42d81021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:39748/0 (socket says 172.21.15.70:39748) 2024-04-02T04:47:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.045+0000 7f42d77fe700 1 -- 172.21.15.70:0/760207020 learned_addr learned my addr 172.21.15.70:0/760207020 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:47:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.045+0000 7f42d7fff700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d80ffc10 0x7f42d8103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.046+0000 7f42d77fe700 1 -- 172.21.15.70:0/760207020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d80ffc10 msgr2=0x7f42d8103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.046+0000 7f42d77fe700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d80ffc10 0x7f42d8103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.046+0000 7f42d77fe700 1 -- 172.21.15.70:0/760207020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80095f0 con 0x7f42d8100e10 2024-04-02T04:47:40.050 INFO:tasks.workunit.client.1.smithi179.stdout:2/56: dread f2 [0,4194304] 0 2024-04-02T04:47:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.046+0000 7f42d77fe700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42d8100e10 0x7f42d81021f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00d8d0 tx=0x7f42cc00dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.046+0000 7f42d57fa700 1 -- 172.21.15.70:0/760207020 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42cc0098a0 con 0x7f42d8100e10 2024-04-02T04:47:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.047+0000 7f42d57fa700 1 -- 172.21.15.70:0/760207020 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f42cc010460 con 0x7f42d8100e10 2024-04-02T04:47:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.047+0000 7f42d57fa700 1 -- 172.21.15.70:0/760207020 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42cc00f5f0 con 0x7f42d8100e10 2024-04-02T04:47:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.047+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d8102730 con 0x7f42d8100e10 2024-04-02T04:47:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.047+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d8102c50 con 0x7f42d8100e10 2024-04-02T04:47:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.049+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c4005320 con 0x7f42d8100e10 2024-04-02T04:47:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.049+0000 7f42d57fa700 1 -- 172.21.15.70:0/760207020 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f42cc00f750 con 0x7f42d8100e10 2024-04-02T04:47:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.050+0000 7f42d57fa700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f42c006d090 0x7f42c006f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.050+0000 7f42d57fa700 1 -- 172.21.15.70:0/760207020 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f42cc08c760 con 0x7f42d8100e10 2024-04-02T04:47:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.050+0000 7f42d7fff700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f42c006d090 0x7f42c006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:40.054 INFO:tasks.workunit.client.0.smithi070.stdout:3/87: mknod d10/d17/d19/c23 0 2024-04-02T04:47:40.054 INFO:tasks.workunit.client.1.smithi179.stdout:3/65: sync 2024-04-02T04:47:40.054 INFO:tasks.workunit.client.1.smithi179.stdout:3/66: fsync f5 0 2024-04-02T04:47:40.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.051+0000 7f42d7fff700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f42c006d090 0x7f42c006f550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006010 tx=0x7f42c800b560 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:40.055 INFO:tasks.workunit.client.0.smithi070.stdout:3/88: dread d10/f22 [0,4194304] 0 2024-04-02T04:47:40.057 INFO:tasks.workunit.client.1.smithi179.stdout:1/175: symlink d3/d31/d35/l3e 0 2024-04-02T04:47:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.056+0000 7f42d57fa700 1 -- 172.21.15.70:0/760207020 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f42cc05ada0 con 0x7f42d8100e10 2024-04-02T04:47:40.059 INFO:tasks.workunit.client.0.smithi070.stdout:4/100: getdents d1 0 2024-04-02T04:47:40.061 INFO:tasks.workunit.client.0.smithi070.stdout:4/101: dread d1/d3/d9/f1a [0,4194304] 0 2024-04-02T04:47:40.061 INFO:tasks.workunit.client.1.smithi179.stdout:0/171: truncate d1/db/fc 2454130 0 2024-04-02T04:47:40.061 INFO:tasks.workunit.client.1.smithi179.stdout:1/176: dread d3/d5/de/f16 [0,4194304] 0 2024-04-02T04:47:40.061 INFO:tasks.workunit.client.1.smithi179.stdout:0/172: chown d1/f30 50795 1 2024-04-02T04:47:40.062 INFO:tasks.workunit.client.1.smithi179.stdout:1/177: write d3/d5/de/f26 [3519847,73596] 0 2024-04-02T04:47:40.062 INFO:tasks.workunit.client.0.smithi070.stdout:5/126: mkdir d5/d17/d2b 0 2024-04-02T04:47:40.062 INFO:tasks.workunit.client.0.smithi070.stdout:5/127: fsync d5/f22 0 2024-04-02T04:47:40.063 INFO:tasks.workunit.client.1.smithi179.stdout:1/178: dread - d3/f1f zero size 2024-04-02T04:47:40.063 INFO:tasks.workunit.client.1.smithi179.stdout:1/179: read d3/d5/fb [1453037,89984] 0 2024-04-02T04:47:40.064 INFO:tasks.workunit.client.1.smithi179.stdout:7/76: rmdir d1/d9 39 2024-04-02T04:47:40.065 INFO:tasks.workunit.client.1.smithi179.stdout:6/64: dread f2 [0,4194304] 0 2024-04-02T04:47:40.065 INFO:tasks.workunit.client.0.smithi070.stdout:8/133: symlink d2/d3/l28 0 2024-04-02T04:47:40.065 INFO:tasks.workunit.client.1.smithi179.stdout:7/77: read - d1/dd/f14 zero size 2024-04-02T04:47:40.066 INFO:tasks.workunit.client.1.smithi179.stdout:8/117: chown d3/d12/d21/l28 8532996 1 2024-04-02T04:47:40.066 INFO:tasks.workunit.client.1.smithi179.stdout:8/118: dread - d3/d12/d21/f25 zero size 2024-04-02T04:47:40.067 INFO:tasks.workunit.client.0.smithi070.stdout:4/102: dwrite d1/d3/d9/f13 [0,4194304] 0 2024-04-02T04:47:40.068 INFO:tasks.workunit.client.0.smithi070.stdout:4/103: stat d1/c1b 0 2024-04-02T04:47:40.069 INFO:tasks.workunit.client.0.smithi070.stdout:6/67: sync 2024-04-02T04:47:40.070 INFO:tasks.workunit.client.0.smithi070.stdout:6/68: read d0/f15 [1371812,2727] 0 2024-04-02T04:47:40.072 INFO:tasks.workunit.client.0.smithi070.stdout:6/69: fsync d0/d3/fb 0 2024-04-02T04:47:40.074 INFO:tasks.workunit.client.0.smithi070.stdout:6/70: truncate d0/f16 452439 0 2024-04-02T04:47:40.075 INFO:tasks.workunit.client.1.smithi179.stdout:2/57: write f2 [340319,78130] 0 2024-04-02T04:47:40.076 INFO:tasks.workunit.client.0.smithi070.stdout:3/89: symlink d10/l24 0 2024-04-02T04:47:40.076 INFO:tasks.workunit.client.1.smithi179.stdout:2/58: write dc/fd [277801,38333] 0 2024-04-02T04:47:40.077 INFO:tasks.workunit.client.0.smithi070.stdout:4/104: dwrite d1/d3/d9/f11 [0,4194304] 0 2024-04-02T04:47:40.077 INFO:tasks.workunit.client.1.smithi179.stdout:3/67: symlink d8/l12 0 2024-04-02T04:47:40.079 INFO:tasks.workunit.client.0.smithi070.stdout:4/105: write d1/f18 [2665609,87039] 0 2024-04-02T04:47:40.079 INFO:tasks.workunit.client.0.smithi070.stdout:7/118: mknod d7/d9/c1f 0 2024-04-02T04:47:40.083 INFO:tasks.workunit.client.0.smithi070.stdout:3/90: dread d10/d17/d19/f1a [0,4194304] 0 2024-04-02T04:47:40.083 INFO:tasks.workunit.client.0.smithi070.stdout:3/91: chown d10/d17 830121 1 2024-04-02T04:47:40.086 INFO:tasks.workunit.client.0.smithi070.stdout:5/128: mkdir d5/d17/d27/d2c 0 2024-04-02T04:47:40.086 INFO:tasks.workunit.client.1.smithi179.stdout:1/180: symlink d3/d31/d35/l3f 0 2024-04-02T04:47:40.087 INFO:tasks.workunit.client.0.smithi070.stdout:5/129: truncate d5/d14/f16 4929459 0 2024-04-02T04:47:40.087 INFO:tasks.workunit.client.1.smithi179.stdout:1/181: dread - d3/d5/de/f24 zero size 2024-04-02T04:47:40.089 INFO:tasks.workunit.client.0.smithi070.stdout:9/143: link f3 d4/d13/d16/d19/f2e 0 2024-04-02T04:47:40.091 INFO:tasks.workunit.client.0.smithi070.stdout:9/144: write d4/d13/d16/d19/f22 [446109,120365] 0 2024-04-02T04:47:40.093 INFO:tasks.workunit.client.1.smithi179.stdout:2/59: mkdir dc/d14 0 2024-04-02T04:47:40.093 INFO:tasks.workunit.client.1.smithi179.stdout:2/60: fdatasync dc/fd 0 2024-04-02T04:47:40.095 INFO:tasks.workunit.client.1.smithi179.stdout:3/68: unlink f4 0 2024-04-02T04:47:40.095 INFO:tasks.workunit.client.0.smithi070.stdout:6/71: write d0/d3/f13 [1193209,64753] 0 2024-04-02T04:47:40.098 INFO:tasks.workunit.client.0.smithi070.stdout:5/130: dwrite d5/fa [4194304,4194304] 0 2024-04-02T04:47:40.098 INFO:tasks.workunit.client.0.smithi070.stdout:6/72: chown d0/f15 61163 1 2024-04-02T04:47:40.099 INFO:tasks.workunit.client.0.smithi070.stdout:5/131: chown d5/d17/d27 0 1 2024-04-02T04:47:40.100 INFO:tasks.workunit.client.1.smithi179.stdout:0/173: mknod d1/c32 0 2024-04-02T04:47:40.102 INFO:tasks.workunit.client.0.smithi070.stdout:9/145: dwrite d4/f8 [4194304,4194304] 0 2024-04-02T04:47:40.104 INFO:tasks.workunit.client.0.smithi070.stdout:9/146: read d4/d13/d16/d19/f22 [513728,104722] 0 2024-04-02T04:47:40.105 INFO:tasks.workunit.client.0.smithi070.stdout:4/106: creat d1/f21 x:0 0 0 2024-04-02T04:47:40.105 INFO:tasks.workunit.client.0.smithi070.stdout:9/147: stat d4/d10/d11/f2d 0 2024-04-02T04:47:40.107 INFO:tasks.workunit.client.1.smithi179.stdout:7/78: mknod d1/c19 0 2024-04-02T04:47:40.108 INFO:tasks.workunit.client.1.smithi179.stdout:1/182: mknod d3/d31/c40 0 2024-04-02T04:47:40.109 INFO:tasks.workunit.client.1.smithi179.stdout:1/183: stat d3/d5/de/l23 0 2024-04-02T04:47:40.112 INFO:tasks.workunit.client.1.smithi179.stdout:6/65: link d3/f1c d3/d12/f21 0 2024-04-02T04:47:40.113 INFO:tasks.workunit.client.0.smithi070.stdout:3/92: mknod d10/d17/c25 0 2024-04-02T04:47:40.113 INFO:tasks.workunit.client.0.smithi070.stdout:9/148: dwrite d4/d10/f18 [0,4194304] 0 2024-04-02T04:47:40.115 INFO:tasks.workunit.client.1.smithi179.stdout:8/119: symlink d3/d12/l2c 0 2024-04-02T04:47:40.122 INFO:tasks.workunit.client.0.smithi070.stdout:6/73: symlink d0/l1a 0 2024-04-02T04:47:40.122 INFO:tasks.workunit.client.1.smithi179.stdout:1/184: mknod d3/d5/d2f/d36/c41 0 2024-04-02T04:47:40.123 INFO:tasks.workunit.client.0.smithi070.stdout:5/132: truncate f4 3947937 0 2024-04-02T04:47:40.124 INFO:tasks.workunit.client.1.smithi179.stdout:6/66: unlink d3/fe 0 2024-04-02T04:47:40.125 INFO:tasks.workunit.client.1.smithi179.stdout:6/67: chown d3/d12/f21 3814793 1 2024-04-02T04:47:40.125 INFO:tasks.workunit.client.1.smithi179.stdout:6/68: readlink d3/l6 0 2024-04-02T04:47:40.125 INFO:tasks.workunit.client.1.smithi179.stdout:8/120: creat d3/d12/d21/d22/f2d x:0 0 0 2024-04-02T04:47:40.126 INFO:tasks.workunit.client.0.smithi070.stdout:7/119: mknod d7/d9/d11/c20 0 2024-04-02T04:47:40.127 INFO:tasks.workunit.client.1.smithi179.stdout:2/61: creat dc/d14/f15 x:0 0 0 2024-04-02T04:47:40.128 INFO:tasks.workunit.client.1.smithi179.stdout:6/69: dread d3/fa [0,4194304] 0 2024-04-02T04:47:40.129 INFO:tasks.workunit.client.0.smithi070.stdout:4/107: creat d1/d3/d9/f22 x:0 0 0 2024-04-02T04:47:40.130 INFO:tasks.workunit.client.0.smithi070.stdout:7/120: dread d7/fc [4194304,4194304] 0 2024-04-02T04:47:40.130 INFO:tasks.workunit.client.0.smithi070.stdout:7/121: readlink d7/l1d 0 2024-04-02T04:47:40.132 INFO:tasks.workunit.client.0.smithi070.stdout:3/93: unlink d10/d17/d19/c23 0 2024-04-02T04:47:40.132 INFO:tasks.workunit.client.0.smithi070.stdout:3/94: write fd [1963273,53969] 0 2024-04-02T04:47:40.133 INFO:tasks.workunit.client.1.smithi179.stdout:4/102: sync 2024-04-02T04:47:40.133 INFO:tasks.workunit.client.0.smithi070.stdout:9/149: mkdir d4/d13/d16/d19/d2f 0 2024-04-02T04:47:40.134 INFO:tasks.workunit.client.1.smithi179.stdout:4/103: chown d1/d3/f13 2102 1 2024-04-02T04:47:40.134 INFO:tasks.workunit.client.1.smithi179.stdout:1/185: mkdir d3/d31/d42 0 2024-04-02T04:47:40.135 INFO:tasks.workunit.client.1.smithi179.stdout:4/104: write d1/fb [429136,20217] 0 2024-04-02T04:47:40.135 INFO:tasks.workunit.client.0.smithi070.stdout:8/134: rename d2/d27 to d2/d8/d29 0 2024-04-02T04:47:40.136 INFO:tasks.workunit.client.1.smithi179.stdout:4/105: write d1/fb [645832,68360] 0 2024-04-02T04:47:40.137 INFO:tasks.workunit.client.0.smithi070.stdout:8/135: write d2/d8/d29/db/f21 [165686,104334] 0 2024-04-02T04:47:40.138 INFO:tasks.workunit.client.0.smithi070.stdout:6/74: creat d0/f1b x:0 0 0 2024-04-02T04:47:40.140 INFO:tasks.workunit.client.1.smithi179.stdout:8/121: symlink d3/l2e 0 2024-04-02T04:47:40.141 INFO:tasks.workunit.client.1.smithi179.stdout:8/122: chown d3/ca 125991050 1 2024-04-02T04:47:40.142 INFO:tasks.workunit.client.0.smithi070.stdout:6/75: dread d0/f15 [0,4194304] 0 2024-04-02T04:47:40.147 INFO:tasks.workunit.client.1.smithi179.stdout:2/62: dwrite dc/f12 [0,4194304] 0 2024-04-02T04:47:40.148 INFO:tasks.workunit.client.1.smithi179.stdout:2/63: truncate f9 8474904 0 2024-04-02T04:47:40.149 INFO:tasks.workunit.client.1.smithi179.stdout:6/70: creat d3/d12/f22 x:0 0 0 2024-04-02T04:47:40.149 INFO:tasks.workunit.client.0.smithi070.stdout:3/95: symlink d10/d17/d19/l26 0 2024-04-02T04:47:40.151 INFO:tasks.workunit.client.1.smithi179.stdout:1/186: symlink d3/d5/de/d33/l43 0 2024-04-02T04:47:40.152 INFO:tasks.workunit.client.1.smithi179.stdout:2/64: dread dc/fd [0,4194304] 0 2024-04-02T04:47:40.152 INFO:tasks.workunit.client.1.smithi179.stdout:6/71: dread d3/f1c [0,4194304] 0 2024-04-02T04:47:40.153 INFO:tasks.workunit.client.1.smithi179.stdout:6/72: fsync d3/f10 0 2024-04-02T04:47:40.154 INFO:tasks.workunit.client.0.smithi070.stdout:8/136: creat d2/d8/d29/f2a x:0 0 0 2024-04-02T04:47:40.159 INFO:tasks.workunit.client.0.smithi070.stdout:3/96: dwrite fe [0,4194304] 0 2024-04-02T04:47:40.159 INFO:tasks.workunit.client.0.smithi070.stdout:6/76: creat d0/f1c x:0 0 0 2024-04-02T04:47:40.161 INFO:tasks.workunit.client.0.smithi070.stdout:8/137: dread f1 [0,4194304] 0 2024-04-02T04:47:40.162 INFO:tasks.workunit.client.1.smithi179.stdout:4/106: creat d1/d3/d19/f21 x:0 0 0 2024-04-02T04:47:40.164 INFO:tasks.workunit.client.0.smithi070.stdout:8/138: write d2/d3/f17 [1291384,84262] 0 2024-04-02T04:47:40.164 INFO:tasks.workunit.client.1.smithi179.stdout:9/118: sync 2024-04-02T04:47:40.164 INFO:tasks.workunit.client.1.smithi179.stdout:5/54: sync 2024-04-02T04:47:40.166 INFO:tasks.workunit.client.0.smithi070.stdout:3/97: dread fd [0,4194304] 0 2024-04-02T04:47:40.169 INFO:tasks.workunit.client.1.smithi179.stdout:3/69: getdents d8 0 2024-04-02T04:47:40.170 INFO:tasks.workunit.client.0.smithi070.stdout:9/150: creat d4/d10/d12/d27/f30 x:0 0 0 2024-04-02T04:47:40.171 INFO:tasks.workunit.client.0.smithi070.stdout:4/108: rmdir d1/d3/d9 39 2024-04-02T04:47:40.173 INFO:tasks.workunit.client.0.smithi070.stdout:5/133: link d5/d9/d10/c15 d5/d17/d2b/c2d 0 2024-04-02T04:47:40.175 INFO:tasks.workunit.client.1.smithi179.stdout:4/107: dwrite d1/f4 [4194304,4194304] 0 2024-04-02T04:47:40.175 INFO:tasks.workunit.client.1.smithi179.stdout:2/65: rename dc/d14 to dc/d16 0 2024-04-02T04:47:40.177 INFO:tasks.workunit.client.0.smithi070.stdout:2/200: sync 2024-04-02T04:47:40.177 INFO:tasks.workunit.client.0.smithi070.stdout:0/271: sync 2024-04-02T04:47:40.177 INFO:tasks.workunit.client.0.smithi070.stdout:1/141: sync 2024-04-02T04:47:40.179 INFO:tasks.workunit.client.1.smithi179.stdout:2/66: write f6 [1344949,125138] 0 2024-04-02T04:47:40.179 INFO:tasks.workunit.client.0.smithi070.stdout:6/77: mknod d0/d3/c1d 0 2024-04-02T04:47:40.181 INFO:tasks.workunit.client.1.smithi179.stdout:4/108: write d1/d3/dc/fe [847316,952] 0 2024-04-02T04:47:40.182 INFO:tasks.workunit.client.1.smithi179.stdout:5/55: dwrite d8/fc [0,4194304] 0 2024-04-02T04:47:40.183 INFO:tasks.workunit.client.1.smithi179.stdout:1/187: rename d3/d5/fb to d3/d5/d2f/d36/f44 0 2024-04-02T04:47:40.183 INFO:tasks.workunit.client.1.smithi179.stdout:9/119: dwrite d0/d1/d12/fe [0,4194304] 0 2024-04-02T04:47:40.189 INFO:tasks.workunit.client.0.smithi070.stdout:1/142: dwrite d0/d7/d1a/f2e [0,4194304] 0 2024-04-02T04:47:40.189 INFO:tasks.workunit.client.0.smithi070.stdout:8/139: mknod d2/d8/d29/c2b 0 2024-04-02T04:47:40.190 INFO:tasks.workunit.client.0.smithi070.stdout:0/272: dread d4/d5/d13/d2a/f20 [0,4194304] 0 2024-04-02T04:47:40.190 INFO:tasks.workunit.client.1.smithi179.stdout:6/73: mkdir d3/d12/d23 0 2024-04-02T04:47:40.191 INFO:tasks.workunit.client.0.smithi070.stdout:0/273: read d4/d5/f14 [665750,60693] 0 2024-04-02T04:47:40.191 INFO:tasks.workunit.client.0.smithi070.stdout:1/143: truncate d0/d9/d1d/f2 1711147 0 2024-04-02T04:47:40.192 INFO:tasks.workunit.client.1.smithi179.stdout:7/79: dwrite d1/d9/fb [0,4194304] 0 2024-04-02T04:47:40.192 INFO:tasks.workunit.client.1.smithi179.stdout:2/67: dread f2 [0,4194304] 0 2024-04-02T04:47:40.192 INFO:tasks.workunit.client.1.smithi179.stdout:6/74: readlink d3/l19 0 2024-04-02T04:47:40.193 INFO:tasks.workunit.client.0.smithi070.stdout:3/98: fdatasync d10/f15 0 2024-04-02T04:47:40.193 INFO:tasks.workunit.client.1.smithi179.stdout:7/80: stat d1/ff 0 2024-04-02T04:47:40.194 INFO:tasks.workunit.client.1.smithi179.stdout:1/188: dread d3/f28 [0,4194304] 0 2024-04-02T04:47:40.196 INFO:tasks.workunit.client.1.smithi179.stdout:0/174: truncate d1/f2 7075560 0 2024-04-02T04:47:40.197 INFO:tasks.workunit.client.0.smithi070.stdout:7/122: creat d7/d9/f21 x:0 0 0 2024-04-02T04:47:40.199 INFO:tasks.workunit.client.1.smithi179.stdout:0/175: readlink d1/da/df/l1f 0 2024-04-02T04:47:40.200 INFO:tasks.workunit.client.0.smithi070.stdout:9/151: mknod d4/d10/d12/c31 0 2024-04-02T04:47:40.202 INFO:tasks.workunit.client.1.smithi179.stdout:8/123: unlink d3/d12/c17 0 2024-04-02T04:47:40.205 INFO:tasks.workunit.client.1.smithi179.stdout:3/70: mknod d8/c13 0 2024-04-02T04:47:40.205 INFO:tasks.workunit.client.1.smithi179.stdout:3/71: truncate f5 4864980 0 2024-04-02T04:47:40.213 INFO:tasks.workunit.client.0.smithi070.stdout:9/152: dread d4/f7 [0,4194304] 0 2024-04-02T04:47:40.215 INFO:tasks.workunit.client.0.smithi070.stdout:6/78: creat d0/d3/f1e x:0 0 0 2024-04-02T04:47:40.216 INFO:tasks.workunit.client.1.smithi179.stdout:5/56: creat d8/f10 x:0 0 0 2024-04-02T04:47:40.216 INFO:tasks.workunit.client.1.smithi179.stdout:5/57: chown d8 0 1 2024-04-02T04:47:40.218 INFO:tasks.workunit.client.1.smithi179.stdout:9/120: fsync d0/fd 0 2024-04-02T04:47:40.219 INFO:tasks.workunit.client.1.smithi179.stdout:9/121: chown d0/d1/d12/c1a 1590272 1 2024-04-02T04:47:40.220 INFO:tasks.workunit.client.0.smithi070.stdout:0/274: chown d4/l28 743 1 2024-04-02T04:47:40.221 INFO:tasks.workunit.client.0.smithi070.stdout:0/275: write d4/f1b [1025205,16581] 0 2024-04-02T04:47:40.222 INFO:tasks.workunit.client.1.smithi179.stdout:2/68: readlink dc/l13 0 2024-04-02T04:47:40.224 INFO:tasks.workunit.client.1.smithi179.stdout:2/69: write dc/f12 [5220841,51842] 0 2024-04-02T04:47:40.226 INFO:tasks.workunit.client.1.smithi179.stdout:7/81: rmdir d1/dd 39 2024-04-02T04:47:40.227 INFO:tasks.workunit.client.0.smithi070.stdout:6/79: dread d0/d3/fb [0,4194304] 0 2024-04-02T04:47:40.227 INFO:tasks.workunit.client.0.smithi070.stdout:6/80: chown d0/d3/fb 13452063 1 2024-04-02T04:47:40.228 INFO:tasks.workunit.client.0.smithi070.stdout:6/81: write d0/d3/f13 [402226,77564] 0 2024-04-02T04:47:40.228 INFO:tasks.workunit.client.0.smithi070.stdout:3/99: dread d10/d17/d19/f1a [0,4194304] 0 2024-04-02T04:47:40.228 INFO:tasks.workunit.client.0.smithi070.stdout:6/82: readlink d0/l12 0 2024-04-02T04:47:40.229 INFO:tasks.workunit.client.1.smithi179.stdout:1/189: symlink d3/d5/de/d33/l45 0 2024-04-02T04:47:40.231 INFO:tasks.workunit.client.0.smithi070.stdout:6/83: write d0/d3/f1e [825393,128719] 0 2024-04-02T04:47:40.234 INFO:tasks.workunit.client.1.smithi179.stdout:7/82: dwrite d1/d9/f12 [0,4194304] 0 2024-04-02T04:47:40.235 INFO:tasks.workunit.client.1.smithi179.stdout:0/176: dread d1/da/df/f20 [0,4194304] 0 2024-04-02T04:47:40.235 INFO:tasks.workunit.client.0.smithi070.stdout:3/100: dwrite fa [0,4194304] 0 2024-04-02T04:47:40.235 INFO:tasks.workunit.client.1.smithi179.stdout:0/177: readlink d1/l2e 0 2024-04-02T04:47:40.236 INFO:tasks.workunit.client.1.smithi179.stdout:8/124: rename d3/c6 to d3/d12/d21/c2f 0 2024-04-02T04:47:40.241 INFO:tasks.workunit.client.0.smithi070.stdout:3/101: dread fa [0,4194304] 0 2024-04-02T04:47:40.241 INFO:tasks.workunit.client.1.smithi179.stdout:3/72: rename d8/lc to d8/l14 0 2024-04-02T04:47:40.242 INFO:tasks.workunit.client.1.smithi179.stdout:3/73: stat d8/l12 0 2024-04-02T04:47:40.243 INFO:tasks.workunit.client.1.smithi179.stdout:3/74: dread - d8/f10 zero size 2024-04-02T04:47:40.244 INFO:tasks.workunit.client.0.smithi070.stdout:2/201: mkdir d2/d10/d21/d3d/d42 0 2024-04-02T04:47:40.248 INFO:tasks.workunit.client.1.smithi179.stdout:5/58: symlink d8/l11 0 2024-04-02T04:47:40.249 INFO:tasks.workunit.client.1.smithi179.stdout:4/109: symlink d1/d3/df/d1f/l22 0 2024-04-02T04:47:40.254 INFO:tasks.workunit.client.1.smithi179.stdout:6/75: getdents d3/d12/d23 0 2024-04-02T04:47:40.256 INFO:tasks.workunit.client.0.smithi070.stdout:0/276: creat d4/d5/d13/d2a/d29/f46 x:0 0 0 2024-04-02T04:47:40.257 INFO:tasks.workunit.client.1.smithi179.stdout:2/70: creat dc/f17 x:0 0 0 2024-04-02T04:47:40.258 INFO:tasks.workunit.client.0.smithi070.stdout:1/144: mknod d0/d7/c2f 0 2024-04-02T04:47:40.259 INFO:tasks.workunit.client.0.smithi070.stdout:2/202: dread d2/fb [0,4194304] 0 2024-04-02T04:47:40.261 INFO:tasks.workunit.client.0.smithi070.stdout:7/123: dwrite d7/d9/f1b [0,4194304] 0 2024-04-02T04:47:40.264 INFO:tasks.workunit.client.0.smithi070.stdout:0/277: dread d4/d5/d13/f32 [0,4194304] 0 2024-04-02T04:47:40.264 INFO:tasks.workunit.client.1.smithi179.stdout:7/83: rename d1 to d1/dd/d1a 22 2024-04-02T04:47:40.265 INFO:tasks.workunit.client.0.smithi070.stdout:0/278: stat d4/d5/d13/d2a/d29/f46 0 2024-04-02T04:47:40.266 INFO:tasks.workunit.client.1.smithi179.stdout:0/178: creat d1/da/df/f33 x:0 0 0 2024-04-02T04:47:40.267 INFO:tasks.workunit.client.1.smithi179.stdout:0/179: chown d1/da/f27 11046 1 2024-04-02T04:47:40.268 INFO:tasks.workunit.client.1.smithi179.stdout:0/180: dread - d1/da/f2d zero size 2024-04-02T04:47:40.269 INFO:tasks.workunit.client.1.smithi179.stdout:0/181: write d1/db/f2f [1582059,94639] 0 2024-04-02T04:47:40.270 INFO:tasks.workunit.client.1.smithi179.stdout:3/75: creat d8/f15 x:0 0 0 2024-04-02T04:47:40.270 INFO:tasks.workunit.client.1.smithi179.stdout:0/182: fsync d1/da/d28/f29 0 2024-04-02T04:47:40.271 INFO:tasks.workunit.client.1.smithi179.stdout:0/183: fdatasync d1/da/df/f11 0 2024-04-02T04:47:40.272 INFO:tasks.workunit.client.0.smithi070.stdout:8/140: truncate d2/d3/d6/f16 89520 0 2024-04-02T04:47:40.273 INFO:tasks.workunit.client.1.smithi179.stdout:5/59: truncate f1 159388 0 2024-04-02T04:47:40.274 INFO:tasks.workunit.client.1.smithi179.stdout:4/110: mknod d1/d3/dc/c23 0 2024-04-02T04:47:40.275 INFO:tasks.workunit.client.0.smithi070.stdout:3/102: creat d10/d17/f27 x:0 0 0 2024-04-02T04:47:40.277 INFO:tasks.workunit.client.1.smithi179.stdout:9/122: symlink d0/d19/d1c/l22 0 2024-04-02T04:47:40.278 INFO:tasks.workunit.client.1.smithi179.stdout:9/123: chown d0 0 1 2024-04-02T04:47:40.279 INFO:tasks.workunit.client.1.smithi179.stdout:1/190: write d3/d5/f25 [585365,4336] 0 2024-04-02T04:47:40.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:39 smithi070 ceph-mon[29212]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:40.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:39 smithi070 ceph-mon[29212]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:40.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:39 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2570037957' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:40.281 INFO:tasks.workunit.client.0.smithi070.stdout:4/109: truncate d1/d3/d9/f13 7929298 0 2024-04-02T04:47:40.281 INFO:tasks.workunit.client.1.smithi179.stdout:5/60: dwrite d8/fc [0,4194304] 0 2024-04-02T04:47:40.286 INFO:tasks.workunit.client.0.smithi070.stdout:2/203: mknod d2/d10/d21/d2b/c43 0 2024-04-02T04:47:40.287 INFO:tasks.workunit.client.0.smithi070.stdout:2/204: write d2/d10/f20 [2034438,78960] 0 2024-04-02T04:47:40.288 INFO:tasks.workunit.client.1.smithi179.stdout:4/111: dwrite d1/d3/dc/fe [0,4194304] 0 2024-04-02T04:47:40.289 INFO:tasks.workunit.client.0.smithi070.stdout:7/124: rmdir d7/dd/df 39 2024-04-02T04:47:40.292 INFO:tasks.workunit.client.1.smithi179.stdout:3/76: unlink d8/ff 0 2024-04-02T04:47:40.294 INFO:tasks.workunit.client.0.smithi070.stdout:6/84: creat d0/d3/d19/f1f x:0 0 0 2024-04-02T04:47:40.295 INFO:tasks.workunit.client.1.smithi179.stdout:9/124: dwrite d0/d1/d12/f20 [0,4194304] 0 2024-04-02T04:47:40.296 INFO:tasks.workunit.client.0.smithi070.stdout:7/125: dwrite d7/f1e [0,4194304] 0 2024-04-02T04:47:40.296 INFO:tasks.workunit.client.0.smithi070.stdout:0/279: symlink d4/d3c/l47 0 2024-04-02T04:47:40.297 INFO:tasks.workunit.client.0.smithi070.stdout:0/280: chown d4 7788358 1 2024-04-02T04:47:40.301 INFO:tasks.workunit.client.0.smithi070.stdout:8/141: write d2/d3/f7 [301748,122881] 0 2024-04-02T04:47:40.303 INFO:tasks.workunit.client.0.smithi070.stdout:1/145: write d0/d7/f8 [5207869,86306] 0 2024-04-02T04:47:40.304 INFO:tasks.workunit.client.0.smithi070.stdout:1/146: chown d0/d9/lf 17 1 2024-04-02T04:47:40.307 INFO:tasks.workunit.client.1.smithi179.stdout:8/125: truncate d3/d12/f19 2603768 0 2024-04-02T04:47:40.308 INFO:tasks.workunit.client.0.smithi070.stdout:5/134: getdents d5 0 2024-04-02T04:47:40.309 INFO:tasks.workunit.client.0.smithi070.stdout:3/103: unlink d10/l21 0 2024-04-02T04:47:40.309 INFO:tasks.workunit.client.0.smithi070.stdout:3/104: chown f0 666723902 1 2024-04-02T04:47:40.313 INFO:tasks.workunit.client.1.smithi179.stdout:7/84: symlink d1/dd/d16/l1b 0 2024-04-02T04:47:40.314 INFO:tasks.workunit.client.1.smithi179.stdout:7/85: truncate d1/f15 767113 0 2024-04-02T04:47:40.314 INFO:tasks.workunit.client.1.smithi179.stdout:5/61: creat d8/f12 x:0 0 0 2024-04-02T04:47:40.315 INFO:tasks.workunit.client.1.smithi179.stdout:7/86: chown d1/dd/d16 478083 1 2024-04-02T04:47:40.315 INFO:tasks.workunit.client.1.smithi179.stdout:5/62: read - d8/fb zero size 2024-04-02T04:47:40.315 INFO:tasks.workunit.client.1.smithi179.stdout:7/87: readlink l0 0 2024-04-02T04:47:40.315 INFO:tasks.workunit.client.1.smithi179.stdout:5/63: chown d8/fb 21839 1 2024-04-02T04:47:40.317 INFO:tasks.workunit.client.0.smithi070.stdout:2/205: symlink d2/d10/d21/d3d/l44 0 2024-04-02T04:47:40.319 INFO:tasks.workunit.client.1.smithi179.stdout:3/77: symlink d8/l16 0 2024-04-02T04:47:40.320 INFO:tasks.workunit.client.1.smithi179.stdout:9/125: mknod d0/c23 0 2024-04-02T04:47:40.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:39 smithi179 ceph-mon[36603]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:40.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:39 smithi179 ceph-mon[36603]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:40.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:39 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2570037957' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:40.324 INFO:tasks.workunit.client.0.smithi070.stdout:1/147: rmdir d0/d9/d1d 39 2024-04-02T04:47:40.326 INFO:tasks.workunit.client.1.smithi179.stdout:7/88: creat d1/dd/f1c x:0 0 0 2024-04-02T04:47:40.327 INFO:tasks.workunit.client.0.smithi070.stdout:5/135: mkdir d5/d17/d27/d2e 0 2024-04-02T04:47:40.327 INFO:tasks.workunit.client.1.smithi179.stdout:4/112: symlink d1/d15/l24 0 2024-04-02T04:47:40.330 INFO:tasks.workunit.client.1.smithi179.stdout:4/113: truncate d1/d3/df/d10/f11 951567 0 2024-04-02T04:47:40.331 INFO:tasks.workunit.client.1.smithi179.stdout:3/78: mknod d8/c17 0 2024-04-02T04:47:40.333 INFO:tasks.workunit.client.1.smithi179.stdout:9/126: truncate d0/d1/d10/f13 2896314 0 2024-04-02T04:47:40.333 INFO:tasks.workunit.client.1.smithi179.stdout:7/89: dwrite d1/d9/fb [4194304,4194304] 0 2024-04-02T04:47:40.336 INFO:tasks.workunit.client.1.smithi179.stdout:2/71: getdents dc 0 2024-04-02T04:47:40.336 INFO:tasks.workunit.client.1.smithi179.stdout:4/114: dread d1/d3/df/d10/f11 [0,4194304] 0 2024-04-02T04:47:40.339 INFO:tasks.workunit.client.1.smithi179.stdout:2/72: dread - dc/d16/f15 zero size 2024-04-02T04:47:40.341 INFO:tasks.workunit.client.1.smithi179.stdout:7/90: write d1/d9/f11 [1726455,82349] 0 2024-04-02T04:47:40.342 INFO:tasks.workunit.client.1.smithi179.stdout:2/73: readlink dc/l13 0 2024-04-02T04:47:40.343 INFO:tasks.workunit.client.1.smithi179.stdout:1/191: dwrite d3/d5/f21 [4194304,4194304] 0 2024-04-02T04:47:40.344 INFO:tasks.workunit.client.1.smithi179.stdout:4/115: mknod d1/d15/c25 0 2024-04-02T04:47:40.347 INFO:tasks.workunit.client.1.smithi179.stdout:7/91: write d1/dd/f17 [850728,42705] 0 2024-04-02T04:47:40.347 INFO:tasks.workunit.client.1.smithi179.stdout:3/79: dwrite d8/f11 [0,4194304] 0 2024-04-02T04:47:40.348 INFO:tasks.workunit.client.1.smithi179.stdout:1/192: read d3/d5/f13 [3736809,63454] 0 2024-04-02T04:47:40.349 INFO:tasks.workunit.client.1.smithi179.stdout:1/193: stat d3/f2d 0 2024-04-02T04:47:40.350 INFO:tasks.workunit.client.1.smithi179.stdout:2/74: mknod dc/c18 0 2024-04-02T04:47:40.356 INFO:tasks.workunit.client.1.smithi179.stdout:0/184: dwrite d1/f2 [4194304,4194304] 0 2024-04-02T04:47:40.357 INFO:tasks.workunit.client.1.smithi179.stdout:3/80: symlink d8/l18 0 2024-04-02T04:47:40.357 INFO:tasks.workunit.client.1.smithi179.stdout:4/116: dread d1/d3/dc/fd [0,4194304] 0 2024-04-02T04:47:40.358 INFO:tasks.workunit.client.1.smithi179.stdout:3/81: rename d8 to d8/d19 22 2024-04-02T04:47:40.358 INFO:tasks.workunit.client.1.smithi179.stdout:7/92: truncate d1/fc 4202323 0 2024-04-02T04:47:40.359 INFO:tasks.workunit.client.1.smithi179.stdout:3/82: dread - d8/f10 zero size 2024-04-02T04:47:40.360 INFO:tasks.workunit.client.1.smithi179.stdout:2/75: mknod dc/c19 0 2024-04-02T04:47:40.363 INFO:tasks.workunit.client.1.smithi179.stdout:7/93: unlink d1/c5 0 2024-04-02T04:47:40.367 INFO:tasks.workunit.client.1.smithi179.stdout:3/83: creat d8/f1a x:0 0 0 2024-04-02T04:47:40.373 INFO:tasks.workunit.client.1.smithi179.stdout:2/76: dwrite dc/d16/f15 [0,4194304] 0 2024-04-02T04:47:40.373 INFO:tasks.workunit.client.1.smithi179.stdout:9/127: truncate d0/d1/d10/f18 365172 0 2024-04-02T04:47:40.374 INFO:tasks.workunit.client.1.smithi179.stdout:0/185: dwrite d1/f14 [0,4194304] 0 2024-04-02T04:47:40.376 INFO:tasks.workunit.client.1.smithi179.stdout:4/117: write d1/d3/df/d10/f11 [1213026,19853] 0 2024-04-02T04:47:40.379 INFO:tasks.workunit.client.1.smithi179.stdout:0/186: mkdir d1/da/d28/d34 0 2024-04-02T04:47:40.380 INFO:tasks.workunit.client.1.smithi179.stdout:3/84: dread f5 [0,4194304] 0 2024-04-02T04:47:40.383 INFO:tasks.workunit.client.1.smithi179.stdout:3/85: chown d8/la 9 1 2024-04-02T04:47:40.383 INFO:tasks.workunit.client.1.smithi179.stdout:7/94: getdents d1 0 2024-04-02T04:47:40.385 INFO:tasks.workunit.client.1.smithi179.stdout:3/86: truncate d8/f1a 901000 0 2024-04-02T04:47:40.389 INFO:tasks.workunit.client.1.smithi179.stdout:4/118: dwrite d1/d3/df/d10/f11 [0,4194304] 0 2024-04-02T04:47:40.389 INFO:tasks.workunit.client.1.smithi179.stdout:0/187: dread d1/f23 [0,4194304] 0 2024-04-02T04:47:40.391 INFO:tasks.workunit.client.1.smithi179.stdout:0/188: readlink d1/db/l31 0 2024-04-02T04:47:40.396 INFO:tasks.workunit.client.1.smithi179.stdout:4/119: mkdir d1/d3/dc/d26 0 2024-04-02T04:47:40.396 INFO:tasks.workunit.client.1.smithi179.stdout:0/189: write d1/da/df/f11 [2115905,126167] 0 2024-04-02T04:47:40.398 INFO:tasks.workunit.client.1.smithi179.stdout:7/95: dwrite d1/dd/f17 [0,4194304] 0 2024-04-02T04:47:40.403 INFO:tasks.workunit.client.1.smithi179.stdout:9/128: dwrite d0/d1/d10/f18 [0,4194304] 0 2024-04-02T04:47:40.404 INFO:tasks.workunit.client.1.smithi179.stdout:9/129: chown d0 861707 1 2024-04-02T04:47:40.404 INFO:tasks.workunit.client.1.smithi179.stdout:3/87: dread f5 [0,4194304] 0 2024-04-02T04:47:40.405 INFO:tasks.workunit.client.1.smithi179.stdout:4/120: rename d1/d3/df/d1f to d1/d3/dc/d27 0 2024-04-02T04:47:40.406 INFO:tasks.workunit.client.1.smithi179.stdout:3/88: read f5 [2613058,100467] 0 2024-04-02T04:47:40.406 INFO:tasks.workunit.client.1.smithi179.stdout:9/130: write d0/d1/d12/f20 [3745601,111939] 0 2024-04-02T04:47:40.406 INFO:tasks.workunit.client.1.smithi179.stdout:9/131: stat d0/c23 0 2024-04-02T04:47:40.407 INFO:tasks.workunit.client.1.smithi179.stdout:0/190: creat d1/da/d28/d34/f35 x:0 0 0 2024-04-02T04:47:40.412 INFO:tasks.workunit.client.1.smithi179.stdout:4/121: link d1/f4 d1/d3/f28 0 2024-04-02T04:47:40.418 INFO:tasks.workunit.client.1.smithi179.stdout:4/122: creat d1/d3/dc/d26/f29 x:0 0 0 2024-04-02T04:47:40.419 INFO:tasks.workunit.client.0.smithi070.stdout:3/105: rename d10/d17/d19/l1b to d10/d17/l28 0 2024-04-02T04:47:40.419 INFO:tasks.workunit.client.1.smithi179.stdout:9/132: getdents d0 0 2024-04-02T04:47:40.420 INFO:tasks.workunit.client.0.smithi070.stdout:9/153: getdents d4/d10 0 2024-04-02T04:47:40.420 INFO:tasks.workunit.client.0.smithi070.stdout:5/136: dread - d5/d9/d10/f11 zero size 2024-04-02T04:47:40.422 INFO:tasks.workunit.client.1.smithi179.stdout:3/89: dwrite f5 [4194304,4194304] 0 2024-04-02T04:47:40.422 INFO:tasks.workunit.client.0.smithi070.stdout:9/154: write d4/d13/f26 [499222,97055] 0 2024-04-02T04:47:40.422 INFO:tasks.workunit.client.0.smithi070.stdout:2/206: chown d2/c16 108 1 2024-04-02T04:47:40.422 INFO:tasks.workunit.client.0.smithi070.stdout:7/126: mknod d7/d1c/c22 0 2024-04-02T04:47:40.424 INFO:tasks.workunit.client.1.smithi179.stdout:0/191: truncate d1/f2 3181412 0 2024-04-02T04:47:40.424 INFO:tasks.workunit.client.1.smithi179.stdout:4/123: dread d1/d3/fa [0,4194304] 0 2024-04-02T04:47:40.424 INFO:tasks.workunit.client.1.smithi179.stdout:7/96: fdatasync d1/dd/f17 0 2024-04-02T04:47:40.425 INFO:tasks.workunit.client.0.smithi070.stdout:9/155: fsync d4/d13/d16/f1d 0 2024-04-02T04:47:40.425 INFO:tasks.workunit.client.0.smithi070.stdout:1/148: write d0/f1c [4565739,11243] 0 2024-04-02T04:47:40.425 INFO:tasks.workunit.client.1.smithi179.stdout:7/97: chown d1/d9 13163008 1 2024-04-02T04:47:40.426 INFO:tasks.workunit.client.0.smithi070.stdout:3/106: creat d10/f29 x:0 0 0 2024-04-02T04:47:40.429 INFO:tasks.workunit.client.1.smithi179.stdout:0/192: write d1/f22 [345770,116691] 0 2024-04-02T04:47:40.430 INFO:tasks.workunit.client.1.smithi179.stdout:9/133: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:47:40.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.428+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 --> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f42c4000bf0 con 0x7f42c006d090 2024-04-02T04:47:40.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.432+0000 7f42d57fa700 1 -- 172.21.15.70:0/760207020 <== mgr.24359 v2:172.21.15.179:6828/1862820759 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f42c4000bf0 con 0x7f42c006d090 2024-04-02T04:47:40.435 INFO:tasks.workunit.client.0.smithi070.stdout:4/110: getdents d1 0 2024-04-02T04:47:40.435 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:47:40.435 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:47:40.435 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:47:40.435 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:47:40.436 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-04-02T04:47:40.436 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "1/2 daemons upgraded", 2024-04-02T04:47:40.436 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-04-02T04:47:40.436 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:47:40.436 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:47:40.436 INFO:tasks.workunit.client.0.smithi070.stdout:7/127: symlink d7/d9/d11/l23 0 2024-04-02T04:47:40.437 INFO:tasks.workunit.client.1.smithi179.stdout:9/134: symlink d0/l24 0 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f42c006d090 msgr2=0x7f42c006f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f42c006d090 0x7f42c006f550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006010 tx=0x7f42c800b560 comp rx=0 tx=0).stop 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42d8100e10 msgr2=0x7f42d81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42d8100e10 0x7f42d81021f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00d8d0 tx=0x7f42cc00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 shutdown_connections 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f42c006d090 0x7f42c006f550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d80ffc10 0x7f42d8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 --2- 172.21.15.70:0/760207020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42d8100e10 0x7f42d81021f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 >> 172.21.15.70:0/760207020 conn(0x7f42d80fb190 msgr2=0x7f42d8068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:40.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.435+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 shutdown_connections 2024-04-02T04:47:40.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:47:40.436+0000 7f42de265700 1 -- 172.21.15.70:0/760207020 wait complete. 2024-04-02T04:47:40.439 INFO:tasks.workunit.client.0.smithi070.stdout:9/156: mkdir d4/d10/d12/d27/d32 0 2024-04-02T04:47:40.440 INFO:tasks.workunit.client.0.smithi070.stdout:9/157: write d4/f9 [8109397,1391] 0 2024-04-02T04:47:40.440 INFO:tasks.workunit.client.1.smithi179.stdout:0/193: rmdir d1/da/df 39 2024-04-02T04:47:40.442 INFO:tasks.workunit.client.0.smithi070.stdout:6/85: getdents d0 0 2024-04-02T04:47:40.443 INFO:tasks.workunit.client.0.smithi070.stdout:6/86: chown d0/f1b 1 1 2024-04-02T04:47:40.443 INFO:tasks.workunit.client.0.smithi070.stdout:6/87: fsync d0/f1c 0 2024-04-02T04:47:40.444 INFO:tasks.workunit.client.1.smithi179.stdout:3/90: truncate d8/f11 3492794 0 2024-04-02T04:47:40.445 INFO:tasks.workunit.client.0.smithi070.stdout:3/107: creat d10/d17/f2a x:0 0 0 2024-04-02T04:47:40.447 INFO:tasks.workunit.client.1.smithi179.stdout:7/98: dwrite d1/fc [0,4194304] 0 2024-04-02T04:47:40.449 INFO:tasks.workunit.client.1.smithi179.stdout:7/99: chown d1/dd/f14 8 1 2024-04-02T04:47:40.451 INFO:tasks.workunit.client.0.smithi070.stdout:0/281: link d4/d5/cc d4/d5/c48 0 2024-04-02T04:47:40.453 INFO:tasks.workunit.client.1.smithi179.stdout:3/91: dwrite d8/f10 [0,4194304] 0 2024-04-02T04:47:40.453 INFO:tasks.workunit.client.0.smithi070.stdout:2/207: rename d2/d10/d19/d30/c36 to d2/d10/c45 0 2024-04-02T04:47:40.453 INFO:tasks.workunit.client.0.smithi070.stdout:2/208: stat d2/c17 0 2024-04-02T04:47:40.457 INFO:tasks.workunit.client.1.smithi179.stdout:9/135: fdatasync d0/d1/d10/f13 0 2024-04-02T04:47:40.459 INFO:tasks.workunit.client.1.smithi179.stdout:0/194: unlink d1/f6 0 2024-04-02T04:47:40.460 INFO:tasks.workunit.client.1.smithi179.stdout:4/124: truncate d1/d3/df/d10/f11 515236 0 2024-04-02T04:47:40.463 INFO:tasks.workunit.client.1.smithi179.stdout:7/100: creat d1/d9/f1d x:0 0 0 2024-04-02T04:47:40.464 INFO:tasks.workunit.client.1.smithi179.stdout:3/92: rmdir d8 39 2024-04-02T04:47:40.469 INFO:tasks.workunit.client.1.smithi179.stdout:9/136: creat d0/d1/d12/f25 x:0 0 0 2024-04-02T04:47:40.473 INFO:tasks.workunit.client.1.smithi179.stdout:4/125: fdatasync d1/f4 0 2024-04-02T04:47:40.474 INFO:tasks.workunit.client.1.smithi179.stdout:7/101: symlink d1/dd/d16/l1e 0 2024-04-02T04:47:40.475 INFO:tasks.workunit.client.1.smithi179.stdout:9/137: dwrite d0/d1/d10/f18 [0,4194304] 0 2024-04-02T04:47:40.478 INFO:tasks.workunit.client.1.smithi179.stdout:4/126: mknod d1/c2a 0 2024-04-02T04:47:40.479 INFO:tasks.workunit.client.1.smithi179.stdout:7/102: dread d1/d9/fb [0,4194304] 0 2024-04-02T04:47:40.482 INFO:tasks.workunit.client.1.smithi179.stdout:3/93: unlink d8/f15 0 2024-04-02T04:47:40.483 INFO:tasks.workunit.client.1.smithi179.stdout:0/195: creat d1/f36 x:0 0 0 2024-04-02T04:47:40.484 INFO:tasks.workunit.client.1.smithi179.stdout:0/196: chown d1/db/f2c 2 1 2024-04-02T04:47:40.484 INFO:tasks.workunit.client.1.smithi179.stdout:0/197: chown d1/f23 7938 1 2024-04-02T04:47:40.485 INFO:tasks.workunit.client.1.smithi179.stdout:0/198: readlink d1/da/l19 0 2024-04-02T04:47:40.487 INFO:tasks.workunit.client.1.smithi179.stdout:4/127: rmdir d1/d3 39 2024-04-02T04:47:40.488 INFO:tasks.workunit.client.1.smithi179.stdout:7/103: symlink d1/l1f 0 2024-04-02T04:47:40.490 INFO:tasks.workunit.client.1.smithi179.stdout:0/199: mkdir d1/da/d28/d37 0 2024-04-02T04:47:40.491 INFO:tasks.workunit.client.1.smithi179.stdout:0/200: readlink d1/da/l19 0 2024-04-02T04:47:40.491 INFO:tasks.workunit.client.1.smithi179.stdout:4/128: fdatasync d1/d3/d19/f21 0 2024-04-02T04:47:40.492 INFO:tasks.workunit.client.1.smithi179.stdout:3/94: fsync d8/f11 0 2024-04-02T04:47:40.492 INFO:tasks.workunit.client.1.smithi179.stdout:4/129: truncate d1/fb 773297 0 2024-04-02T04:47:40.493 INFO:tasks.workunit.client.1.smithi179.stdout:9/138: rename d0/fd to d0/d1/f26 0 2024-04-02T04:47:40.493 INFO:tasks.workunit.client.1.smithi179.stdout:3/95: truncate d8/f10 4488068 0 2024-04-02T04:47:40.494 INFO:tasks.workunit.client.1.smithi179.stdout:9/139: stat d0/d1/d12/f20 0 2024-04-02T04:47:40.496 INFO:tasks.workunit.client.1.smithi179.stdout:9/140: mkdir d0/d19/d27 0 2024-04-02T04:47:40.498 INFO:tasks.workunit.client.1.smithi179.stdout:0/201: rename d1/l18 to d1/da/d28/d34/l38 0 2024-04-02T04:47:40.499 INFO:tasks.workunit.client.1.smithi179.stdout:4/130: creat d1/f2b x:0 0 0 2024-04-02T04:47:40.499 INFO:tasks.workunit.client.1.smithi179.stdout:0/202: write d1/da/f1d [735411,71999] 0 2024-04-02T04:47:40.500 INFO:tasks.workunit.client.1.smithi179.stdout:0/203: fdatasync d1/da/d28/f29 0 2024-04-02T04:47:40.504 INFO:tasks.workunit.client.1.smithi179.stdout:0/204: dread - d1/da/df/f21 zero size 2024-04-02T04:47:40.505 INFO:tasks.workunit.client.1.smithi179.stdout:7/104: truncate d1/d9/f12 2972877 0 2024-04-02T04:47:40.508 INFO:tasks.workunit.client.1.smithi179.stdout:0/205: mkdir d1/da/d28/d34/d39 0 2024-04-02T04:47:40.509 INFO:tasks.workunit.client.1.smithi179.stdout:4/131: dwrite d1/d3/dc/f18 [0,4194304] 0 2024-04-02T04:47:40.510 INFO:tasks.workunit.client.1.smithi179.stdout:9/141: dwrite d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:40.510 INFO:tasks.workunit.client.1.smithi179.stdout:7/105: creat d1/dd/f20 x:0 0 0 2024-04-02T04:47:40.513 INFO:tasks.workunit.client.1.smithi179.stdout:9/142: write d0/d1/d10/f18 [1380187,72971] 0 2024-04-02T04:47:40.514 INFO:tasks.workunit.client.1.smithi179.stdout:4/132: write d1/d3/f7 [2792164,8445] 0 2024-04-02T04:47:40.517 INFO:tasks.workunit.client.0.smithi070.stdout:5/137: truncate d5/f1d 136995 0 2024-04-02T04:47:40.518 INFO:tasks.workunit.client.1.smithi179.stdout:7/106: rename d1/dd/d16/l1b to d1/dd/l21 0 2024-04-02T04:47:40.518 INFO:tasks.workunit.client.1.smithi179.stdout:9/143: write d0/d1/d12/f20 [1496532,62374] 0 2024-04-02T04:47:40.518 INFO:tasks.workunit.client.0.smithi070.stdout:4/111: dread d1/d3/d9/f13 [0,4194304] 0 2024-04-02T04:47:40.519 INFO:tasks.workunit.client.1.smithi179.stdout:4/133: dread d1/d3/f13 [0,4194304] 0 2024-04-02T04:47:40.520 INFO:tasks.workunit.client.0.smithi070.stdout:1/149: fsync d0/d9/d1d/d22/f25 0 2024-04-02T04:47:40.521 INFO:tasks.workunit.client.1.smithi179.stdout:0/206: dwrite d1/da/df/f33 [0,4194304] 0 2024-04-02T04:47:40.523 INFO:tasks.workunit.client.0.smithi070.stdout:6/88: mkdir d0/d20 0 2024-04-02T04:47:40.526 INFO:tasks.workunit.client.0.smithi070.stdout:2/209: symlink d2/d10/d21/l46 0 2024-04-02T04:47:40.528 INFO:tasks.workunit.client.1.smithi179.stdout:9/144: dwrite d0/d1/d10/f18 [4194304,4194304] 0 2024-04-02T04:47:40.529 INFO:tasks.workunit.client.0.smithi070.stdout:7/128: symlink d7/d9/l24 0 2024-04-02T04:47:40.532 INFO:tasks.workunit.client.1.smithi179.stdout:4/134: creat d1/d3/d19/f2c x:0 0 0 2024-04-02T04:47:40.533 INFO:tasks.workunit.client.0.smithi070.stdout:4/112: unlink d1/f18 0 2024-04-02T04:47:40.533 INFO:tasks.workunit.client.0.smithi070.stdout:7/129: chown d7/c15 2 1 2024-04-02T04:47:40.535 INFO:tasks.workunit.client.0.smithi070.stdout:0/282: creat d4/d5/d13/d43/f49 x:0 0 0 2024-04-02T04:47:40.536 INFO:tasks.workunit.client.1.smithi179.stdout:0/207: dread d1/db/fc [0,4194304] 0 2024-04-02T04:47:40.537 INFO:tasks.workunit.client.1.smithi179.stdout:9/145: symlink d0/d1/d10/l28 0 2024-04-02T04:47:40.537 INFO:tasks.workunit.client.1.smithi179.stdout:0/208: chown d1/db/f2c 414 1 2024-04-02T04:47:40.541 INFO:tasks.workunit.client.1.smithi179.stdout:4/135: truncate d1/d3/f28 7100164 0 2024-04-02T04:47:40.541 INFO:tasks.workunit.client.1.smithi179.stdout:4/136: dread - d1/d3/d19/f21 zero size 2024-04-02T04:47:40.545 INFO:tasks.workunit.client.0.smithi070.stdout:5/138: dwrite d5/d17/d27/f28 [0,4194304] 0 2024-04-02T04:47:40.549 INFO:tasks.workunit.client.0.smithi070.stdout:3/108: dwrite d10/f22 [0,4194304] 0 2024-04-02T04:47:40.552 INFO:tasks.workunit.client.0.smithi070.stdout:4/113: rmdir d1/d3 39 2024-04-02T04:47:40.552 INFO:tasks.workunit.client.0.smithi070.stdout:1/150: dwrite d0/d9/d1d/d22/f2d [0,4194304] 0 2024-04-02T04:47:40.553 INFO:tasks.workunit.client.1.smithi179.stdout:0/209: truncate d1/f23 678923 0 2024-04-02T04:47:40.554 INFO:tasks.workunit.client.1.smithi179.stdout:0/210: readlink d1/l10 0 2024-04-02T04:47:40.555 INFO:tasks.workunit.client.0.smithi070.stdout:5/139: write d5/fa [3537036,113466] 0 2024-04-02T04:47:40.557 INFO:tasks.workunit.client.0.smithi070.stdout:2/210: dwrite d2/d24/f3f [0,4194304] 0 2024-04-02T04:47:40.557 INFO:tasks.workunit.client.0.smithi070.stdout:6/89: dwrite d0/d3/fb [4194304,4194304] 0 2024-04-02T04:47:40.558 INFO:tasks.workunit.client.1.smithi179.stdout:4/137: creat d1/d3/df/d10/f2d x:0 0 0 2024-04-02T04:47:40.560 INFO:tasks.workunit.client.0.smithi070.stdout:9/158: getdents d4/d10/d12 0 2024-04-02T04:47:40.562 INFO:tasks.workunit.client.1.smithi179.stdout:9/146: mkdir d0/d1/d11/d29 0 2024-04-02T04:47:40.565 INFO:tasks.workunit.client.1.smithi179.stdout:0/211: creat d1/da/d28/f3a x:0 0 0 2024-04-02T04:47:40.568 INFO:tasks.workunit.client.1.smithi179.stdout:4/138: creat d1/d3/dc/d27/f2e x:0 0 0 2024-04-02T04:47:40.568 INFO:tasks.workunit.client.0.smithi070.stdout:6/90: dwrite d0/d3/f1e [0,4194304] 0 2024-04-02T04:47:40.569 INFO:tasks.workunit.client.0.smithi070.stdout:6/91: dread - d0/d3/d19/f1f zero size 2024-04-02T04:47:40.571 INFO:tasks.workunit.client.0.smithi070.stdout:3/109: creat d10/d17/f2b x:0 0 0 2024-04-02T04:47:40.572 INFO:tasks.workunit.client.0.smithi070.stdout:3/110: dread - d10/d17/f2b zero size 2024-04-02T04:47:40.572 INFO:tasks.workunit.client.0.smithi070.stdout:3/111: dread - fb zero size 2024-04-02T04:47:40.573 INFO:tasks.workunit.client.1.smithi179.stdout:4/139: mkdir d1/d3/df/d10/d2f 0 2024-04-02T04:47:40.573 INFO:tasks.workunit.client.0.smithi070.stdout:3/112: readlink d10/d17/d19/l26 0 2024-04-02T04:47:40.577 INFO:tasks.workunit.client.1.smithi179.stdout:4/140: mknod d1/c30 0 2024-04-02T04:47:40.578 INFO:tasks.workunit.client.1.smithi179.stdout:4/141: write d1/d3/d19/f21 [944778,129801] 0 2024-04-02T04:47:40.579 INFO:tasks.workunit.client.0.smithi070.stdout:4/114: read d1/d3/d9/f1a [102542,103998] 0 2024-04-02T04:47:40.579 INFO:tasks.workunit.client.1.smithi179.stdout:4/142: write d1/d3/df/d10/f2d [486406,76345] 0 2024-04-02T04:47:40.580 INFO:tasks.workunit.client.1.smithi179.stdout:4/143: dread - d1/d3/dc/f1c zero size 2024-04-02T04:47:40.581 INFO:tasks.workunit.client.0.smithi070.stdout:9/159: creat d4/d10/d12/d27/d32/f33 x:0 0 0 2024-04-02T04:47:40.583 INFO:tasks.workunit.client.1.smithi179.stdout:4/144: creat d1/d15/f31 x:0 0 0 2024-04-02T04:47:40.583 INFO:tasks.workunit.client.0.smithi070.stdout:1/151: creat d0/f30 x:0 0 0 2024-04-02T04:47:40.584 INFO:tasks.workunit.client.0.smithi070.stdout:1/152: write d0/f1c [4157546,6675] 0 2024-04-02T04:47:40.585 INFO:tasks.workunit.client.0.smithi070.stdout:2/211: rmdir d2/d10/d19/d30 0 2024-04-02T04:47:40.587 INFO:tasks.workunit.client.0.smithi070.stdout:6/92: link d0/d3/l18 d0/l21 0 2024-04-02T04:47:40.587 INFO:tasks.workunit.client.0.smithi070.stdout:6/93: readlink d0/l1a 0 2024-04-02T04:47:40.587 INFO:tasks.workunit.client.1.smithi179.stdout:4/145: dread d1/d3/dc/fd [0,4194304] 0 2024-04-02T04:47:40.587 INFO:tasks.workunit.client.1.smithi179.stdout:4/146: dread - d1/d15/f31 zero size 2024-04-02T04:47:40.588 INFO:tasks.workunit.client.0.smithi070.stdout:1/153: chown d0/d9/lf 9723 1 2024-04-02T04:47:40.589 INFO:tasks.workunit.client.1.smithi179.stdout:4/147: symlink d1/d15/l32 0 2024-04-02T04:47:40.589 INFO:tasks.workunit.client.0.smithi070.stdout:9/160: mkdir d4/d34 0 2024-04-02T04:47:40.589 INFO:tasks.workunit.client.0.smithi070.stdout:1/154: write d0/d7/f8 [5485098,88207] 0 2024-04-02T04:47:40.590 INFO:tasks.workunit.client.0.smithi070.stdout:2/212: creat d2/d10/d21/d3d/f47 x:0 0 0 2024-04-02T04:47:40.592 INFO:tasks.workunit.client.0.smithi070.stdout:1/155: read d0/d7/d1a/f24 [136797,15736] 0 2024-04-02T04:47:40.592 INFO:tasks.workunit.client.0.smithi070.stdout:6/94: rmdir d0/d3/d19 39 2024-04-02T04:47:40.592 INFO:tasks.workunit.client.0.smithi070.stdout:2/213: write d2/d10/f20 [398960,58895] 0 2024-04-02T04:47:40.594 INFO:tasks.workunit.client.0.smithi070.stdout:6/95: write d0/d3/f13 [1015854,104820] 0 2024-04-02T04:47:40.595 INFO:tasks.workunit.client.0.smithi070.stdout:9/161: dread d4/d10/f18 [0,4194304] 0 2024-04-02T04:47:40.595 INFO:tasks.workunit.client.0.smithi070.stdout:2/214: truncate d2/d10/d21/f33 254541 0 2024-04-02T04:47:40.596 INFO:tasks.workunit.client.0.smithi070.stdout:9/162: chown d4/d34 0 1 2024-04-02T04:47:40.597 INFO:tasks.workunit.client.1.smithi179.stdout:4/148: getdents d1/d3/dc/d26 0 2024-04-02T04:47:40.599 INFO:tasks.workunit.client.0.smithi070.stdout:2/215: mkdir d2/d12/d48 0 2024-04-02T04:47:40.599 INFO:tasks.workunit.client.0.smithi070.stdout:2/216: chown d2/d12/f13 566 1 2024-04-02T04:47:40.600 INFO:tasks.workunit.client.0.smithi070.stdout:9/163: creat d4/d13/d16/f35 x:0 0 0 2024-04-02T04:47:40.601 INFO:tasks.workunit.client.0.smithi070.stdout:1/156: rename d0/d9/d1d/f2 to d0/d9/f31 0 2024-04-02T04:47:40.603 INFO:tasks.workunit.client.0.smithi070.stdout:6/96: rename d0/d3 to d0/d3/d22 22 2024-04-02T04:47:40.604 INFO:tasks.workunit.client.0.smithi070.stdout:2/217: rename d2/d10/d19/l23 to d2/d24/l49 0 2024-04-02T04:47:40.605 INFO:tasks.workunit.client.0.smithi070.stdout:9/164: creat d4/f36 x:0 0 0 2024-04-02T04:47:40.607 INFO:tasks.workunit.client.0.smithi070.stdout:9/165: creat d4/d13/f37 x:0 0 0 2024-04-02T04:47:40.608 INFO:tasks.workunit.client.0.smithi070.stdout:2/218: link d2/d10/d21/d2b/c43 d2/d12/d3e/c4a 0 2024-04-02T04:47:40.609 INFO:tasks.workunit.client.0.smithi070.stdout:9/166: unlink d4/c5 0 2024-04-02T04:47:40.610 INFO:tasks.workunit.client.0.smithi070.stdout:9/167: fdatasync d4/d10/d11/f21 0 2024-04-02T04:47:40.617 INFO:tasks.workunit.client.1.smithi179.stdout:7/107: write d1/d9/f12 [2229598,25] 0 2024-04-02T04:47:40.619 INFO:tasks.workunit.client.1.smithi179.stdout:7/108: mknod d1/d9/c22 0 2024-04-02T04:47:40.621 INFO:tasks.workunit.client.1.smithi179.stdout:7/109: mkdir d1/dd/d23 0 2024-04-02T04:47:40.621 INFO:tasks.workunit.client.1.smithi179.stdout:4/149: dread d1/d3/d19/f21 [0,4194304] 0 2024-04-02T04:47:40.622 INFO:tasks.workunit.client.1.smithi179.stdout:9/147: getdents d0/d1/d11 0 2024-04-02T04:47:40.623 INFO:tasks.workunit.client.1.smithi179.stdout:4/150: write d1/d3/dc/d26/f29 [420082,65520] 0 2024-04-02T04:47:40.623 INFO:tasks.workunit.client.1.smithi179.stdout:9/148: chown d0/d1/d10 6489 1 2024-04-02T04:47:40.624 INFO:tasks.workunit.client.1.smithi179.stdout:7/110: write d1/ff [571635,40078] 0 2024-04-02T04:47:40.625 INFO:tasks.workunit.client.1.smithi179.stdout:4/151: chown d1/d3/df/d10/c1d 41 1 2024-04-02T04:47:40.626 INFO:tasks.workunit.client.1.smithi179.stdout:9/149: mknod d0/d1/d12/c2a 0 2024-04-02T04:47:40.627 INFO:tasks.workunit.client.1.smithi179.stdout:7/111: creat d1/dd/d23/f24 x:0 0 0 2024-04-02T04:47:40.628 INFO:tasks.workunit.client.1.smithi179.stdout:4/152: symlink d1/d3/d19/l33 0 2024-04-02T04:47:40.629 INFO:tasks.workunit.client.1.smithi179.stdout:9/150: creat d0/d1/d10/f2b x:0 0 0 2024-04-02T04:47:40.630 INFO:tasks.workunit.client.1.smithi179.stdout:7/112: creat d1/dd/d23/f25 x:0 0 0 2024-04-02T04:47:40.630 INFO:tasks.workunit.client.1.smithi179.stdout:7/113: stat d1/d9/ce 0 2024-04-02T04:47:40.630 INFO:tasks.workunit.client.1.smithi179.stdout:4/153: truncate d1/f5 379042 0 2024-04-02T04:47:40.632 INFO:tasks.workunit.client.1.smithi179.stdout:7/114: symlink d1/d9/l26 0 2024-04-02T04:47:40.633 INFO:tasks.workunit.client.1.smithi179.stdout:4/154: mkdir d1/d3/df/d34 0 2024-04-02T04:47:40.633 INFO:tasks.workunit.client.1.smithi179.stdout:9/151: mknod d0/d19/d27/c2c 0 2024-04-02T04:47:40.634 INFO:tasks.workunit.client.1.smithi179.stdout:7/115: rmdir d1/dd/d16 39 2024-04-02T04:47:40.635 INFO:tasks.workunit.client.1.smithi179.stdout:7/116: fsync d1/dd/f17 0 2024-04-02T04:47:40.638 INFO:tasks.workunit.client.1.smithi179.stdout:7/117: mkdir d1/d9/d27 0 2024-04-02T04:47:40.644 INFO:tasks.workunit.client.0.smithi070.stdout:2/219: link d2/d10/f20 d2/d10/d21/d3d/d42/f4b 0 2024-04-02T04:47:40.646 INFO:tasks.workunit.client.0.smithi070.stdout:3/113: fsync d10/d17/f2b 0 2024-04-02T04:47:40.647 INFO:tasks.workunit.client.0.smithi070.stdout:5/140: write f4 [3358335,68962] 0 2024-04-02T04:47:40.649 INFO:tasks.workunit.client.0.smithi070.stdout:3/114: fdatasync d10/f29 0 2024-04-02T04:47:40.649 INFO:tasks.workunit.client.1.smithi179.stdout:7/118: rmdir d1/dd 39 2024-04-02T04:47:40.654 INFO:tasks.workunit.client.0.smithi070.stdout:5/141: dread - d5/d9/d10/f19 zero size 2024-04-02T04:47:40.656 INFO:tasks.workunit.client.1.smithi179.stdout:7/119: symlink d1/dd/d23/l28 0 2024-04-02T04:47:40.657 INFO:tasks.workunit.client.1.smithi179.stdout:9/152: truncate d0/d1/d12/fe 2481249 0 2024-04-02T04:47:40.658 INFO:tasks.workunit.client.0.smithi070.stdout:9/168: dwrite d4/d10/d11/f2d [0,4194304] 0 2024-04-02T04:47:40.659 INFO:tasks.workunit.client.0.smithi070.stdout:7/130: dwrite d7/d9/f17 [0,4194304] 0 2024-04-02T04:47:40.660 INFO:tasks.workunit.client.1.smithi179.stdout:4/155: dwrite d1/d3/df/d10/f11 [0,4194304] 0 2024-04-02T04:47:40.661 INFO:tasks.workunit.client.1.smithi179.stdout:7/120: rmdir d1/dd/d23 39 2024-04-02T04:47:40.661 INFO:tasks.workunit.client.0.smithi070.stdout:5/142: creat d5/f2f x:0 0 0 2024-04-02T04:47:40.663 INFO:tasks.workunit.client.0.smithi070.stdout:4/115: dwrite d1/d3/d9/f1a [0,4194304] 0 2024-04-02T04:47:40.663 INFO:tasks.workunit.client.0.smithi070.stdout:2/220: write d2/d10/d21/f31 [1265078,54340] 0 2024-04-02T04:47:40.665 INFO:tasks.workunit.client.0.smithi070.stdout:0/283: dread d4/f1b [0,4194304] 0 2024-04-02T04:47:40.667 INFO:tasks.workunit.client.0.smithi070.stdout:3/115: dread ff [0,4194304] 0 2024-04-02T04:47:40.669 INFO:tasks.workunit.client.1.smithi179.stdout:4/156: symlink d1/d3/l35 0 2024-04-02T04:47:40.670 INFO:tasks.workunit.client.0.smithi070.stdout:7/131: rename d7/c15 to d7/d9/d19/c25 0 2024-04-02T04:47:40.672 INFO:tasks.workunit.client.0.smithi070.stdout:4/116: symlink d1/d3/d9/l23 0 2024-04-02T04:47:40.682 INFO:tasks.workunit.client.1.smithi179.stdout:7/121: symlink d1/dd/d23/l29 0 2024-04-02T04:47:40.683 INFO:tasks.workunit.client.0.smithi070.stdout:9/169: rename d4/d10/d11/c1e to d4/d10/d12/c38 0 2024-04-02T04:47:40.685 INFO:tasks.workunit.client.0.smithi070.stdout:7/132: mknod d7/c26 0 2024-04-02T04:47:40.685 INFO:tasks.workunit.client.0.smithi070.stdout:9/170: truncate d4/d10/d12/d27/d32/f33 38464 0 2024-04-02T04:47:40.685 INFO:tasks.workunit.client.0.smithi070.stdout:7/133: chown d7/d9/fa 128622556 1 2024-04-02T04:47:40.687 INFO:tasks.workunit.client.0.smithi070.stdout:2/221: mknod d2/d12/d48/c4c 0 2024-04-02T04:47:40.688 INFO:tasks.workunit.client.1.smithi179.stdout:7/122: creat d1/dd/d16/f2a x:0 0 0 2024-04-02T04:47:40.688 INFO:tasks.workunit.client.1.smithi179.stdout:7/123: chown l0 259626 1 2024-04-02T04:47:40.692 INFO:tasks.workunit.client.1.smithi179.stdout:7/124: symlink d1/dd/l2b 0 2024-04-02T04:47:40.692 INFO:tasks.workunit.client.1.smithi179.stdout:7/125: chown d1/dd/f20 6145 1 2024-04-02T04:47:40.692 INFO:tasks.workunit.client.1.smithi179.stdout:6/76: sync 2024-04-02T04:47:40.693 INFO:tasks.workunit.client.0.smithi070.stdout:2/222: symlink d2/d10/l4d 0 2024-04-02T04:47:40.695 INFO:tasks.workunit.client.0.smithi070.stdout:5/143: dwrite d5/d14/f16 [0,4194304] 0 2024-04-02T04:47:40.696 INFO:tasks.workunit.client.1.smithi179.stdout:7/126: chown d1/l2 106466536 1 2024-04-02T04:47:40.696 INFO:tasks.workunit.client.0.smithi070.stdout:4/117: link d1/d3/d9/f13 d1/d3/d9/f24 0 2024-04-02T04:47:40.699 INFO:tasks.workunit.client.1.smithi179.stdout:9/153: dwrite d0/d1/d10/f13 [0,4194304] 0 2024-04-02T04:47:40.701 INFO:tasks.workunit.client.0.smithi070.stdout:9/171: mkdir d4/d13/d16/d19/d2f/d39 0 2024-04-02T04:47:40.703 INFO:tasks.workunit.client.1.smithi179.stdout:7/127: creat d1/dd/d16/f2c x:0 0 0 2024-04-02T04:47:40.704 INFO:tasks.workunit.client.1.smithi179.stdout:4/157: write d1/d3/fa [489521,72274] 0 2024-04-02T04:47:40.705 INFO:tasks.workunit.client.1.smithi179.stdout:4/158: readlink d1/d3/df/d10/l17 0 2024-04-02T04:47:40.705 INFO:tasks.workunit.client.0.smithi070.stdout:9/172: chown d4/d13/f37 17998 1 2024-04-02T04:47:40.706 INFO:tasks.workunit.client.0.smithi070.stdout:4/118: dwrite d1/f21 [0,4194304] 0 2024-04-02T04:47:40.707 INFO:tasks.workunit.client.1.smithi179.stdout:9/154: creat d0/d1/d12/f2d x:0 0 0 2024-04-02T04:47:40.707 INFO:tasks.workunit.client.0.smithi070.stdout:7/134: dread d7/fc [0,4194304] 0 2024-04-02T04:47:40.708 INFO:tasks.workunit.client.0.smithi070.stdout:4/119: readlink d1/d3/l7 0 2024-04-02T04:47:40.708 INFO:tasks.workunit.client.0.smithi070.stdout:7/135: read d7/d9/f17 [1654476,28782] 0 2024-04-02T04:47:40.711 INFO:tasks.workunit.client.0.smithi070.stdout:0/284: truncate d4/d5/f3f 1830740 0 2024-04-02T04:47:40.719 INFO:tasks.workunit.client.1.smithi179.stdout:6/77: creat d3/f24 x:0 0 0 2024-04-02T04:47:40.722 INFO:tasks.workunit.client.0.smithi070.stdout:7/136: unlink d7/d9/d11/c20 0 2024-04-02T04:47:40.722 INFO:tasks.workunit.client.1.smithi179.stdout:6/78: chown d3/f18 1 1 2024-04-02T04:47:40.722 INFO:tasks.workunit.client.1.smithi179.stdout:7/128: mknod d1/c2d 0 2024-04-02T04:47:40.722 INFO:tasks.workunit.client.1.smithi179.stdout:4/159: mkdir d1/d3/df/d10/d2f/d36 0 2024-04-02T04:47:40.723 INFO:tasks.workunit.client.1.smithi179.stdout:4/160: truncate d1/fb 1723624 0 2024-04-02T04:47:40.723 INFO:tasks.workunit.client.0.smithi070.stdout:5/144: mkdir d5/d9/d23/d30 0 2024-04-02T04:47:40.725 INFO:tasks.workunit.client.0.smithi070.stdout:9/173: symlink d4/d13/d16/d19/d2f/d39/l3a 0 2024-04-02T04:47:40.726 INFO:tasks.workunit.client.1.smithi179.stdout:7/129: dread - d1/d9/f1d zero size 2024-04-02T04:47:40.726 INFO:tasks.workunit.client.0.smithi070.stdout:7/137: dwrite d7/f1e [0,4194304] 0 2024-04-02T04:47:40.727 INFO:tasks.workunit.client.1.smithi179.stdout:9/155: link d0/d1/l14 d0/d19/d1c/l2e 0 2024-04-02T04:47:40.728 INFO:tasks.workunit.client.0.smithi070.stdout:7/138: write d7/f1e [2315124,43897] 0 2024-04-02T04:47:40.728 INFO:tasks.workunit.client.1.smithi179.stdout:9/156: write d0/d1/d12/f2d [601697,30154] 0 2024-04-02T04:47:40.729 INFO:tasks.workunit.client.0.smithi070.stdout:2/223: link d2/c16 d2/d10/d19/c4e 0 2024-04-02T04:47:40.729 INFO:tasks.workunit.client.0.smithi070.stdout:2/224: chown d2/f1b 314 1 2024-04-02T04:47:40.729 INFO:tasks.workunit.client.1.smithi179.stdout:9/157: fdatasync d0/d1/f26 0 2024-04-02T04:47:40.730 INFO:tasks.workunit.client.1.smithi179.stdout:4/161: symlink d1/d3/dc/l37 0 2024-04-02T04:47:40.733 INFO:tasks.workunit.client.1.smithi179.stdout:7/130: stat d1/c19 0 2024-04-02T04:47:40.738 INFO:tasks.workunit.client.1.smithi179.stdout:4/162: creat d1/d3/d19/f38 x:0 0 0 2024-04-02T04:47:40.741 INFO:tasks.workunit.client.1.smithi179.stdout:4/163: symlink d1/d3/dc/d26/l39 0 2024-04-02T04:47:40.742 INFO:tasks.workunit.client.1.smithi179.stdout:6/79: write d3/f1c [3566594,71732] 0 2024-04-02T04:47:40.743 INFO:tasks.workunit.client.1.smithi179.stdout:4/164: write d1/d3/dc/f1c [246,78945] 0 2024-04-02T04:47:40.743 INFO:tasks.workunit.client.1.smithi179.stdout:7/131: getdents d1/dd/d23 0 2024-04-02T04:47:40.744 INFO:tasks.workunit.client.1.smithi179.stdout:4/165: chown d1/d3/dc/d26/f29 921156819 1 2024-04-02T04:47:40.745 INFO:tasks.workunit.client.1.smithi179.stdout:4/166: stat d1/d3/dc/c23 0 2024-04-02T04:47:40.748 INFO:tasks.workunit.client.1.smithi179.stdout:4/167: write d1/d3/dc/f1c [567224,9692] 0 2024-04-02T04:47:40.749 INFO:tasks.workunit.client.1.smithi179.stdout:6/80: dwrite d3/d12/f21 [4194304,4194304] 0 2024-04-02T04:47:40.749 INFO:tasks.workunit.client.1.smithi179.stdout:7/132: dread d1/d9/f12 [0,4194304] 0 2024-04-02T04:47:40.750 INFO:tasks.workunit.client.1.smithi179.stdout:7/133: read - d1/dd/d16/f2c zero size 2024-04-02T04:47:40.753 INFO:tasks.workunit.client.1.smithi179.stdout:6/81: dread f2 [0,4194304] 0 2024-04-02T04:47:40.754 INFO:tasks.workunit.client.1.smithi179.stdout:7/134: rename d1/l8 to d1/dd/d16/l2e 0 2024-04-02T04:47:40.756 INFO:tasks.workunit.client.1.smithi179.stdout:7/135: creat d1/dd/d23/f2f x:0 0 0 2024-04-02T04:47:40.759 INFO:tasks.workunit.client.1.smithi179.stdout:7/136: write d1/d9/fb [1885535,52927] 0 2024-04-02T04:47:40.760 INFO:tasks.workunit.client.1.smithi179.stdout:4/168: dread d1/fb [0,4194304] 0 2024-04-02T04:47:40.761 INFO:tasks.workunit.client.1.smithi179.stdout:6/82: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:40.762 INFO:tasks.workunit.client.1.smithi179.stdout:7/137: symlink d1/dd/d16/l30 0 2024-04-02T04:47:40.763 INFO:tasks.workunit.client.1.smithi179.stdout:6/83: fsync d3/fa 0 2024-04-02T04:47:40.765 INFO:tasks.workunit.client.1.smithi179.stdout:7/138: truncate d1/dd/d23/f25 915056 0 2024-04-02T04:47:40.766 INFO:tasks.workunit.client.1.smithi179.stdout:6/84: rmdir d3 39 2024-04-02T04:47:40.767 INFO:tasks.workunit.client.1.smithi179.stdout:7/139: mknod d1/d9/c31 0 2024-04-02T04:47:40.775 INFO:tasks.workunit.client.0.smithi070.stdout:4/120: write d1/d3/d9/f13 [3297824,8112] 0 2024-04-02T04:47:40.776 INFO:tasks.workunit.client.1.smithi179.stdout:4/169: dread d1/d3/f28 [0,4194304] 0 2024-04-02T04:47:40.777 INFO:tasks.workunit.client.0.smithi070.stdout:7/139: truncate d7/d9/f17 2944079 0 2024-04-02T04:47:40.779 INFO:tasks.workunit.client.1.smithi179.stdout:6/85: rename d3/d12/f1b to d3/d12/f25 0 2024-04-02T04:47:40.782 INFO:tasks.workunit.client.1.smithi179.stdout:7/140: dwrite d1/fc [4194304,4194304] 0 2024-04-02T04:47:40.782 INFO:tasks.workunit.client.0.smithi070.stdout:2/225: dread d2/d10/f20 [0,4194304] 0 2024-04-02T04:47:40.783 INFO:tasks.workunit.client.1.smithi179.stdout:4/170: mkdir d1/d3/df/d10/d3a 0 2024-04-02T04:47:40.784 INFO:tasks.workunit.client.1.smithi179.stdout:7/141: truncate d1/dd/f1c 126926 0 2024-04-02T04:47:40.784 INFO:tasks.workunit.client.1.smithi179.stdout:7/142: fdatasync d1/dd/d16/f2a 0 2024-04-02T04:47:40.785 INFO:tasks.workunit.client.1.smithi179.stdout:4/171: truncate d1/d3/dc/d27/f2e 893822 0 2024-04-02T04:47:40.788 INFO:tasks.workunit.client.0.smithi070.stdout:4/121: unlink d1/fc 0 2024-04-02T04:47:40.792 INFO:tasks.workunit.client.1.smithi179.stdout:6/86: dwrite d3/d12/f25 [0,4194304] 0 2024-04-02T04:47:40.792 INFO:tasks.workunit.client.1.smithi179.stdout:6/87: fdatasync d3/ff 0 2024-04-02T04:47:40.794 INFO:tasks.workunit.client.0.smithi070.stdout:2/226: chown d2/d24/l49 125 1 2024-04-02T04:47:40.794 INFO:tasks.workunit.client.0.smithi070.stdout:9/174: write d4/d10/d11/f14 [3731765,17312] 0 2024-04-02T04:47:40.795 INFO:tasks.workunit.client.0.smithi070.stdout:5/145: symlink d5/d17/d2b/l31 0 2024-04-02T04:47:40.799 INFO:tasks.workunit.client.1.smithi179.stdout:7/143: creat d1/f32 x:0 0 0 2024-04-02T04:47:40.801 INFO:tasks.workunit.client.0.smithi070.stdout:4/122: creat d1/d3/d9/f25 x:0 0 0 2024-04-02T04:47:40.801 INFO:tasks.workunit.client.0.smithi070.stdout:5/146: dwrite d5/f2f [0,4194304] 0 2024-04-02T04:47:40.802 INFO:tasks.workunit.client.1.smithi179.stdout:6/88: mkdir d3/d12/d23/d26 0 2024-04-02T04:47:40.803 INFO:tasks.workunit.client.1.smithi179.stdout:7/144: dread d1/dd/f17 [0,4194304] 0 2024-04-02T04:47:40.804 INFO:tasks.workunit.client.0.smithi070.stdout:2/227: rmdir d2 39 2024-04-02T04:47:40.805 INFO:tasks.workunit.client.1.smithi179.stdout:6/89: dread f2 [0,4194304] 0 2024-04-02T04:47:40.805 INFO:tasks.workunit.client.0.smithi070.stdout:5/147: dread d5/d14/f16 [0,4194304] 0 2024-04-02T04:47:40.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/148: chown f4 2363185 1 2024-04-02T04:47:40.807 INFO:tasks.workunit.client.1.smithi179.stdout:7/145: creat d1/dd/d16/f33 x:0 0 0 2024-04-02T04:47:40.808 INFO:tasks.workunit.client.1.smithi179.stdout:7/146: dread - d1/f32 zero size 2024-04-02T04:47:40.809 INFO:tasks.workunit.client.1.smithi179.stdout:7/147: chown d1/dd/f20 1847 1 2024-04-02T04:47:40.810 INFO:tasks.workunit.client.1.smithi179.stdout:7/148: write d1/dd/f20 [849168,83648] 0 2024-04-02T04:47:40.811 INFO:tasks.workunit.client.0.smithi070.stdout:7/140: creat d7/d9/f27 x:0 0 0 2024-04-02T04:47:40.812 INFO:tasks.workunit.client.0.smithi070.stdout:7/141: write d7/d9/f27 [666686,44240] 0 2024-04-02T04:47:40.813 INFO:tasks.workunit.client.0.smithi070.stdout:7/142: write d7/d9/f27 [754923,25548] 0 2024-04-02T04:47:40.815 INFO:tasks.workunit.client.1.smithi179.stdout:7/149: rmdir d1 39 2024-04-02T04:47:40.815 INFO:tasks.workunit.client.1.smithi179.stdout:6/90: write d3/fb [2698124,51387] 0 2024-04-02T04:47:40.816 INFO:tasks.workunit.client.1.smithi179.stdout:6/91: read d3/fa [1733991,76435] 0 2024-04-02T04:47:40.816 INFO:tasks.workunit.client.1.smithi179.stdout:6/92: chown d3/ff 128307 1 2024-04-02T04:47:40.817 INFO:tasks.workunit.client.0.smithi070.stdout:5/149: creat d5/d17/f32 x:0 0 0 2024-04-02T04:47:40.817 INFO:tasks.workunit.client.1.smithi179.stdout:6/93: dread - d3/ff zero size 2024-04-02T04:47:40.817 INFO:tasks.workunit.client.0.smithi070.stdout:5/150: chown d5/d9/d10/f11 207098244 1 2024-04-02T04:47:40.819 INFO:tasks.workunit.client.1.smithi179.stdout:7/150: unlink d1/c2d 0 2024-04-02T04:47:40.820 INFO:tasks.workunit.client.1.smithi179.stdout:7/151: readlink d1/l2 0 2024-04-02T04:47:40.821 INFO:tasks.workunit.client.0.smithi070.stdout:5/151: mkdir d5/d9/d23/d33 0 2024-04-02T04:47:40.822 INFO:tasks.workunit.client.0.smithi070.stdout:4/123: rename d1/d3/d9/ff to d1/f26 0 2024-04-02T04:47:40.823 INFO:tasks.workunit.client.1.smithi179.stdout:6/94: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:40.824 INFO:tasks.workunit.client.1.smithi179.stdout:7/152: mknod d1/d9/c34 0 2024-04-02T04:47:40.824 INFO:tasks.workunit.client.0.smithi070.stdout:9/175: getdents d4/d13/d16/d19/d2f 0 2024-04-02T04:47:40.831 INFO:tasks.workunit.client.1.smithi179.stdout:6/95: dwrite d3/d12/f22 [0,4194304] 0 2024-04-02T04:47:40.832 INFO:tasks.workunit.client.0.smithi070.stdout:0/285: dread d4/d5/d13/f1f [0,4194304] 0 2024-04-02T04:47:40.833 INFO:tasks.workunit.client.0.smithi070.stdout:5/152: rmdir d5/d14 39 2024-04-02T04:47:40.834 INFO:tasks.workunit.client.0.smithi070.stdout:0/286: write d4/d5/d13/d23/f36 [1572285,31149] 0 2024-04-02T04:47:40.837 INFO:tasks.workunit.client.1.smithi179.stdout:7/153: mkdir d1/d9/d35 0 2024-04-02T04:47:40.840 INFO:tasks.workunit.client.0.smithi070.stdout:0/287: dread d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:40.844 INFO:tasks.workunit.client.1.smithi179.stdout:7/154: mknod d1/dd/d23/c36 0 2024-04-02T04:47:40.845 INFO:tasks.workunit.client.0.smithi070.stdout:7/143: dread d7/d9/f27 [0,4194304] 0 2024-04-02T04:47:40.846 INFO:tasks.workunit.client.1.smithi179.stdout:6/96: creat d3/d12/d23/d26/f27 x:0 0 0 2024-04-02T04:47:40.848 INFO:tasks.workunit.client.0.smithi070.stdout:2/228: rename d2/d10/d21/l2a to d2/d10/d21/d3d/d42/l4f 0 2024-04-02T04:47:40.849 INFO:tasks.workunit.client.0.smithi070.stdout:4/124: mknod d1/c27 0 2024-04-02T04:47:40.850 INFO:tasks.workunit.client.0.smithi070.stdout:4/125: write d1/fa [4620326,110245] 0 2024-04-02T04:47:40.850 INFO:tasks.workunit.client.1.smithi179.stdout:7/155: symlink d1/d9/d27/l37 0 2024-04-02T04:47:40.853 INFO:tasks.workunit.client.1.smithi179.stdout:6/97: symlink d3/l28 0 2024-04-02T04:47:40.853 INFO:tasks.workunit.client.1.smithi179.stdout:6/98: stat d3/l28 0 2024-04-02T04:47:40.854 INFO:tasks.workunit.client.0.smithi070.stdout:9/176: dread d4/f8 [0,4194304] 0 2024-04-02T04:47:40.855 INFO:tasks.workunit.client.1.smithi179.stdout:7/156: fsync d1/f15 0 2024-04-02T04:47:40.856 INFO:tasks.workunit.client.0.smithi070.stdout:2/229: fdatasync d2/d10/d21/f34 0 2024-04-02T04:47:40.856 INFO:tasks.workunit.client.0.smithi070.stdout:9/177: write d4/d13/f26 [211847,68286] 0 2024-04-02T04:47:40.856 INFO:tasks.workunit.client.1.smithi179.stdout:7/157: write d1/dd/f1c [1034993,43738] 0 2024-04-02T04:47:40.858 INFO:tasks.workunit.client.0.smithi070.stdout:7/144: read d7/dd/df/f10 [3462,19903] 0 2024-04-02T04:47:40.859 INFO:tasks.workunit.client.1.smithi179.stdout:6/99: dwrite d3/ff [0,4194304] 0 2024-04-02T04:47:40.862 INFO:tasks.workunit.client.1.smithi179.stdout:7/158: creat d1/dd/d16/f38 x:0 0 0 2024-04-02T04:47:40.863 INFO:tasks.workunit.client.0.smithi070.stdout:9/178: dwrite d4/d10/d11/f14 [0,4194304] 0 2024-04-02T04:47:40.869 INFO:tasks.workunit.client.0.smithi070.stdout:2/230: mknod d2/d10/c50 0 2024-04-02T04:47:40.871 INFO:tasks.workunit.client.1.smithi179.stdout:6/100: link d3/l7 d3/d12/d23/d26/l29 0 2024-04-02T04:47:40.872 INFO:tasks.workunit.client.1.smithi179.stdout:7/159: dwrite d1/d9/f12 [0,4194304] 0 2024-04-02T04:47:40.872 INFO:tasks.workunit.client.0.smithi070.stdout:7/145: mknod d7/d9/d11/c28 0 2024-04-02T04:47:40.873 INFO:tasks.workunit.client.1.smithi179.stdout:7/160: fsync d1/dd/d23/f2f 0 2024-04-02T04:47:40.873 INFO:tasks.workunit.client.0.smithi070.stdout:0/288: link d4/d5/d13/d2a/d3a/c3e d4/d5/d13/d23/c4a 0 2024-04-02T04:47:40.874 INFO:tasks.workunit.client.1.smithi179.stdout:7/161: chown d1/d9/c22 240510 1 2024-04-02T04:47:40.874 INFO:tasks.workunit.client.1.smithi179.stdout:7/162: chown d1/dd/d23/l28 0 1 2024-04-02T04:47:40.877 INFO:tasks.workunit.client.0.smithi070.stdout:5/153: write d5/f6 [481959,64650] 0 2024-04-02T04:47:40.877 INFO:tasks.workunit.client.0.smithi070.stdout:2/231: dwrite d2/d10/d21/f34 [0,4194304] 0 2024-04-02T04:47:40.878 INFO:tasks.workunit.client.1.smithi179.stdout:7/163: symlink d1/d9/l39 0 2024-04-02T04:47:40.884 INFO:tasks.workunit.client.0.smithi070.stdout:4/126: creat d1/f28 x:0 0 0 2024-04-02T04:47:40.884 INFO:tasks.workunit.client.1.smithi179.stdout:6/101: link d3/d12/d23/d26/l29 d3/d12/d23/l2a 0 2024-04-02T04:47:40.885 INFO:tasks.workunit.client.0.smithi070.stdout:4/127: write d1/f28 [628254,127237] 0 2024-04-02T04:47:40.885 INFO:tasks.workunit.client.1.smithi179.stdout:8/126: sync 2024-04-02T04:47:40.886 INFO:tasks.workunit.client.0.smithi070.stdout:7/146: mkdir d7/d1c/d29 0 2024-04-02T04:47:40.887 INFO:tasks.workunit.client.1.smithi179.stdout:7/164: creat d1/d9/d35/f3a x:0 0 0 2024-04-02T04:47:40.889 INFO:tasks.workunit.client.1.smithi179.stdout:6/102: creat d3/d12/d23/f2b x:0 0 0 2024-04-02T04:47:40.890 INFO:tasks.workunit.client.1.smithi179.stdout:7/165: truncate d1/dd/d16/f38 100218 0 2024-04-02T04:47:40.890 INFO:tasks.workunit.client.0.smithi070.stdout:9/179: mknod d4/d34/c3b 0 2024-04-02T04:47:40.891 INFO:tasks.workunit.client.1.smithi179.stdout:7/166: write d1/dd/f1c [116655,101394] 0 2024-04-02T04:47:40.892 INFO:tasks.workunit.client.0.smithi070.stdout:2/232: unlink d2/d12/d48/c4c 0 2024-04-02T04:47:40.892 INFO:tasks.workunit.client.1.smithi179.stdout:8/127: dwrite d3/f16 [4194304,4194304] 0 2024-04-02T04:47:40.894 INFO:tasks.workunit.client.0.smithi070.stdout:4/128: dwrite d1/d3/d9/f1d [0,4194304] 0 2024-04-02T04:47:40.894 INFO:tasks.workunit.client.1.smithi179.stdout:6/103: unlink d3/fc 0 2024-04-02T04:47:40.894 INFO:tasks.workunit.client.1.smithi179.stdout:7/167: fdatasync d1/dd/f1c 0 2024-04-02T04:47:40.896 INFO:tasks.workunit.client.0.smithi070.stdout:7/147: rmdir d7/dd 39 2024-04-02T04:47:40.896 INFO:tasks.workunit.client.0.smithi070.stdout:7/148: readlink d7/l1d 0 2024-04-02T04:47:40.898 INFO:tasks.workunit.client.0.smithi070.stdout:0/289: mknod d4/d5/da/c4b 0 2024-04-02T04:47:40.900 INFO:tasks.workunit.client.0.smithi070.stdout:0/290: read d4/d5/d13/f2b [464523,70226] 0 2024-04-02T04:47:40.903 INFO:tasks.workunit.client.0.smithi070.stdout:5/154: rmdir d5/d9/d10 39 2024-04-02T04:47:40.905 INFO:tasks.workunit.client.0.smithi070.stdout:9/180: creat d4/d13/d16/d19/d2f/f3c x:0 0 0 2024-04-02T04:47:40.906 INFO:tasks.workunit.client.0.smithi070.stdout:9/181: stat d4/d10/d12/d27/f30 0 2024-04-02T04:47:40.906 INFO:tasks.workunit.client.1.smithi179.stdout:7/168: readlink d1/dd/d16/l2e 0 2024-04-02T04:47:40.908 INFO:tasks.workunit.client.0.smithi070.stdout:2/233: symlink d2/d12/d3e/l51 0 2024-04-02T04:47:40.911 INFO:tasks.workunit.client.1.smithi179.stdout:2/77: sync 2024-04-02T04:47:40.911 INFO:tasks.workunit.client.1.smithi179.stdout:1/194: sync 2024-04-02T04:47:40.911 INFO:tasks.workunit.client.1.smithi179.stdout:3/96: sync 2024-04-02T04:47:40.911 INFO:tasks.workunit.client.1.smithi179.stdout:0/212: sync 2024-04-02T04:47:40.911 INFO:tasks.workunit.client.1.smithi179.stdout:9/158: sync 2024-04-02T04:47:40.912 INFO:tasks.workunit.client.1.smithi179.stdout:4/172: sync 2024-04-02T04:47:40.912 INFO:tasks.workunit.client.1.smithi179.stdout:5/64: sync 2024-04-02T04:47:40.913 INFO:tasks.workunit.client.1.smithi179.stdout:8/128: mkdir d3/d12/d30 0 2024-04-02T04:47:40.913 INFO:tasks.workunit.client.0.smithi070.stdout:9/182: dwrite d4/d13/d16/d19/d2f/f3c [0,4194304] 0 2024-04-02T04:47:40.915 INFO:tasks.workunit.client.1.smithi179.stdout:7/169: creat d1/d9/d35/f3b x:0 0 0 2024-04-02T04:47:40.920 INFO:tasks.workunit.client.1.smithi179.stdout:2/78: fdatasync f6 0 2024-04-02T04:47:40.921 INFO:tasks.workunit.client.0.smithi070.stdout:9/183: dwrite d4/d10/d11/f21 [0,4194304] 0 2024-04-02T04:47:40.923 INFO:tasks.workunit.client.1.smithi179.stdout:8/129: dwrite d3/d1b/f23 [0,4194304] 0 2024-04-02T04:47:40.925 INFO:tasks.workunit.client.1.smithi179.stdout:1/195: symlink d3/l46 0 2024-04-02T04:47:40.925 INFO:tasks.workunit.client.1.smithi179.stdout:1/196: stat d3/f2d 0 2024-04-02T04:47:40.926 INFO:tasks.workunit.client.1.smithi179.stdout:8/130: dread - d3/f7 zero size 2024-04-02T04:47:40.926 INFO:tasks.workunit.client.0.smithi070.stdout:2/234: creat d2/d12/f52 x:0 0 0 2024-04-02T04:47:40.926 INFO:tasks.workunit.client.0.smithi070.stdout:2/235: chown d2/l3a 6799 1 2024-04-02T04:47:40.926 INFO:tasks.workunit.client.0.smithi070.stdout:0/291: dwrite d4/d5/d13/f32 [0,4194304] 0 2024-04-02T04:47:40.927 INFO:tasks.workunit.client.0.smithi070.stdout:2/236: write d2/d10/d21/f31 [3218605,43652] 0 2024-04-02T04:47:40.927 INFO:tasks.workunit.client.1.smithi179.stdout:1/197: truncate d3/d5/f25 1019345 0 2024-04-02T04:47:40.928 INFO:tasks.workunit.client.1.smithi179.stdout:5/65: creat d8/f13 x:0 0 0 2024-04-02T04:47:40.931 INFO:tasks.workunit.client.1.smithi179.stdout:5/66: write d8/f13 [169012,91749] 0 2024-04-02T04:47:40.933 INFO:tasks.workunit.client.1.smithi179.stdout:9/159: mknod d0/d19/d27/c2f 0 2024-04-02T04:47:40.934 INFO:tasks.workunit.client.1.smithi179.stdout:3/97: creat d8/f1b x:0 0 0 2024-04-02T04:47:40.935 INFO:tasks.workunit.client.1.smithi179.stdout:4/173: truncate d1/d3/f28 5236189 0 2024-04-02T04:47:40.937 INFO:tasks.workunit.client.1.smithi179.stdout:7/170: symlink d1/dd/d16/l3c 0 2024-04-02T04:47:40.938 INFO:tasks.workunit.client.0.smithi070.stdout:5/155: mknod d5/c34 0 2024-04-02T04:47:40.939 INFO:tasks.workunit.client.0.smithi070.stdout:4/129: dwrite d1/fb [0,4194304] 0 2024-04-02T04:47:40.940 INFO:tasks.workunit.client.1.smithi179.stdout:5/67: dread f6 [0,4194304] 0 2024-04-02T04:47:40.941 INFO:tasks.workunit.client.1.smithi179.stdout:4/174: write d1/d3/dc/d26/f29 [1032627,90119] 0 2024-04-02T04:47:40.941 INFO:tasks.workunit.client.0.smithi070.stdout:9/184: creat d4/d13/d16/d19/d2f/d39/f3d x:0 0 0 2024-04-02T04:47:40.942 INFO:tasks.workunit.client.0.smithi070.stdout:4/130: truncate d1/d3/d9/f25 508557 0 2024-04-02T04:47:40.945 INFO:tasks.workunit.client.0.smithi070.stdout:0/292: creat d4/d5/da/d2d/f4c x:0 0 0 2024-04-02T04:47:40.945 INFO:tasks.workunit.client.1.smithi179.stdout:1/198: dwrite d3/d31/f37 [0,4194304] 0 2024-04-02T04:47:40.946 INFO:tasks.workunit.client.0.smithi070.stdout:4/131: dread d1/d3/d9/f25 [0,4194304] 0 2024-04-02T04:47:40.947 INFO:tasks.workunit.client.0.smithi070.stdout:4/132: chown d1/f1e 13054 1 2024-04-02T04:47:40.947 INFO:tasks.workunit.client.1.smithi179.stdout:2/79: rename dc/fd to dc/f1a 0 2024-04-02T04:47:40.947 INFO:tasks.workunit.client.1.smithi179.stdout:9/160: dread d0/f5 [0,4194304] 0 2024-04-02T04:47:40.947 INFO:tasks.workunit.client.1.smithi179.stdout:2/80: chown dc/f17 2685599 1 2024-04-02T04:47:40.948 INFO:tasks.workunit.client.0.smithi070.stdout:4/133: write d1/d3/d9/f25 [390697,64003] 0 2024-04-02T04:47:40.948 INFO:tasks.workunit.client.0.smithi070.stdout:2/237: write d2/d10/d21/d3d/d42/f4b [2483860,20568] 0 2024-04-02T04:47:40.951 INFO:tasks.workunit.client.0.smithi070.stdout:2/238: write d2/d10/d21/d3d/d42/f4b [195721,79295] 0 2024-04-02T04:47:40.951 INFO:tasks.workunit.client.0.smithi070.stdout:4/134: truncate d1/d3/d9/f13 8712427 0 2024-04-02T04:47:40.952 INFO:tasks.workunit.client.0.smithi070.stdout:2/239: fsync d2/d24/f3f 0 2024-04-02T04:47:40.955 INFO:tasks.workunit.client.0.smithi070.stdout:2/240: write d2/d10/d21/f34 [1660350,74590] 0 2024-04-02T04:47:40.958 INFO:tasks.workunit.client.0.smithi070.stdout:2/241: chown d2/l15 314964 1 2024-04-02T04:47:40.959 INFO:tasks.workunit.client.0.smithi070.stdout:0/293: dread d4/d5/d13/d2a/f20 [0,4194304] 0 2024-04-02T04:47:40.959 INFO:tasks.workunit.client.0.smithi070.stdout:0/294: readlink d4/d5/d13/d23/l2e 0 2024-04-02T04:47:40.959 INFO:tasks.workunit.client.1.smithi179.stdout:3/98: rename d8/f1a to d8/f1c 0 2024-04-02T04:47:40.962 INFO:tasks.workunit.client.1.smithi179.stdout:6/104: getdents d3/d12/d23/d26 0 2024-04-02T04:47:40.963 INFO:tasks.workunit.client.0.smithi070.stdout:4/135: creat d1/d3/d9/f29 x:0 0 0 2024-04-02T04:47:40.963 INFO:tasks.workunit.client.1.smithi179.stdout:5/68: mknod d8/c14 0 2024-04-02T04:47:40.965 INFO:tasks.workunit.client.1.smithi179.stdout:4/175: fsync d1/fb 0 2024-04-02T04:47:40.969 INFO:tasks.workunit.client.1.smithi179.stdout:5/69: dwrite d8/f12 [0,4194304] 0 2024-04-02T04:47:40.972 INFO:tasks.workunit.client.0.smithi070.stdout:2/242: mkdir d2/d10/d53 0 2024-04-02T04:47:40.974 INFO:tasks.workunit.client.0.smithi070.stdout:4/136: write d1/d3/d9/f25 [166655,111066] 0 2024-04-02T04:47:40.975 INFO:tasks.workunit.client.1.smithi179.stdout:3/99: rename d8/f10 to d8/f1d 0 2024-04-02T04:47:40.976 INFO:tasks.workunit.client.0.smithi070.stdout:7/149: getdents d7/dd/df 0 2024-04-02T04:47:40.977 INFO:tasks.workunit.client.0.smithi070.stdout:7/150: write d7/d9/f21 [807333,120565] 0 2024-04-02T04:47:40.977 INFO:tasks.workunit.client.1.smithi179.stdout:7/171: symlink d1/l3d 0 2024-04-02T04:47:40.978 INFO:tasks.workunit.client.0.smithi070.stdout:5/156: mkdir d5/d17/d27/d2c/d35 0 2024-04-02T04:47:40.979 INFO:tasks.workunit.client.1.smithi179.stdout:4/176: dread d1/d3/f7 [0,4194304] 0 2024-04-02T04:47:40.980 INFO:tasks.workunit.client.0.smithi070.stdout:2/243: unlink d2/d24/f3f 0 2024-04-02T04:47:40.984 INFO:tasks.workunit.client.0.smithi070.stdout:4/137: creat d1/d3/d9/f2a x:0 0 0 2024-04-02T04:47:40.986 INFO:tasks.workunit.client.0.smithi070.stdout:7/151: creat d7/d9/d11/f2a x:0 0 0 2024-04-02T04:47:40.988 INFO:tasks.workunit.client.1.smithi179.stdout:9/161: unlink d0/f5 0 2024-04-02T04:47:40.988 INFO:tasks.workunit.client.0.smithi070.stdout:4/138: dread d1/f12 [0,4194304] 0 2024-04-02T04:47:40.989 INFO:tasks.workunit.client.0.smithi070.stdout:5/157: read d5/ff [2227574,103261] 0 2024-04-02T04:47:40.990 INFO:tasks.workunit.client.1.smithi179.stdout:5/70: mknod d8/c15 0 2024-04-02T04:47:40.993 INFO:tasks.workunit.client.1.smithi179.stdout:0/213: getdents d1/da/d28 0 2024-04-02T04:47:40.993 INFO:tasks.workunit.client.0.smithi070.stdout:2/244: read d2/f4 [361845,63152] 0 2024-04-02T04:47:40.995 INFO:tasks.workunit.client.0.smithi070.stdout:7/152: symlink d7/d9/d19/l2b 0 2024-04-02T04:47:40.996 INFO:tasks.workunit.client.1.smithi179.stdout:7/172: rename d1/d9/d27/l37 to d1/d9/d27/l3e 0 2024-04-02T04:47:40.997 INFO:tasks.workunit.client.1.smithi179.stdout:6/105: mkdir d3/d2c 0 2024-04-02T04:47:40.998 INFO:tasks.workunit.client.1.smithi179.stdout:4/177: fdatasync d1/fb 0 2024-04-02T04:47:41.005 INFO:tasks.workunit.client.1.smithi179.stdout:8/131: getdents d3/d12/d21/d22 0 2024-04-02T04:47:41.006 INFO:tasks.workunit.client.1.smithi179.stdout:8/132: truncate d3/f18 900449 0 2024-04-02T04:47:41.007 INFO:tasks.workunit.client.0.smithi070.stdout:9/185: dwrite d4/d13/d16/d19/f22 [0,4194304] 0 2024-04-02T04:47:41.008 INFO:tasks.workunit.client.0.smithi070.stdout:9/186: stat d4/d10/d12/d27/d32 0 2024-04-02T04:47:41.010 INFO:tasks.workunit.client.1.smithi179.stdout:7/173: unlink d1/d9/c31 0 2024-04-02T04:47:41.010 INFO:tasks.workunit.client.0.smithi070.stdout:2/245: mknod d2/d10/d53/c54 0 2024-04-02T04:47:41.011 INFO:tasks.workunit.client.0.smithi070.stdout:2/246: fdatasync d2/f32 0 2024-04-02T04:47:41.012 INFO:tasks.workunit.client.0.smithi070.stdout:7/153: symlink d7/l2c 0 2024-04-02T04:47:41.015 INFO:tasks.workunit.client.0.smithi070.stdout:4/139: creat d1/d3/f2b x:0 0 0 2024-04-02T04:47:41.016 INFO:tasks.workunit.client.0.smithi070.stdout:2/247: read d2/d12/f13 [1351413,45118] 0 2024-04-02T04:47:41.018 INFO:tasks.workunit.client.1.smithi179.stdout:7/174: dwrite d1/d9/f12 [0,4194304] 0 2024-04-02T04:47:41.019 INFO:tasks.workunit.client.1.smithi179.stdout:8/133: rename d3/d12/d1e to d3/d31 0 2024-04-02T04:47:41.020 INFO:tasks.workunit.client.0.smithi070.stdout:9/187: symlink d4/d13/l3e 0 2024-04-02T04:47:41.021 INFO:tasks.workunit.client.1.smithi179.stdout:8/134: write d3/f16 [1999518,96927] 0 2024-04-02T04:47:41.025 INFO:tasks.workunit.client.1.smithi179.stdout:7/175: dread d1/d9/fb [0,4194304] 0 2024-04-02T04:47:41.025 INFO:tasks.workunit.client.1.smithi179.stdout:7/176: chown d1 132477482 1 2024-04-02T04:47:41.025 INFO:tasks.workunit.client.0.smithi070.stdout:0/295: dwrite d4/d5/f3f [0,4194304] 0 2024-04-02T04:47:41.027 INFO:tasks.workunit.client.0.smithi070.stdout:7/154: chown d7/d9/f17 7322749 1 2024-04-02T04:47:41.028 INFO:tasks.workunit.client.0.smithi070.stdout:7/155: chown d7/d1c 28585099 1 2024-04-02T04:47:41.030 INFO:tasks.workunit.client.1.smithi179.stdout:6/106: mkdir d3/d2d 0 2024-04-02T04:47:41.030 INFO:tasks.workunit.client.1.smithi179.stdout:5/71: rmdir d8 39 2024-04-02T04:47:41.032 INFO:tasks.workunit.client.0.smithi070.stdout:4/140: chown d1/d3/l6 61 1 2024-04-02T04:47:41.033 INFO:tasks.workunit.client.0.smithi070.stdout:0/296: dwrite f0 [0,4194304] 0 2024-04-02T04:47:41.034 INFO:tasks.workunit.client.0.smithi070.stdout:4/141: truncate d1/f28 1382886 0 2024-04-02T04:47:41.034 INFO:tasks.workunit.client.0.smithi070.stdout:2/248: creat d2/d10/d53/f55 x:0 0 0 2024-04-02T04:47:41.038 INFO:tasks.workunit.client.1.smithi179.stdout:4/178: symlink d1/l3b 0 2024-04-02T04:47:41.039 INFO:tasks.workunit.client.0.smithi070.stdout:5/158: getdents d5 0 2024-04-02T04:47:41.039 INFO:tasks.workunit.client.1.smithi179.stdout:3/100: write d8/f11 [2488036,7857] 0 2024-04-02T04:47:41.040 INFO:tasks.workunit.client.0.smithi070.stdout:5/159: write d5/fa [2436531,68306] 0 2024-04-02T04:47:41.042 INFO:tasks.workunit.client.0.smithi070.stdout:2/249: dwrite d2/d12/f38 [0,4194304] 0 2024-04-02T04:47:41.042 INFO:tasks.workunit.client.0.smithi070.stdout:9/188: unlink d4/d13/d16/d19/d2f/d39/l3a 0 2024-04-02T04:47:41.043 INFO:tasks.workunit.client.0.smithi070.stdout:2/250: readlink d2/l15 0 2024-04-02T04:47:41.046 INFO:tasks.workunit.client.0.smithi070.stdout:9/189: dread d4/d10/f18 [4194304,4194304] 0 2024-04-02T04:47:41.048 INFO:tasks.workunit.client.0.smithi070.stdout:9/190: write d4/d13/d16/d19/d2f/f3c [3557642,126519] 0 2024-04-02T04:47:41.049 INFO:tasks.workunit.client.1.smithi179.stdout:0/214: link d1/f22 d1/f3b 0 2024-04-02T04:47:41.051 INFO:tasks.workunit.client.1.smithi179.stdout:8/135: rmdir d3/d12/d21/d22 39 2024-04-02T04:47:41.052 INFO:tasks.workunit.client.0.smithi070.stdout:0/297: unlink d4/d5/c3b 0 2024-04-02T04:47:41.054 INFO:tasks.workunit.client.1.smithi179.stdout:7/177: mknod d1/d9/d35/c3f 0 2024-04-02T04:47:41.056 INFO:tasks.workunit.client.1.smithi179.stdout:8/136: dread d3/d1b/f23 [0,4194304] 0 2024-04-02T04:47:41.056 INFO:tasks.workunit.client.1.smithi179.stdout:6/107: rename d3/f24 to d3/d2d/f2e 0 2024-04-02T04:47:41.057 INFO:tasks.workunit.client.1.smithi179.stdout:9/162: write d0/d1/d12/fe [1405105,108652] 0 2024-04-02T04:47:41.060 INFO:tasks.workunit.client.1.smithi179.stdout:3/101: rmdir d8 39 2024-04-02T04:47:41.063 INFO:tasks.workunit.client.0.smithi070.stdout:2/251: symlink d2/d12/d48/l56 0 2024-04-02T04:47:41.065 INFO:tasks.workunit.client.0.smithi070.stdout:2/252: read d2/d10/d19/f39 [3588153,89744] 0 2024-04-02T04:47:41.065 INFO:tasks.workunit.client.0.smithi070.stdout:2/253: fdatasync d2/d24/f37 0 2024-04-02T04:47:41.067 INFO:tasks.workunit.client.0.smithi070.stdout:7/156: creat d7/d1c/d29/f2d x:0 0 0 2024-04-02T04:47:41.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:40 smithi179 ceph-mon[36603]: pgmap v9: 65 pgs: 65 active+clean; 518 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 915 KiB/s rd, 56 MiB/s wr, 252 op/s 2024-04-02T04:47:41.071 INFO:tasks.workunit.client.1.smithi179.stdout:8/137: rename d3/d12 to d3/d32 0 2024-04-02T04:47:41.074 INFO:tasks.workunit.client.1.smithi179.stdout:6/108: mkdir d3/d12/d23/d2f 0 2024-04-02T04:47:41.077 INFO:tasks.workunit.client.0.smithi070.stdout:2/254: mknod d2/d24/c57 0 2024-04-02T04:47:41.083 INFO:tasks.workunit.client.1.smithi179.stdout:8/138: stat d3/c8 0 2024-04-02T04:47:41.085 INFO:tasks.workunit.client.1.smithi179.stdout:6/109: creat d3/d12/d23/f30 x:0 0 0 2024-04-02T04:47:41.086 INFO:tasks.workunit.client.1.smithi179.stdout:5/72: write f1 [833748,67163] 0 2024-04-02T04:47:41.086 INFO:tasks.workunit.client.1.smithi179.stdout:6/110: dread - d3/d12/d23/d26/f27 zero size 2024-04-02T04:47:41.088 INFO:tasks.workunit.client.1.smithi179.stdout:3/102: creat d8/f1e x:0 0 0 2024-04-02T04:47:41.089 INFO:tasks.workunit.client.1.smithi179.stdout:0/215: write d1/fe [1762859,111702] 0 2024-04-02T04:47:41.089 INFO:tasks.workunit.client.1.smithi179.stdout:3/103: dread - d8/f1b zero size 2024-04-02T04:47:41.091 INFO:tasks.workunit.client.1.smithi179.stdout:4/179: rename d1/d3/d19/f38 to d1/f3c 0 2024-04-02T04:47:41.092 INFO:tasks.workunit.client.1.smithi179.stdout:6/111: unlink d3/d2d/f2e 0 2024-04-02T04:47:41.092 INFO:tasks.workunit.client.1.smithi179.stdout:9/163: link d0/d1/d12/l1f d0/d1/d12/l30 0 2024-04-02T04:47:41.093 INFO:tasks.workunit.client.1.smithi179.stdout:9/164: readlink d0/d1/d10/l28 0 2024-04-02T04:47:41.093 INFO:tasks.workunit.client.1.smithi179.stdout:8/139: mkdir d3/d32/d21/d22/d33 0 2024-04-02T04:47:41.094 INFO:tasks.workunit.client.1.smithi179.stdout:3/104: unlink d8/f1e 0 2024-04-02T04:47:41.095 INFO:tasks.workunit.client.1.smithi179.stdout:3/105: chown c7 260394 1 2024-04-02T04:47:41.095 INFO:tasks.workunit.client.1.smithi179.stdout:3/106: write d8/f1b [315629,38327] 0 2024-04-02T04:47:41.096 INFO:tasks.workunit.client.1.smithi179.stdout:5/73: rename d8/le to d8/l16 0 2024-04-02T04:47:41.096 INFO:tasks.workunit.client.1.smithi179.stdout:4/180: symlink d1/d3/df/d10/d2f/l3d 0 2024-04-02T04:47:41.097 INFO:tasks.workunit.client.1.smithi179.stdout:4/181: chown d1/d3/df/d10/f2d 1 1 2024-04-02T04:47:41.099 INFO:tasks.workunit.client.1.smithi179.stdout:3/107: symlink d8/l1f 0 2024-04-02T04:47:41.099 INFO:tasks.workunit.client.1.smithi179.stdout:5/74: symlink d8/l17 0 2024-04-02T04:47:41.101 INFO:tasks.workunit.client.1.smithi179.stdout:9/165: link d0/d1/f26 d0/d19/d1c/f31 0 2024-04-02T04:47:41.102 INFO:tasks.workunit.client.1.smithi179.stdout:8/140: creat d3/d32/d21/d22/d33/f34 x:0 0 0 2024-04-02T04:47:41.103 INFO:tasks.workunit.client.1.smithi179.stdout:8/141: dread - d3/d32/d21/f25 zero size 2024-04-02T04:47:41.103 INFO:tasks.workunit.client.1.smithi179.stdout:7/178: rename d1/d9/d35/c3f to d1/c40 0 2024-04-02T04:47:41.104 INFO:tasks.workunit.client.1.smithi179.stdout:3/108: creat d8/f20 x:0 0 0 2024-04-02T04:47:41.104 INFO:tasks.workunit.client.1.smithi179.stdout:7/179: write d1/dd/d23/f25 [1142841,8186] 0 2024-04-02T04:47:41.105 INFO:tasks.workunit.client.1.smithi179.stdout:5/75: rmdir d8 39 2024-04-02T04:47:41.105 INFO:tasks.workunit.client.1.smithi179.stdout:7/180: write d1/d9/d35/f3b [787017,85334] 0 2024-04-02T04:47:41.112 INFO:tasks.workunit.client.1.smithi179.stdout:2/81: sync 2024-04-02T04:47:41.113 INFO:tasks.workunit.client.1.smithi179.stdout:1/199: sync 2024-04-02T04:47:41.113 INFO:tasks.workunit.client.0.smithi070.stdout:4/142: creat d1/d3/f2c x:0 0 0 2024-04-02T04:47:41.113 INFO:tasks.workunit.client.1.smithi179.stdout:2/82: chown dc 1691 1 2024-04-02T04:47:41.115 INFO:tasks.workunit.client.1.smithi179.stdout:5/76: creat d8/f18 x:0 0 0 2024-04-02T04:47:41.118 INFO:tasks.workunit.client.1.smithi179.stdout:2/83: dwrite f9 [4194304,4194304] 0 2024-04-02T04:47:41.124 INFO:tasks.workunit.client.0.smithi070.stdout:2/255: dwrite d2/d10/f20 [0,4194304] 0 2024-04-02T04:47:41.125 INFO:tasks.workunit.client.0.smithi070.stdout:5/160: dwrite d5/d14/f16 [0,4194304] 0 2024-04-02T04:47:41.129 INFO:tasks.workunit.client.0.smithi070.stdout:5/161: readlink d5/l21 0 2024-04-02T04:47:41.131 INFO:tasks.workunit.client.0.smithi070.stdout:0/298: dwrite d4/f25 [0,4194304] 0 2024-04-02T04:47:41.133 INFO:tasks.workunit.client.1.smithi179.stdout:0/216: rename d1/fe to d1/da/f3c 0 2024-04-02T04:47:41.134 INFO:tasks.workunit.client.0.smithi070.stdout:7/157: dwrite d7/d9/f17 [0,4194304] 0 2024-04-02T04:47:41.134 INFO:tasks.workunit.client.0.smithi070.stdout:5/162: stat d5/d17/d27/d2c 0 2024-04-02T04:47:41.134 INFO:tasks.workunit.client.1.smithi179.stdout:8/142: rename d3/d1b to d3/d1b/d2a/d35 22 2024-04-02T04:47:41.135 INFO:tasks.workunit.client.1.smithi179.stdout:1/200: rmdir d3/d5/d2f/d36 39 2024-04-02T04:47:41.135 INFO:tasks.workunit.client.1.smithi179.stdout:4/182: write d1/d3/f7 [289629,25656] 0 2024-04-02T04:47:41.136 INFO:tasks.workunit.client.0.smithi070.stdout:5/163: fsync f2 0 2024-04-02T04:47:41.136 INFO:tasks.workunit.client.0.smithi070.stdout:4/143: dwrite d1/fe [0,4194304] 0 2024-04-02T04:47:41.137 INFO:tasks.workunit.client.1.smithi179.stdout:5/77: creat d8/f19 x:0 0 0 2024-04-02T04:47:41.140 INFO:tasks.workunit.client.0.smithi070.stdout:4/144: write d1/f1e [2716225,42452] 0 2024-04-02T04:47:41.141 INFO:tasks.workunit.client.1.smithi179.stdout:9/166: getdents d0/d1/d10 0 2024-04-02T04:47:41.142 INFO:tasks.workunit.client.0.smithi070.stdout:4/145: truncate d1/d3/d9/f2a 40342 0 2024-04-02T04:47:41.142 INFO:tasks.workunit.client.0.smithi070.stdout:0/299: creat d4/d5/d13/d23/f4d x:0 0 0 2024-04-02T04:47:41.142 INFO:tasks.workunit.client.1.smithi179.stdout:6/112: dwrite d3/fb [0,4194304] 0 2024-04-02T04:47:41.143 INFO:tasks.workunit.client.1.smithi179.stdout:3/109: rmdir d8 39 2024-04-02T04:47:41.143 INFO:tasks.workunit.client.1.smithi179.stdout:0/217: fdatasync d1/f22 0 2024-04-02T04:47:41.146 INFO:tasks.workunit.client.0.smithi070.stdout:2/256: fsync d2/f1c 0 2024-04-02T04:47:41.151 INFO:tasks.workunit.client.0.smithi070.stdout:4/146: dwrite d1/d3/f2c [0,4194304] 0 2024-04-02T04:47:41.151 INFO:tasks.workunit.client.1.smithi179.stdout:5/78: rename d8/f19 to d8/f1a 0 2024-04-02T04:47:41.154 INFO:tasks.workunit.client.0.smithi070.stdout:0/300: symlink d4/d5/d13/d23/l4e 0 2024-04-02T04:47:41.157 INFO:tasks.workunit.client.0.smithi070.stdout:5/164: getdents d5/d17/d27/d2c/d35 0 2024-04-02T04:47:41.158 INFO:tasks.workunit.client.1.smithi179.stdout:9/167: dwrite d0/d1/d10/f2b [0,4194304] 0 2024-04-02T04:47:41.158 INFO:tasks.workunit.client.1.smithi179.stdout:4/183: dread d1/f5 [0,4194304] 0 2024-04-02T04:47:41.158 INFO:tasks.workunit.client.1.smithi179.stdout:9/168: stat d0/l24 0 2024-04-02T04:47:41.161 INFO:tasks.workunit.client.1.smithi179.stdout:4/184: write d1/d3/dc/f18 [996780,112533] 0 2024-04-02T04:47:41.161 INFO:tasks.workunit.client.0.smithi070.stdout:2/257: truncate d2/d12/f13 3976791 0 2024-04-02T04:47:41.162 INFO:tasks.workunit.client.1.smithi179.stdout:8/143: unlink d3/d32/d21/c2f 0 2024-04-02T04:47:41.164 INFO:tasks.workunit.client.1.smithi179.stdout:8/144: chown d3/d32/l20 924717 1 2024-04-02T04:47:41.164 INFO:tasks.workunit.client.1.smithi179.stdout:4/185: truncate d1/d3/dc/f1c 844478 0 2024-04-02T04:47:41.164 INFO:tasks.workunit.client.0.smithi070.stdout:0/301: read d4/d5/d13/f2b [2163416,9790] 0 2024-04-02T04:47:41.165 INFO:tasks.workunit.client.0.smithi070.stdout:5/165: symlink d5/d17/l36 0 2024-04-02T04:47:41.165 INFO:tasks.workunit.client.0.smithi070.stdout:5/166: readlink d5/d17/l20 0 2024-04-02T04:47:41.167 INFO:tasks.workunit.client.1.smithi179.stdout:5/79: write d8/f1a [521107,1847] 0 2024-04-02T04:47:41.169 INFO:tasks.workunit.client.1.smithi179.stdout:3/110: mknod d8/c21 0 2024-04-02T04:47:41.172 INFO:tasks.workunit.client.1.smithi179.stdout:2/84: write dc/fe [1280498,105105] 0 2024-04-02T04:47:41.172 INFO:tasks.workunit.client.1.smithi179.stdout:2/85: stat dc/f17 0 2024-04-02T04:47:41.173 INFO:tasks.workunit.client.1.smithi179.stdout:2/86: chown f6 17341326 1 2024-04-02T04:47:41.174 INFO:tasks.workunit.client.0.smithi070.stdout:5/167: creat d5/d17/d27/d2c/f37 x:0 0 0 2024-04-02T04:47:41.175 INFO:tasks.workunit.client.1.smithi179.stdout:0/218: dread d1/da/df/f11 [0,4194304] 0 2024-04-02T04:47:41.177 INFO:tasks.workunit.client.0.smithi070.stdout:4/147: rename d1/f21 to d1/f2d 0 2024-04-02T04:47:41.178 INFO:tasks.workunit.client.0.smithi070.stdout:7/158: dwrite d7/dd/df/f10 [0,4194304] 0 2024-04-02T04:47:41.179 INFO:tasks.workunit.client.1.smithi179.stdout:9/169: creat d0/d1/d11/f32 x:0 0 0 2024-04-02T04:47:41.179 INFO:tasks.workunit.client.0.smithi070.stdout:7/159: chown d7/l2c 179 1 2024-04-02T04:47:41.180 INFO:tasks.workunit.client.0.smithi070.stdout:7/160: chown d7/d9/f27 6 1 2024-04-02T04:47:41.181 INFO:tasks.workunit.client.0.smithi070.stdout:0/302: rename d4/d5/d13/d23/l31 to d4/d5/d13/d2a/d29/l4f 0 2024-04-02T04:47:41.181 INFO:tasks.workunit.client.0.smithi070.stdout:7/161: write d7/dd/df/f10 [3085378,114480] 0 2024-04-02T04:47:41.182 INFO:tasks.workunit.client.1.smithi179.stdout:1/201: creat d3/d5/d2f/f47 x:0 0 0 2024-04-02T04:47:41.184 INFO:tasks.workunit.client.0.smithi070.stdout:4/148: dwrite d1/d3/f2b [0,4194304] 0 2024-04-02T04:47:41.186 INFO:tasks.workunit.client.1.smithi179.stdout:5/80: mkdir d8/d1b 0 2024-04-02T04:47:41.186 INFO:tasks.workunit.client.1.smithi179.stdout:5/81: fsync f6 0 2024-04-02T04:47:41.188 INFO:tasks.workunit.client.1.smithi179.stdout:5/82: dread f6 [0,4194304] 0 2024-04-02T04:47:41.189 INFO:tasks.workunit.client.1.smithi179.stdout:2/87: mknod dc/c1b 0 2024-04-02T04:47:41.190 INFO:tasks.workunit.client.0.smithi070.stdout:5/168: creat d5/d17/d27/d2c/d35/f38 x:0 0 0 2024-04-02T04:47:41.191 INFO:tasks.workunit.client.0.smithi070.stdout:5/169: write d5/f22 [4955951,104823] 0 2024-04-02T04:47:41.192 INFO:tasks.workunit.client.1.smithi179.stdout:0/219: chown d1/da/d28/d34/l38 8868656 1 2024-04-02T04:47:41.193 INFO:tasks.workunit.client.1.smithi179.stdout:9/170: mknod d0/d1/d10/c33 0 2024-04-02T04:47:41.194 INFO:tasks.workunit.client.0.smithi070.stdout:4/149: rmdir d1/d3 39 2024-04-02T04:47:41.196 INFO:tasks.workunit.client.1.smithi179.stdout:1/202: rename d3/d5/de/c15 to d3/d5/de/d33/c48 0 2024-04-02T04:47:41.197 INFO:tasks.workunit.client.1.smithi179.stdout:5/83: dwrite d8/fb [0,4194304] 0 2024-04-02T04:47:41.197 INFO:tasks.workunit.client.1.smithi179.stdout:9/171: write d0/d1/f4 [3493241,82919] 0 2024-04-02T04:47:41.198 INFO:tasks.workunit.client.1.smithi179.stdout:0/220: dread d1/f22 [0,4194304] 0 2024-04-02T04:47:41.198 INFO:tasks.workunit.client.1.smithi179.stdout:3/111: link d8/f1d d8/f22 0 2024-04-02T04:47:41.199 INFO:tasks.workunit.client.1.smithi179.stdout:1/203: write d3/d5/de/f3c [369407,20754] 0 2024-04-02T04:47:41.200 INFO:tasks.workunit.client.0.smithi070.stdout:7/162: rename d7/f1e to d7/dd/f2e 0 2024-04-02T04:47:41.201 INFO:tasks.workunit.client.0.smithi070.stdout:7/163: dread - d7/d9/d11/f2a zero size 2024-04-02T04:47:41.203 INFO:tasks.workunit.client.1.smithi179.stdout:1/204: write d3/d5/de/f3c [85372,25409] 0 2024-04-02T04:47:41.203 INFO:tasks.workunit.client.0.smithi070.stdout:7/164: dread d7/d9/f27 [0,4194304] 0 2024-04-02T04:47:41.207 INFO:tasks.workunit.client.0.smithi070.stdout:2/258: write d2/f14 [1051769,23426] 0 2024-04-02T04:47:41.209 INFO:tasks.workunit.client.1.smithi179.stdout:6/113: truncate d3/f1c 3381428 0 2024-04-02T04:47:41.210 INFO:tasks.workunit.client.1.smithi179.stdout:0/221: dread d1/da/f27 [0,4194304] 0 2024-04-02T04:47:41.211 INFO:tasks.workunit.client.1.smithi179.stdout:6/114: fsync d3/d12/f25 0 2024-04-02T04:47:41.212 INFO:tasks.workunit.client.1.smithi179.stdout:3/112: dwrite d8/f20 [0,4194304] 0 2024-04-02T04:47:41.213 INFO:tasks.workunit.client.0.smithi070.stdout:2/259: truncate d2/fa 654298 0 2024-04-02T04:47:41.217 INFO:tasks.workunit.client.1.smithi179.stdout:5/84: mknod d8/d1b/c1c 0 2024-04-02T04:47:41.217 INFO:tasks.workunit.client.1.smithi179.stdout:1/205: dwrite d3/f2d [0,4194304] 0 2024-04-02T04:47:41.218 INFO:tasks.workunit.client.0.smithi070.stdout:2/260: symlink d2/l58 0 2024-04-02T04:47:41.219 INFO:tasks.workunit.client.0.smithi070.stdout:2/261: chown d2 72966954 1 2024-04-02T04:47:41.221 INFO:tasks.workunit.client.1.smithi179.stdout:9/172: creat d0/d1/d11/d29/f34 x:0 0 0 2024-04-02T04:47:41.224 INFO:tasks.workunit.client.1.smithi179.stdout:0/222: mknod d1/db/c3d 0 2024-04-02T04:47:41.224 INFO:tasks.workunit.client.1.smithi179.stdout:4/186: dwrite d1/f5 [0,4194304] 0 2024-04-02T04:47:41.225 INFO:tasks.workunit.client.1.smithi179.stdout:5/85: dread d8/fb [0,4194304] 0 2024-04-02T04:47:41.226 INFO:tasks.workunit.client.1.smithi179.stdout:6/115: unlink d3/c13 0 2024-04-02T04:47:41.227 INFO:tasks.workunit.client.0.smithi070.stdout:2/262: dwrite d2/d12/f52 [0,4194304] 0 2024-04-02T04:47:41.228 INFO:tasks.workunit.client.1.smithi179.stdout:0/223: dread d1/da/d28/f29 [0,4194304] 0 2024-04-02T04:47:41.231 INFO:tasks.workunit.client.1.smithi179.stdout:3/113: rename d8/f20 to d8/f23 0 2024-04-02T04:47:41.237 INFO:tasks.workunit.client.0.smithi070.stdout:2/263: dwrite d2/d24/f2d [0,4194304] 0 2024-04-02T04:47:41.245 INFO:tasks.workunit.client.0.smithi070.stdout:5/170: truncate d5/fa 793968 0 2024-04-02T04:47:41.246 INFO:tasks.workunit.client.1.smithi179.stdout:1/206: mkdir d3/d31/d35/d49 0 2024-04-02T04:47:41.246 INFO:tasks.workunit.client.1.smithi179.stdout:1/207: stat d3/d31/d35/d49 0 2024-04-02T04:47:41.247 INFO:tasks.workunit.client.1.smithi179.stdout:1/208: chown d3/d5/de/c29 712 1 2024-04-02T04:47:41.248 INFO:tasks.workunit.client.0.smithi070.stdout:2/264: dread d2/f14 [0,4194304] 0 2024-04-02T04:47:41.249 INFO:tasks.workunit.client.0.smithi070.stdout:2/265: truncate d2/d10/d21/d3d/d42/f4b 4507738 0 2024-04-02T04:47:41.249 INFO:tasks.workunit.client.0.smithi070.stdout:2/266: readlink d2/l3a 0 2024-04-02T04:47:41.250 INFO:tasks.workunit.client.0.smithi070.stdout:2/267: fdatasync d2/d10/f20 0 2024-04-02T04:47:41.250 INFO:tasks.workunit.client.0.smithi070.stdout:5/171: rename d5/c34 to d5/d17/d27/c39 0 2024-04-02T04:47:41.254 INFO:tasks.workunit.client.1.smithi179.stdout:4/187: mkdir d1/d3/df/d10/d3e 0 2024-04-02T04:47:41.255 INFO:tasks.workunit.client.1.smithi179.stdout:2/88: truncate f6 1418078 0 2024-04-02T04:47:41.257 INFO:tasks.workunit.client.1.smithi179.stdout:5/86: creat d8/f1d x:0 0 0 2024-04-02T04:47:41.258 INFO:tasks.workunit.client.1.smithi179.stdout:5/87: chown d8/f12 1030545 1 2024-04-02T04:47:41.268 INFO:tasks.workunit.client.0.smithi070.stdout:2/268: getdents d2/d10/d53 0 2024-04-02T04:47:41.269 INFO:tasks.workunit.client.1.smithi179.stdout:6/116: getdents d3/d2c 0 2024-04-02T04:47:41.269 INFO:tasks.workunit.client.1.smithi179.stdout:5/88: write d8/fb [2529840,31825] 0 2024-04-02T04:47:41.270 INFO:tasks.workunit.client.1.smithi179.stdout:5/89: readlink d8/l11 0 2024-04-02T04:47:41.270 INFO:tasks.workunit.client.1.smithi179.stdout:0/224: creat d1/da/d28/d37/f3e x:0 0 0 2024-04-02T04:47:41.271 INFO:tasks.workunit.client.1.smithi179.stdout:5/90: dread - d8/f10 zero size 2024-04-02T04:47:41.272 INFO:tasks.workunit.client.1.smithi179.stdout:9/173: link d0/c9 d0/d1/d11/c35 0 2024-04-02T04:47:41.273 INFO:tasks.workunit.client.1.smithi179.stdout:4/188: symlink d1/d3/df/d10/d3a/l3f 0 2024-04-02T04:47:41.274 INFO:tasks.workunit.client.1.smithi179.stdout:4/189: readlink d1/d3/d19/l33 0 2024-04-02T04:47:41.274 INFO:tasks.workunit.client.1.smithi179.stdout:6/117: dread d3/fa [0,4194304] 0 2024-04-02T04:47:41.276 INFO:tasks.workunit.client.1.smithi179.stdout:5/91: mknod d8/c1e 0 2024-04-02T04:47:41.276 INFO:tasks.workunit.client.1.smithi179.stdout:4/190: truncate d1/d3/df/d10/f11 4279962 0 2024-04-02T04:47:41.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:40 smithi070 ceph-mon[29212]: pgmap v9: 65 pgs: 65 active+clean; 518 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 915 KiB/s rd, 56 MiB/s wr, 252 op/s 2024-04-02T04:47:41.280 INFO:tasks.workunit.client.0.smithi070.stdout:2/269: dread d2/d10/d19/f39 [0,4194304] 0 2024-04-02T04:47:41.280 INFO:tasks.workunit.client.1.smithi179.stdout:0/225: link d1/da/df/f11 d1/da/d28/d37/f3f 0 2024-04-02T04:47:41.280 INFO:tasks.workunit.client.1.smithi179.stdout:0/226: fdatasync d1/f2a 0 2024-04-02T04:47:41.281 INFO:tasks.workunit.client.0.smithi070.stdout:2/270: write d2/d10/d21/f33 [617795,89380] 0 2024-04-02T04:47:41.281 INFO:tasks.workunit.client.1.smithi179.stdout:0/227: rename d1/da/df to d1/da/df/d40 22 2024-04-02T04:47:41.284 INFO:tasks.workunit.client.1.smithi179.stdout:4/191: symlink d1/d15/l40 0 2024-04-02T04:47:41.285 INFO:tasks.workunit.client.1.smithi179.stdout:2/89: read f6 [71637,48076] 0 2024-04-02T04:47:41.288 INFO:tasks.workunit.client.1.smithi179.stdout:0/228: rename d1/da/d28/f29 to d1/da/d28/f41 0 2024-04-02T04:47:41.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/271: readlink d2/d10/d21/d3d/d42/l4f 0 2024-04-02T04:47:41.292 INFO:tasks.workunit.client.0.smithi070.stdout:5/172: unlink d5/d17/d2b/l31 0 2024-04-02T04:47:41.294 INFO:tasks.workunit.client.1.smithi179.stdout:3/114: dwrite d8/f22 [0,4194304] 0 2024-04-02T04:47:41.295 INFO:tasks.workunit.client.0.smithi070.stdout:2/272: mkdir d2/d12/d59 0 2024-04-02T04:47:41.295 INFO:tasks.workunit.client.1.smithi179.stdout:4/192: write d1/f3c [713900,94516] 0 2024-04-02T04:47:41.297 INFO:tasks.workunit.client.0.smithi070.stdout:5/173: creat d5/d17/d2b/f3a x:0 0 0 2024-04-02T04:47:41.298 INFO:tasks.workunit.client.1.smithi179.stdout:2/90: symlink dc/d16/l1c 0 2024-04-02T04:47:41.299 INFO:tasks.workunit.client.0.smithi070.stdout:2/273: dread d2/d10/d19/f1e [0,4194304] 0 2024-04-02T04:47:41.299 INFO:tasks.workunit.client.1.smithi179.stdout:2/91: write dc/f12 [1716170,50958] 0 2024-04-02T04:47:41.300 INFO:tasks.workunit.client.1.smithi179.stdout:4/193: dread d1/d3/dc/fd [0,4194304] 0 2024-04-02T04:47:41.300 INFO:tasks.workunit.client.1.smithi179.stdout:2/92: chown dc/fe 336530 1 2024-04-02T04:47:41.300 INFO:tasks.workunit.client.0.smithi070.stdout:2/274: write d2/d10/d53/f55 [508716,83309] 0 2024-04-02T04:47:41.302 INFO:tasks.workunit.client.0.smithi070.stdout:2/275: write d2/d10/d21/d3d/f47 [42537,88336] 0 2024-04-02T04:47:41.303 INFO:tasks.workunit.client.1.smithi179.stdout:9/174: write d0/d19/d1c/f31 [1758686,32090] 0 2024-04-02T04:47:41.304 INFO:tasks.workunit.client.1.smithi179.stdout:3/115: mknod d8/c24 0 2024-04-02T04:47:41.308 INFO:tasks.workunit.client.0.smithi070.stdout:5/174: dwrite d5/f6 [0,4194304] 0 2024-04-02T04:47:41.309 INFO:tasks.workunit.client.0.smithi070.stdout:5/175: read - d5/d17/d27/d2c/f37 zero size 2024-04-02T04:47:41.310 INFO:tasks.workunit.client.0.smithi070.stdout:2/276: fsync d2/fb 0 2024-04-02T04:47:41.311 INFO:tasks.workunit.client.1.smithi179.stdout:5/92: truncate f2 7338237 0 2024-04-02T04:47:41.312 INFO:tasks.workunit.client.1.smithi179.stdout:5/93: fsync d8/fb 0 2024-04-02T04:47:41.314 INFO:tasks.workunit.client.1.smithi179.stdout:4/194: rename d1/d3/df/d10/l17 to d1/d15/l41 0 2024-04-02T04:47:41.315 INFO:tasks.workunit.client.1.smithi179.stdout:4/195: chown d1/d3/dc/d27/l22 6954198 1 2024-04-02T04:47:41.317 INFO:tasks.workunit.client.1.smithi179.stdout:3/116: mknod d8/c25 0 2024-04-02T04:47:41.318 INFO:tasks.workunit.client.1.smithi179.stdout:3/117: write f5 [6494124,72566] 0 2024-04-02T04:47:41.319 INFO:tasks.workunit.client.1.smithi179.stdout:3/118: chown d8/l9 24867 1 2024-04-02T04:47:41.320 INFO:tasks.workunit.client.0.smithi070.stdout:2/277: mknod d2/d10/d21/d3d/d42/c5a 0 2024-04-02T04:47:41.322 INFO:tasks.workunit.client.1.smithi179.stdout:5/94: creat d8/d1b/f1f x:0 0 0 2024-04-02T04:47:41.325 INFO:tasks.workunit.client.1.smithi179.stdout:4/196: unlink d1/d3/df/d10/c1d 0 2024-04-02T04:47:41.326 INFO:tasks.workunit.client.1.smithi179.stdout:4/197: write d1/d15/f31 [565649,34303] 0 2024-04-02T04:47:41.328 INFO:tasks.workunit.client.0.smithi070.stdout:5/176: dwrite d5/d9/d10/f19 [0,4194304] 0 2024-04-02T04:47:41.329 INFO:tasks.workunit.client.1.smithi179.stdout:3/119: mkdir d8/d26 0 2024-04-02T04:47:41.331 INFO:tasks.workunit.client.0.smithi070.stdout:5/177: rmdir d5/d17/d27 39 2024-04-02T04:47:41.332 INFO:tasks.workunit.client.0.smithi070.stdout:5/178: write d5/f2f [4812074,95691] 0 2024-04-02T04:47:41.333 INFO:tasks.workunit.client.1.smithi179.stdout:9/175: creat d0/d1/f36 x:0 0 0 2024-04-02T04:47:41.335 INFO:tasks.workunit.client.1.smithi179.stdout:4/198: chown d1/d3/f28 42 1 2024-04-02T04:47:41.336 INFO:tasks.workunit.client.1.smithi179.stdout:2/93: getdents dc 0 2024-04-02T04:47:41.337 INFO:tasks.workunit.client.0.smithi070.stdout:5/179: dread d5/f1d [0,4194304] 0 2024-04-02T04:47:41.338 INFO:tasks.workunit.client.1.smithi179.stdout:3/120: write d8/f1c [751042,25094] 0 2024-04-02T04:47:41.339 INFO:tasks.workunit.client.1.smithi179.stdout:9/176: creat d0/d1/d11/d29/f37 x:0 0 0 2024-04-02T04:47:41.340 INFO:tasks.workunit.client.1.smithi179.stdout:4/199: readlink d1/d15/l41 0 2024-04-02T04:47:41.341 INFO:tasks.workunit.client.1.smithi179.stdout:4/200: fdatasync d1/d3/dc/fe 0 2024-04-02T04:47:41.341 INFO:tasks.workunit.client.0.smithi070.stdout:2/278: write d2/fa [1681212,7299] 0 2024-04-02T04:47:41.343 INFO:tasks.workunit.client.0.smithi070.stdout:2/279: write d2/d12/f52 [4381337,108099] 0 2024-04-02T04:47:41.344 INFO:tasks.workunit.client.1.smithi179.stdout:3/121: dread d8/f1c [0,4194304] 0 2024-04-02T04:47:41.345 INFO:tasks.workunit.client.0.smithi070.stdout:2/280: dread d2/d10/d21/d3d/f47 [0,4194304] 0 2024-04-02T04:47:41.346 INFO:tasks.workunit.client.1.smithi179.stdout:5/95: getdents d8 0 2024-04-02T04:47:41.346 INFO:tasks.workunit.client.1.smithi179.stdout:5/96: readlink l4 0 2024-04-02T04:47:41.348 INFO:tasks.workunit.client.1.smithi179.stdout:9/177: dwrite d0/d1/d11/d29/f37 [0,4194304] 0 2024-04-02T04:47:41.349 INFO:tasks.workunit.client.0.smithi070.stdout:5/180: creat d5/d17/d27/d2e/f3b x:0 0 0 2024-04-02T04:47:41.349 INFO:tasks.workunit.client.0.smithi070.stdout:5/181: chown d5/d17/d27/f28 199 1 2024-04-02T04:47:41.350 INFO:tasks.workunit.client.0.smithi070.stdout:2/281: mknod d2/d10/d21/d3d/c5b 0 2024-04-02T04:47:41.351 INFO:tasks.workunit.client.1.smithi179.stdout:9/178: write d0/d19/d1c/f31 [399779,119941] 0 2024-04-02T04:47:41.352 INFO:tasks.workunit.client.1.smithi179.stdout:3/122: mknod d8/c27 0 2024-04-02T04:47:41.355 INFO:tasks.workunit.client.1.smithi179.stdout:3/123: read d8/f11 [3425714,9494] 0 2024-04-02T04:47:41.355 INFO:tasks.workunit.client.1.smithi179.stdout:4/201: dwrite d1/d3/d19/f21 [0,4194304] 0 2024-04-02T04:47:41.356 INFO:tasks.workunit.client.1.smithi179.stdout:3/124: chown d8/l12 328327 1 2024-04-02T04:47:41.356 INFO:tasks.workunit.client.0.smithi070.stdout:2/282: mkdir d2/d10/d5c 0 2024-04-02T04:47:41.357 INFO:tasks.workunit.client.1.smithi179.stdout:3/125: chown d8/f1d 1425953 1 2024-04-02T04:47:41.357 INFO:tasks.workunit.client.1.smithi179.stdout:4/202: truncate d1/d3/d19/f2c 382909 0 2024-04-02T04:47:41.360 INFO:tasks.workunit.client.0.smithi070.stdout:5/182: dwrite d5/d14/f16 [0,4194304] 0 2024-04-02T04:47:41.361 INFO:tasks.workunit.client.0.smithi070.stdout:5/183: read - d5/d17/f32 zero size 2024-04-02T04:47:41.362 INFO:tasks.workunit.client.1.smithi179.stdout:4/203: dread d1/d3/f7 [0,4194304] 0 2024-04-02T04:47:41.362 INFO:tasks.workunit.client.0.smithi070.stdout:2/283: unlink d2/fa 0 2024-04-02T04:47:41.363 INFO:tasks.workunit.client.1.smithi179.stdout:4/204: readlink d1/d3/d19/l33 0 2024-04-02T04:47:41.363 INFO:tasks.workunit.client.1.smithi179.stdout:9/179: mknod d0/d1/d10/c38 0 2024-04-02T04:47:41.365 INFO:tasks.workunit.client.0.smithi070.stdout:5/184: truncate d5/f1d 512164 0 2024-04-02T04:47:41.366 INFO:tasks.workunit.client.0.smithi070.stdout:5/185: chown d5/d14 72075 1 2024-04-02T04:47:41.367 INFO:tasks.workunit.client.0.smithi070.stdout:5/186: write f2 [2089884,26097] 0 2024-04-02T04:47:41.368 INFO:tasks.workunit.client.0.smithi070.stdout:5/187: dread - d5/d9/d10/f11 zero size 2024-04-02T04:47:41.368 INFO:tasks.workunit.client.1.smithi179.stdout:3/126: creat d8/f28 x:0 0 0 2024-04-02T04:47:41.368 INFO:tasks.workunit.client.0.smithi070.stdout:5/188: fsync f4 0 2024-04-02T04:47:41.371 INFO:tasks.workunit.client.0.smithi070.stdout:2/284: truncate d2/d10/d21/d3d/d42/f4b 1965448 0 2024-04-02T04:47:41.373 INFO:tasks.workunit.client.0.smithi070.stdout:5/189: mknod d5/d17/d27/d2c/d35/c3c 0 2024-04-02T04:47:41.375 INFO:tasks.workunit.client.0.smithi070.stdout:2/285: creat d2/d10/d21/d3d/f5d x:0 0 0 2024-04-02T04:47:41.377 INFO:tasks.workunit.client.0.smithi070.stdout:5/190: creat d5/d17/d27/d2c/d35/f3d x:0 0 0 2024-04-02T04:47:41.379 INFO:tasks.workunit.client.1.smithi179.stdout:5/97: getdents d8/d1b 0 2024-04-02T04:47:41.379 INFO:tasks.workunit.client.1.smithi179.stdout:5/98: readlink l3 0 2024-04-02T04:47:41.380 INFO:tasks.workunit.client.1.smithi179.stdout:5/99: write d8/f18 [713728,16038] 0 2024-04-02T04:47:41.380 INFO:tasks.workunit.client.0.smithi070.stdout:5/191: chown d5/c7 49764 1 2024-04-02T04:47:41.380 INFO:tasks.workunit.client.1.smithi179.stdout:8/145: sync 2024-04-02T04:47:41.381 INFO:tasks.workunit.client.1.smithi179.stdout:7/181: sync 2024-04-02T04:47:41.384 INFO:tasks.workunit.client.1.smithi179.stdout:3/127: mknod d8/d26/c29 0 2024-04-02T04:47:41.386 INFO:tasks.workunit.client.0.smithi070.stdout:2/286: dread d2/d24/f37 [0,4194304] 0 2024-04-02T04:47:41.386 INFO:tasks.workunit.client.1.smithi179.stdout:5/100: symlink d8/l20 0 2024-04-02T04:47:41.387 INFO:tasks.workunit.client.0.smithi070.stdout:2/287: read d2/f1b [163066,95140] 0 2024-04-02T04:47:41.388 INFO:tasks.workunit.client.0.smithi070.stdout:2/288: chown d2/d10/d21 1557024316 1 2024-04-02T04:47:41.390 INFO:tasks.workunit.client.1.smithi179.stdout:4/205: rename d1/d3/d19/f21 to d1/f42 0 2024-04-02T04:47:41.391 INFO:tasks.workunit.client.1.smithi179.stdout:8/146: mknod d3/d31/c36 0 2024-04-02T04:47:41.391 INFO:tasks.workunit.client.1.smithi179.stdout:8/147: write d3/f27 [510042,107884] 0 2024-04-02T04:47:41.392 INFO:tasks.workunit.client.1.smithi179.stdout:3/128: mknod d8/c2a 0 2024-04-02T04:47:41.394 INFO:tasks.workunit.client.1.smithi179.stdout:9/180: truncate d0/d1/d11/d29/f37 4180744 0 2024-04-02T04:47:41.395 INFO:tasks.workunit.client.1.smithi179.stdout:9/181: dread - d0/d1/d11/d29/f34 zero size 2024-04-02T04:47:41.396 INFO:tasks.workunit.client.1.smithi179.stdout:7/182: rename d1/l1f to d1/d9/d35/l41 0 2024-04-02T04:47:41.397 INFO:tasks.workunit.client.1.smithi179.stdout:9/182: chown d0/d1/d11/d29/f34 8024 1 2024-04-02T04:47:41.398 INFO:tasks.workunit.client.1.smithi179.stdout:8/148: dwrite d3/f16 [0,4194304] 0 2024-04-02T04:47:41.398 INFO:tasks.workunit.client.1.smithi179.stdout:9/183: readlink d0/l24 0 2024-04-02T04:47:41.399 INFO:tasks.workunit.client.1.smithi179.stdout:8/149: chown d3/d31 91436 1 2024-04-02T04:47:41.399 INFO:tasks.workunit.client.1.smithi179.stdout:9/184: fsync d0/d1/f4 0 2024-04-02T04:47:41.401 INFO:tasks.workunit.client.0.smithi070.stdout:5/192: dread d5/f22 [0,4194304] 0 2024-04-02T04:47:41.404 INFO:tasks.workunit.client.1.smithi179.stdout:9/185: dwrite d0/d1/d10/f18 [8388608,4194304] 0 2024-04-02T04:47:41.410 INFO:tasks.workunit.client.1.smithi179.stdout:9/186: dwrite d0/d19/d1c/f31 [0,4194304] 0 2024-04-02T04:47:41.411 INFO:tasks.workunit.client.1.smithi179.stdout:9/187: fsync d0/d1/d11/d29/f34 0 2024-04-02T04:47:41.412 INFO:tasks.workunit.client.1.smithi179.stdout:9/188: chown d0/d19/d27/c2f 217 1 2024-04-02T04:47:41.420 INFO:tasks.workunit.client.0.smithi070.stdout:3/116: sync 2024-04-02T04:47:41.420 INFO:tasks.workunit.client.0.smithi070.stdout:1/157: sync 2024-04-02T04:47:41.420 INFO:tasks.workunit.client.0.smithi070.stdout:8/142: sync 2024-04-02T04:47:41.420 INFO:tasks.workunit.client.0.smithi070.stdout:6/97: sync 2024-04-02T04:47:41.422 INFO:tasks.workunit.client.0.smithi070.stdout:3/117: write fe [2961530,80932] 0 2024-04-02T04:47:41.426 INFO:tasks.workunit.client.0.smithi070.stdout:5/193: mkdir d5/d17/d2b/d3e 0 2024-04-02T04:47:41.427 INFO:tasks.workunit.client.0.smithi070.stdout:5/194: fsync f2 0 2024-04-02T04:47:41.429 INFO:tasks.workunit.client.1.smithi179.stdout:5/101: getdents d8/d1b 0 2024-04-02T04:47:41.430 INFO:tasks.workunit.client.0.smithi070.stdout:8/143: dread d2/d8/d29/f15 [0,4194304] 0 2024-04-02T04:47:41.432 INFO:tasks.workunit.client.1.smithi179.stdout:3/129: dwrite d8/f1c [0,4194304] 0 2024-04-02T04:47:41.432 INFO:tasks.workunit.client.1.smithi179.stdout:7/183: creat d1/d9/f42 x:0 0 0 2024-04-02T04:47:41.433 INFO:tasks.workunit.client.0.smithi070.stdout:3/118: dwrite d10/d17/f2a [0,4194304] 0 2024-04-02T04:47:41.434 INFO:tasks.workunit.client.0.smithi070.stdout:1/158: rename d0/d9/d1d/d22/c23 to d0/d9/d1d/d22/c32 0 2024-04-02T04:47:41.438 INFO:tasks.workunit.client.0.smithi070.stdout:3/119: dread ff [0,4194304] 0 2024-04-02T04:47:41.443 INFO:tasks.workunit.client.0.smithi070.stdout:1/159: dwrite d0/f1c [0,4194304] 0 2024-04-02T04:47:41.449 INFO:tasks.workunit.client.0.smithi070.stdout:6/98: dread d0/f15 [0,4194304] 0 2024-04-02T04:47:41.454 INFO:tasks.workunit.client.0.smithi070.stdout:8/144: rename d2/d8/d29/f2a to d2/d8/d29/d22/f2c 0 2024-04-02T04:47:41.455 INFO:tasks.workunit.client.1.smithi179.stdout:5/102: symlink d8/d1b/l21 0 2024-04-02T04:47:41.459 INFO:tasks.workunit.client.1.smithi179.stdout:7/184: creat d1/d9/f43 x:0 0 0 2024-04-02T04:47:41.460 INFO:tasks.workunit.client.0.smithi070.stdout:5/195: mkdir d5/d9/d3f 0 2024-04-02T04:47:41.460 INFO:tasks.workunit.client.1.smithi179.stdout:7/185: write d1/dd/f20 [20399,57094] 0 2024-04-02T04:47:41.460 INFO:tasks.workunit.client.0.smithi070.stdout:5/196: readlink d5/d17/d27/l29 0 2024-04-02T04:47:41.461 INFO:tasks.workunit.client.0.smithi070.stdout:1/160: symlink d0/d9/d1d/d22/l33 0 2024-04-02T04:47:41.464 INFO:tasks.workunit.client.1.smithi179.stdout:5/103: chown f2 194318 1 2024-04-02T04:47:41.465 INFO:tasks.workunit.client.1.smithi179.stdout:5/104: truncate f6 1648270 0 2024-04-02T04:47:41.465 INFO:tasks.workunit.client.0.smithi070.stdout:6/99: rename d0/f17 to d0/d20/f23 0 2024-04-02T04:47:41.467 INFO:tasks.workunit.client.0.smithi070.stdout:8/145: mknod d2/d8/d29/db/c2d 0 2024-04-02T04:47:41.469 INFO:tasks.workunit.client.1.smithi179.stdout:3/130: link d8/f1b d8/d26/f2b 0 2024-04-02T04:47:41.470 INFO:tasks.workunit.client.0.smithi070.stdout:5/197: chown d5/cc 61724 1 2024-04-02T04:47:41.470 INFO:tasks.workunit.client.0.smithi070.stdout:3/120: dwrite f1 [0,4194304] 0 2024-04-02T04:47:41.471 INFO:tasks.workunit.client.0.smithi070.stdout:5/198: stat d5/d17/l36 0 2024-04-02T04:47:41.472 INFO:tasks.workunit.client.0.smithi070.stdout:1/161: mknod d0/d7/d1a/c34 0 2024-04-02T04:47:41.472 INFO:tasks.workunit.client.1.smithi179.stdout:8/150: creat d3/d32/d21/f37 x:0 0 0 2024-04-02T04:47:41.474 INFO:tasks.workunit.client.1.smithi179.stdout:4/206: getdents d1/d3/dc 0 2024-04-02T04:47:41.475 INFO:tasks.workunit.client.1.smithi179.stdout:5/105: mknod d8/c22 0 2024-04-02T04:47:41.475 INFO:tasks.workunit.client.1.smithi179.stdout:3/131: dwrite d8/f28 [0,4194304] 0 2024-04-02T04:47:41.477 INFO:tasks.workunit.client.0.smithi070.stdout:6/100: creat d0/f24 x:0 0 0 2024-04-02T04:47:41.477 INFO:tasks.workunit.client.1.smithi179.stdout:7/186: mknod d1/c44 0 2024-04-02T04:47:41.479 INFO:tasks.workunit.client.0.smithi070.stdout:5/199: dwrite d5/d9/d10/f11 [0,4194304] 0 2024-04-02T04:47:41.479 INFO:tasks.workunit.client.0.smithi070.stdout:3/121: creat d10/f2c x:0 0 0 2024-04-02T04:47:41.480 INFO:tasks.workunit.client.1.smithi179.stdout:3/132: dread d8/f1c [0,4194304] 0 2024-04-02T04:47:41.481 INFO:tasks.workunit.client.1.smithi179.stdout:3/133: stat d8/f11 0 2024-04-02T04:47:41.481 INFO:tasks.workunit.client.1.smithi179.stdout:8/151: symlink d3/d1b/l38 0 2024-04-02T04:47:41.481 INFO:tasks.workunit.client.1.smithi179.stdout:3/134: chown c7 136 1 2024-04-02T04:47:41.482 INFO:tasks.workunit.client.0.smithi070.stdout:3/122: dread d10/d17/d19/f1a [0,4194304] 0 2024-04-02T04:47:41.485 INFO:tasks.workunit.client.0.smithi070.stdout:1/162: rename d0/f30 to d0/d9/d1d/f35 0 2024-04-02T04:47:41.485 INFO:tasks.workunit.client.1.smithi179.stdout:5/106: mkdir d8/d1b/d23 0 2024-04-02T04:47:41.489 INFO:tasks.workunit.client.0.smithi070.stdout:6/101: write d0/f15 [442488,66559] 0 2024-04-02T04:47:41.492 INFO:tasks.workunit.client.1.smithi179.stdout:4/207: rename d1/d3/dc/d27/f2e to d1/d3/df/d10/f43 0 2024-04-02T04:47:41.492 INFO:tasks.workunit.client.0.smithi070.stdout:8/146: symlink d2/d8/l2e 0 2024-04-02T04:47:41.493 INFO:tasks.workunit.client.0.smithi070.stdout:8/147: fsync d2/f1f 0 2024-04-02T04:47:41.496 INFO:tasks.workunit.client.1.smithi179.stdout:8/152: mknod d3/d32/d21/d22/d33/c39 0 2024-04-02T04:47:41.498 INFO:tasks.workunit.client.0.smithi070.stdout:5/200: rename d5 to d5/d40 22 2024-04-02T04:47:41.500 INFO:tasks.workunit.client.1.smithi179.stdout:5/107: creat d8/d1b/f24 x:0 0 0 2024-04-02T04:47:41.500 INFO:tasks.workunit.client.0.smithi070.stdout:1/163: mknod d0/d9/d1d/c36 0 2024-04-02T04:47:41.503 INFO:tasks.workunit.client.1.smithi179.stdout:8/153: dwrite d3/d32/d21/f25 [0,4194304] 0 2024-04-02T04:47:41.503 INFO:tasks.workunit.client.0.smithi070.stdout:6/102: creat d0/f25 x:0 0 0 2024-04-02T04:47:41.503 INFO:tasks.workunit.client.0.smithi070.stdout:6/103: stat d0/f15 0 2024-04-02T04:47:41.503 INFO:tasks.workunit.client.0.smithi070.stdout:6/104: readlink d0/l11 0 2024-04-02T04:47:41.505 INFO:tasks.workunit.client.0.smithi070.stdout:8/148: mknod d2/d8/d29/d22/c2f 0 2024-04-02T04:47:41.506 INFO:tasks.workunit.client.1.smithi179.stdout:4/208: mkdir d1/d3/dc/d44 0 2024-04-02T04:47:41.511 INFO:tasks.workunit.client.0.smithi070.stdout:8/149: mkdir d2/d8/d29/d22/d30 0 2024-04-02T04:47:41.511 INFO:tasks.workunit.client.1.smithi179.stdout:8/154: symlink d3/l3a 0 2024-04-02T04:47:41.514 INFO:tasks.workunit.client.1.smithi179.stdout:4/209: rename d1/f5 to d1/d3/df/d10/d3e/f45 0 2024-04-02T04:47:41.514 INFO:tasks.workunit.client.0.smithi070.stdout:1/164: mknod d0/d9/d1d/d22/d2c/c37 0 2024-04-02T04:47:41.516 INFO:tasks.workunit.client.1.smithi179.stdout:4/210: chown d1/d3/df/d10/d2f/d36 28152 1 2024-04-02T04:47:41.517 INFO:tasks.workunit.client.0.smithi070.stdout:3/123: write d10/f15 [4227127,118563] 0 2024-04-02T04:47:41.518 INFO:tasks.workunit.client.1.smithi179.stdout:3/135: write d8/f1b [1396449,33867] 0 2024-04-02T04:47:41.519 INFO:tasks.workunit.client.1.smithi179.stdout:3/136: chown c7 53 1 2024-04-02T04:47:41.519 INFO:tasks.workunit.client.0.smithi070.stdout:8/150: creat d2/d8/d23/f31 x:0 0 0 2024-04-02T04:47:41.520 INFO:tasks.workunit.client.0.smithi070.stdout:8/151: rename d2/d8 to d2/d8/d23/d24/d32 22 2024-04-02T04:47:41.520 INFO:tasks.workunit.client.1.smithi179.stdout:8/155: dwrite d3/d32/d21/f37 [0,4194304] 0 2024-04-02T04:47:41.521 INFO:tasks.workunit.client.1.smithi179.stdout:4/211: mknod d1/d3/df/d10/c46 0 2024-04-02T04:47:41.522 INFO:tasks.workunit.client.0.smithi070.stdout:5/201: getdents d5 0 2024-04-02T04:47:41.525 INFO:tasks.workunit.client.1.smithi179.stdout:3/137: rename d8/f1d to d8/d26/f2c 0 2024-04-02T04:47:41.525 INFO:tasks.workunit.client.0.smithi070.stdout:5/202: write d5/d17/d27/f28 [3737601,49667] 0 2024-04-02T04:47:41.526 INFO:tasks.workunit.client.0.smithi070.stdout:5/203: stat d5/d17/d2b/d3e 0 2024-04-02T04:47:41.528 INFO:tasks.workunit.client.0.smithi070.stdout:8/152: dwrite d2/d8/d29/db/f18 [0,4194304] 0 2024-04-02T04:47:41.529 INFO:tasks.workunit.client.1.smithi179.stdout:4/212: dwrite d1/d3/d19/f2c [0,4194304] 0 2024-04-02T04:47:41.533 INFO:tasks.workunit.client.0.smithi070.stdout:5/204: dwrite f4 [0,4194304] 0 2024-04-02T04:47:41.534 INFO:tasks.workunit.client.1.smithi179.stdout:4/213: dread d1/f3c [0,4194304] 0 2024-04-02T04:47:41.535 INFO:tasks.workunit.client.1.smithi179.stdout:4/214: chown d1/d3/df/d10/d2f 383523 1 2024-04-02T04:47:41.608 INFO:tasks.workunit.client.1.smithi179.stdout:1/209: sync 2024-04-02T04:47:41.608 INFO:tasks.workunit.client.1.smithi179.stdout:6/118: sync 2024-04-02T04:47:41.609 INFO:tasks.workunit.client.1.smithi179.stdout:6/119: chown d3/ld 3380218 1 2024-04-02T04:47:41.612 INFO:tasks.workunit.client.1.smithi179.stdout:7/187: truncate d1/d9/d35/f3b 179982 0 2024-04-02T04:47:41.612 INFO:tasks.workunit.client.0.smithi070.stdout:9/191: sync 2024-04-02T04:47:41.616 INFO:tasks.workunit.client.1.smithi179.stdout:8/156: creat d3/d32/d30/f3b x:0 0 0 2024-04-02T04:47:41.617 INFO:tasks.workunit.client.0.smithi070.stdout:3/124: link d10/d17/l20 d10/d17/l2d 0 2024-04-02T04:47:41.618 INFO:tasks.workunit.client.1.smithi179.stdout:3/138: creat d8/f2d x:0 0 0 2024-04-02T04:47:41.619 INFO:tasks.workunit.client.0.smithi070.stdout:6/105: truncate d0/d3/fb 7366143 0 2024-04-02T04:47:41.619 INFO:tasks.workunit.client.0.smithi070.stdout:6/106: fdatasync d0/f1c 0 2024-04-02T04:47:41.620 INFO:tasks.workunit.client.0.smithi070.stdout:8/153: creat d2/d3/d6/f33 x:0 0 0 2024-04-02T04:47:41.624 INFO:tasks.workunit.client.0.smithi070.stdout:9/192: rename d4/f7 to d4/d10/d11/f3f 0 2024-04-02T04:47:41.626 INFO:tasks.workunit.client.0.smithi070.stdout:9/193: dread d4/d10/d12/d27/d32/f33 [0,4194304] 0 2024-04-02T04:47:41.644 INFO:tasks.workunit.client.1.smithi179.stdout:0/229: sync 2024-04-02T04:47:41.644 INFO:tasks.workunit.client.1.smithi179.stdout:2/94: sync 2024-04-02T04:47:41.644 INFO:tasks.workunit.client.1.smithi179.stdout:4/215: fdatasync d1/d3/df/d10/d3e/f45 0 2024-04-02T04:47:41.645 INFO:tasks.workunit.client.0.smithi070.stdout:3/125: mknod d10/c2e 0 2024-04-02T04:47:41.647 INFO:tasks.workunit.client.0.smithi070.stdout:6/107: chown d0/d3/l18 9 1 2024-04-02T04:47:41.648 INFO:tasks.workunit.client.1.smithi179.stdout:6/120: symlink d3/d2d/l31 0 2024-04-02T04:47:41.649 INFO:tasks.workunit.client.0.smithi070.stdout:8/154: write d2/d8/d29/db/f13 [1545151,63499] 0 2024-04-02T04:47:41.651 INFO:tasks.workunit.client.1.smithi179.stdout:7/188: rmdir d1/d9 39 2024-04-02T04:47:41.652 INFO:tasks.workunit.client.1.smithi179.stdout:7/189: write d1/dd/d23/f24 [712889,77120] 0 2024-04-02T04:47:41.652 INFO:tasks.workunit.client.1.smithi179.stdout:7/190: dread - d1/dd/d16/f2a zero size 2024-04-02T04:47:41.653 INFO:tasks.workunit.client.0.smithi070.stdout:5/205: creat d5/d9/d3f/f41 x:0 0 0 2024-04-02T04:47:41.653 INFO:tasks.workunit.client.1.smithi179.stdout:7/191: chown d1/dd/d16/f38 19918 1 2024-04-02T04:47:41.654 INFO:tasks.workunit.client.1.smithi179.stdout:8/157: mkdir d3/d32/d21/d22/d33/d3c 0 2024-04-02T04:47:41.654 INFO:tasks.workunit.client.1.smithi179.stdout:8/158: readlink d3/d32/l2c 0 2024-04-02T04:47:41.655 INFO:tasks.workunit.client.0.smithi070.stdout:6/108: dwrite d0/f1b [0,4194304] 0 2024-04-02T04:47:41.657 INFO:tasks.workunit.client.1.smithi179.stdout:3/139: mknod d8/c2e 0 2024-04-02T04:47:41.660 INFO:tasks.workunit.client.0.smithi070.stdout:9/194: mknod d4/d10/d12/d27/c40 0 2024-04-02T04:47:41.660 INFO:tasks.workunit.client.1.smithi179.stdout:2/95: mknod dc/d16/c1d 0 2024-04-02T04:47:41.661 INFO:tasks.workunit.client.1.smithi179.stdout:5/108: sync 2024-04-02T04:47:41.661 INFO:tasks.workunit.client.1.smithi179.stdout:9/189: sync 2024-04-02T04:47:41.661 INFO:tasks.workunit.client.0.smithi070.stdout:3/126: chown d10/d17/l28 383276 1 2024-04-02T04:47:41.662 INFO:tasks.workunit.client.0.smithi070.stdout:3/127: write f1 [1647412,119258] 0 2024-04-02T04:47:41.663 INFO:tasks.workunit.client.1.smithi179.stdout:0/230: mknod d1/db/c42 0 2024-04-02T04:47:41.666 INFO:tasks.workunit.client.1.smithi179.stdout:4/216: symlink d1/d15/l47 0 2024-04-02T04:47:41.667 INFO:tasks.workunit.client.0.smithi070.stdout:5/206: creat d5/d9/d23/f42 x:0 0 0 2024-04-02T04:47:41.668 INFO:tasks.workunit.client.0.smithi070.stdout:6/109: creat d0/f26 x:0 0 0 2024-04-02T04:47:41.669 INFO:tasks.workunit.client.1.smithi179.stdout:4/217: readlink d1/d15/l41 0 2024-04-02T04:47:41.669 INFO:tasks.workunit.client.0.smithi070.stdout:6/110: fsync d0/d3/f1e 0 2024-04-02T04:47:41.671 INFO:tasks.workunit.client.1.smithi179.stdout:8/159: mknod d3/c3d 0 2024-04-02T04:47:41.671 INFO:tasks.workunit.client.0.smithi070.stdout:9/195: stat d4/d10/d12/c38 0 2024-04-02T04:47:41.672 INFO:tasks.workunit.client.0.smithi070.stdout:9/196: chown d4/d10/f2c 178613437 1 2024-04-02T04:47:41.673 INFO:tasks.workunit.client.1.smithi179.stdout:3/140: mknod d8/c2f 0 2024-04-02T04:47:41.675 INFO:tasks.workunit.client.1.smithi179.stdout:8/160: read d3/f16 [7351233,77721] 0 2024-04-02T04:47:41.675 INFO:tasks.workunit.client.1.smithi179.stdout:1/210: dwrite d3/f28 [0,4194304] 0 2024-04-02T04:47:41.676 INFO:tasks.workunit.client.0.smithi070.stdout:3/128: symlink d10/d17/l2f 0 2024-04-02T04:47:41.677 INFO:tasks.workunit.client.1.smithi179.stdout:5/109: dwrite d8/f1d [0,4194304] 0 2024-04-02T04:47:41.678 INFO:tasks.workunit.client.1.smithi179.stdout:2/96: dwrite dc/d16/f15 [0,4194304] 0 2024-04-02T04:47:41.679 INFO:tasks.workunit.client.1.smithi179.stdout:8/161: stat d3/d31/c24 0 2024-04-02T04:47:41.680 INFO:tasks.workunit.client.1.smithi179.stdout:2/97: chown dc/d16 57 1 2024-04-02T04:47:41.681 INFO:tasks.workunit.client.0.smithi070.stdout:8/155: link d2/d8/d29/db/f21 d2/d8/d23/d24/f34 0 2024-04-02T04:47:41.681 INFO:tasks.workunit.client.1.smithi179.stdout:0/231: dwrite d1/f36 [0,4194304] 0 2024-04-02T04:47:41.682 INFO:tasks.workunit.client.0.smithi070.stdout:8/156: chown d2/d3/l28 43642 1 2024-04-02T04:47:41.683 INFO:tasks.workunit.client.1.smithi179.stdout:8/162: read d3/d32/d21/f25 [3575312,17060] 0 2024-04-02T04:47:41.684 INFO:tasks.workunit.client.1.smithi179.stdout:9/190: rmdir d0/d19/d27 39 2024-04-02T04:47:41.687 INFO:tasks.workunit.client.1.smithi179.stdout:6/121: rename d3/d12/d23/d26/l29 to d3/d12/d23/l32 0 2024-04-02T04:47:41.688 INFO:tasks.workunit.client.0.smithi070.stdout:8/157: dread d2/d8/d23/d24/f34 [0,4194304] 0 2024-04-02T04:47:41.689 INFO:tasks.workunit.client.1.smithi179.stdout:6/122: fsync d3/ff 0 2024-04-02T04:47:41.690 INFO:tasks.workunit.client.0.smithi070.stdout:8/158: write d2/d3/fd [5179647,55586] 0 2024-04-02T04:47:41.692 INFO:tasks.workunit.client.1.smithi179.stdout:8/163: dwrite d3/f7 [0,4194304] 0 2024-04-02T04:47:41.694 INFO:tasks.workunit.client.0.smithi070.stdout:9/197: unlink d4/d34/c3b 0 2024-04-02T04:47:41.697 INFO:tasks.workunit.client.1.smithi179.stdout:4/218: creat d1/d3/df/f48 x:0 0 0 2024-04-02T04:47:41.699 INFO:tasks.workunit.client.0.smithi070.stdout:3/129: creat d10/d17/f30 x:0 0 0 2024-04-02T04:47:41.709 INFO:tasks.workunit.client.0.smithi070.stdout:8/159: mkdir d2/d3/d35 0 2024-04-02T04:47:41.710 INFO:tasks.workunit.client.0.smithi070.stdout:8/160: chown d2/d3 6121234 1 2024-04-02T04:47:41.711 INFO:tasks.workunit.client.1.smithi179.stdout:2/98: mkdir dc/d16/d1e 0 2024-04-02T04:47:41.715 INFO:tasks.workunit.client.0.smithi070.stdout:3/130: symlink d10/d17/l31 0 2024-04-02T04:47:41.716 INFO:tasks.workunit.client.1.smithi179.stdout:7/192: creat d1/f45 x:0 0 0 2024-04-02T04:47:41.716 INFO:tasks.workunit.client.0.smithi070.stdout:8/161: dwrite d2/d3/d6/f25 [0,4194304] 0 2024-04-02T04:47:41.718 INFO:tasks.workunit.client.1.smithi179.stdout:3/141: dwrite d8/f11 [0,4194304] 0 2024-04-02T04:47:41.721 INFO:tasks.workunit.client.0.smithi070.stdout:6/111: truncate d0/f1b 508225 0 2024-04-02T04:47:41.722 INFO:tasks.workunit.client.0.smithi070.stdout:3/131: unlink d10/d17/d19/f1a 0 2024-04-02T04:47:41.726 INFO:tasks.workunit.client.0.smithi070.stdout:9/198: truncate d4/d10/d11/f3f 4188056 0 2024-04-02T04:47:41.728 INFO:tasks.workunit.client.0.smithi070.stdout:6/112: dwrite d0/f16 [0,4194304] 0 2024-04-02T04:47:41.729 INFO:tasks.workunit.client.1.smithi179.stdout:4/219: unlink d1/d3/d19/c1a 0 2024-04-02T04:47:41.731 INFO:tasks.workunit.client.0.smithi070.stdout:3/132: mkdir d10/d32 0 2024-04-02T04:47:41.734 INFO:tasks.workunit.client.1.smithi179.stdout:5/110: link d8/f18 d8/d1b/f25 0 2024-04-02T04:47:41.737 INFO:tasks.workunit.client.1.smithi179.stdout:1/211: truncate d3/d5/f1d 1469277 0 2024-04-02T04:47:41.739 INFO:tasks.workunit.client.1.smithi179.stdout:0/232: symlink d1/da/l43 0 2024-04-02T04:47:41.740 INFO:tasks.workunit.client.1.smithi179.stdout:0/233: dread - d1/da/d28/d37/f3e zero size 2024-04-02T04:47:41.740 INFO:tasks.workunit.client.1.smithi179.stdout:0/234: chown d1/f36 5067 1 2024-04-02T04:47:41.741 INFO:tasks.workunit.client.1.smithi179.stdout:7/193: chown d1/d9/ce 472453 1 2024-04-02T04:47:41.742 INFO:tasks.workunit.client.1.smithi179.stdout:7/194: dread - d1/dd/d23/f2f zero size 2024-04-02T04:47:41.742 INFO:tasks.workunit.client.0.smithi070.stdout:8/162: getdents d2/d3 0 2024-04-02T04:47:41.743 INFO:tasks.workunit.client.1.smithi179.stdout:3/142: stat d8/l14 0 2024-04-02T04:47:41.743 INFO:tasks.workunit.client.0.smithi070.stdout:8/163: dread d2/d8/d23/d24/f34 [0,4194304] 0 2024-04-02T04:47:41.744 INFO:tasks.workunit.client.1.smithi179.stdout:6/123: fdatasync d3/f1c 0 2024-04-02T04:47:41.745 INFO:tasks.workunit.client.1.smithi179.stdout:6/124: read d3/ff [2963452,109442] 0 2024-04-02T04:47:41.745 INFO:tasks.workunit.client.0.smithi070.stdout:8/164: mkdir d2/d8/d23/d36 0 2024-04-02T04:47:41.746 INFO:tasks.workunit.client.1.smithi179.stdout:4/220: creat d1/d3/f49 x:0 0 0 2024-04-02T04:47:41.747 INFO:tasks.workunit.client.1.smithi179.stdout:5/111: rmdir d8 39 2024-04-02T04:47:41.747 INFO:tasks.workunit.client.1.smithi179.stdout:4/221: fdatasync d1/d3/df/f48 0 2024-04-02T04:47:41.749 INFO:tasks.workunit.client.1.smithi179.stdout:1/212: unlink d3/d5/de/f26 0 2024-04-02T04:47:41.751 INFO:tasks.workunit.client.0.smithi070.stdout:8/165: dwrite d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:41.751 INFO:tasks.workunit.client.0.smithi070.stdout:6/113: rename d0/f1b to d0/d20/f27 0 2024-04-02T04:47:41.752 INFO:tasks.workunit.client.0.smithi070.stdout:6/114: dread - d0/f25 zero size 2024-04-02T04:47:41.753 INFO:tasks.workunit.client.0.smithi070.stdout:8/166: dread d2/d8/d29/db/f21 [0,4194304] 0 2024-04-02T04:47:41.753 INFO:tasks.workunit.client.0.smithi070.stdout:8/167: read - d2/f1f zero size 2024-04-02T04:47:41.757 INFO:tasks.workunit.client.0.smithi070.stdout:3/133: rmdir d10 39 2024-04-02T04:47:41.760 INFO:tasks.workunit.client.0.smithi070.stdout:8/168: creat d2/d8/d23/d24/f37 x:0 0 0 2024-04-02T04:47:41.761 INFO:tasks.workunit.client.0.smithi070.stdout:3/134: dread ff [0,4194304] 0 2024-04-02T04:47:41.761 INFO:tasks.workunit.client.1.smithi179.stdout:7/195: dwrite d1/ff [0,4194304] 0 2024-04-02T04:47:41.761 INFO:tasks.workunit.client.0.smithi070.stdout:8/169: chown d2/c19 3236 1 2024-04-02T04:47:41.763 INFO:tasks.workunit.client.1.smithi179.stdout:3/143: dwrite d8/d26/f2c [0,4194304] 0 2024-04-02T04:47:41.765 INFO:tasks.workunit.client.1.smithi179.stdout:3/144: write d8/f22 [3196,34753] 0 2024-04-02T04:47:41.765 INFO:tasks.workunit.client.1.smithi179.stdout:4/222: creat d1/d3/df/d10/d3e/f4a x:0 0 0 2024-04-02T04:47:41.766 INFO:tasks.workunit.client.0.smithi070.stdout:8/170: write d2/d8/d29/d22/f2c [195009,81792] 0 2024-04-02T04:47:41.768 INFO:tasks.workunit.client.0.smithi070.stdout:8/171: truncate d2/d8/d23/d24/f37 962998 0 2024-04-02T04:47:41.769 INFO:tasks.workunit.client.0.smithi070.stdout:9/199: truncate d4/d10/d11/f21 3055951 0 2024-04-02T04:47:41.771 INFO:tasks.workunit.client.1.smithi179.stdout:0/235: symlink d1/da/l44 0 2024-04-02T04:47:41.771 INFO:tasks.workunit.client.1.smithi179.stdout:4/223: dread d1/d15/f31 [0,4194304] 0 2024-04-02T04:47:41.771 INFO:tasks.workunit.client.1.smithi179.stdout:9/191: truncate d0/d1/f7 2581909 0 2024-04-02T04:47:41.772 INFO:tasks.workunit.client.1.smithi179.stdout:0/236: chown d1/da/df/f20 57364 1 2024-04-02T04:47:41.774 INFO:tasks.workunit.client.0.smithi070.stdout:3/135: link d10/f2c d10/d17/f33 0 2024-04-02T04:47:41.776 INFO:tasks.workunit.client.1.smithi179.stdout:6/125: mknod d3/d2c/c33 0 2024-04-02T04:47:41.777 INFO:tasks.workunit.client.1.smithi179.stdout:5/112: rmdir d8 39 2024-04-02T04:47:41.779 INFO:tasks.workunit.client.1.smithi179.stdout:9/192: dwrite d0/d1/d10/f13 [0,4194304] 0 2024-04-02T04:47:41.779 INFO:tasks.workunit.client.0.smithi070.stdout:3/136: mknod d10/c34 0 2024-04-02T04:47:41.779 INFO:tasks.workunit.client.1.smithi179.stdout:6/126: write d3/d12/d23/f30 [830795,25121] 0 2024-04-02T04:47:41.782 INFO:tasks.workunit.client.1.smithi179.stdout:8/164: dwrite d3/d32/f19 [0,4194304] 0 2024-04-02T04:47:41.783 INFO:tasks.workunit.client.1.smithi179.stdout:7/196: mknod d1/d9/c46 0 2024-04-02T04:47:41.783 INFO:tasks.workunit.client.0.smithi070.stdout:3/137: creat d10/d17/d19/f35 x:0 0 0 2024-04-02T04:47:41.786 INFO:tasks.workunit.client.0.smithi070.stdout:8/172: link d2/l1a d2/d8/d29/d22/l38 0 2024-04-02T04:47:41.787 INFO:tasks.workunit.client.0.smithi070.stdout:8/173: write d2/d8/d23/f31 [214196,122554] 0 2024-04-02T04:47:41.789 INFO:tasks.workunit.client.0.smithi070.stdout:8/174: stat d2/d8/d29/db/f18 0 2024-04-02T04:47:41.789 INFO:tasks.workunit.client.1.smithi179.stdout:4/224: mknod d1/d3/dc/d27/c4b 0 2024-04-02T04:47:41.791 INFO:tasks.workunit.client.1.smithi179.stdout:4/225: truncate d1/d3/dc/fe 4205761 0 2024-04-02T04:47:41.792 INFO:tasks.workunit.client.1.smithi179.stdout:4/226: write d1/f2b [766023,29871] 0 2024-04-02T04:47:41.794 INFO:tasks.workunit.client.1.smithi179.stdout:9/193: symlink d0/d1/d12/l39 0 2024-04-02T04:47:41.796 INFO:tasks.workunit.client.1.smithi179.stdout:6/127: creat d3/d12/f34 x:0 0 0 2024-04-02T04:47:41.799 INFO:tasks.workunit.client.0.smithi070.stdout:8/175: dwrite d2/d8/d29/db/f21 [0,4194304] 0 2024-04-02T04:47:41.801 INFO:tasks.workunit.client.1.smithi179.stdout:1/213: creat d3/d5/f4a x:0 0 0 2024-04-02T04:47:41.801 INFO:tasks.workunit.client.1.smithi179.stdout:5/113: dwrite d8/f1a [0,4194304] 0 2024-04-02T04:47:41.801 INFO:tasks.workunit.client.1.smithi179.stdout:0/237: symlink d1/l45 0 2024-04-02T04:47:41.802 INFO:tasks.workunit.client.0.smithi070.stdout:3/138: dread f0 [0,4194304] 0 2024-04-02T04:47:41.803 INFO:tasks.workunit.client.0.smithi070.stdout:3/139: readlink d10/l18 0 2024-04-02T04:47:41.804 INFO:tasks.workunit.client.0.smithi070.stdout:8/176: mkdir d2/d8/d23/d39 0 2024-04-02T04:47:41.804 INFO:tasks.workunit.client.1.smithi179.stdout:4/227: dwrite d1/d3/dc/d26/f29 [0,4194304] 0 2024-04-02T04:47:41.807 INFO:tasks.workunit.client.1.smithi179.stdout:6/128: dread d3/f18 [0,4194304] 0 2024-04-02T04:47:41.809 INFO:tasks.workunit.client.1.smithi179.stdout:7/197: mknod d1/c47 0 2024-04-02T04:47:41.812 INFO:tasks.workunit.client.1.smithi179.stdout:2/99: sync 2024-04-02T04:47:41.812 INFO:tasks.workunit.client.0.smithi070.stdout:8/177: unlink f1 0 2024-04-02T04:47:41.813 INFO:tasks.workunit.client.1.smithi179.stdout:7/198: dread - d1/dd/d16/f2a zero size 2024-04-02T04:47:41.814 INFO:tasks.workunit.client.0.smithi070.stdout:8/178: write d2/d3/f17 [1386313,11653] 0 2024-04-02T04:47:41.815 INFO:tasks.workunit.client.0.smithi070.stdout:8/179: fsync d2/d8/d29/db/f13 0 2024-04-02T04:47:41.816 INFO:tasks.workunit.client.1.smithi179.stdout:8/165: creat d3/d1b/d2a/f3e x:0 0 0 2024-04-02T04:47:41.816 INFO:tasks.workunit.client.1.smithi179.stdout:6/129: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:41.817 INFO:tasks.workunit.client.0.smithi070.stdout:3/140: creat d10/d32/f36 x:0 0 0 2024-04-02T04:47:41.818 INFO:tasks.workunit.client.1.smithi179.stdout:8/166: dread - d3/d1b/d2a/f3e zero size 2024-04-02T04:47:41.819 INFO:tasks.workunit.client.1.smithi179.stdout:6/130: chown d3/d12/l17 2 1 2024-04-02T04:47:41.820 INFO:tasks.workunit.client.1.smithi179.stdout:8/167: chown l2 848 1 2024-04-02T04:47:41.821 INFO:tasks.workunit.client.1.smithi179.stdout:2/100: dwrite f9 [0,4194304] 0 2024-04-02T04:47:41.821 INFO:tasks.workunit.client.1.smithi179.stdout:6/131: fdatasync d3/d12/f22 0 2024-04-02T04:47:41.823 INFO:tasks.workunit.client.1.smithi179.stdout:5/114: creat d8/d1b/f26 x:0 0 0 2024-04-02T04:47:41.823 INFO:tasks.workunit.client.1.smithi179.stdout:8/168: write d3/d32/f19 [998583,76354] 0 2024-04-02T04:47:41.831 INFO:tasks.workunit.client.1.smithi179.stdout:5/115: dread f1 [0,4194304] 0 2024-04-02T04:47:41.831 INFO:tasks.workunit.client.0.smithi070.stdout:8/180: mkdir d2/d3/d35/d3a 0 2024-04-02T04:47:41.832 INFO:tasks.workunit.client.0.smithi070.stdout:3/141: dread fa [0,4194304] 0 2024-04-02T04:47:41.833 INFO:tasks.workunit.client.0.smithi070.stdout:3/142: fsync d10/f22 0 2024-04-02T04:47:41.834 INFO:tasks.workunit.client.0.smithi070.stdout:3/143: write fe [3704671,422] 0 2024-04-02T04:47:41.834 INFO:tasks.workunit.client.1.smithi179.stdout:7/199: creat d1/dd/d16/f48 x:0 0 0 2024-04-02T04:47:41.834 INFO:tasks.workunit.client.1.smithi179.stdout:7/200: chown d1/d9/c22 5729690 1 2024-04-02T04:47:41.835 INFO:tasks.workunit.client.0.smithi070.stdout:3/144: dread - d10/d17/f27 zero size 2024-04-02T04:47:41.836 INFO:tasks.workunit.client.0.smithi070.stdout:3/145: write d10/f15 [4946083,59002] 0 2024-04-02T04:47:41.839 INFO:tasks.workunit.client.0.smithi070.stdout:8/181: rename d2/d8/d29/db/f18 to d2/d8/d29/f3b 0 2024-04-02T04:47:41.841 INFO:tasks.workunit.client.1.smithi179.stdout:3/145: truncate f5 2158326 0 2024-04-02T04:47:41.844 INFO:tasks.workunit.client.1.smithi179.stdout:6/132: mknod d3/d12/c35 0 2024-04-02T04:47:41.845 INFO:tasks.workunit.client.1.smithi179.stdout:3/146: dread d8/f22 [0,4194304] 0 2024-04-02T04:47:41.845 INFO:tasks.workunit.client.0.smithi070.stdout:3/146: rename d10/l24 to d10/d32/l37 0 2024-04-02T04:47:41.846 INFO:tasks.workunit.client.0.smithi070.stdout:3/147: read - d10/d17/d19/f1f zero size 2024-04-02T04:47:41.846 INFO:tasks.workunit.client.0.smithi070.stdout:3/148: dread - d10/d32/f36 zero size 2024-04-02T04:47:41.847 INFO:tasks.workunit.client.0.smithi070.stdout:8/182: creat d2/d3/d6/f3c x:0 0 0 2024-04-02T04:47:41.847 INFO:tasks.workunit.client.0.smithi070.stdout:8/183: stat d2/d8/d29/c2b 0 2024-04-02T04:47:41.849 INFO:tasks.workunit.client.1.smithi179.stdout:1/214: symlink d3/d31/d35/d49/l4b 0 2024-04-02T04:47:41.850 INFO:tasks.workunit.client.0.smithi070.stdout:3/149: rmdir d10/d17/d19 39 2024-04-02T04:47:41.851 INFO:tasks.workunit.client.0.smithi070.stdout:8/184: write d2/d8/d29/f3b [4289219,22014] 0 2024-04-02T04:47:41.854 INFO:tasks.workunit.client.1.smithi179.stdout:5/116: creat d8/f27 x:0 0 0 2024-04-02T04:47:41.857 INFO:tasks.workunit.client.1.smithi179.stdout:9/194: dwrite d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:41.858 INFO:tasks.workunit.client.1.smithi179.stdout:2/101: mknod dc/d16/d1e/c1f 0 2024-04-02T04:47:41.859 INFO:tasks.workunit.client.1.smithi179.stdout:2/102: stat f9 0 2024-04-02T04:47:41.860 INFO:tasks.workunit.client.0.smithi070.stdout:3/150: creat d10/d17/d19/f38 x:0 0 0 2024-04-02T04:47:41.863 INFO:tasks.workunit.client.1.smithi179.stdout:4/228: dwrite d1/d3/f28 [4194304,4194304] 0 2024-04-02T04:47:41.864 INFO:tasks.workunit.client.0.smithi070.stdout:3/151: mknod d10/d17/d19/c39 0 2024-04-02T04:47:41.864 INFO:tasks.workunit.client.1.smithi179.stdout:4/229: dread - d1/d3/df/f48 zero size 2024-04-02T04:47:41.865 INFO:tasks.workunit.client.1.smithi179.stdout:4/230: write d1/d3/f1b [4065628,1127] 0 2024-04-02T04:47:41.866 INFO:tasks.workunit.client.1.smithi179.stdout:4/231: write d1/d3/dc/f1c [1507442,29314] 0 2024-04-02T04:47:41.867 INFO:tasks.workunit.client.1.smithi179.stdout:4/232: dread - d1/d3/f49 zero size 2024-04-02T04:47:41.868 INFO:tasks.workunit.client.1.smithi179.stdout:3/147: mknod d8/c30 0 2024-04-02T04:47:41.868 INFO:tasks.workunit.client.1.smithi179.stdout:4/233: chown d1/d3/d19/l33 101468865 1 2024-04-02T04:47:41.873 INFO:tasks.workunit.client.0.smithi070.stdout:3/152: dwrite d10/d17/d19/f38 [0,4194304] 0 2024-04-02T04:47:41.873 INFO:tasks.workunit.client.0.smithi070.stdout:8/185: link d2/l1a d2/d8/d23/d36/l3d 0 2024-04-02T04:47:41.874 INFO:tasks.workunit.client.0.smithi070.stdout:8/186: chown d2/d8/d29/db 92 1 2024-04-02T04:47:41.876 INFO:tasks.workunit.client.1.smithi179.stdout:1/215: creat d3/d31/f4c x:0 0 0 2024-04-02T04:47:41.880 INFO:tasks.workunit.client.0.smithi070.stdout:3/153: fdatasync ff 0 2024-04-02T04:47:41.881 INFO:tasks.workunit.client.0.smithi070.stdout:3/154: write d10/d17/f2b [278374,89894] 0 2024-04-02T04:47:41.882 INFO:tasks.workunit.client.1.smithi179.stdout:7/201: link d1/d9/d35/f3a d1/d9/d27/f49 0 2024-04-02T04:47:41.883 INFO:tasks.workunit.client.1.smithi179.stdout:1/216: dwrite d3/d5/d2f/f34 [0,4194304] 0 2024-04-02T04:47:41.885 INFO:tasks.workunit.client.1.smithi179.stdout:2/103: rmdir dc/d16 39 2024-04-02T04:47:41.890 INFO:tasks.workunit.client.1.smithi179.stdout:0/238: getdents d1/da/d28 0 2024-04-02T04:47:41.891 INFO:tasks.workunit.client.1.smithi179.stdout:8/169: dwrite d3/d1b/f1c [0,4194304] 0 2024-04-02T04:47:41.891 INFO:tasks.workunit.client.1.smithi179.stdout:0/239: readlink d1/l2e 0 2024-04-02T04:47:41.896 INFO:tasks.workunit.client.0.smithi070.stdout:3/155: getdents d10/d17/d19 0 2024-04-02T04:47:41.897 INFO:tasks.workunit.client.1.smithi179.stdout:6/133: dwrite d3/f1c [0,4194304] 0 2024-04-02T04:47:41.902 INFO:tasks.workunit.client.0.smithi070.stdout:3/156: creat d10/d17/d19/f3a x:0 0 0 2024-04-02T04:47:41.904 INFO:tasks.workunit.client.1.smithi179.stdout:5/117: symlink d8/d1b/d23/l28 0 2024-04-02T04:47:41.907 INFO:tasks.workunit.client.0.smithi070.stdout:3/157: creat d10/d17/d19/f3b x:0 0 0 2024-04-02T04:47:41.909 INFO:tasks.workunit.client.0.smithi070.stdout:8/187: write d2/d3/f9 [1757315,27814] 0 2024-04-02T04:47:41.909 INFO:tasks.workunit.client.1.smithi179.stdout:9/195: truncate d0/d1/f4 2783456 0 2024-04-02T04:47:41.910 INFO:tasks.workunit.client.0.smithi070.stdout:8/188: write d2/d8/d29/d22/f2c [247000,97456] 0 2024-04-02T04:47:41.911 INFO:tasks.workunit.client.0.smithi070.stdout:8/189: fdatasync d2/d8/d23/f31 0 2024-04-02T04:47:41.915 INFO:tasks.workunit.client.0.smithi070.stdout:3/158: rename d10/d17/d19/l1c to d10/d17/d19/l3c 0 2024-04-02T04:47:41.916 INFO:tasks.workunit.client.0.smithi070.stdout:3/159: dread - d10/d17/d19/f1f zero size 2024-04-02T04:47:41.917 INFO:tasks.workunit.client.0.smithi070.stdout:3/160: fdatasync d10/d17/f2a 0 2024-04-02T04:47:41.918 INFO:tasks.workunit.client.0.smithi070.stdout:3/161: write fb [216301,58646] 0 2024-04-02T04:47:41.919 INFO:tasks.workunit.client.0.smithi070.stdout:3/162: write d10/f29 [739630,104763] 0 2024-04-02T04:47:41.919 INFO:tasks.workunit.client.1.smithi179.stdout:4/234: dwrite d1/d3/f13 [0,4194304] 0 2024-04-02T04:47:41.920 INFO:tasks.workunit.client.0.smithi070.stdout:3/163: read d10/f22 [610521,16599] 0 2024-04-02T04:47:41.921 INFO:tasks.workunit.client.0.smithi070.stdout:8/190: mknod d2/d3/d35/c3e 0 2024-04-02T04:47:41.922 INFO:tasks.workunit.client.0.smithi070.stdout:8/191: fdatasync d2/d3/f9 0 2024-04-02T04:47:41.924 INFO:tasks.workunit.client.0.smithi070.stdout:8/192: truncate d2/d3/d6/f33 442959 0 2024-04-02T04:47:41.924 INFO:tasks.workunit.client.1.smithi179.stdout:1/217: write d3/d5/de/f16 [3796404,128574] 0 2024-04-02T04:47:41.924 INFO:tasks.workunit.client.0.smithi070.stdout:8/193: chown d2/c19 135804178 1 2024-04-02T04:47:41.925 INFO:tasks.workunit.client.1.smithi179.stdout:1/218: write d3/f2d [2680176,28434] 0 2024-04-02T04:47:41.926 INFO:tasks.workunit.client.0.smithi070.stdout:8/194: write d2/d3/d6/f3c [857885,98094] 0 2024-04-02T04:47:41.926 INFO:tasks.workunit.client.1.smithi179.stdout:1/219: fdatasync d3/f28 0 2024-04-02T04:47:41.927 INFO:tasks.workunit.client.1.smithi179.stdout:6/134: write d3/f18 [51209,90640] 0 2024-04-02T04:47:41.928 INFO:tasks.workunit.client.1.smithi179.stdout:1/220: write d3/d5/de/f3c [232752,34479] 0 2024-04-02T04:47:41.929 INFO:tasks.workunit.client.1.smithi179.stdout:6/135: write d3/d12/f22 [1151599,66301] 0 2024-04-02T04:47:41.932 INFO:tasks.workunit.client.0.smithi070.stdout:8/195: unlink d2/d3/f17 0 2024-04-02T04:47:41.933 INFO:tasks.workunit.client.1.smithi179.stdout:5/118: mknod d8/d1b/c29 0 2024-04-02T04:47:41.934 INFO:tasks.workunit.client.1.smithi179.stdout:5/119: readlink d8/l11 0 2024-04-02T04:47:41.935 INFO:tasks.workunit.client.1.smithi179.stdout:9/196: mkdir d0/d1/d11/d3a 0 2024-04-02T04:47:41.937 INFO:tasks.workunit.client.0.smithi070.stdout:3/164: link d10/d17/d19/l26 d10/l3d 0 2024-04-02T04:47:41.938 INFO:tasks.workunit.client.1.smithi179.stdout:6/136: dread d3/ff [0,4194304] 0 2024-04-02T04:47:41.938 INFO:tasks.workunit.client.1.smithi179.stdout:1/221: dwrite d3/f28 [4194304,4194304] 0 2024-04-02T04:47:41.939 INFO:tasks.workunit.client.1.smithi179.stdout:2/104: rmdir dc/d16/d1e 39 2024-04-02T04:47:41.940 INFO:tasks.workunit.client.1.smithi179.stdout:1/222: stat d3/d5/d2f/f47 0 2024-04-02T04:47:41.940 INFO:tasks.workunit.client.1.smithi179.stdout:9/197: write d0/d1/d10/f2b [3467318,68535] 0 2024-04-02T04:47:41.942 INFO:tasks.workunit.client.0.smithi070.stdout:8/196: link d2/d8/d29/d22/c2f d2/d8/d29/c3f 0 2024-04-02T04:47:41.943 INFO:tasks.workunit.client.1.smithi179.stdout:8/170: symlink d3/d32/d21/l3f 0 2024-04-02T04:47:41.946 INFO:tasks.workunit.client.1.smithi179.stdout:4/235: symlink d1/d3/d19/l4c 0 2024-04-02T04:47:41.946 INFO:tasks.workunit.client.1.smithi179.stdout:0/240: getdents d1/da/d28/d34/d39 0 2024-04-02T04:47:41.949 INFO:tasks.workunit.client.0.smithi070.stdout:3/165: rename d10/c34 to d10/d17/c3e 0 2024-04-02T04:47:41.950 INFO:tasks.workunit.client.0.smithi070.stdout:3/166: write d10/f29 [633939,95604] 0 2024-04-02T04:47:41.951 INFO:tasks.workunit.client.0.smithi070.stdout:8/197: symlink d2/l40 0 2024-04-02T04:47:41.953 INFO:tasks.workunit.client.1.smithi179.stdout:3/148: truncate d8/f23 13877 0 2024-04-02T04:47:41.954 INFO:tasks.workunit.client.0.smithi070.stdout:8/198: write d2/d8/d23/d24/f37 [289939,72895] 0 2024-04-02T04:47:41.957 INFO:tasks.workunit.client.1.smithi179.stdout:7/202: rename d1/c44 to d1/c4a 0 2024-04-02T04:47:41.958 INFO:tasks.workunit.client.1.smithi179.stdout:5/120: mknod d8/c2a 0 2024-04-02T04:47:41.958 INFO:tasks.workunit.client.0.smithi070.stdout:3/167: dwrite d10/d32/f36 [0,4194304] 0 2024-04-02T04:47:41.960 INFO:tasks.workunit.client.0.smithi070.stdout:8/199: mknod d2/d3/d6/c41 0 2024-04-02T04:47:41.963 INFO:tasks.workunit.client.0.smithi070.stdout:8/200: dread d2/d3/d6/f33 [0,4194304] 0 2024-04-02T04:47:41.963 INFO:tasks.workunit.client.1.smithi179.stdout:5/121: dwrite d8/fc [0,4194304] 0 2024-04-02T04:47:41.969 INFO:tasks.workunit.client.0.smithi070.stdout:8/201: unlink d2/d8/d29/d22/f2c 0 2024-04-02T04:47:41.974 INFO:tasks.workunit.client.0.smithi070.stdout:8/202: mknod d2/d8/d23/d24/c42 0 2024-04-02T04:47:41.975 INFO:tasks.workunit.client.0.smithi070.stdout:8/203: stat d2/d8/d23/d24/f37 0 2024-04-02T04:47:41.975 INFO:tasks.workunit.client.1.smithi179.stdout:2/105: dwrite dc/fe [0,4194304] 0 2024-04-02T04:47:41.975 INFO:tasks.workunit.client.1.smithi179.stdout:5/122: dwrite d8/fb [4194304,4194304] 0 2024-04-02T04:47:41.977 INFO:tasks.workunit.client.1.smithi179.stdout:2/106: chown dc/c19 29384 1 2024-04-02T04:47:41.978 INFO:tasks.workunit.client.1.smithi179.stdout:7/203: truncate d1/d9/fb 4883620 0 2024-04-02T04:47:41.981 INFO:tasks.workunit.client.0.smithi070.stdout:8/204: dwrite d2/d3/f9 [0,4194304] 0 2024-04-02T04:47:41.982 INFO:tasks.workunit.client.1.smithi179.stdout:7/204: fsync d1/d9/f42 0 2024-04-02T04:47:41.982 INFO:tasks.workunit.client.1.smithi179.stdout:6/137: symlink d3/l36 0 2024-04-02T04:47:41.985 INFO:tasks.workunit.client.1.smithi179.stdout:1/223: mknod d3/d5/c4d 0 2024-04-02T04:47:41.986 INFO:tasks.workunit.client.1.smithi179.stdout:1/224: readlink d3/l9 0 2024-04-02T04:47:41.987 INFO:tasks.workunit.client.1.smithi179.stdout:6/138: truncate d3/d12/f22 5202040 0 2024-04-02T04:47:41.987 INFO:tasks.workunit.client.1.smithi179.stdout:8/171: creat d3/d32/d21/d22/d33/d3c/f40 x:0 0 0 2024-04-02T04:47:41.988 INFO:tasks.workunit.client.1.smithi179.stdout:1/225: write d3/f2d [2288209,47230] 0 2024-04-02T04:47:41.988 INFO:tasks.workunit.client.1.smithi179.stdout:6/139: write d3/d12/f22 [2134111,31014] 0 2024-04-02T04:47:41.989 INFO:tasks.workunit.client.1.smithi179.stdout:6/140: fsync d3/d12/f25 0 2024-04-02T04:47:41.990 INFO:tasks.workunit.client.0.smithi070.stdout:8/205: creat d2/d3/d6/f43 x:0 0 0 2024-04-02T04:47:41.991 INFO:tasks.workunit.client.0.smithi070.stdout:8/206: fsync d2/d3/d6/f25 0 2024-04-02T04:47:41.991 INFO:tasks.workunit.client.1.smithi179.stdout:4/236: creat d1/d3/dc/d44/f4d x:0 0 0 2024-04-02T04:47:41.991 INFO:tasks.workunit.client.0.smithi070.stdout:8/207: stat d2/d8/d29/d22/d30 0 2024-04-02T04:47:41.992 INFO:tasks.workunit.client.0.smithi070.stdout:8/208: write d2/d3/d6/f43 [1041009,79728] 0 2024-04-02T04:47:41.993 INFO:tasks.workunit.client.0.smithi070.stdout:3/168: write d10/f2c [491767,56833] 0 2024-04-02T04:47:41.995 INFO:tasks.workunit.client.0.smithi070.stdout:3/169: fsync d10/d17/d19/f1f 0 2024-04-02T04:47:41.999 INFO:tasks.workunit.client.1.smithi179.stdout:5/123: creat d8/d1b/d23/f2b x:0 0 0 2024-04-02T04:47:42.000 INFO:tasks.workunit.client.1.smithi179.stdout:7/205: rmdir d1/dd 39 2024-04-02T04:47:42.000 INFO:tasks.workunit.client.0.smithi070.stdout:8/209: dwrite d2/d3/d6/f43 [0,4194304] 0 2024-04-02T04:47:42.003 INFO:tasks.workunit.client.0.smithi070.stdout:3/170: dwrite fb [0,4194304] 0 2024-04-02T04:47:42.006 INFO:tasks.workunit.client.0.smithi070.stdout:8/210: rmdir d2/d8/d29/db 39 2024-04-02T04:47:42.008 INFO:tasks.workunit.client.1.smithi179.stdout:8/172: mkdir d3/d1b/d2a/d41 0 2024-04-02T04:47:42.009 INFO:tasks.workunit.client.1.smithi179.stdout:8/173: write d3/f18 [1685933,47537] 0 2024-04-02T04:47:42.011 INFO:tasks.workunit.client.1.smithi179.stdout:8/174: write d3/d1b/f1c [391265,29627] 0 2024-04-02T04:47:42.013 INFO:tasks.workunit.client.1.smithi179.stdout:1/226: creat d3/d5/de/f4e x:0 0 0 2024-04-02T04:47:42.015 INFO:tasks.workunit.client.1.smithi179.stdout:1/227: chown d3/c11 488775 1 2024-04-02T04:47:42.017 INFO:tasks.workunit.client.1.smithi179.stdout:1/228: dread - d3/d5/d2f/f47 zero size 2024-04-02T04:47:42.017 INFO:tasks.workunit.client.1.smithi179.stdout:0/241: creat d1/da/f46 x:0 0 0 2024-04-02T04:47:42.019 INFO:tasks.workunit.client.1.smithi179.stdout:2/107: mknod dc/d16/d1e/c20 0 2024-04-02T04:47:42.020 INFO:tasks.workunit.client.0.smithi070.stdout:8/211: creat d2/d8/f44 x:0 0 0 2024-04-02T04:47:42.020 INFO:tasks.workunit.client.1.smithi179.stdout:2/108: stat dc/d16/f15 0 2024-04-02T04:47:42.021 INFO:tasks.workunit.client.0.smithi070.stdout:8/212: write d2/d8/d23/f31 [19897,53097] 0 2024-04-02T04:47:42.021 INFO:tasks.workunit.client.1.smithi179.stdout:4/237: creat d1/d3/df/d10/d2f/f4e x:0 0 0 2024-04-02T04:47:42.022 INFO:tasks.workunit.client.1.smithi179.stdout:4/238: chown d1/d3/df/d10/d2f 2849727 1 2024-04-02T04:47:42.022 INFO:tasks.workunit.client.0.smithi070.stdout:3/171: getdents d10/d32 0 2024-04-02T04:47:42.022 INFO:tasks.workunit.client.0.smithi070.stdout:3/172: chown d10/d17 0 1 2024-04-02T04:47:42.023 INFO:tasks.workunit.client.0.smithi070.stdout:3/173: chown d10/d17 978914 1 2024-04-02T04:47:42.023 INFO:tasks.workunit.client.1.smithi179.stdout:9/198: link d0/d1/d12/l39 d0/d19/d27/l3b 0 2024-04-02T04:47:42.024 INFO:tasks.workunit.client.1.smithi179.stdout:9/199: chown d0/d19/d27/c2c 797 1 2024-04-02T04:47:42.028 INFO:tasks.workunit.client.1.smithi179.stdout:0/242: read d1/da/df/f12 [5924830,122363] 0 2024-04-02T04:47:42.028 INFO:tasks.workunit.client.0.smithi070.stdout:3/174: rename d10/d17/d19 to d10/d32/d3f 0 2024-04-02T04:47:42.031 INFO:tasks.workunit.client.1.smithi179.stdout:8/175: rename d3/f7 to d3/d1b/d2a/d41/f42 0 2024-04-02T04:47:42.035 INFO:tasks.workunit.client.1.smithi179.stdout:2/109: dread dc/f12 [0,4194304] 0 2024-04-02T04:47:42.035 INFO:tasks.workunit.client.1.smithi179.stdout:6/141: link d3/fb d3/d12/d23/d26/f37 0 2024-04-02T04:47:42.037 INFO:tasks.workunit.client.1.smithi179.stdout:3/149: dread d8/f23 [0,4194304] 0 2024-04-02T04:47:42.037 INFO:tasks.workunit.client.1.smithi179.stdout:5/124: dread f2 [4194304,4194304] 0 2024-04-02T04:47:42.040 INFO:tasks.workunit.client.1.smithi179.stdout:4/239: symlink d1/d3/df/d10/d3a/l4f 0 2024-04-02T04:47:42.042 INFO:tasks.workunit.client.1.smithi179.stdout:4/240: chown d1/d3 504445 1 2024-04-02T04:47:42.042 INFO:tasks.workunit.client.1.smithi179.stdout:7/206: unlink d1/dd/d16/f38 0 2024-04-02T04:47:42.046 INFO:tasks.workunit.client.1.smithi179.stdout:0/243: symlink d1/db/l47 0 2024-04-02T04:47:42.048 INFO:tasks.workunit.client.0.smithi070.stdout:3/175: dread d10/f15 [0,4194304] 0 2024-04-02T04:47:42.048 INFO:tasks.workunit.client.1.smithi179.stdout:4/241: read d1/d3/dc/f18 [2390400,73405] 0 2024-04-02T04:47:42.050 INFO:tasks.workunit.client.1.smithi179.stdout:3/150: dwrite d8/f11 [0,4194304] 0 2024-04-02T04:47:42.052 INFO:tasks.workunit.client.1.smithi179.stdout:0/244: write d1/da/f1b [101098,71554] 0 2024-04-02T04:47:42.053 INFO:tasks.workunit.client.0.smithi070.stdout:8/213: truncate d2/d8/d29/f3b 2514728 0 2024-04-02T04:47:42.054 INFO:tasks.workunit.client.1.smithi179.stdout:5/125: dwrite d8/d1b/f26 [0,4194304] 0 2024-04-02T04:47:42.055 INFO:tasks.workunit.client.0.smithi070.stdout:3/176: mknod d10/d17/c40 0 2024-04-02T04:47:42.055 INFO:tasks.workunit.client.0.smithi070.stdout:3/177: dread - d10/d17/f27 zero size 2024-04-02T04:47:42.055 INFO:tasks.workunit.client.1.smithi179.stdout:8/176: write d3/d1b/d2a/d41/f42 [501245,15671] 0 2024-04-02T04:47:42.056 INFO:tasks.workunit.client.0.smithi070.stdout:3/178: write d10/d17/f30 [652297,105949] 0 2024-04-02T04:47:42.058 INFO:tasks.workunit.client.1.smithi179.stdout:9/200: getdents d0/d1/d11/d3a 0 2024-04-02T04:47:42.058 INFO:tasks.workunit.client.0.smithi070.stdout:8/214: symlink d2/d8/d23/d24/l45 0 2024-04-02T04:47:42.060 INFO:tasks.workunit.client.0.smithi070.stdout:8/215: write d2/d3/f7 [2522783,12390] 0 2024-04-02T04:47:42.061 INFO:tasks.workunit.client.0.smithi070.stdout:3/179: stat d10/c14 0 2024-04-02T04:47:42.063 INFO:tasks.workunit.client.0.smithi070.stdout:3/180: write d10/f29 [1816505,45847] 0 2024-04-02T04:47:42.063 INFO:tasks.workunit.client.0.smithi070.stdout:3/181: read f8 [2608363,99599] 0 2024-04-02T04:47:42.065 INFO:tasks.workunit.client.1.smithi179.stdout:6/142: dread d3/d12/f22 [0,4194304] 0 2024-04-02T04:47:42.066 INFO:tasks.workunit.client.1.smithi179.stdout:9/201: write d0/d1/d10/f2b [4002554,53586] 0 2024-04-02T04:47:42.067 INFO:tasks.workunit.client.0.smithi070.stdout:3/182: dread d10/d17/f2a [0,4194304] 0 2024-04-02T04:47:42.068 INFO:tasks.workunit.client.1.smithi179.stdout:7/207: dwrite d1/dd/d23/f2f [0,4194304] 0 2024-04-02T04:47:42.074 INFO:tasks.workunit.client.1.smithi179.stdout:9/202: readlink d0/d1/l15 0 2024-04-02T04:47:42.079 INFO:tasks.workunit.client.1.smithi179.stdout:5/126: dwrite d8/d1b/f1f [0,4194304] 0 2024-04-02T04:47:42.081 INFO:tasks.workunit.client.1.smithi179.stdout:5/127: chown d8/f12 0 1 2024-04-02T04:47:42.085 INFO:tasks.workunit.client.0.smithi070.stdout:8/216: creat d2/d8/d29/d22/f46 x:0 0 0 2024-04-02T04:47:42.088 INFO:tasks.workunit.client.0.smithi070.stdout:4/150: sync 2024-04-02T04:47:42.088 INFO:tasks.workunit.client.0.smithi070.stdout:0/303: sync 2024-04-02T04:47:42.088 INFO:tasks.workunit.client.0.smithi070.stdout:7/165: sync 2024-04-02T04:47:42.088 INFO:tasks.workunit.client.1.smithi179.stdout:0/245: dwrite d1/f2a [0,4194304] 0 2024-04-02T04:47:42.090 INFO:tasks.workunit.client.0.smithi070.stdout:0/304: dread d4/d5/d13/d2a/f20 [0,4194304] 0 2024-04-02T04:47:42.091 INFO:tasks.workunit.client.0.smithi070.stdout:0/305: chown d4/d5/da/c4b 16285 1 2024-04-02T04:47:42.095 INFO:tasks.workunit.client.0.smithi070.stdout:0/306: dread d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:42.105 INFO:tasks.workunit.client.1.smithi179.stdout:8/177: mknod d3/d32/d21/d22/d33/d3c/c43 0 2024-04-02T04:47:42.106 INFO:tasks.workunit.client.1.smithi179.stdout:8/178: chown d3/c13 21 1 2024-04-02T04:47:42.106 INFO:tasks.workunit.client.1.smithi179.stdout:4/242: symlink d1/d3/l50 0 2024-04-02T04:47:42.112 INFO:tasks.workunit.client.0.smithi070.stdout:0/307: mknod d4/d5/d13/d43/c50 0 2024-04-02T04:47:42.112 INFO:tasks.workunit.client.0.smithi070.stdout:0/308: chown d4/d5/da/d2d 63799 1 2024-04-02T04:47:42.113 INFO:tasks.workunit.client.0.smithi070.stdout:0/309: truncate d4/d5/d13/d43/f49 722975 0 2024-04-02T04:47:42.114 INFO:tasks.workunit.client.0.smithi070.stdout:8/217: creat d2/d3/d35/d3a/f47 x:0 0 0 2024-04-02T04:47:42.122 INFO:tasks.workunit.client.0.smithi070.stdout:0/310: mkdir d4/d5/d13/d17/d51 0 2024-04-02T04:47:42.123 INFO:tasks.workunit.client.1.smithi179.stdout:9/203: mknod d0/d1/d11/d29/c3c 0 2024-04-02T04:47:42.124 INFO:tasks.workunit.client.0.smithi070.stdout:4/151: rmdir d1/d3/d9 39 2024-04-02T04:47:42.124 INFO:tasks.workunit.client.1.smithi179.stdout:9/204: readlink d0/l3 0 2024-04-02T04:47:42.125 INFO:tasks.workunit.client.1.smithi179.stdout:5/128: creat d8/f2c x:0 0 0 2024-04-02T04:47:42.127 INFO:tasks.workunit.client.0.smithi070.stdout:7/166: rename d7/d1c/d29 to d7/d2f 0 2024-04-02T04:47:42.127 INFO:tasks.workunit.client.0.smithi070.stdout:7/167: chown d7/d1c/c22 431 1 2024-04-02T04:47:42.130 INFO:tasks.workunit.client.0.smithi070.stdout:4/152: chown d1/d3/d9/f1d 215459 1 2024-04-02T04:47:42.132 INFO:tasks.workunit.client.1.smithi179.stdout:2/110: rename dc/d16/l1c to dc/d16/l21 0 2024-04-02T04:47:42.134 INFO:tasks.workunit.client.1.smithi179.stdout:8/179: write d3/f4 [4396946,79143] 0 2024-04-02T04:47:42.134 INFO:tasks.workunit.client.0.smithi070.stdout:7/168: rename d7/dd to d7/dd/d30 22 2024-04-02T04:47:42.135 INFO:tasks.workunit.client.0.smithi070.stdout:7/169: fdatasync d7/d9/fa 0 2024-04-02T04:47:42.136 INFO:tasks.workunit.client.1.smithi179.stdout:1/229: rename d3/d31 to d3/d5/d2f/d4f 0 2024-04-02T04:47:42.136 INFO:tasks.workunit.client.1.smithi179.stdout:3/151: rename d8 to d8/d26/d31 22 2024-04-02T04:47:42.137 INFO:tasks.workunit.client.1.smithi179.stdout:3/152: stat c7 0 2024-04-02T04:47:42.138 INFO:tasks.workunit.client.0.smithi070.stdout:4/153: symlink d1/d3/d9/l2e 0 2024-04-02T04:47:42.138 INFO:tasks.workunit.client.1.smithi179.stdout:4/243: creat d1/d3/df/f51 x:0 0 0 2024-04-02T04:47:42.138 INFO:tasks.workunit.client.1.smithi179.stdout:3/153: stat d8/f22 0 2024-04-02T04:47:42.139 INFO:tasks.workunit.client.1.smithi179.stdout:4/244: fdatasync d1/d3/dc/d26/f29 0 2024-04-02T04:47:42.140 INFO:tasks.workunit.client.1.smithi179.stdout:4/245: chown d1/d15/c25 123 1 2024-04-02T04:47:42.141 INFO:tasks.workunit.client.1.smithi179.stdout:4/246: stat d1/d3/dc 0 2024-04-02T04:47:42.142 INFO:tasks.workunit.client.0.smithi070.stdout:7/170: creat d7/d2f/f31 x:0 0 0 2024-04-02T04:47:42.143 INFO:tasks.workunit.client.1.smithi179.stdout:1/230: dwrite d3/d5/f27 [0,4194304] 0 2024-04-02T04:47:42.143 INFO:tasks.workunit.client.1.smithi179.stdout:4/247: read d1/d3/f28 [1731729,36537] 0 2024-04-02T04:47:42.144 INFO:tasks.workunit.client.1.smithi179.stdout:4/248: read - d1/d3/df/f48 zero size 2024-04-02T04:47:42.145 INFO:tasks.workunit.client.1.smithi179.stdout:5/129: mkdir d8/d2d 0 2024-04-02T04:47:42.146 INFO:tasks.workunit.client.1.smithi179.stdout:5/130: truncate d8/f27 203680 0 2024-04-02T04:47:42.146 INFO:tasks.workunit.client.1.smithi179.stdout:7/208: link d1/dd/d16/f48 d1/d9/d27/f4b 0 2024-04-02T04:47:42.147 INFO:tasks.workunit.client.1.smithi179.stdout:7/209: stat d1/d9/c22 0 2024-04-02T04:47:42.147 INFO:tasks.workunit.client.1.smithi179.stdout:0/246: mkdir d1/da/d48 0 2024-04-02T04:47:42.148 INFO:tasks.workunit.client.0.smithi070.stdout:7/171: dwrite d7/d9/d11/f2a [0,4194304] 0 2024-04-02T04:47:42.154 INFO:tasks.workunit.client.0.smithi070.stdout:7/172: chown d7/l1d 8 1 2024-04-02T04:47:42.154 INFO:tasks.workunit.client.0.smithi070.stdout:7/173: fdatasync d7/d9/f21 0 2024-04-02T04:47:42.155 INFO:tasks.workunit.client.1.smithi179.stdout:8/180: unlink l0 0 2024-04-02T04:47:42.158 INFO:tasks.workunit.client.0.smithi070.stdout:7/174: creat d7/dd/df/f32 x:0 0 0 2024-04-02T04:47:42.159 INFO:tasks.workunit.client.0.smithi070.stdout:7/175: readlink d7/dd/l13 0 2024-04-02T04:47:42.163 INFO:tasks.workunit.client.0.smithi070.stdout:0/311: mkdir d4/d5/d13/d2a/d29/d52 0 2024-04-02T04:47:42.163 INFO:tasks.workunit.client.0.smithi070.stdout:7/176: symlink d7/d9/d11/l33 0 2024-04-02T04:47:42.164 INFO:tasks.workunit.client.1.smithi179.stdout:3/154: dread d8/f22 [0,4194304] 0 2024-04-02T04:47:42.164 INFO:tasks.workunit.client.1.smithi179.stdout:9/205: mknod d0/d19/c3d 0 2024-04-02T04:47:42.165 INFO:tasks.workunit.client.1.smithi179.stdout:3/155: read - d8/f2d zero size 2024-04-02T04:47:42.166 INFO:tasks.workunit.client.0.smithi070.stdout:0/312: creat d4/d5/d13/d43/f53 x:0 0 0 2024-04-02T04:47:42.170 INFO:tasks.workunit.client.1.smithi179.stdout:4/249: rmdir d1/d3/dc/d27 39 2024-04-02T04:47:42.172 INFO:tasks.workunit.client.0.smithi070.stdout:0/313: mknod d4/d5/d21/c54 0 2024-04-02T04:47:42.173 INFO:tasks.workunit.client.0.smithi070.stdout:0/314: truncate d4/d5/d13/d2a/d29/f46 972730 0 2024-04-02T04:47:42.174 INFO:tasks.workunit.client.1.smithi179.stdout:5/131: unlink d8/fc 0 2024-04-02T04:47:42.175 INFO:tasks.workunit.client.1.smithi179.stdout:7/210: mknod d1/dd/d16/c4c 0 2024-04-02T04:47:42.177 INFO:tasks.workunit.client.1.smithi179.stdout:0/247: creat d1/da/d28/d37/f49 x:0 0 0 2024-04-02T04:47:42.177 INFO:tasks.workunit.client.1.smithi179.stdout:4/250: dwrite d1/d3/df/d10/d3e/f45 [0,4194304] 0 2024-04-02T04:47:42.180 INFO:tasks.workunit.client.1.smithi179.stdout:0/248: dread - d1/da/d28/d34/f35 zero size 2024-04-02T04:47:42.181 INFO:tasks.workunit.client.1.smithi179.stdout:8/181: creat d3/d32/d21/d22/d33/d3c/f44 x:0 0 0 2024-04-02T04:47:42.181 INFO:tasks.workunit.client.1.smithi179.stdout:6/143: getdents d3/d12/d23 0 2024-04-02T04:47:42.183 INFO:tasks.workunit.client.1.smithi179.stdout:9/206: symlink d0/d1/d12/l3e 0 2024-04-02T04:47:42.183 INFO:tasks.workunit.client.1.smithi179.stdout:0/249: chown d1/da/l16 63 1 2024-04-02T04:47:42.184 INFO:tasks.workunit.client.1.smithi179.stdout:8/182: read d3/f4 [3265612,111099] 0 2024-04-02T04:47:42.185 INFO:tasks.workunit.client.1.smithi179.stdout:3/156: creat d8/f32 x:0 0 0 2024-04-02T04:47:42.185 INFO:tasks.workunit.client.1.smithi179.stdout:0/250: stat d1/db/c25 0 2024-04-02T04:47:42.187 INFO:tasks.workunit.client.1.smithi179.stdout:7/211: dwrite d1/f45 [0,4194304] 0 2024-04-02T04:47:42.192 INFO:tasks.workunit.client.1.smithi179.stdout:4/251: mknod d1/d3/df/c52 0 2024-04-02T04:47:42.192 INFO:tasks.workunit.client.1.smithi179.stdout:7/212: write d1/dd/f14 [620748,11590] 0 2024-04-02T04:47:42.193 INFO:tasks.workunit.client.1.smithi179.stdout:6/144: mkdir d3/d2d/d38 0 2024-04-02T04:47:42.193 INFO:tasks.workunit.client.1.smithi179.stdout:4/252: readlink d1/d15/l40 0 2024-04-02T04:47:42.194 INFO:tasks.workunit.client.1.smithi179.stdout:6/145: readlink d3/l20 0 2024-04-02T04:47:42.195 INFO:tasks.workunit.client.1.smithi179.stdout:6/146: write d3/f10 [1462204,19656] 0 2024-04-02T04:47:42.195 INFO:tasks.workunit.client.1.smithi179.stdout:6/147: fsync d3/d12/f25 0 2024-04-02T04:47:42.205 INFO:tasks.workunit.client.1.smithi179.stdout:3/157: creat d8/d26/f33 x:0 0 0 2024-04-02T04:47:42.205 INFO:tasks.workunit.client.1.smithi179.stdout:3/158: chown d8/l1f 4518750 1 2024-04-02T04:47:42.207 INFO:tasks.workunit.client.1.smithi179.stdout:7/213: readlink d1/dd/l21 0 2024-04-02T04:47:42.211 INFO:tasks.workunit.client.1.smithi179.stdout:8/183: mkdir d3/d32/d21/d45 0 2024-04-02T04:47:42.212 INFO:tasks.workunit.client.1.smithi179.stdout:2/111: getdents dc/d16 0 2024-04-02T04:47:42.213 INFO:tasks.workunit.client.1.smithi179.stdout:7/214: mkdir d1/dd/d16/d4d 0 2024-04-02T04:47:42.214 INFO:tasks.workunit.client.1.smithi179.stdout:4/253: unlink d1/d3/dc/d27/l22 0 2024-04-02T04:47:42.217 INFO:tasks.workunit.client.1.smithi179.stdout:8/184: symlink d3/d31/l46 0 2024-04-02T04:47:42.218 INFO:tasks.workunit.client.1.smithi179.stdout:8/185: write d3/d1b/f1c [1196616,61370] 0 2024-04-02T04:47:42.219 INFO:tasks.workunit.client.1.smithi179.stdout:9/207: getdents d0 0 2024-04-02T04:47:42.221 INFO:tasks.workunit.client.1.smithi179.stdout:2/112: creat dc/d16/f22 x:0 0 0 2024-04-02T04:47:42.225 INFO:tasks.workunit.client.1.smithi179.stdout:8/186: dwrite d3/d32/d21/d22/d33/d3c/f40 [0,4194304] 0 2024-04-02T04:47:42.226 INFO:tasks.workunit.client.1.smithi179.stdout:8/187: write d3/d32/f19 [4346761,97658] 0 2024-04-02T04:47:42.230 INFO:tasks.workunit.client.1.smithi179.stdout:4/254: dwrite d1/fb [0,4194304] 0 2024-04-02T04:47:42.232 INFO:tasks.workunit.client.1.smithi179.stdout:4/255: readlink d1/d3/d19/l33 0 2024-04-02T04:47:42.232 INFO:tasks.workunit.client.1.smithi179.stdout:2/113: chown dc/d16/d1e/c20 203305 1 2024-04-02T04:47:42.235 INFO:tasks.workunit.client.0.smithi070.stdout:4/154: link d1/d3/d9/l2e d1/l2f 0 2024-04-02T04:47:42.235 INFO:tasks.workunit.client.0.smithi070.stdout:4/155: chown d1/f28 3 1 2024-04-02T04:47:42.239 INFO:tasks.workunit.client.1.smithi179.stdout:2/114: creat dc/d16/d1e/f23 x:0 0 0 2024-04-02T04:47:42.240 INFO:tasks.workunit.client.1.smithi179.stdout:8/188: dwrite d3/d1b/d2a/d41/f42 [0,4194304] 0 2024-04-02T04:47:42.244 INFO:tasks.workunit.client.0.smithi070.stdout:7/177: rename d7/d9/d11 to d7/dd/df/d34 0 2024-04-02T04:47:42.245 INFO:tasks.workunit.client.0.smithi070.stdout:7/178: readlink d7/dd/df/d34/l23 0 2024-04-02T04:47:42.245 INFO:tasks.workunit.client.1.smithi179.stdout:4/256: symlink d1/l53 0 2024-04-02T04:47:42.245 INFO:tasks.workunit.client.1.smithi179.stdout:4/257: readlink d1/d3/d19/l4c 0 2024-04-02T04:47:42.248 INFO:tasks.workunit.client.0.smithi070.stdout:4/156: creat d1/d3/f30 x:0 0 0 2024-04-02T04:47:42.249 INFO:tasks.workunit.client.0.smithi070.stdout:4/157: stat d1/d3/d9/f29 0 2024-04-02T04:47:42.250 INFO:tasks.workunit.client.0.smithi070.stdout:4/158: chown d1/d3/f2b 15390 1 2024-04-02T04:47:42.251 INFO:tasks.workunit.client.1.smithi179.stdout:8/189: write d3/d1b/d2a/d41/f42 [559250,104506] 0 2024-04-02T04:47:42.251 INFO:tasks.workunit.client.1.smithi179.stdout:4/258: creat d1/d3/dc/d44/f54 x:0 0 0 2024-04-02T04:47:42.251 INFO:tasks.workunit.client.0.smithi070.stdout:4/159: fdatasync d1/f1e 0 2024-04-02T04:47:42.253 INFO:tasks.workunit.client.1.smithi179.stdout:4/259: truncate d1/d3/df/d10/d3e/f4a 963562 0 2024-04-02T04:47:42.253 INFO:tasks.workunit.client.1.smithi179.stdout:1/231: write d3/fa [80930,106846] 0 2024-04-02T04:47:42.253 INFO:tasks.workunit.client.0.smithi070.stdout:7/179: dwrite d7/d2f/f2d [0,4194304] 0 2024-04-02T04:47:42.257 INFO:tasks.workunit.client.1.smithi179.stdout:2/115: rename f9 to dc/d16/f24 0 2024-04-02T04:47:42.257 INFO:tasks.workunit.client.1.smithi179.stdout:2/116: dread - dc/f17 zero size 2024-04-02T04:47:42.263 INFO:tasks.workunit.client.1.smithi179.stdout:2/117: chown dc/d16/f15 899125477 1 2024-04-02T04:47:42.265 INFO:tasks.workunit.client.0.smithi070.stdout:8/218: link d2/d8/f44 d2/f48 0 2024-04-02T04:47:42.265 INFO:tasks.workunit.client.1.smithi179.stdout:4/260: getdents d1/d3/df/d10/d2f/d36 0 2024-04-02T04:47:42.266 INFO:tasks.workunit.client.1.smithi179.stdout:4/261: chown d1/d3/dc/l37 30598358 1 2024-04-02T04:47:42.266 INFO:tasks.workunit.client.0.smithi070.stdout:0/315: dwrite d4/f9 [0,4194304] 0 2024-04-02T04:47:42.269 INFO:tasks.workunit.client.1.smithi179.stdout:8/190: link d3/d32/d21/d22/d33/d3c/f40 d3/d32/f47 0 2024-04-02T04:47:42.272 INFO:tasks.workunit.client.0.smithi070.stdout:8/219: dread d2/d3/f9 [4194304,4194304] 0 2024-04-02T04:47:42.273 INFO:tasks.workunit.client.1.smithi179.stdout:4/262: creat d1/d3/df/d10/d3e/f55 x:0 0 0 2024-04-02T04:47:42.276 INFO:tasks.workunit.client.1.smithi179.stdout:8/191: write d3/d32/f47 [3796250,16098] 0 2024-04-02T04:47:42.277 INFO:tasks.workunit.client.1.smithi179.stdout:4/263: unlink d1/f4 0 2024-04-02T04:47:42.278 INFO:tasks.workunit.client.1.smithi179.stdout:4/264: dread - d1/d3/df/d10/d2f/f4e zero size 2024-04-02T04:47:42.279 INFO:tasks.workunit.client.0.smithi070.stdout:8/220: creat d2/d3/f49 x:0 0 0 2024-04-02T04:47:42.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:42 smithi070 ceph-mon[29212]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:42.280 INFO:tasks.workunit.client.1.smithi179.stdout:8/192: mknod d3/d32/d30/c48 0 2024-04-02T04:47:42.281 INFO:tasks.workunit.client.1.smithi179.stdout:8/193: truncate d3/f27 755889 0 2024-04-02T04:47:42.282 INFO:tasks.workunit.client.1.smithi179.stdout:4/265: unlink d1/d15/l24 0 2024-04-02T04:47:42.282 INFO:tasks.workunit.client.1.smithi179.stdout:5/132: dwrite f1 [0,4194304] 0 2024-04-02T04:47:42.282 INFO:tasks.workunit.client.0.smithi070.stdout:8/221: dread d2/d3/d6/f25 [0,4194304] 0 2024-04-02T04:47:42.284 INFO:tasks.workunit.client.0.smithi070.stdout:8/222: readlink d2/l1a 0 2024-04-02T04:47:42.284 INFO:tasks.workunit.client.1.smithi179.stdout:0/251: write d1/f2 [3996166,113505] 0 2024-04-02T04:47:42.285 INFO:tasks.workunit.client.1.smithi179.stdout:5/133: mknod d8/d1b/d23/c2e 0 2024-04-02T04:47:42.289 INFO:tasks.workunit.client.1.smithi179.stdout:8/194: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:42.290 INFO:tasks.workunit.client.1.smithi179.stdout:6/148: truncate d3/f10 444504 0 2024-04-02T04:47:42.290 INFO:tasks.workunit.client.1.smithi179.stdout:6/149: chown d3/ff 31 1 2024-04-02T04:47:42.292 INFO:tasks.workunit.client.1.smithi179.stdout:4/266: link d1/d3/dc/f1c d1/d15/f56 0 2024-04-02T04:47:42.292 INFO:tasks.workunit.client.1.smithi179.stdout:3/159: write f5 [1574687,101397] 0 2024-04-02T04:47:42.293 INFO:tasks.workunit.client.1.smithi179.stdout:4/267: fsync d1/d3/df/d10/f11 0 2024-04-02T04:47:42.293 INFO:tasks.workunit.client.1.smithi179.stdout:5/134: creat d8/d1b/d23/f2f x:0 0 0 2024-04-02T04:47:42.294 INFO:tasks.workunit.client.1.smithi179.stdout:5/135: truncate d8/f10 297691 0 2024-04-02T04:47:42.295 INFO:tasks.workunit.client.1.smithi179.stdout:3/160: truncate d8/d26/f33 185770 0 2024-04-02T04:47:42.295 INFO:tasks.workunit.client.1.smithi179.stdout:6/150: mkdir d3/d12/d23/d39 0 2024-04-02T04:47:42.297 INFO:tasks.workunit.client.1.smithi179.stdout:3/161: write f5 [360034,85994] 0 2024-04-02T04:47:42.299 INFO:tasks.workunit.client.1.smithi179.stdout:4/268: fdatasync d1/f3c 0 2024-04-02T04:47:42.301 INFO:tasks.workunit.client.1.smithi179.stdout:0/252: creat d1/da/f4a x:0 0 0 2024-04-02T04:47:42.303 INFO:tasks.workunit.client.1.smithi179.stdout:8/195: mkdir d3/d32/d21/d49 0 2024-04-02T04:47:42.305 INFO:tasks.workunit.client.1.smithi179.stdout:6/151: fdatasync d3/d12/d23/d26/f37 0 2024-04-02T04:47:42.308 INFO:tasks.workunit.client.1.smithi179.stdout:9/208: dwrite d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:42.308 INFO:tasks.workunit.client.1.smithi179.stdout:7/215: dwrite d1/d9/d35/f3b [0,4194304] 0 2024-04-02T04:47:42.311 INFO:tasks.workunit.client.1.smithi179.stdout:7/216: readlink d1/dd/l2b 0 2024-04-02T04:47:42.312 INFO:tasks.workunit.client.1.smithi179.stdout:8/196: write d3/d1b/d2a/d41/f42 [4537661,127240] 0 2024-04-02T04:47:42.315 INFO:tasks.workunit.client.1.smithi179.stdout:3/162: truncate d8/f22 5031924 0 2024-04-02T04:47:42.316 INFO:tasks.workunit.client.0.smithi070.stdout:7/180: truncate d7/d9/f17 789912 0 2024-04-02T04:47:42.317 INFO:tasks.workunit.client.0.smithi070.stdout:7/181: truncate d7/d9/f27 837189 0 2024-04-02T04:47:42.319 INFO:tasks.workunit.client.1.smithi179.stdout:4/269: dwrite d1/d3/f1b [0,4194304] 0 2024-04-02T04:47:42.320 INFO:tasks.workunit.client.1.smithi179.stdout:5/136: rmdir d8/d2d 0 2024-04-02T04:47:42.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:42 smithi179 ceph-mon[36603]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:42.322 INFO:tasks.workunit.client.0.smithi070.stdout:7/182: dread d7/d9/f21 [0,4194304] 0 2024-04-02T04:47:42.322 INFO:tasks.workunit.client.0.smithi070.stdout:7/183: stat d7/l1d 0 2024-04-02T04:47:42.323 INFO:tasks.workunit.client.0.smithi070.stdout:7/184: dread - d7/d2f/f31 zero size 2024-04-02T04:47:42.323 INFO:tasks.workunit.client.1.smithi179.stdout:6/152: dwrite d3/d12/f34 [0,4194304] 0 2024-04-02T04:47:42.324 INFO:tasks.workunit.client.0.smithi070.stdout:7/185: dread - d7/dd/df/f32 zero size 2024-04-02T04:47:42.325 INFO:tasks.workunit.client.0.smithi070.stdout:7/186: stat d7/d9/d19/l2b 0 2024-04-02T04:47:42.325 INFO:tasks.workunit.client.1.smithi179.stdout:8/197: dread d3/d1b/f1c [0,4194304] 0 2024-04-02T04:47:42.325 INFO:tasks.workunit.client.0.smithi070.stdout:7/187: fsync d7/d2f/f31 0 2024-04-02T04:47:42.327 INFO:tasks.workunit.client.0.smithi070.stdout:0/316: write d4/f25 [3897901,71120] 0 2024-04-02T04:47:42.329 INFO:tasks.workunit.client.1.smithi179.stdout:5/137: mkdir d8/d1b/d23/d30 0 2024-04-02T04:47:42.330 INFO:tasks.workunit.client.1.smithi179.stdout:5/138: chown d8/d1b/f1f 2001 1 2024-04-02T04:47:42.331 INFO:tasks.workunit.client.1.smithi179.stdout:6/153: dwrite d3/ff [0,4194304] 0 2024-04-02T04:47:42.332 INFO:tasks.workunit.client.0.smithi070.stdout:7/188: dread d7/dd/df/d34/f2a [0,4194304] 0 2024-04-02T04:47:42.335 INFO:tasks.workunit.client.0.smithi070.stdout:0/317: mkdir d4/d5/d13/d23/d55 0 2024-04-02T04:47:42.343 INFO:tasks.workunit.client.0.smithi070.stdout:0/318: dwrite d4/d5/d13/f19 [4194304,4194304] 0 2024-04-02T04:47:42.343 INFO:tasks.workunit.client.1.smithi179.stdout:7/217: creat d1/f4e x:0 0 0 2024-04-02T04:47:42.346 INFO:tasks.workunit.client.1.smithi179.stdout:4/270: link d1/f3c d1/d3/df/f57 0 2024-04-02T04:47:42.347 INFO:tasks.workunit.client.0.smithi070.stdout:8/223: dwrite d2/d8/d29/f15 [0,4194304] 0 2024-04-02T04:47:42.347 INFO:tasks.workunit.client.0.smithi070.stdout:8/224: chown d2/c20 114247691 1 2024-04-02T04:47:42.348 INFO:tasks.workunit.client.1.smithi179.stdout:5/139: symlink d8/d1b/d23/l31 0 2024-04-02T04:47:42.349 INFO:tasks.workunit.client.0.smithi070.stdout:0/319: dread d4/f9 [0,4194304] 0 2024-04-02T04:47:42.354 INFO:tasks.workunit.client.0.smithi070.stdout:7/189: rename d7/d1c to d7/dd/d35 0 2024-04-02T04:47:42.354 INFO:tasks.workunit.client.1.smithi179.stdout:5/140: dread d8/f13 [0,4194304] 0 2024-04-02T04:47:42.357 INFO:tasks.workunit.client.1.smithi179.stdout:5/141: fsync d8/f13 0 2024-04-02T04:47:42.358 INFO:tasks.workunit.client.1.smithi179.stdout:5/142: fdatasync d8/fb 0 2024-04-02T04:47:42.358 INFO:tasks.workunit.client.0.smithi070.stdout:8/225: rmdir d2 39 2024-04-02T04:47:42.358 INFO:tasks.workunit.client.1.smithi179.stdout:8/198: symlink d3/d32/d21/l4a 0 2024-04-02T04:47:42.359 INFO:tasks.workunit.client.1.smithi179.stdout:7/218: mknod d1/d9/d35/c4f 0 2024-04-02T04:47:42.360 INFO:tasks.workunit.client.1.smithi179.stdout:0/253: write d1/f22 [308232,79929] 0 2024-04-02T04:47:42.363 INFO:tasks.workunit.client.1.smithi179.stdout:3/163: dread d8/d26/f2c [0,4194304] 0 2024-04-02T04:47:42.364 INFO:tasks.workunit.client.1.smithi179.stdout:6/154: creat d3/d12/d23/d2f/f3a x:0 0 0 2024-04-02T04:47:42.365 INFO:tasks.workunit.client.0.smithi070.stdout:0/320: mknod d4/d5/c56 0 2024-04-02T04:47:42.366 INFO:tasks.workunit.client.1.smithi179.stdout:9/209: link d0/d19/d27/l3b d0/d1/d11/d29/l3f 0 2024-04-02T04:47:42.366 INFO:tasks.workunit.client.0.smithi070.stdout:0/321: write d4/d5/d13/d2a/d29/f46 [120187,56644] 0 2024-04-02T04:47:42.367 INFO:tasks.workunit.client.0.smithi070.stdout:7/190: mkdir d7/d36 0 2024-04-02T04:47:42.367 INFO:tasks.workunit.client.0.smithi070.stdout:7/191: chown d7/dd 2479 1 2024-04-02T04:47:42.370 INFO:tasks.workunit.client.0.smithi070.stdout:8/226: rename d2/d8/d23/d24/c42 to d2/d8/d23/d36/c4a 0 2024-04-02T04:47:42.371 INFO:tasks.workunit.client.1.smithi179.stdout:5/143: mkdir d8/d1b/d23/d32 0 2024-04-02T04:47:42.373 INFO:tasks.workunit.client.1.smithi179.stdout:8/199: fsync d3/d1b/f1c 0 2024-04-02T04:47:42.373 INFO:tasks.workunit.client.0.smithi070.stdout:0/322: dwrite d4/d5/d13/d23/f36 [0,4194304] 0 2024-04-02T04:47:42.374 INFO:tasks.workunit.client.1.smithi179.stdout:8/200: stat d3/d1b/d2a/d41 0 2024-04-02T04:47:42.375 INFO:tasks.workunit.client.1.smithi179.stdout:7/219: creat d1/d9/d35/f50 x:0 0 0 2024-04-02T04:47:42.376 INFO:tasks.workunit.client.1.smithi179.stdout:7/220: chown l0 3285 1 2024-04-02T04:47:42.377 INFO:tasks.workunit.client.1.smithi179.stdout:7/221: dread - d1/d9/d35/f50 zero size 2024-04-02T04:47:42.379 INFO:tasks.workunit.client.0.smithi070.stdout:9/200: sync 2024-04-02T04:47:42.379 INFO:tasks.workunit.client.0.smithi070.stdout:2/289: sync 2024-04-02T04:47:42.379 INFO:tasks.workunit.client.0.smithi070.stdout:1/165: sync 2024-04-02T04:47:42.379 INFO:tasks.workunit.client.0.smithi070.stdout:5/207: sync 2024-04-02T04:47:42.379 INFO:tasks.workunit.client.0.smithi070.stdout:6/115: sync 2024-04-02T04:47:42.381 INFO:tasks.workunit.client.0.smithi070.stdout:5/208: write d5/d17/d27/f28 [2866850,119956] 0 2024-04-02T04:47:42.383 INFO:tasks.workunit.client.0.smithi070.stdout:8/227: creat d2/d8/d23/d24/f4b x:0 0 0 2024-04-02T04:47:42.388 INFO:tasks.workunit.client.1.smithi179.stdout:6/155: rename d3/l1a to d3/d2c/l3b 0 2024-04-02T04:47:42.388 INFO:tasks.workunit.client.0.smithi070.stdout:2/290: dread d2/fb [0,4194304] 0 2024-04-02T04:47:42.389 INFO:tasks.workunit.client.0.smithi070.stdout:7/192: mknod d7/c37 0 2024-04-02T04:47:42.390 INFO:tasks.workunit.client.1.smithi179.stdout:9/210: mkdir d0/d1/d11/d29/d40 0 2024-04-02T04:47:42.390 INFO:tasks.workunit.client.1.smithi179.stdout:9/211: chown d0/d19/d1c/f31 32356 1 2024-04-02T04:47:42.393 INFO:tasks.workunit.client.1.smithi179.stdout:5/144: unlink d8/l20 0 2024-04-02T04:47:42.394 INFO:tasks.workunit.client.1.smithi179.stdout:5/145: write d8/f12 [4173462,56522] 0 2024-04-02T04:47:42.397 INFO:tasks.workunit.client.1.smithi179.stdout:7/222: readlink d1/d9/d27/l3e 0 2024-04-02T04:47:42.398 INFO:tasks.workunit.client.1.smithi179.stdout:7/223: read - d1/d9/f1d zero size 2024-04-02T04:47:42.398 INFO:tasks.workunit.client.1.smithi179.stdout:7/224: dread - d1/d9/f42 zero size 2024-04-02T04:47:42.399 INFO:tasks.workunit.client.1.smithi179.stdout:0/254: mkdir d1/da/d28/d34/d39/d4b 0 2024-04-02T04:47:42.400 INFO:tasks.workunit.client.1.smithi179.stdout:0/255: write d1/db/f2f [4614900,99511] 0 2024-04-02T04:47:42.400 INFO:tasks.workunit.client.1.smithi179.stdout:4/271: chown d1/d3/fa 377162 1 2024-04-02T04:47:42.402 INFO:tasks.workunit.client.1.smithi179.stdout:6/156: creat d3/d12/d23/d2f/f3c x:0 0 0 2024-04-02T04:47:42.402 INFO:tasks.workunit.client.1.smithi179.stdout:6/157: chown d3/d12/f21 9014 1 2024-04-02T04:47:42.403 INFO:tasks.workunit.client.1.smithi179.stdout:9/212: creat d0/d19/d1c/f41 x:0 0 0 2024-04-02T04:47:42.404 INFO:tasks.workunit.client.1.smithi179.stdout:6/158: write d3/f18 [727142,103663] 0 2024-04-02T04:47:42.404 INFO:tasks.workunit.client.1.smithi179.stdout:8/201: mknod d3/d32/d21/d49/c4b 0 2024-04-02T04:47:42.404 INFO:tasks.workunit.client.1.smithi179.stdout:7/225: symlink d1/d9/l51 0 2024-04-02T04:47:42.406 INFO:tasks.workunit.client.1.smithi179.stdout:0/256: mknod d1/da/d28/d37/c4c 0 2024-04-02T04:47:42.407 INFO:tasks.workunit.client.1.smithi179.stdout:6/159: write d3/d12/d23/d26/f27 [882265,103598] 0 2024-04-02T04:47:42.407 INFO:tasks.workunit.client.1.smithi179.stdout:7/226: dread - d1/dd/d16/f2c zero size 2024-04-02T04:47:42.408 INFO:tasks.workunit.client.1.smithi179.stdout:8/202: chown d3/d1b/d2a/d41/f42 3430 1 2024-04-02T04:47:42.408 INFO:tasks.workunit.client.1.smithi179.stdout:5/146: getdents d8/d1b/d23/d32 0 2024-04-02T04:47:42.409 INFO:tasks.workunit.client.1.smithi179.stdout:6/160: stat d3/d12/d23/d26/f37 0 2024-04-02T04:47:42.411 INFO:tasks.workunit.client.1.smithi179.stdout:7/227: write d1/dd/f20 [369248,109241] 0 2024-04-02T04:47:42.411 INFO:tasks.workunit.client.0.smithi070.stdout:1/166: creat d0/d9/d1d/f38 x:0 0 0 2024-04-02T04:47:42.412 INFO:tasks.workunit.client.0.smithi070.stdout:6/116: rename d0/f24 to d0/d20/f28 0 2024-04-02T04:47:42.412 INFO:tasks.workunit.client.1.smithi179.stdout:8/203: creat d3/d1b/d2a/f4c x:0 0 0 2024-04-02T04:47:42.413 INFO:tasks.workunit.client.0.smithi070.stdout:5/209: rmdir d5 39 2024-04-02T04:47:42.413 INFO:tasks.workunit.client.0.smithi070.stdout:0/323: dwrite d4/f1b [0,4194304] 0 2024-04-02T04:47:42.413 INFO:tasks.workunit.client.1.smithi179.stdout:9/213: dwrite d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:42.414 INFO:tasks.workunit.client.1.smithi179.stdout:7/228: truncate d1/f4e 490831 0 2024-04-02T04:47:42.415 INFO:tasks.workunit.client.0.smithi070.stdout:8/228: write d2/d3/f9 [3312506,93167] 0 2024-04-02T04:47:42.415 INFO:tasks.workunit.client.0.smithi070.stdout:0/324: stat d4/d5/d13/d23/l4e 0 2024-04-02T04:47:42.416 INFO:tasks.workunit.client.0.smithi070.stdout:2/291: fsync d2/f14 0 2024-04-02T04:47:42.417 INFO:tasks.workunit.client.0.smithi070.stdout:0/325: write d4/d5/da/d2d/f4c [867431,105833] 0 2024-04-02T04:47:42.418 INFO:tasks.workunit.client.1.smithi179.stdout:6/161: dread d3/d12/d23/d26/f27 [0,4194304] 0 2024-04-02T04:47:42.419 INFO:tasks.workunit.client.1.smithi179.stdout:6/162: write d3/d12/f25 [2581485,81642] 0 2024-04-02T04:47:42.420 INFO:tasks.workunit.client.0.smithi070.stdout:1/167: rmdir d0/d9 39 2024-04-02T04:47:42.424 INFO:tasks.workunit.client.0.smithi070.stdout:8/229: dwrite d2/d3/fd [4194304,4194304] 0 2024-04-02T04:47:42.425 INFO:tasks.workunit.client.1.smithi179.stdout:9/214: unlink d0/d1/f26 0 2024-04-02T04:47:42.425 INFO:tasks.workunit.client.0.smithi070.stdout:8/230: chown d2/d8/d29/d22/f46 4 1 2024-04-02T04:47:42.426 INFO:tasks.workunit.client.1.smithi179.stdout:9/215: readlink d0/d1/d12/l3e 0 2024-04-02T04:47:42.426 INFO:tasks.workunit.client.0.smithi070.stdout:8/231: stat d2/d3/d35 0 2024-04-02T04:47:42.427 INFO:tasks.workunit.client.0.smithi070.stdout:8/232: truncate d2/f48 823788 0 2024-04-02T04:47:42.428 INFO:tasks.workunit.client.1.smithi179.stdout:0/257: creat d1/f4d x:0 0 0 2024-04-02T04:47:42.431 INFO:tasks.workunit.client.0.smithi070.stdout:0/326: symlink d4/d5/d13/d2a/d29/l57 0 2024-04-02T04:47:42.432 INFO:tasks.workunit.client.1.smithi179.stdout:6/163: creat d3/d12/d23/d2f/f3d x:0 0 0 2024-04-02T04:47:42.434 INFO:tasks.workunit.client.1.smithi179.stdout:4/272: creat d1/d3/df/d10/d2f/d36/f58 x:0 0 0 2024-04-02T04:47:42.434 INFO:tasks.workunit.client.0.smithi070.stdout:6/117: symlink d0/d3/l29 0 2024-04-02T04:47:42.436 INFO:tasks.workunit.client.1.smithi179.stdout:9/216: creat d0/d1/d11/f42 x:0 0 0 2024-04-02T04:47:42.437 INFO:tasks.workunit.client.1.smithi179.stdout:7/229: symlink d1/dd/d16/d4d/l52 0 2024-04-02T04:47:42.439 INFO:tasks.workunit.client.0.smithi070.stdout:2/292: creat d2/d12/d59/f5e x:0 0 0 2024-04-02T04:47:42.441 INFO:tasks.workunit.client.0.smithi070.stdout:1/168: mknod d0/d9/d1d/d22/c39 0 2024-04-02T04:47:42.442 INFO:tasks.workunit.client.0.smithi070.stdout:5/210: dread d5/ff [0,4194304] 0 2024-04-02T04:47:42.442 INFO:tasks.workunit.client.0.smithi070.stdout:5/211: readlink d5/d17/l20 0 2024-04-02T04:47:42.443 INFO:tasks.workunit.client.1.smithi179.stdout:4/273: rename d1/d3/df/d10/f43 to d1/d3/df/f59 0 2024-04-02T04:47:42.444 INFO:tasks.workunit.client.0.smithi070.stdout:6/118: chown d0/d3/d19/f1f 1 1 2024-04-02T04:47:42.445 INFO:tasks.workunit.client.0.smithi070.stdout:1/169: dread d0/d9/d1d/f21 [0,4194304] 0 2024-04-02T04:47:42.445 INFO:tasks.workunit.client.0.smithi070.stdout:7/193: link d7/dd/df/f32 d7/dd/f38 0 2024-04-02T04:47:42.446 INFO:tasks.workunit.client.1.smithi179.stdout:0/258: mknod d1/da/d48/c4e 0 2024-04-02T04:47:42.448 INFO:tasks.workunit.client.1.smithi179.stdout:7/230: rename d1/dd/d23/c36 to d1/dd/d23/c53 0 2024-04-02T04:47:42.449 INFO:tasks.workunit.client.1.smithi179.stdout:5/147: getdents d8/d1b/d23 0 2024-04-02T04:47:42.450 INFO:tasks.workunit.client.1.smithi179.stdout:5/148: dread - d8/d1b/d23/f2f zero size 2024-04-02T04:47:42.450 INFO:tasks.workunit.client.0.smithi070.stdout:2/293: symlink d2/d10/d21/l5f 0 2024-04-02T04:47:42.452 INFO:tasks.workunit.client.0.smithi070.stdout:0/327: mkdir d4/d5/d58 0 2024-04-02T04:47:42.453 INFO:tasks.workunit.client.1.smithi179.stdout:4/274: creat d1/d3/df/f5a x:0 0 0 2024-04-02T04:47:42.457 INFO:tasks.workunit.client.1.smithi179.stdout:7/231: mknod d1/dd/d16/c54 0 2024-04-02T04:47:42.460 INFO:tasks.workunit.client.1.smithi179.stdout:6/164: link d3/d2c/c33 d3/d12/d23/c3e 0 2024-04-02T04:47:42.461 INFO:tasks.workunit.client.1.smithi179.stdout:6/165: dread - d3/d12/d23/d2f/f3d zero size 2024-04-02T04:47:42.462 INFO:tasks.workunit.client.0.smithi070.stdout:7/194: fsync d7/fc 0 2024-04-02T04:47:42.463 INFO:tasks.workunit.client.0.smithi070.stdout:5/212: dwrite d5/f22 [0,4194304] 0 2024-04-02T04:47:42.465 INFO:tasks.workunit.client.0.smithi070.stdout:0/328: dread d4/d5/d13/f1f [0,4194304] 0 2024-04-02T04:47:42.466 INFO:tasks.workunit.client.1.smithi179.stdout:4/275: rmdir d1/d3/df/d10 39 2024-04-02T04:47:42.470 INFO:tasks.workunit.client.0.smithi070.stdout:7/195: dwrite d7/d2f/f2d [0,4194304] 0 2024-04-02T04:47:42.470 INFO:tasks.workunit.client.0.smithi070.stdout:7/196: rename d7 to d7/dd/df/d39 22 2024-04-02T04:47:42.471 INFO:tasks.workunit.client.0.smithi070.stdout:8/233: truncate d2/d3/f9 8238703 0 2024-04-02T04:47:42.474 INFO:tasks.workunit.client.0.smithi070.stdout:6/119: mknod d0/d3/c2a 0 2024-04-02T04:47:42.477 INFO:tasks.workunit.client.1.smithi179.stdout:5/149: creat d8/d1b/d23/d32/f33 x:0 0 0 2024-04-02T04:47:42.479 INFO:tasks.workunit.client.0.smithi070.stdout:5/213: mknod d5/d14/c43 0 2024-04-02T04:47:42.479 INFO:tasks.workunit.client.0.smithi070.stdout:5/214: chown d5/c7 37 1 2024-04-02T04:47:42.481 INFO:tasks.workunit.client.1.smithi179.stdout:0/259: truncate d1/f36 226820 0 2024-04-02T04:47:42.482 INFO:tasks.workunit.client.1.smithi179.stdout:0/260: truncate d1/da/df/f21 381420 0 2024-04-02T04:47:42.483 INFO:tasks.workunit.client.0.smithi070.stdout:2/294: mknod d2/d10/d5c/c60 0 2024-04-02T04:47:42.484 INFO:tasks.workunit.client.1.smithi179.stdout:0/261: write d1/f22 [1056635,107780] 0 2024-04-02T04:47:42.484 INFO:tasks.workunit.client.1.smithi179.stdout:6/166: mkdir d3/d12/d23/d39/d3f 0 2024-04-02T04:47:42.484 INFO:tasks.workunit.client.1.smithi179.stdout:6/167: stat d3/d2c 0 2024-04-02T04:47:42.488 INFO:tasks.workunit.client.0.smithi070.stdout:5/215: dwrite d5/d17/f32 [0,4194304] 0 2024-04-02T04:47:42.488 INFO:tasks.workunit.client.0.smithi070.stdout:6/120: read d0/d3/f1e [2372777,61830] 0 2024-04-02T04:47:42.490 INFO:tasks.workunit.client.0.smithi070.stdout:5/216: fsync f4 0 2024-04-02T04:47:42.492 INFO:tasks.workunit.client.1.smithi179.stdout:0/262: dread d1/da/f1d [0,4194304] 0 2024-04-02T04:47:42.492 INFO:tasks.workunit.client.1.smithi179.stdout:9/217: dwrite d0/d19/d1c/f31 [0,4194304] 0 2024-04-02T04:47:42.493 INFO:tasks.workunit.client.0.smithi070.stdout:8/234: dwrite d2/d3/f7 [0,4194304] 0 2024-04-02T04:47:42.493 INFO:tasks.workunit.client.1.smithi179.stdout:4/276: write d1/d15/f56 [884388,97564] 0 2024-04-02T04:47:42.494 INFO:tasks.workunit.client.1.smithi179.stdout:7/232: getdents d1/dd 0 2024-04-02T04:47:42.495 INFO:tasks.workunit.client.1.smithi179.stdout:7/233: readlink d1/l3d 0 2024-04-02T04:47:42.496 INFO:tasks.workunit.client.0.smithi070.stdout:1/170: truncate d0/d9/d1d/d22/f2d 3662114 0 2024-04-02T04:47:42.497 INFO:tasks.workunit.client.1.smithi179.stdout:7/234: truncate d1/dd/f14 1307337 0 2024-04-02T04:47:42.498 INFO:tasks.workunit.client.0.smithi070.stdout:6/121: symlink d0/d3/d19/l2b 0 2024-04-02T04:47:42.499 INFO:tasks.workunit.client.1.smithi179.stdout:6/168: mknod d3/d12/d23/d39/d3f/c40 0 2024-04-02T04:47:42.502 INFO:tasks.workunit.client.1.smithi179.stdout:9/218: symlink d0/d19/d1c/l43 0 2024-04-02T04:47:42.503 INFO:tasks.workunit.client.0.smithi070.stdout:8/235: rmdir d2/d8/d23/d36 39 2024-04-02T04:47:42.503 INFO:tasks.workunit.client.0.smithi070.stdout:3/183: sync 2024-04-02T04:47:42.504 INFO:tasks.workunit.client.0.smithi070.stdout:6/122: dread d0/d3/f13 [0,4194304] 0 2024-04-02T04:47:42.504 INFO:tasks.workunit.client.0.smithi070.stdout:1/171: dwrite d0/d9/d1d/f26 [0,4194304] 0 2024-04-02T04:47:42.506 INFO:tasks.workunit.client.0.smithi070.stdout:6/123: dread - d0/f1c zero size 2024-04-02T04:47:42.506 INFO:tasks.workunit.client.0.smithi070.stdout:3/184: dread f8 [4194304,4194304] 0 2024-04-02T04:47:42.510 INFO:tasks.workunit.client.0.smithi070.stdout:2/295: dwrite d2/f1b [0,4194304] 0 2024-04-02T04:47:42.510 INFO:tasks.workunit.client.1.smithi179.stdout:7/235: mkdir d1/d9/d55 0 2024-04-02T04:47:42.512 INFO:tasks.workunit.client.1.smithi179.stdout:7/236: write d1/fc [7961458,99831] 0 2024-04-02T04:47:42.513 INFO:tasks.workunit.client.0.smithi070.stdout:2/296: write d2/d10/d53/f55 [549305,77594] 0 2024-04-02T04:47:42.517 INFO:tasks.workunit.client.0.smithi070.stdout:6/124: write d0/d20/f28 [544689,48644] 0 2024-04-02T04:47:42.519 INFO:tasks.workunit.client.1.smithi179.stdout:6/169: mknod d3/d2d/d38/c41 0 2024-04-02T04:47:42.521 INFO:tasks.workunit.client.0.smithi070.stdout:8/236: creat d2/d8/d23/d36/f4c x:0 0 0 2024-04-02T04:47:42.525 INFO:tasks.workunit.client.0.smithi070.stdout:6/125: mknod d0/c2c 0 2024-04-02T04:47:42.525 INFO:tasks.workunit.client.0.smithi070.stdout:5/217: getdents d5/d17/d27/d2c 0 2024-04-02T04:47:42.526 INFO:tasks.workunit.client.0.smithi070.stdout:8/237: dread d2/d3/d6/f3c [0,4194304] 0 2024-04-02T04:47:42.526 INFO:tasks.workunit.client.1.smithi179.stdout:7/237: creat d1/d9/d55/f56 x:0 0 0 2024-04-02T04:47:42.527 INFO:tasks.workunit.client.0.smithi070.stdout:5/218: write d5/d17/d27/d2c/d35/f3d [387509,128488] 0 2024-04-02T04:47:42.527 INFO:tasks.workunit.client.0.smithi070.stdout:8/238: write d2/d3/d6/f3c [854602,30481] 0 2024-04-02T04:47:42.527 INFO:tasks.workunit.client.1.smithi179.stdout:9/219: creat d0/d1/f44 x:0 0 0 2024-04-02T04:47:42.529 INFO:tasks.workunit.client.1.smithi179.stdout:7/238: read d1/dd/d23/f25 [1126033,22302] 0 2024-04-02T04:47:42.530 INFO:tasks.workunit.client.1.smithi179.stdout:4/277: getdents d1 0 2024-04-02T04:47:42.531 INFO:tasks.workunit.client.0.smithi070.stdout:2/297: rename d2/d10/d21/d3d/d42/f4b to d2/d10/d19/f61 0 2024-04-02T04:47:42.532 INFO:tasks.workunit.client.0.smithi070.stdout:2/298: chown d2/d10/c50 280887 1 2024-04-02T04:47:42.532 INFO:tasks.workunit.client.1.smithi179.stdout:6/170: mkdir d3/d42 0 2024-04-02T04:47:42.532 INFO:tasks.workunit.client.0.smithi070.stdout:6/126: rmdir d0/d3/d19 39 2024-04-02T04:47:42.533 INFO:tasks.workunit.client.0.smithi070.stdout:2/299: truncate d2/d12/f52 4550973 0 2024-04-02T04:47:42.534 INFO:tasks.workunit.client.0.smithi070.stdout:3/185: getdents d10 0 2024-04-02T04:47:42.535 INFO:tasks.workunit.client.0.smithi070.stdout:2/300: write d2/d24/f37 [674157,7018] 0 2024-04-02T04:47:42.535 INFO:tasks.workunit.client.1.smithi179.stdout:9/220: mkdir d0/d19/d1c/d45 0 2024-04-02T04:47:42.536 INFO:tasks.workunit.client.0.smithi070.stdout:2/301: chown d2/d10/d21/l29 6229076 1 2024-04-02T04:47:42.536 INFO:tasks.workunit.client.0.smithi070.stdout:3/186: chown d10/d32/l37 1 1 2024-04-02T04:47:42.536 INFO:tasks.workunit.client.0.smithi070.stdout:8/239: chown d2/d8/d29/db/f21 29 1 2024-04-02T04:47:42.537 INFO:tasks.workunit.client.0.smithi070.stdout:3/187: write d10/d17/f2b [1349955,87032] 0 2024-04-02T04:47:42.538 INFO:tasks.workunit.client.1.smithi179.stdout:7/239: creat d1/dd/d23/f57 x:0 0 0 2024-04-02T04:47:42.539 INFO:tasks.workunit.client.0.smithi070.stdout:8/240: truncate d2/d8/d29/db/f13 4832276 0 2024-04-02T04:47:42.539 INFO:tasks.workunit.client.1.smithi179.stdout:7/240: dread - d1/d9/f42 zero size 2024-04-02T04:47:42.542 INFO:tasks.workunit.client.0.smithi070.stdout:6/127: write d0/d20/f23 [546044,127246] 0 2024-04-02T04:47:42.542 INFO:tasks.workunit.client.1.smithi179.stdout:9/221: dwrite d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:42.543 INFO:tasks.workunit.client.1.smithi179.stdout:9/222: dread - d0/d1/d11/f42 zero size 2024-04-02T04:47:42.544 INFO:tasks.workunit.client.0.smithi070.stdout:6/128: dread - d0/f25 zero size 2024-04-02T04:47:42.545 INFO:tasks.workunit.client.0.smithi070.stdout:3/188: dwrite fb [0,4194304] 0 2024-04-02T04:47:42.552 INFO:tasks.workunit.client.0.smithi070.stdout:3/189: dwrite d10/d32/f36 [0,4194304] 0 2024-04-02T04:47:42.553 INFO:tasks.workunit.client.0.smithi070.stdout:5/219: mknod d5/d9/d23/d30/c44 0 2024-04-02T04:47:42.554 INFO:tasks.workunit.client.0.smithi070.stdout:5/220: stat d5/d17/f32 0 2024-04-02T04:47:42.555 INFO:tasks.workunit.client.0.smithi070.stdout:3/190: read fd [83551,117372] 0 2024-04-02T04:47:42.556 INFO:tasks.workunit.client.0.smithi070.stdout:8/241: fdatasync d2/d3/d6/f3c 0 2024-04-02T04:47:42.556 INFO:tasks.workunit.client.1.smithi179.stdout:0/263: truncate d1/f3b 811784 0 2024-04-02T04:47:42.556 INFO:tasks.workunit.client.0.smithi070.stdout:5/221: write d5/d17/d27/d2c/d35/f38 [646676,76044] 0 2024-04-02T04:47:42.557 INFO:tasks.workunit.client.0.smithi070.stdout:3/191: dread - d10/d32/d3f/f3b zero size 2024-04-02T04:47:42.558 INFO:tasks.workunit.client.1.smithi179.stdout:4/278: creat d1/d3/f5b x:0 0 0 2024-04-02T04:47:42.558 INFO:tasks.workunit.client.0.smithi070.stdout:5/222: write d5/f2f [954527,83966] 0 2024-04-02T04:47:42.560 INFO:tasks.workunit.client.0.smithi070.stdout:5/223: chown d5/d17 868 1 2024-04-02T04:47:42.560 INFO:tasks.workunit.client.0.smithi070.stdout:5/224: chown d5/d17/d27/f28 664787 1 2024-04-02T04:47:42.561 INFO:tasks.workunit.client.1.smithi179.stdout:6/171: dread f2 [0,4194304] 0 2024-04-02T04:47:42.562 INFO:tasks.workunit.client.0.smithi070.stdout:5/225: write d5/d9/d10/f11 [1642088,124595] 0 2024-04-02T04:47:42.562 INFO:tasks.workunit.client.0.smithi070.stdout:2/302: rename d2/d12/d3e/c4a to d2/d12/c62 0 2024-04-02T04:47:42.567 INFO:tasks.workunit.client.0.smithi070.stdout:5/226: dwrite d5/d17/d2b/f3a [0,4194304] 0 2024-04-02T04:47:42.568 INFO:tasks.workunit.client.0.smithi070.stdout:5/227: write d5/f6 [1620910,52010] 0 2024-04-02T04:47:42.575 INFO:tasks.workunit.client.0.smithi070.stdout:5/228: dwrite d5/d9/d23/f42 [0,4194304] 0 2024-04-02T04:47:42.576 INFO:tasks.workunit.client.0.smithi070.stdout:5/229: write d5/d17/d27/d2e/f3b [427630,16491] 0 2024-04-02T04:47:42.577 INFO:tasks.workunit.client.0.smithi070.stdout:5/230: dread - d5/d17/d27/d2c/f37 zero size 2024-04-02T04:47:42.587 INFO:tasks.workunit.client.0.smithi070.stdout:2/303: read d2/f32 [3125906,115203] 0 2024-04-02T04:47:42.648 INFO:tasks.workunit.client.1.smithi179.stdout:9/223: getdents d0/d1/d11/d29/d40 0 2024-04-02T04:47:42.650 INFO:tasks.workunit.client.1.smithi179.stdout:0/264: rename d1/f4d to d1/da/df/f4f 0 2024-04-02T04:47:42.651 INFO:tasks.workunit.client.0.smithi070.stdout:8/242: mkdir d2/d8/d29/d22/d4d 0 2024-04-02T04:47:42.652 INFO:tasks.workunit.client.1.smithi179.stdout:4/279: dread d1/d15/f31 [0,4194304] 0 2024-04-02T04:47:42.654 INFO:tasks.workunit.client.0.smithi070.stdout:8/243: dread d2/d8/d29/db/f21 [0,4194304] 0 2024-04-02T04:47:42.655 INFO:tasks.workunit.client.1.smithi179.stdout:6/172: mkdir d3/d2d/d43 0 2024-04-02T04:47:42.659 INFO:tasks.workunit.client.1.smithi179.stdout:0/265: write d1/da/f27 [1013627,74083] 0 2024-04-02T04:47:42.661 INFO:tasks.workunit.client.1.smithi179.stdout:0/266: write d1/da/d28/d37/f49 [93462,111298] 0 2024-04-02T04:47:42.661 INFO:tasks.workunit.client.1.smithi179.stdout:4/280: rmdir d1 39 2024-04-02T04:47:42.663 INFO:tasks.workunit.client.1.smithi179.stdout:6/173: mkdir d3/d12/d44 0 2024-04-02T04:47:42.664 INFO:tasks.workunit.client.0.smithi070.stdout:5/231: chown d5/d17/d27/c39 76 1 2024-04-02T04:47:42.664 INFO:tasks.workunit.client.0.smithi070.stdout:5/232: read d5/f22 [1204100,115313] 0 2024-04-02T04:47:42.665 INFO:tasks.workunit.client.0.smithi070.stdout:5/233: fsync d5/d17/d27/d2c/f37 0 2024-04-02T04:47:42.666 INFO:tasks.workunit.client.0.smithi070.stdout:2/304: creat d2/d10/d21/d3d/f63 x:0 0 0 2024-04-02T04:47:42.667 INFO:tasks.workunit.client.0.smithi070.stdout:2/305: stat d2/d10/d21/d3d/d42 0 2024-04-02T04:47:42.668 INFO:tasks.workunit.client.0.smithi070.stdout:2/306: chown d2/d10/d21/f34 3 1 2024-04-02T04:47:42.668 INFO:tasks.workunit.client.1.smithi179.stdout:7/241: dwrite d1/dd/d16/f48 [0,4194304] 0 2024-04-02T04:47:42.671 INFO:tasks.workunit.client.1.smithi179.stdout:0/267: mkdir d1/da/d48/d50 0 2024-04-02T04:47:42.673 INFO:tasks.workunit.client.0.smithi070.stdout:8/244: unlink d2/d3/d6/f25 0 2024-04-02T04:47:42.676 INFO:tasks.workunit.client.0.smithi070.stdout:6/129: dwrite d0/d3/f13 [0,4194304] 0 2024-04-02T04:47:42.678 INFO:tasks.workunit.client.0.smithi070.stdout:8/245: chown d2/d8/d23/d24 47223066 1 2024-04-02T04:47:42.678 INFO:tasks.workunit.client.0.smithi070.stdout:3/192: unlink d10/d32/d3f/l3c 0 2024-04-02T04:47:42.679 INFO:tasks.workunit.client.0.smithi070.stdout:3/193: chown c6 3 1 2024-04-02T04:47:42.681 INFO:tasks.workunit.client.0.smithi070.stdout:2/307: dwrite d2/d10/d21/d3d/f63 [0,4194304] 0 2024-04-02T04:47:42.681 INFO:tasks.workunit.client.0.smithi070.stdout:3/194: read ff [3549135,27466] 0 2024-04-02T04:47:42.688 INFO:tasks.workunit.client.0.smithi070.stdout:4/160: sync 2024-04-02T04:47:42.689 INFO:tasks.workunit.client.0.smithi070.stdout:5/234: creat d5/d17/d27/f45 x:0 0 0 2024-04-02T04:47:42.690 INFO:tasks.workunit.client.1.smithi179.stdout:7/242: mknod d1/dd/d23/c58 0 2024-04-02T04:47:42.691 INFO:tasks.workunit.client.0.smithi070.stdout:4/161: write d1/d3/d9/f22 [252943,78942] 0 2024-04-02T04:47:42.692 INFO:tasks.workunit.client.1.smithi179.stdout:0/268: mknod d1/da/d28/d34/c51 0 2024-04-02T04:47:42.692 INFO:tasks.workunit.client.0.smithi070.stdout:6/130: dwrite d0/f1c [0,4194304] 0 2024-04-02T04:47:42.696 INFO:tasks.workunit.client.0.smithi070.stdout:5/235: dread d5/d17/d27/d2c/d35/f38 [0,4194304] 0 2024-04-02T04:47:42.697 INFO:tasks.workunit.client.1.smithi179.stdout:9/224: link d0/d1/d12/l1f d0/d1/d11/l46 0 2024-04-02T04:47:42.697 INFO:tasks.workunit.client.0.smithi070.stdout:5/236: write f4 [1684672,64419] 0 2024-04-02T04:47:42.697 INFO:tasks.workunit.client.1.smithi179.stdout:1/232: sync 2024-04-02T04:47:42.698 INFO:tasks.workunit.client.0.smithi070.stdout:5/237: dread d5/f1d [0,4194304] 0 2024-04-02T04:47:42.700 INFO:tasks.workunit.client.1.smithi179.stdout:4/281: symlink d1/d3/l5c 0 2024-04-02T04:47:42.700 INFO:tasks.workunit.client.1.smithi179.stdout:1/233: write d3/d5/f27 [4536312,116546] 0 2024-04-02T04:47:42.703 INFO:tasks.workunit.client.1.smithi179.stdout:1/234: chown d3/d5/d2f/d4f/d42 4620223 1 2024-04-02T04:47:42.704 INFO:tasks.workunit.client.1.smithi179.stdout:7/243: readlink d1/l3 0 2024-04-02T04:47:42.705 INFO:tasks.workunit.client.0.smithi070.stdout:5/238: dwrite d5/d17/d2b/f3a [0,4194304] 0 2024-04-02T04:47:42.705 INFO:tasks.workunit.client.0.smithi070.stdout:3/195: rename fa to d10/f41 0 2024-04-02T04:47:42.706 INFO:tasks.workunit.client.1.smithi179.stdout:0/269: dread d1/da/f3c [0,4194304] 0 2024-04-02T04:47:42.707 INFO:tasks.workunit.client.1.smithi179.stdout:6/174: dwrite d3/d12/d23/d26/f27 [0,4194304] 0 2024-04-02T04:47:42.708 INFO:tasks.workunit.client.0.smithi070.stdout:2/308: unlink d2/d12/c18 0 2024-04-02T04:47:42.709 INFO:tasks.workunit.client.1.smithi179.stdout:1/235: truncate d3/d5/d2f/d4f/f4c 290711 0 2024-04-02T04:47:42.710 INFO:tasks.workunit.client.1.smithi179.stdout:9/225: creat d0/d19/d1c/f47 x:0 0 0 2024-04-02T04:47:42.712 INFO:tasks.workunit.client.1.smithi179.stdout:1/236: fdatasync d3/f1f 0 2024-04-02T04:47:42.713 INFO:tasks.workunit.client.1.smithi179.stdout:9/226: fdatasync d0/d19/d1c/f31 0 2024-04-02T04:47:42.717 INFO:tasks.workunit.client.1.smithi179.stdout:7/244: dread d1/dd/d16/f48 [0,4194304] 0 2024-04-02T04:47:42.720 INFO:tasks.workunit.client.1.smithi179.stdout:7/245: dread - d1/dd/d16/f2a zero size 2024-04-02T04:47:42.721 INFO:tasks.workunit.client.0.smithi070.stdout:6/131: rmdir d0 39 2024-04-02T04:47:42.722 INFO:tasks.workunit.client.0.smithi070.stdout:8/246: dwrite d2/f48 [0,4194304] 0 2024-04-02T04:47:42.724 INFO:tasks.workunit.client.1.smithi179.stdout:7/246: dread - d1/d9/d35/f3a zero size 2024-04-02T04:47:42.726 INFO:tasks.workunit.client.1.smithi179.stdout:6/175: dwrite d3/d12/d23/d2f/f3d [0,4194304] 0 2024-04-02T04:47:42.727 INFO:tasks.workunit.client.0.smithi070.stdout:3/196: rmdir d10 39 2024-04-02T04:47:42.727 INFO:tasks.workunit.client.1.smithi179.stdout:7/247: chown d1/dd/d16/l3c 441770880 1 2024-04-02T04:47:42.729 INFO:tasks.workunit.client.0.smithi070.stdout:5/239: symlink d5/d9/d3f/l46 0 2024-04-02T04:47:42.730 INFO:tasks.workunit.client.1.smithi179.stdout:7/248: truncate d1/dd/f1c 1601563 0 2024-04-02T04:47:42.731 INFO:tasks.workunit.client.1.smithi179.stdout:1/237: symlink d3/d5/d2f/l50 0 2024-04-02T04:47:42.732 INFO:tasks.workunit.client.1.smithi179.stdout:9/227: dwrite d0/d1/d12/fe [0,4194304] 0 2024-04-02T04:47:42.733 INFO:tasks.workunit.client.1.smithi179.stdout:7/249: truncate d1/dd/f1c 2081878 0 2024-04-02T04:47:42.736 INFO:tasks.workunit.client.1.smithi179.stdout:6/176: mknod d3/d2d/c45 0 2024-04-02T04:47:42.739 INFO:tasks.workunit.client.1.smithi179.stdout:9/228: chown d0/d1/d11/l46 38610759 1 2024-04-02T04:47:42.740 INFO:tasks.workunit.client.0.smithi070.stdout:4/162: symlink d1/d3/l31 0 2024-04-02T04:47:42.741 INFO:tasks.workunit.client.1.smithi179.stdout:0/270: dwrite d1/da/df/f11 [4194304,4194304] 0 2024-04-02T04:47:42.742 INFO:tasks.workunit.client.1.smithi179.stdout:9/229: write d0/d1/d11/d29/f34 [678018,19546] 0 2024-04-02T04:47:42.744 INFO:tasks.workunit.client.1.smithi179.stdout:1/238: creat d3/d5/de/d33/f51 x:0 0 0 2024-04-02T04:47:42.744 INFO:tasks.workunit.client.1.smithi179.stdout:7/250: symlink d1/d9/l59 0 2024-04-02T04:47:42.749 INFO:tasks.workunit.client.1.smithi179.stdout:6/177: unlink d3/d12/d23/f30 0 2024-04-02T04:47:42.751 INFO:tasks.workunit.client.0.smithi070.stdout:5/240: rename d5/d14/c43 to d5/d17/d27/d2c/d35/c47 0 2024-04-02T04:47:42.753 INFO:tasks.workunit.client.0.smithi070.stdout:2/309: mknod d2/c64 0 2024-04-02T04:47:42.755 INFO:tasks.workunit.client.1.smithi179.stdout:9/230: rmdir d0/d1/d11 39 2024-04-02T04:47:42.756 INFO:tasks.workunit.client.0.smithi070.stdout:6/132: stat d0/d3/fb 0 2024-04-02T04:47:42.757 INFO:tasks.workunit.client.0.smithi070.stdout:4/163: fsync d1/f2d 0 2024-04-02T04:47:42.758 INFO:tasks.workunit.client.1.smithi179.stdout:1/239: mknod d3/d5/de/d33/c52 0 2024-04-02T04:47:42.759 INFO:tasks.workunit.client.0.smithi070.stdout:8/247: link d2/d3/d6/f43 d2/d8/d29/d22/d4d/f4e 0 2024-04-02T04:47:42.761 INFO:tasks.workunit.client.1.smithi179.stdout:7/251: unlink d1/d9/f42 0 2024-04-02T04:47:42.762 INFO:tasks.workunit.client.1.smithi179.stdout:7/252: read - d1/d9/d55/f56 zero size 2024-04-02T04:47:42.766 INFO:tasks.workunit.client.1.smithi179.stdout:1/240: dwrite d3/d5/de/f16 [0,4194304] 0 2024-04-02T04:47:42.766 INFO:tasks.workunit.client.0.smithi070.stdout:3/197: rename d10/d17/f30 to d10/d32/f42 0 2024-04-02T04:47:42.766 INFO:tasks.workunit.client.0.smithi070.stdout:8/248: dread d2/d3/f7 [4194304,4194304] 0 2024-04-02T04:47:42.767 INFO:tasks.workunit.client.0.smithi070.stdout:8/249: fdatasync d2/f1f 0 2024-04-02T04:47:42.767 INFO:tasks.workunit.client.0.smithi070.stdout:5/241: creat d5/d9/d23/f48 x:0 0 0 2024-04-02T04:47:42.767 INFO:tasks.workunit.client.0.smithi070.stdout:9/201: sync 2024-04-02T04:47:42.767 INFO:tasks.workunit.client.0.smithi070.stdout:0/329: sync 2024-04-02T04:47:42.767 INFO:tasks.workunit.client.1.smithi179.stdout:1/241: dread d3/d5/d2f/f34 [0,4194304] 0 2024-04-02T04:47:42.770 INFO:tasks.workunit.client.0.smithi070.stdout:6/133: creat d0/f2d x:0 0 0 2024-04-02T04:47:42.772 INFO:tasks.workunit.client.1.smithi179.stdout:9/231: mkdir d0/d19/d27/d48 0 2024-04-02T04:47:42.775 INFO:tasks.workunit.client.0.smithi070.stdout:2/310: dread d2/f32 [0,4194304] 0 2024-04-02T04:47:42.776 INFO:tasks.workunit.client.0.smithi070.stdout:5/242: dwrite f2 [0,4194304] 0 2024-04-02T04:47:42.779 INFO:tasks.workunit.client.0.smithi070.stdout:0/330: dwrite d4/d5/f14 [4194304,4194304] 0 2024-04-02T04:47:42.779 INFO:tasks.workunit.client.0.smithi070.stdout:3/198: rename fe to d10/d32/d3f/f43 0 2024-04-02T04:47:42.783 INFO:tasks.workunit.client.1.smithi179.stdout:7/253: mknod d1/d9/d55/c5a 0 2024-04-02T04:47:42.785 INFO:tasks.workunit.client.0.smithi070.stdout:9/202: mknod d4/d13/d16/d19/d2f/c41 0 2024-04-02T04:47:42.787 INFO:tasks.workunit.client.0.smithi070.stdout:3/199: write d10/d32/d3f/f35 [113864,11343] 0 2024-04-02T04:47:42.788 INFO:tasks.workunit.client.0.smithi070.stdout:5/243: dwrite d5/d9/d23/f48 [0,4194304] 0 2024-04-02T04:47:42.789 INFO:tasks.workunit.client.0.smithi070.stdout:6/134: unlink d0/d3/c1d 0 2024-04-02T04:47:42.789 INFO:tasks.workunit.client.0.smithi070.stdout:9/203: fdatasync d4/d10/d11/f14 0 2024-04-02T04:47:42.790 INFO:tasks.workunit.client.0.smithi070.stdout:6/135: fdatasync d0/f16 0 2024-04-02T04:47:42.792 INFO:tasks.workunit.client.1.smithi179.stdout:0/271: rmdir d1/da/d28/d34/d39/d4b 0 2024-04-02T04:47:42.796 INFO:tasks.workunit.client.1.smithi179.stdout:5/150: sync 2024-04-02T04:47:42.796 INFO:tasks.workunit.client.1.smithi179.stdout:3/164: sync 2024-04-02T04:47:42.796 INFO:tasks.workunit.client.0.smithi070.stdout:0/331: unlink d4/d3c/f3d 0 2024-04-02T04:47:42.798 INFO:tasks.workunit.client.1.smithi179.stdout:2/118: sync 2024-04-02T04:47:42.798 INFO:tasks.workunit.client.1.smithi179.stdout:8/204: sync 2024-04-02T04:47:42.799 INFO:tasks.workunit.client.1.smithi179.stdout:8/205: stat d3/d31/c24 0 2024-04-02T04:47:42.799 INFO:tasks.workunit.client.1.smithi179.stdout:3/165: dread d8/d26/f33 [0,4194304] 0 2024-04-02T04:47:42.800 INFO:tasks.workunit.client.0.smithi070.stdout:8/250: rename d2/d8/d23/d39 to d2/d3/d35/d3a/d4f 0 2024-04-02T04:47:42.801 INFO:tasks.workunit.client.1.smithi179.stdout:8/206: chown d3/f9 3939701 1 2024-04-02T04:47:42.801 INFO:tasks.workunit.client.1.smithi179.stdout:9/232: link d0/d1/f7 d0/d1/d11/d29/d40/f49 0 2024-04-02T04:47:42.803 INFO:tasks.workunit.client.1.smithi179.stdout:8/207: fdatasync d3/d1b/d2a/f3e 0 2024-04-02T04:47:42.804 INFO:tasks.workunit.client.0.smithi070.stdout:3/200: unlink d10/f15 0 2024-04-02T04:47:42.804 INFO:tasks.workunit.client.1.smithi179.stdout:8/208: chown d3/d32/d21/l4a 209900 1 2024-04-02T04:47:42.806 INFO:tasks.workunit.client.1.smithi179.stdout:5/151: dwrite d8/fb [0,4194304] 0 2024-04-02T04:47:42.807 INFO:tasks.workunit.client.1.smithi179.stdout:8/209: fdatasync d3/d1b/d2a/f3e 0 2024-04-02T04:47:42.810 INFO:tasks.workunit.client.0.smithi070.stdout:9/204: fdatasync d4/d10/d12/d27/d32/f33 0 2024-04-02T04:47:42.812 INFO:tasks.workunit.client.1.smithi179.stdout:7/254: link d1/d9/l39 d1/d9/d35/l5b 0 2024-04-02T04:47:42.814 INFO:tasks.workunit.client.1.smithi179.stdout:9/233: dwrite d0/d1/d10/f13 [4194304,4194304] 0 2024-04-02T04:47:42.814 INFO:tasks.workunit.client.0.smithi070.stdout:2/311: unlink d2/d10/d19/f61 0 2024-04-02T04:47:42.816 INFO:tasks.workunit.client.1.smithi179.stdout:9/234: write d0/d1/d11/f42 [342337,24376] 0 2024-04-02T04:47:42.820 INFO:tasks.workunit.client.0.smithi070.stdout:0/332: creat d4/d5/d13/d2a/d29/f59 x:0 0 0 2024-04-02T04:47:42.824 INFO:tasks.workunit.client.1.smithi179.stdout:6/178: truncate d3/d12/f22 2903345 0 2024-04-02T04:47:42.825 INFO:tasks.workunit.client.1.smithi179.stdout:7/255: mknod d1/d9/c5c 0 2024-04-02T04:47:42.825 INFO:tasks.workunit.client.1.smithi179.stdout:7/256: dread - d1/dd/d16/f2a zero size 2024-04-02T04:47:42.826 INFO:tasks.workunit.client.0.smithi070.stdout:4/164: truncate d1/fa 4382192 0 2024-04-02T04:47:42.830 INFO:tasks.workunit.client.0.smithi070.stdout:8/251: dread d2/d3/f9 [4194304,4194304] 0 2024-04-02T04:47:42.831 INFO:tasks.workunit.client.0.smithi070.stdout:2/312: dread d2/d10/d21/f34 [0,4194304] 0 2024-04-02T04:47:42.831 INFO:tasks.workunit.client.1.smithi179.stdout:2/119: unlink dc/d16/c1d 0 2024-04-02T04:47:42.833 INFO:tasks.workunit.client.0.smithi070.stdout:9/205: creat d4/d10/d11/f42 x:0 0 0 2024-04-02T04:47:42.837 INFO:tasks.workunit.client.1.smithi179.stdout:0/272: getdents d1/da/d28 0 2024-04-02T04:47:42.839 INFO:tasks.workunit.client.0.smithi070.stdout:2/313: dread d2/f14 [0,4194304] 0 2024-04-02T04:47:42.840 INFO:tasks.workunit.client.1.smithi179.stdout:7/257: truncate d1/d9/d35/f3a 59796 0 2024-04-02T04:47:42.842 INFO:tasks.workunit.client.1.smithi179.stdout:5/152: truncate d8/d1b/f25 669351 0 2024-04-02T04:47:42.843 INFO:tasks.workunit.client.0.smithi070.stdout:4/165: rmdir d1/d3 39 2024-04-02T04:47:42.843 INFO:tasks.workunit.client.1.smithi179.stdout:0/273: dwrite d1/da/d28/d34/f35 [0,4194304] 0 2024-04-02T04:47:42.844 INFO:tasks.workunit.client.1.smithi179.stdout:5/153: write f1 [3720373,27727] 0 2024-04-02T04:47:42.847 INFO:tasks.workunit.client.1.smithi179.stdout:5/154: write d8/d1b/d23/d32/f33 [718320,127888] 0 2024-04-02T04:47:42.854 INFO:tasks.workunit.client.0.smithi070.stdout:6/136: truncate d0/f15 2965367 0 2024-04-02T04:47:42.856 INFO:tasks.workunit.client.1.smithi179.stdout:6/179: write d3/fa [2226870,65448] 0 2024-04-02T04:47:42.860 INFO:tasks.workunit.client.1.smithi179.stdout:3/166: getdents d8 0 2024-04-02T04:47:42.861 INFO:tasks.workunit.client.1.smithi179.stdout:2/120: write dc/f12 [6139983,19538] 0 2024-04-02T04:47:42.862 INFO:tasks.workunit.client.1.smithi179.stdout:2/121: write dc/f17 [429592,1750] 0 2024-04-02T04:47:42.863 INFO:tasks.workunit.client.0.smithi070.stdout:3/201: rename d10/l12 to d10/d17/l44 0 2024-04-02T04:47:42.864 INFO:tasks.workunit.client.0.smithi070.stdout:3/202: fdatasync d10/d17/f27 0 2024-04-02T04:47:42.864 INFO:tasks.workunit.client.0.smithi070.stdout:3/203: chown l3 169 1 2024-04-02T04:47:42.866 INFO:tasks.workunit.client.1.smithi179.stdout:9/235: link d0/d19/d27/c2c d0/d1/d11/d3a/c4a 0 2024-04-02T04:47:42.871 INFO:tasks.workunit.client.0.smithi070.stdout:6/137: rename d0/d3/d19 to d0/d3/d19/d2e 22 2024-04-02T04:47:42.872 INFO:tasks.workunit.client.1.smithi179.stdout:6/180: creat d3/d12/f46 x:0 0 0 2024-04-02T04:47:42.873 INFO:tasks.workunit.client.0.smithi070.stdout:0/333: rmdir d4/d5/d13/d17/d51 0 2024-04-02T04:47:42.877 INFO:tasks.workunit.client.1.smithi179.stdout:2/122: dread dc/d16/f15 [0,4194304] 0 2024-04-02T04:47:42.878 INFO:tasks.workunit.client.1.smithi179.stdout:0/274: truncate d1/da/df/f12 5157597 0 2024-04-02T04:47:42.879 INFO:tasks.workunit.client.1.smithi179.stdout:6/181: dwrite d3/d12/f46 [0,4194304] 0 2024-04-02T04:47:42.882 INFO:tasks.workunit.client.1.smithi179.stdout:2/123: dread f6 [0,4194304] 0 2024-04-02T04:47:42.884 INFO:tasks.workunit.client.1.smithi179.stdout:9/236: unlink d0/d19/d27/c2f 0 2024-04-02T04:47:42.892 INFO:tasks.workunit.client.1.smithi179.stdout:9/237: dwrite d0/d1/d11/d29/f34 [0,4194304] 0 2024-04-02T04:47:42.893 INFO:tasks.workunit.client.1.smithi179.stdout:9/238: write d0/d19/d1c/f41 [591351,63173] 0 2024-04-02T04:47:42.894 INFO:tasks.workunit.client.1.smithi179.stdout:7/258: rename d1/d9/d35/f3a to d1/f5d 0 2024-04-02T04:47:42.895 INFO:tasks.workunit.client.1.smithi179.stdout:6/182: creat d3/d2d/d43/f47 x:0 0 0 2024-04-02T04:47:42.896 INFO:tasks.workunit.client.1.smithi179.stdout:0/275: mkdir d1/da/d52 0 2024-04-02T04:47:42.897 INFO:tasks.workunit.client.1.smithi179.stdout:9/239: chown d0/d1/f7 1 1 2024-04-02T04:47:42.898 INFO:tasks.workunit.client.1.smithi179.stdout:5/155: truncate d8/ff 1772622 0 2024-04-02T04:47:42.899 INFO:tasks.workunit.client.1.smithi179.stdout:3/167: rename d8/f23 to d8/d26/f34 0 2024-04-02T04:47:42.899 INFO:tasks.workunit.client.1.smithi179.stdout:9/240: dread - d0/d1/f36 zero size 2024-04-02T04:47:42.900 INFO:tasks.workunit.client.1.smithi179.stdout:7/259: mkdir d1/dd/d23/d5e 0 2024-04-02T04:47:42.900 INFO:tasks.workunit.client.1.smithi179.stdout:5/156: write d8/d1b/d23/f2b [379074,81821] 0 2024-04-02T04:47:42.902 INFO:tasks.workunit.client.1.smithi179.stdout:2/124: dwrite dc/f1a [4194304,4194304] 0 2024-04-02T04:47:42.903 INFO:tasks.workunit.client.1.smithi179.stdout:0/276: symlink d1/da/df/l53 0 2024-04-02T04:47:42.903 INFO:tasks.workunit.client.1.smithi179.stdout:7/260: chown d1/dd/d23/f2f 4674 1 2024-04-02T04:47:42.905 INFO:tasks.workunit.client.1.smithi179.stdout:5/157: chown c7 285 1 2024-04-02T04:47:42.905 INFO:tasks.workunit.client.1.smithi179.stdout:7/261: read - d1/d9/d55/f56 zero size 2024-04-02T04:47:42.906 INFO:tasks.workunit.client.1.smithi179.stdout:6/183: mknod d3/d12/d44/c48 0 2024-04-02T04:47:42.907 INFO:tasks.workunit.client.1.smithi179.stdout:5/158: dread d8/f27 [0,4194304] 0 2024-04-02T04:47:42.908 INFO:tasks.workunit.client.1.smithi179.stdout:6/184: chown d3/c1f 132026496 1 2024-04-02T04:47:42.908 INFO:tasks.workunit.client.1.smithi179.stdout:0/277: creat d1/da/d48/f54 x:0 0 0 2024-04-02T04:47:42.908 INFO:tasks.workunit.client.1.smithi179.stdout:4/282: sync 2024-04-02T04:47:42.909 INFO:tasks.workunit.client.1.smithi179.stdout:9/241: mknod d0/d1/d11/d29/d40/c4b 0 2024-04-02T04:47:42.910 INFO:tasks.workunit.client.1.smithi179.stdout:0/278: readlink d1/db/l47 0 2024-04-02T04:47:42.913 INFO:tasks.workunit.client.1.smithi179.stdout:0/279: write d1/da/d28/f3a [177378,77705] 0 2024-04-02T04:47:42.915 INFO:tasks.workunit.client.1.smithi179.stdout:9/242: rmdir d0/d19 39 2024-04-02T04:47:42.915 INFO:tasks.workunit.client.1.smithi179.stdout:7/262: dread d1/d9/d35/f3b [0,4194304] 0 2024-04-02T04:47:42.916 INFO:tasks.workunit.client.1.smithi179.stdout:9/243: dread - d0/d1/f44 zero size 2024-04-02T04:47:42.917 INFO:tasks.workunit.client.1.smithi179.stdout:9/244: fsync d0/d1/f36 0 2024-04-02T04:47:42.917 INFO:tasks.workunit.client.1.smithi179.stdout:4/283: truncate d1/d15/f31 451421 0 2024-04-02T04:47:42.919 INFO:tasks.workunit.client.0.smithi070.stdout:1/172: sync 2024-04-02T04:47:42.919 INFO:tasks.workunit.client.0.smithi070.stdout:7/197: sync 2024-04-02T04:47:42.919 INFO:tasks.workunit.client.1.smithi179.stdout:2/125: creat dc/f25 x:0 0 0 2024-04-02T04:47:42.919 INFO:tasks.workunit.client.1.smithi179.stdout:0/280: write d1/da/f46 [784646,70567] 0 2024-04-02T04:47:42.920 INFO:tasks.workunit.client.0.smithi070.stdout:8/252: truncate d2/d8/d23/d24/f37 432418 0 2024-04-02T04:47:42.920 INFO:tasks.workunit.client.0.smithi070.stdout:3/204: mknod d10/d32/d3f/c45 0 2024-04-02T04:47:42.921 INFO:tasks.workunit.client.0.smithi070.stdout:0/334: write d4/d5/d13/d43/f49 [324343,115517] 0 2024-04-02T04:47:42.921 INFO:tasks.workunit.client.0.smithi070.stdout:9/206: write d4/d10/d11/f21 [589472,39497] 0 2024-04-02T04:47:42.921 INFO:tasks.workunit.client.1.smithi179.stdout:7/263: creat d1/d9/d27/f5f x:0 0 0 2024-04-02T04:47:42.922 INFO:tasks.workunit.client.1.smithi179.stdout:3/168: dread f5 [0,4194304] 0 2024-04-02T04:47:42.923 INFO:tasks.workunit.client.1.smithi179.stdout:5/159: creat d8/d1b/d23/d30/f34 x:0 0 0 2024-04-02T04:47:42.923 INFO:tasks.workunit.client.0.smithi070.stdout:6/138: creat d0/d3/d19/f2f x:0 0 0 2024-04-02T04:47:42.924 INFO:tasks.workunit.client.0.smithi070.stdout:8/253: dread - d2/d3/f49 zero size 2024-04-02T04:47:42.925 INFO:tasks.workunit.client.0.smithi070.stdout:7/198: read d7/dd/f2e [468256,115785] 0 2024-04-02T04:47:42.926 INFO:tasks.workunit.client.0.smithi070.stdout:6/139: fdatasync d0/d20/f28 0 2024-04-02T04:47:42.926 INFO:tasks.workunit.client.0.smithi070.stdout:8/254: readlink d2/l40 0 2024-04-02T04:47:42.927 INFO:tasks.workunit.client.1.smithi179.stdout:9/245: write d0/d19/d1c/f47 [242048,56849] 0 2024-04-02T04:47:42.928 INFO:tasks.workunit.client.1.smithi179.stdout:9/246: fsync d0/d1/d12/f25 0 2024-04-02T04:47:42.929 INFO:tasks.workunit.client.0.smithi070.stdout:0/335: dread - d4/d5/d13/d2a/d29/f59 zero size 2024-04-02T04:47:42.929 INFO:tasks.workunit.client.0.smithi070.stdout:6/140: chown d0/f2d 0 1 2024-04-02T04:47:42.929 INFO:tasks.workunit.client.1.smithi179.stdout:6/185: dread d3/fb [0,4194304] 0 2024-04-02T04:47:42.930 INFO:tasks.workunit.client.1.smithi179.stdout:5/160: dwrite d8/f1a [0,4194304] 0 2024-04-02T04:47:42.931 INFO:tasks.workunit.client.0.smithi070.stdout:6/141: chown d0/d3/c2a 2152486 1 2024-04-02T04:47:42.931 INFO:tasks.workunit.client.0.smithi070.stdout:3/205: dwrite d10/d32/d3f/f38 [0,4194304] 0 2024-04-02T04:47:42.931 INFO:tasks.workunit.client.1.smithi179.stdout:2/126: mknod dc/d16/d1e/c26 0 2024-04-02T04:47:42.931 INFO:tasks.workunit.client.1.smithi179.stdout:5/161: stat f6 0 2024-04-02T04:47:42.947 INFO:tasks.workunit.client.0.smithi070.stdout:9/207: rename d4/d10/d12/c31 to d4/d10/d12/d27/d32/c43 0 2024-04-02T04:47:42.951 INFO:tasks.workunit.client.0.smithi070.stdout:6/142: rmdir d0 39 2024-04-02T04:47:42.953 INFO:tasks.workunit.client.0.smithi070.stdout:3/206: mknod d10/d32/c46 0 2024-04-02T04:47:42.954 INFO:tasks.workunit.client.0.smithi070.stdout:9/208: dwrite d4/d13/f37 [0,4194304] 0 2024-04-02T04:47:42.956 INFO:tasks.workunit.client.0.smithi070.stdout:9/209: write d4/d13/d16/f35 [839599,116210] 0 2024-04-02T04:47:42.958 INFO:tasks.workunit.client.0.smithi070.stdout:7/199: mknod d7/d9/c3a 0 2024-04-02T04:47:42.958 INFO:tasks.workunit.client.1.smithi179.stdout:6/186: truncate d3/d12/f22 1100213 0 2024-04-02T04:47:42.960 INFO:tasks.workunit.client.0.smithi070.stdout:4/166: getdents d1/d3/d9 0 2024-04-02T04:47:42.961 INFO:tasks.workunit.client.0.smithi070.stdout:8/255: symlink d2/d8/l50 0 2024-04-02T04:47:42.963 INFO:tasks.workunit.client.1.smithi179.stdout:3/169: write d8/f22 [1327480,18123] 0 2024-04-02T04:47:42.964 INFO:tasks.workunit.client.0.smithi070.stdout:0/336: mknod d4/d5/d13/c5a 0 2024-04-02T04:47:42.964 INFO:tasks.workunit.client.1.smithi179.stdout:3/170: dread - d8/f2d zero size 2024-04-02T04:47:42.965 INFO:tasks.workunit.client.0.smithi070.stdout:1/173: dwrite d0/d9/d1d/f21 [0,4194304] 0 2024-04-02T04:47:42.973 INFO:tasks.workunit.client.0.smithi070.stdout:3/207: creat d10/d32/d3f/f47 x:0 0 0 2024-04-02T04:47:42.975 INFO:tasks.workunit.client.0.smithi070.stdout:9/210: chown d4/d10/c28 2 1 2024-04-02T04:47:42.976 INFO:tasks.workunit.client.0.smithi070.stdout:4/167: dread d1/d3/f2b [0,4194304] 0 2024-04-02T04:47:42.976 INFO:tasks.workunit.client.1.smithi179.stdout:6/187: readlink d3/l7 0 2024-04-02T04:47:42.978 INFO:tasks.workunit.client.0.smithi070.stdout:7/200: dread d7/fc [4194304,4194304] 0 2024-04-02T04:47:42.979 INFO:tasks.workunit.client.1.smithi179.stdout:2/127: creat dc/d16/f27 x:0 0 0 2024-04-02T04:47:42.979 INFO:tasks.workunit.client.0.smithi070.stdout:7/201: readlink d7/d9/d19/l2b 0 2024-04-02T04:47:42.981 INFO:tasks.workunit.client.0.smithi070.stdout:8/256: fdatasync d2/d3/f7 0 2024-04-02T04:47:42.983 INFO:tasks.workunit.client.1.smithi179.stdout:9/247: truncate d0/d1/d11/f42 178040 0 2024-04-02T04:47:42.985 INFO:tasks.workunit.client.1.smithi179.stdout:3/171: unlink d8/c2f 0 2024-04-02T04:47:42.986 INFO:tasks.workunit.client.0.smithi070.stdout:6/143: mkdir d0/d30 0 2024-04-02T04:47:42.987 INFO:tasks.workunit.client.1.smithi179.stdout:7/264: link d1/c47 d1/c60 0 2024-04-02T04:47:42.987 INFO:tasks.workunit.client.1.smithi179.stdout:4/284: getdents d1/d3/dc/d27 0 2024-04-02T04:47:42.988 INFO:tasks.workunit.client.1.smithi179.stdout:7/265: write d1/f32 [832784,31374] 0 2024-04-02T04:47:42.989 INFO:tasks.workunit.client.0.smithi070.stdout:9/211: rmdir d4/d10 39 2024-04-02T04:47:42.990 INFO:tasks.workunit.client.0.smithi070.stdout:9/212: chown d4/d13/d16/d19/f2e 126965766 1 2024-04-02T04:47:42.992 INFO:tasks.workunit.client.1.smithi179.stdout:5/162: getdents d8 0 2024-04-02T04:47:42.993 INFO:tasks.workunit.client.1.smithi179.stdout:3/172: rename d8/l1f to d8/l35 0 2024-04-02T04:47:42.993 INFO:tasks.workunit.client.0.smithi070.stdout:7/202: chown d7/dd/df/d34/c12 12275796 1 2024-04-02T04:47:42.994 INFO:tasks.workunit.client.1.smithi179.stdout:7/266: dread d1/dd/d23/f24 [0,4194304] 0 2024-04-02T04:47:42.994 INFO:tasks.workunit.client.0.smithi070.stdout:7/203: fdatasync d7/dd/df/f10 0 2024-04-02T04:47:42.995 INFO:tasks.workunit.client.1.smithi179.stdout:4/285: dwrite d1/d3/dc/fe [0,4194304] 0 2024-04-02T04:47:42.995 INFO:tasks.workunit.client.1.smithi179.stdout:7/267: chown d1/d9/d55/c5a 143357 1 2024-04-02T04:47:42.996 INFO:tasks.workunit.client.0.smithi070.stdout:0/337: link d4/d5/d13/d43/f53 d4/d5/d13/d23/f5b 0 2024-04-02T04:47:42.997 INFO:tasks.workunit.client.0.smithi070.stdout:1/174: mknod d0/d7/d28/c3a 0 2024-04-02T04:47:43.000 INFO:tasks.workunit.client.1.smithi179.stdout:3/173: mkdir d8/d26/d36 0 2024-04-02T04:47:43.001 INFO:tasks.workunit.client.0.smithi070.stdout:9/213: write d4/d10/d11/f42 [779068,57314] 0 2024-04-02T04:47:43.002 INFO:tasks.workunit.client.1.smithi179.stdout:4/286: fdatasync d1/d3/df/d10/d2f/f4e 0 2024-04-02T04:47:43.003 INFO:tasks.workunit.client.0.smithi070.stdout:4/168: truncate d1/fa 2301342 0 2024-04-02T04:47:43.004 INFO:tasks.workunit.client.1.smithi179.stdout:7/268: dread d1/f5d [0,4194304] 0 2024-04-02T04:47:43.004 INFO:tasks.workunit.client.1.smithi179.stdout:6/188: link d3/l6 d3/d2d/d38/l49 0 2024-04-02T04:47:43.005 INFO:tasks.workunit.client.1.smithi179.stdout:7/269: fsync d1/d9/d35/f50 0 2024-04-02T04:47:43.006 INFO:tasks.workunit.client.1.smithi179.stdout:6/189: truncate d3/d2d/d43/f47 77464 0 2024-04-02T04:47:43.007 INFO:tasks.workunit.client.0.smithi070.stdout:7/204: creat d7/d2f/f3b x:0 0 0 2024-04-02T04:47:43.009 INFO:tasks.workunit.client.0.smithi070.stdout:1/175: write d0/d9/f31 [1898463,119720] 0 2024-04-02T04:47:43.009 INFO:tasks.workunit.client.1.smithi179.stdout:9/248: getdents d0/d19 0 2024-04-02T04:47:43.010 INFO:tasks.workunit.client.0.smithi070.stdout:1/176: chown d0/d9/d1d/f13 21 1 2024-04-02T04:47:43.011 INFO:tasks.workunit.client.0.smithi070.stdout:7/205: dread d7/d9/f27 [0,4194304] 0 2024-04-02T04:47:43.012 INFO:tasks.workunit.client.1.smithi179.stdout:7/270: creat d1/d9/d55/f61 x:0 0 0 2024-04-02T04:47:43.012 INFO:tasks.workunit.client.0.smithi070.stdout:7/206: readlink d7/d9/d19/l2b 0 2024-04-02T04:47:43.012 INFO:tasks.workunit.client.0.smithi070.stdout:1/177: truncate d0/d9/d1d/f38 754405 0 2024-04-02T04:47:43.013 INFO:tasks.workunit.client.1.smithi179.stdout:3/174: mknod d8/d26/d36/c37 0 2024-04-02T04:47:43.013 INFO:tasks.workunit.client.0.smithi070.stdout:3/208: creat d10/d17/f48 x:0 0 0 2024-04-02T04:47:43.014 INFO:tasks.workunit.client.0.smithi070.stdout:1/178: write d0/d9/d1d/f26 [2291801,94385] 0 2024-04-02T04:47:43.015 INFO:tasks.workunit.client.1.smithi179.stdout:6/190: rename d3/d12/d23/d2f/f3d to d3/d12/d23/d2f/f4a 0 2024-04-02T04:47:43.016 INFO:tasks.workunit.client.1.smithi179.stdout:6/191: fsync d3/f1c 0 2024-04-02T04:47:43.017 INFO:tasks.workunit.client.0.smithi070.stdout:9/214: creat d4/d13/d16/d19/d2f/d39/f44 x:0 0 0 2024-04-02T04:47:43.020 INFO:tasks.workunit.client.0.smithi070.stdout:3/209: dread d10/d17/f2b [0,4194304] 0 2024-04-02T04:47:43.024 INFO:tasks.workunit.client.1.smithi179.stdout:7/271: creat d1/dd/d16/f62 x:0 0 0 2024-04-02T04:47:43.024 INFO:tasks.workunit.client.1.smithi179.stdout:7/272: dread - d1/d9/d55/f56 zero size 2024-04-02T04:47:43.025 INFO:tasks.workunit.client.0.smithi070.stdout:5/244: sync 2024-04-02T04:47:43.025 INFO:tasks.workunit.client.0.smithi070.stdout:0/338: unlink d4/f25 0 2024-04-02T04:47:43.026 INFO:tasks.workunit.client.1.smithi179.stdout:3/175: unlink d8/c2a 0 2024-04-02T04:47:43.031 INFO:tasks.workunit.client.0.smithi070.stdout:0/339: dwrite d4/d5/d13/d2a/d29/f46 [0,4194304] 0 2024-04-02T04:47:43.032 INFO:tasks.workunit.client.0.smithi070.stdout:0/340: stat d4/d5/d13/d17 0 2024-04-02T04:47:43.035 INFO:tasks.workunit.client.1.smithi179.stdout:4/287: rmdir d1/d3/df/d34 0 2024-04-02T04:47:43.035 INFO:tasks.workunit.client.0.smithi070.stdout:6/144: truncate d0/d3/f13 582310 0 2024-04-02T04:47:43.036 INFO:tasks.workunit.client.0.smithi070.stdout:9/215: truncate d4/fc 772271 0 2024-04-02T04:47:43.037 INFO:tasks.workunit.client.0.smithi070.stdout:9/216: truncate d4/f36 894070 0 2024-04-02T04:47:43.039 INFO:tasks.workunit.client.1.smithi179.stdout:8/210: sync 2024-04-02T04:47:43.039 INFO:tasks.workunit.client.1.smithi179.stdout:1/242: sync 2024-04-02T04:47:43.039 INFO:tasks.workunit.client.0.smithi070.stdout:3/210: mknod d10/d32/d3f/c49 0 2024-04-02T04:47:43.040 INFO:tasks.workunit.client.0.smithi070.stdout:3/211: write d10/d32/d3f/f3a [232082,9432] 0 2024-04-02T04:47:43.042 INFO:tasks.workunit.client.1.smithi179.stdout:3/176: creat d8/d26/d36/f38 x:0 0 0 2024-04-02T04:47:43.042 INFO:tasks.workunit.client.0.smithi070.stdout:8/257: link d2/d3/d6/f43 d2/d8/d23/f51 0 2024-04-02T04:47:43.043 INFO:tasks.workunit.client.1.smithi179.stdout:3/177: dread - d8/f32 zero size 2024-04-02T04:47:43.048 INFO:tasks.workunit.client.1.smithi179.stdout:1/243: dwrite d3/d5/de/f16 [4194304,4194304] 0 2024-04-02T04:47:43.049 INFO:tasks.workunit.client.1.smithi179.stdout:9/249: creat d0/f4c x:0 0 0 2024-04-02T04:47:43.050 INFO:tasks.workunit.client.1.smithi179.stdout:1/244: truncate d3/d5/f27 5530817 0 2024-04-02T04:47:43.050 INFO:tasks.workunit.client.0.smithi070.stdout:8/258: dwrite d2/d3/f49 [0,4194304] 0 2024-04-02T04:47:43.051 INFO:tasks.workunit.client.0.smithi070.stdout:4/169: dwrite d1/f26 [0,4194304] 0 2024-04-02T04:47:43.052 INFO:tasks.workunit.client.1.smithi179.stdout:7/273: unlink d1/c4a 0 2024-04-02T04:47:43.053 INFO:tasks.workunit.client.0.smithi070.stdout:7/207: write d7/dd/df/d34/f2a [2848318,86339] 0 2024-04-02T04:47:43.054 INFO:tasks.workunit.client.1.smithi179.stdout:7/274: stat d1/d9/d55/f56 0 2024-04-02T04:47:43.056 INFO:tasks.workunit.client.1.smithi179.stdout:7/275: fdatasync d1/f4e 0 2024-04-02T04:47:43.058 INFO:tasks.workunit.client.1.smithi179.stdout:9/250: dwrite d0/d1/d10/f2b [0,4194304] 0 2024-04-02T04:47:43.062 INFO:tasks.workunit.client.1.smithi179.stdout:8/211: rename d3/d32/l20 to d3/d32/d21/d45/l4d 0 2024-04-02T04:47:43.064 INFO:tasks.workunit.client.0.smithi070.stdout:4/170: dwrite d1/d3/f30 [0,4194304] 0 2024-04-02T04:47:43.066 INFO:tasks.workunit.client.0.smithi070.stdout:0/341: dwrite d4/d5/d13/f1f [4194304,4194304] 0 2024-04-02T04:47:43.068 INFO:tasks.workunit.client.0.smithi070.stdout:3/212: dread d10/d17/f2b [0,4194304] 0 2024-04-02T04:47:43.071 INFO:tasks.workunit.client.1.smithi179.stdout:7/276: dwrite d1/d9/d35/f50 [0,4194304] 0 2024-04-02T04:47:43.073 INFO:tasks.workunit.client.1.smithi179.stdout:3/178: mkdir d8/d26/d36/d39 0 2024-04-02T04:47:43.075 INFO:tasks.workunit.client.0.smithi070.stdout:3/213: dread d10/d32/d3f/f35 [0,4194304] 0 2024-04-02T04:47:43.076 INFO:tasks.workunit.client.0.smithi070.stdout:0/342: dwrite d4/d5/d13/d43/f49 [0,4194304] 0 2024-04-02T04:47:43.078 INFO:tasks.workunit.client.0.smithi070.stdout:8/259: rename d2/d3/d6 to d2/d3/d52 0 2024-04-02T04:47:43.078 INFO:tasks.workunit.client.0.smithi070.stdout:3/214: chown d10/d32/d3f/c49 0 1 2024-04-02T04:47:43.080 INFO:tasks.workunit.client.1.smithi179.stdout:7/277: dwrite d1/dd/d16/f2c [0,4194304] 0 2024-04-02T04:47:43.081 INFO:tasks.workunit.client.1.smithi179.stdout:6/192: write d3/f10 [1164822,40608] 0 2024-04-02T04:47:43.082 INFO:tasks.workunit.client.0.smithi070.stdout:7/208: rename d7/c37 to d7/dd/df/d34/c3c 0 2024-04-02T04:47:43.084 INFO:tasks.workunit.client.1.smithi179.stdout:1/245: fsync d3/d5/f13 0 2024-04-02T04:47:43.086 INFO:tasks.workunit.client.1.smithi179.stdout:7/278: write d1/d9/d35/f50 [3598688,69150] 0 2024-04-02T04:47:43.092 INFO:tasks.workunit.client.1.smithi179.stdout:7/279: write d1/d9/f43 [217882,55676] 0 2024-04-02T04:47:43.093 INFO:tasks.workunit.client.0.smithi070.stdout:0/343: symlink d4/d3c/l5c 0 2024-04-02T04:47:43.094 INFO:tasks.workunit.client.0.smithi070.stdout:8/260: creat d2/d3/f53 x:0 0 0 2024-04-02T04:47:43.094 INFO:tasks.workunit.client.0.smithi070.stdout:0/344: fdatasync d4/d5/d13/f32 0 2024-04-02T04:47:43.095 INFO:tasks.workunit.client.1.smithi179.stdout:1/246: dwrite d3/d5/d2f/d4f/f4c [0,4194304] 0 2024-04-02T04:47:43.095 INFO:tasks.workunit.client.1.smithi179.stdout:8/212: creat d3/d32/d30/f4e x:0 0 0 2024-04-02T04:47:43.095 INFO:tasks.workunit.client.1.smithi179.stdout:3/179: rename d8/la to d8/d26/l3a 0 2024-04-02T04:47:43.096 INFO:tasks.workunit.client.1.smithi179.stdout:6/193: creat d3/d2d/d43/f4b x:0 0 0 2024-04-02T04:47:43.098 INFO:tasks.workunit.client.0.smithi070.stdout:1/179: getdents d0/d9/d1d/d22 0 2024-04-02T04:47:43.100 INFO:tasks.workunit.client.1.smithi179.stdout:3/180: mknod d8/d26/d36/c3b 0 2024-04-02T04:47:43.102 INFO:tasks.workunit.client.0.smithi070.stdout:7/209: dwrite d7/d9/f1b [0,4194304] 0 2024-04-02T04:47:43.104 INFO:tasks.workunit.client.0.smithi070.stdout:8/261: mknod d2/d8/d23/d24/c54 0 2024-04-02T04:47:43.105 INFO:tasks.workunit.client.1.smithi179.stdout:8/213: unlink d3/d1b/f1c 0 2024-04-02T04:47:43.105 INFO:tasks.workunit.client.0.smithi070.stdout:8/262: dread - d2/d8/d23/d36/f4c zero size 2024-04-02T04:47:43.105 INFO:tasks.workunit.client.1.smithi179.stdout:6/194: symlink d3/l4c 0 2024-04-02T04:47:43.107 INFO:tasks.workunit.client.0.smithi070.stdout:6/145: getdents d0/d20 0 2024-04-02T04:47:43.109 INFO:tasks.workunit.client.0.smithi070.stdout:6/146: read d0/d20/f23 [576578,119261] 0 2024-04-02T04:47:43.109 INFO:tasks.workunit.client.1.smithi179.stdout:1/247: dwrite d3/d5/d2f/f47 [0,4194304] 0 2024-04-02T04:47:43.110 INFO:tasks.workunit.client.0.smithi070.stdout:4/171: link d1/f28 d1/f32 0 2024-04-02T04:47:43.111 INFO:tasks.workunit.client.0.smithi070.stdout:6/147: readlink d0/d3/d19/l2b 0 2024-04-02T04:47:43.113 INFO:tasks.workunit.client.1.smithi179.stdout:6/195: fsync d3/d12/d23/d2f/f4a 0 2024-04-02T04:47:43.114 INFO:tasks.workunit.client.0.smithi070.stdout:8/263: dwrite d2/d3/d52/f3c [0,4194304] 0 2024-04-02T04:47:43.114 INFO:tasks.workunit.client.0.smithi070.stdout:6/148: chown d0/f16 473394 1 2024-04-02T04:47:43.115 INFO:tasks.workunit.client.0.smithi070.stdout:3/215: creat d10/d17/f4a x:0 0 0 2024-04-02T04:47:43.119 INFO:tasks.workunit.client.1.smithi179.stdout:1/248: mkdir d3/d5/d2f/d4f/d42/d53 0 2024-04-02T04:47:43.121 INFO:tasks.workunit.client.0.smithi070.stdout:1/180: symlink d0/d7/l3b 0 2024-04-02T04:47:43.123 INFO:tasks.workunit.client.0.smithi070.stdout:6/149: dread d0/f16 [0,4194304] 0 2024-04-02T04:47:43.123 INFO:tasks.workunit.client.0.smithi070.stdout:8/264: creat d2/d3/d52/f55 x:0 0 0 2024-04-02T04:47:43.124 INFO:tasks.workunit.client.1.smithi179.stdout:4/288: dread d1/d3/d19/f2c [0,4194304] 0 2024-04-02T04:47:43.126 INFO:tasks.workunit.client.0.smithi070.stdout:3/216: mkdir d10/d32/d4b 0 2024-04-02T04:47:43.127 INFO:tasks.workunit.client.0.smithi070.stdout:3/217: write d10/f2c [282035,83396] 0 2024-04-02T04:47:43.127 INFO:tasks.workunit.client.1.smithi179.stdout:6/196: mknod d3/d42/c4d 0 2024-04-02T04:47:43.128 INFO:tasks.workunit.client.0.smithi070.stdout:0/345: rmdir d4/d5/d21/d44 0 2024-04-02T04:47:43.129 INFO:tasks.workunit.client.1.smithi179.stdout:6/197: dread - d3/d12/d23/d2f/f3c zero size 2024-04-02T04:47:43.130 INFO:tasks.workunit.client.0.smithi070.stdout:4/172: dread d1/d3/d9/f2a [0,4194304] 0 2024-04-02T04:47:43.131 INFO:tasks.workunit.client.1.smithi179.stdout:1/249: mknod d3/d5/c54 0 2024-04-02T04:47:43.132 INFO:tasks.workunit.client.0.smithi070.stdout:1/181: fdatasync d0/d9/fc 0 2024-04-02T04:47:43.133 INFO:tasks.workunit.client.0.smithi070.stdout:1/182: chown d0/d9/d1d/d22/c32 11 1 2024-04-02T04:47:43.133 INFO:tasks.workunit.client.0.smithi070.stdout:1/183: stat d0/d9/lf 0 2024-04-02T04:47:43.134 INFO:tasks.workunit.client.1.smithi179.stdout:4/289: read d1/d15/f31 [112997,96414] 0 2024-04-02T04:47:43.134 INFO:tasks.workunit.client.1.smithi179.stdout:6/198: mkdir d3/d12/d23/d39/d4e 0 2024-04-02T04:47:43.135 INFO:tasks.workunit.client.0.smithi070.stdout:8/265: rmdir d2/d8/d29/db 39 2024-04-02T04:47:43.136 INFO:tasks.workunit.client.0.smithi070.stdout:6/150: mkdir d0/d3/d19/d31 0 2024-04-02T04:47:43.139 INFO:tasks.workunit.client.0.smithi070.stdout:0/346: unlink d4/d5/cc 0 2024-04-02T04:47:43.140 INFO:tasks.workunit.client.0.smithi070.stdout:4/173: fdatasync d1/f12 0 2024-04-02T04:47:43.142 INFO:tasks.workunit.client.1.smithi179.stdout:6/199: dwrite d3/d12/f34 [4194304,4194304] 0 2024-04-02T04:47:43.143 INFO:tasks.workunit.client.1.smithi179.stdout:4/290: mkdir d1/d3/df/d10/d3e/d5d 0 2024-04-02T04:47:43.143 INFO:tasks.workunit.client.0.smithi070.stdout:1/184: symlink d0/d9/d1d/l3c 0 2024-04-02T04:47:43.143 INFO:tasks.workunit.client.0.smithi070.stdout:8/266: creat d2/d3/d35/d3a/f56 x:0 0 0 2024-04-02T04:47:43.145 INFO:tasks.workunit.client.0.smithi070.stdout:6/151: write d0/d20/f23 [1596632,49145] 0 2024-04-02T04:47:43.147 INFO:tasks.workunit.client.1.smithi179.stdout:6/200: creat d3/d12/d23/d26/f4f x:0 0 0 2024-04-02T04:47:43.147 INFO:tasks.workunit.client.0.smithi070.stdout:3/218: mkdir d10/d17/d4c 0 2024-04-02T04:47:43.148 INFO:tasks.workunit.client.0.smithi070.stdout:3/219: write d10/d17/f2a [1046253,62703] 0 2024-04-02T04:47:43.151 INFO:tasks.workunit.client.0.smithi070.stdout:4/174: getdents d1/d3/d9 0 2024-04-02T04:47:43.153 INFO:tasks.workunit.client.1.smithi179.stdout:4/291: dwrite d1/d3/df/d10/d2f/d36/f58 [0,4194304] 0 2024-04-02T04:47:43.153 INFO:tasks.workunit.client.1.smithi179.stdout:6/201: creat d3/d12/d23/d39/d3f/f50 x:0 0 0 2024-04-02T04:47:43.153 INFO:tasks.workunit.client.0.smithi070.stdout:8/267: symlink d2/d8/d29/d22/d4d/l57 0 2024-04-02T04:47:43.153 INFO:tasks.workunit.client.1.smithi179.stdout:4/292: chown d1/d3/dc 66590424 1 2024-04-02T04:47:43.154 INFO:tasks.workunit.client.0.smithi070.stdout:8/268: write d2/d3/d35/d3a/f56 [901613,96059] 0 2024-04-02T04:47:43.155 INFO:tasks.workunit.client.0.smithi070.stdout:8/269: write d2/f48 [1016467,55205] 0 2024-04-02T04:47:43.156 INFO:tasks.workunit.client.0.smithi070.stdout:3/220: creat d10/d32/d3f/f4d x:0 0 0 2024-04-02T04:47:43.157 INFO:tasks.workunit.client.0.smithi070.stdout:4/175: chown d1/d3/d9/l2e 4051 1 2024-04-02T04:47:43.163 INFO:tasks.workunit.client.1.smithi179.stdout:4/293: getdents d1/d15 0 2024-04-02T04:47:43.164 INFO:tasks.workunit.client.0.smithi070.stdout:4/176: dwrite d1/f26 [0,4194304] 0 2024-04-02T04:47:43.165 INFO:tasks.workunit.client.0.smithi070.stdout:4/177: write d1/d3/d9/f24 [6597550,14085] 0 2024-04-02T04:47:43.166 INFO:tasks.workunit.client.0.smithi070.stdout:4/178: chown d1/d3/l31 2230723 1 2024-04-02T04:47:43.167 INFO:tasks.workunit.client.1.smithi179.stdout:4/294: creat d1/d3/dc/d44/f5e x:0 0 0 2024-04-02T04:47:43.168 INFO:tasks.workunit.client.1.smithi179.stdout:4/295: write d1/d3/df/d10/f11 [1760125,51344] 0 2024-04-02T04:47:43.170 INFO:tasks.workunit.client.1.smithi179.stdout:4/296: chown d1/d3/df/d10/d2f 2022411 1 2024-04-02T04:47:43.173 INFO:tasks.workunit.client.0.smithi070.stdout:4/179: dwrite d1/f28 [0,4194304] 0 2024-04-02T04:47:43.175 INFO:tasks.workunit.client.1.smithi179.stdout:6/202: dread d3/fa [0,4194304] 0 2024-04-02T04:47:43.176 INFO:tasks.workunit.client.1.smithi179.stdout:6/203: chown d3/d2d/d38 622321475 1 2024-04-02T04:47:43.177 INFO:tasks.workunit.client.0.smithi070.stdout:4/180: creat d1/f33 x:0 0 0 2024-04-02T04:47:43.179 INFO:tasks.workunit.client.1.smithi179.stdout:4/297: dwrite d1/d3/df/d10/d3e/f55 [0,4194304] 0 2024-04-02T04:47:43.179 INFO:tasks.workunit.client.0.smithi070.stdout:4/181: symlink d1/d3/l34 0 2024-04-02T04:47:43.180 INFO:tasks.workunit.client.1.smithi179.stdout:4/298: readlink d1/d3/d19/l33 0 2024-04-02T04:47:43.181 INFO:tasks.workunit.client.0.smithi070.stdout:4/182: mknod d1/d3/c35 0 2024-04-02T04:47:43.181 INFO:tasks.workunit.client.0.smithi070.stdout:4/183: readlink d1/d3/d9/l23 0 2024-04-02T04:47:43.182 INFO:tasks.workunit.client.0.smithi070.stdout:4/184: chown d1/d3/d9/f1d 467572303 1 2024-04-02T04:47:43.183 INFO:tasks.workunit.client.0.smithi070.stdout:4/185: readlink d1/d3/d9/l2e 0 2024-04-02T04:47:43.184 INFO:tasks.workunit.client.1.smithi179.stdout:4/299: symlink d1/d3/df/d10/d3e/l5f 0 2024-04-02T04:47:43.185 INFO:tasks.workunit.client.1.smithi179.stdout:4/300: write d1/fb [4998704,46469] 0 2024-04-02T04:47:43.187 INFO:tasks.workunit.client.1.smithi179.stdout:6/204: rename d3/d2c/l3b to d3/d12/d44/l51 0 2024-04-02T04:47:43.187 INFO:tasks.workunit.client.1.smithi179.stdout:6/205: chown d3/d42 494 1 2024-04-02T04:47:43.189 INFO:tasks.workunit.client.1.smithi179.stdout:4/301: rmdir d1/d3/df/d10 39 2024-04-02T04:47:43.189 INFO:tasks.workunit.client.0.smithi070.stdout:4/186: dwrite d1/d3/f2c [0,4194304] 0 2024-04-02T04:47:43.192 INFO:tasks.workunit.client.1.smithi179.stdout:6/206: truncate d3/d12/d23/d26/f37 1287233 0 2024-04-02T04:47:43.193 INFO:tasks.workunit.client.0.smithi070.stdout:7/210: dread d7/dd/df/f10 [0,4194304] 0 2024-04-02T04:47:43.194 INFO:tasks.workunit.client.1.smithi179.stdout:4/302: write d1/d3/df/d10/d3e/f55 [2829514,29018] 0 2024-04-02T04:47:43.194 INFO:tasks.workunit.client.1.smithi179.stdout:4/303: chown d1/fb 11 1 2024-04-02T04:47:43.195 INFO:tasks.workunit.client.1.smithi179.stdout:4/304: write d1/d3/f5b [754525,79222] 0 2024-04-02T04:47:43.197 INFO:tasks.workunit.client.0.smithi070.stdout:7/211: write d7/d9/f21 [122656,121094] 0 2024-04-02T04:47:43.198 INFO:tasks.workunit.client.0.smithi070.stdout:1/185: dread d0/d7/d1a/f1b [0,4194304] 0 2024-04-02T04:47:43.199 INFO:tasks.workunit.client.1.smithi179.stdout:6/207: unlink d3/l28 0 2024-04-02T04:47:43.200 INFO:tasks.workunit.client.1.smithi179.stdout:6/208: dread - d3/d2d/d43/f4b zero size 2024-04-02T04:47:43.200 INFO:tasks.workunit.client.0.smithi070.stdout:1/186: write d0/d9/fc [199625,4273] 0 2024-04-02T04:47:43.203 INFO:tasks.workunit.client.1.smithi179.stdout:0/281: sync 2024-04-02T04:47:43.203 INFO:tasks.workunit.client.0.smithi070.stdout:7/212: mknod d7/d2f/c3d 0 2024-04-02T04:47:43.205 INFO:tasks.workunit.client.1.smithi179.stdout:6/209: creat d3/d12/d44/f52 x:0 0 0 2024-04-02T04:47:43.205 INFO:tasks.workunit.client.0.smithi070.stdout:2/314: sync 2024-04-02T04:47:43.206 INFO:tasks.workunit.client.0.smithi070.stdout:1/187: creat d0/d9/d1d/f3d x:0 0 0 2024-04-02T04:47:43.207 INFO:tasks.workunit.client.0.smithi070.stdout:8/270: fsync d2/f48 0 2024-04-02T04:47:43.208 INFO:tasks.workunit.client.0.smithi070.stdout:7/213: symlink d7/d9/d19/l3e 0 2024-04-02T04:47:43.208 INFO:tasks.workunit.client.0.smithi070.stdout:4/187: dread d1/f12 [0,4194304] 0 2024-04-02T04:47:43.210 INFO:tasks.workunit.client.1.smithi179.stdout:4/305: dwrite d1/d3/f13 [0,4194304] 0 2024-04-02T04:47:43.212 INFO:tasks.workunit.client.1.smithi179.stdout:6/210: symlink d3/d2c/l53 0 2024-04-02T04:47:43.216 INFO:tasks.workunit.client.1.smithi179.stdout:4/306: mkdir d1/d3/df/d10/d2f/d60 0 2024-04-02T04:47:43.218 INFO:tasks.workunit.client.1.smithi179.stdout:6/211: unlink d3/c1e 0 2024-04-02T04:47:43.218 INFO:tasks.workunit.client.1.smithi179.stdout:0/282: dwrite d1/f14 [4194304,4194304] 0 2024-04-02T04:47:43.220 INFO:tasks.workunit.client.1.smithi179.stdout:4/307: mknod d1/d3/d19/c61 0 2024-04-02T04:47:43.220 INFO:tasks.workunit.client.1.smithi179.stdout:6/212: stat d3/d12/c35 0 2024-04-02T04:47:43.222 INFO:tasks.workunit.client.1.smithi179.stdout:1/250: write d3/d5/d2f/f47 [4663704,58411] 0 2024-04-02T04:47:43.223 INFO:tasks.workunit.client.1.smithi179.stdout:3/181: truncate d8/f28 3818915 0 2024-04-02T04:47:43.227 INFO:tasks.workunit.client.1.smithi179.stdout:1/251: write d3/d5/f13 [3406539,38574] 0 2024-04-02T04:47:43.227 INFO:tasks.workunit.client.1.smithi179.stdout:6/213: chown d3/d12/f25 550 1 2024-04-02T04:47:43.227 INFO:tasks.workunit.client.1.smithi179.stdout:0/283: symlink d1/da/df/l55 0 2024-04-02T04:47:43.229 INFO:tasks.workunit.client.1.smithi179.stdout:3/182: dread d8/d26/f2c [4194304,4194304] 0 2024-04-02T04:47:43.232 INFO:tasks.workunit.client.1.smithi179.stdout:9/251: dwrite d0/d1/d11/d29/f37 [0,4194304] 0 2024-04-02T04:47:43.236 INFO:tasks.workunit.client.1.smithi179.stdout:7/280: dwrite d1/dd/d23/f25 [0,4194304] 0 2024-04-02T04:47:43.238 INFO:tasks.workunit.client.1.smithi179.stdout:1/252: mkdir d3/d5/d2f/d4f/d42/d55 0 2024-04-02T04:47:43.243 INFO:tasks.workunit.client.1.smithi179.stdout:0/284: rename d1/da/d48/d50 to d1/da/df/d56 0 2024-04-02T04:47:43.244 INFO:tasks.workunit.client.1.smithi179.stdout:1/253: fsync d3/d5/de/f32 0 2024-04-02T04:47:43.244 INFO:tasks.workunit.client.1.smithi179.stdout:6/214: dread d3/f10 [0,4194304] 0 2024-04-02T04:47:43.245 INFO:tasks.workunit.client.1.smithi179.stdout:4/308: dwrite d1/d3/df/f59 [0,4194304] 0 2024-04-02T04:47:43.245 INFO:tasks.workunit.client.1.smithi179.stdout:7/281: creat d1/d9/f63 x:0 0 0 2024-04-02T04:47:43.246 INFO:tasks.workunit.client.1.smithi179.stdout:6/215: chown d3/d12/d44 3982 1 2024-04-02T04:47:43.246 INFO:tasks.workunit.client.1.smithi179.stdout:0/285: write d1/da/d28/d37/f3f [290253,126861] 0 2024-04-02T04:47:43.247 INFO:tasks.workunit.client.1.smithi179.stdout:7/282: fdatasync d1/d9/f12 0 2024-04-02T04:47:43.249 INFO:tasks.workunit.client.1.smithi179.stdout:6/216: dread - d3/d12/d23/d39/d3f/f50 zero size 2024-04-02T04:47:43.250 INFO:tasks.workunit.client.1.smithi179.stdout:3/183: rmdir d8/d26/d36/d39 0 2024-04-02T04:47:43.253 INFO:tasks.workunit.client.0.smithi070.stdout:4/188: write d1/d3/d9/f25 [846090,118217] 0 2024-04-02T04:47:43.253 INFO:tasks.workunit.client.1.smithi179.stdout:7/283: creat d1/d9/d55/f64 x:0 0 0 2024-04-02T04:47:43.254 INFO:tasks.workunit.client.0.smithi070.stdout:8/271: mknod d2/d3/d35/d3a/d4f/c58 0 2024-04-02T04:47:43.256 INFO:tasks.workunit.client.0.smithi070.stdout:2/315: rename d2/d10/c3b to d2/d10/d53/c65 0 2024-04-02T04:47:43.258 INFO:tasks.workunit.client.1.smithi179.stdout:4/309: rename d1/d3/dc/d44 to d1/d3/dc/d27/d62 0 2024-04-02T04:47:43.259 INFO:tasks.workunit.client.0.smithi070.stdout:4/189: chown d1/f1e 32165780 1 2024-04-02T04:47:43.259 INFO:tasks.workunit.client.0.smithi070.stdout:1/188: dwrite d0/d9/d1d/f3d [0,4194304] 0 2024-04-02T04:47:43.261 INFO:tasks.workunit.client.1.smithi179.stdout:1/254: symlink d3/d5/d2f/d4f/d42/d55/l56 0 2024-04-02T04:47:43.261 INFO:tasks.workunit.client.0.smithi070.stdout:6/152: write d0/d20/f27 [314151,122322] 0 2024-04-02T04:47:43.263 INFO:tasks.workunit.client.0.smithi070.stdout:4/190: dread - d1/f33 zero size 2024-04-02T04:47:43.265 INFO:tasks.workunit.client.0.smithi070.stdout:4/191: chown d1/d3/d9 26700094 1 2024-04-02T04:47:43.266 INFO:tasks.workunit.client.1.smithi179.stdout:7/284: unlink d1/d9/f63 0 2024-04-02T04:47:43.267 INFO:tasks.workunit.client.1.smithi179.stdout:3/184: dwrite d8/d26/f34 [0,4194304] 0 2024-04-02T04:47:43.267 INFO:tasks.workunit.client.0.smithi070.stdout:8/272: mkdir d2/d3/d35/d3a/d4f/d59 0 2024-04-02T04:47:43.270 INFO:tasks.workunit.client.0.smithi070.stdout:3/221: dwrite f0 [4194304,4194304] 0 2024-04-02T04:47:43.272 INFO:tasks.workunit.client.0.smithi070.stdout:3/222: rename d10 to d10/d32/d4b/d4e 22 2024-04-02T04:47:43.273 INFO:tasks.workunit.client.0.smithi070.stdout:2/316: symlink d2/d12/d48/l66 0 2024-04-02T04:47:43.275 INFO:tasks.workunit.client.1.smithi179.stdout:4/310: mknod d1/d3/df/d10/d2f/d36/c63 0 2024-04-02T04:47:43.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:43 smithi070 ceph-mon[29212]: pgmap v10: 65 pgs: 65 active+clean; 518 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 838 KiB/s rd, 51 MiB/s wr, 231 op/s 2024-04-02T04:47:43.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:43 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:43.281 INFO:tasks.workunit.client.1.smithi179.stdout:6/217: dread d3/d12/f25 [0,4194304] 0 2024-04-02T04:47:43.281 INFO:tasks.workunit.client.1.smithi179.stdout:4/311: dread d1/d3/df/f59 [0,4194304] 0 2024-04-02T04:47:43.281 INFO:tasks.workunit.client.1.smithi179.stdout:6/218: readlink d3/l11 0 2024-04-02T04:47:43.282 INFO:tasks.workunit.client.1.smithi179.stdout:6/219: stat d3/d2c 0 2024-04-02T04:47:43.284 INFO:tasks.workunit.client.1.smithi179.stdout:7/285: symlink d1/dd/d23/l65 0 2024-04-02T04:47:43.289 INFO:tasks.workunit.client.1.smithi179.stdout:3/185: rename c7 to d8/d26/c3c 0 2024-04-02T04:47:43.290 INFO:tasks.workunit.client.1.smithi179.stdout:9/252: write d0/d1/d11/d29/d40/f49 [4321338,81419] 0 2024-04-02T04:47:43.290 INFO:tasks.workunit.client.0.smithi070.stdout:2/317: dread d2/d24/f2d [0,4194304] 0 2024-04-02T04:47:43.292 INFO:tasks.workunit.client.1.smithi179.stdout:9/253: write d0/d1/f36 [625373,110886] 0 2024-04-02T04:47:43.293 INFO:tasks.workunit.client.1.smithi179.stdout:9/254: write d0/d1/d11/d29/f37 [2217868,112030] 0 2024-04-02T04:47:43.294 INFO:tasks.workunit.client.0.smithi070.stdout:8/273: mkdir d2/d8/d29/d22/d5a 0 2024-04-02T04:47:43.297 INFO:tasks.workunit.client.1.smithi179.stdout:7/286: symlink d1/d9/d55/l66 0 2024-04-02T04:47:43.298 INFO:tasks.workunit.client.0.smithi070.stdout:3/223: creat d10/f4f x:0 0 0 2024-04-02T04:47:43.299 INFO:tasks.workunit.client.1.smithi179.stdout:6/220: dread d3/d12/f21 [0,4194304] 0 2024-04-02T04:47:43.300 INFO:tasks.workunit.client.1.smithi179.stdout:6/221: readlink d3/d2c/l53 0 2024-04-02T04:47:43.303 INFO:tasks.workunit.client.0.smithi070.stdout:7/214: write d7/d9/f17 [77486,21150] 0 2024-04-02T04:47:43.305 INFO:tasks.workunit.client.1.smithi179.stdout:4/312: mknod d1/d3/df/d10/d2f/d60/c64 0 2024-04-02T04:47:43.305 INFO:tasks.workunit.client.0.smithi070.stdout:3/224: dwrite d10/d32/d3f/f1f [0,4194304] 0 2024-04-02T04:47:43.307 INFO:tasks.workunit.client.1.smithi179.stdout:2/128: sync 2024-04-02T04:47:43.307 INFO:tasks.workunit.client.0.smithi070.stdout:3/225: write d10/d32/d3f/f1f [3478096,91967] 0 2024-04-02T04:47:43.308 INFO:tasks.workunit.client.1.smithi179.stdout:2/129: chown dc/d16/d1e/c1f 1317 1 2024-04-02T04:47:43.308 INFO:tasks.workunit.client.0.smithi070.stdout:3/226: fdatasync d10/f4f 0 2024-04-02T04:47:43.309 INFO:tasks.workunit.client.0.smithi070.stdout:4/192: mkdir d1/d3/d36 0 2024-04-02T04:47:43.309 INFO:tasks.workunit.client.1.smithi179.stdout:9/255: rename d0 to d0/d4d 22 2024-04-02T04:47:43.310 INFO:tasks.workunit.client.0.smithi070.stdout:7/215: dwrite d7/d9/fa [4194304,4194304] 0 2024-04-02T04:47:43.311 INFO:tasks.workunit.client.0.smithi070.stdout:3/227: truncate d10/d32/d3f/f47 898111 0 2024-04-02T04:47:43.312 INFO:tasks.workunit.client.1.smithi179.stdout:7/287: symlink d1/dd/d16/l67 0 2024-04-02T04:47:43.313 INFO:tasks.workunit.client.0.smithi070.stdout:8/274: creat d2/d8/d23/d24/f5b x:0 0 0 2024-04-02T04:47:43.313 INFO:tasks.workunit.client.1.smithi179.stdout:7/288: fdatasync d1/dd/d16/f2c 0 2024-04-02T04:47:43.314 INFO:tasks.workunit.client.1.smithi179.stdout:7/289: write d1/ff [2307869,33782] 0 2024-04-02T04:47:43.319 INFO:tasks.workunit.client.1.smithi179.stdout:6/222: unlink d3/c15 0 2024-04-02T04:47:43.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:43 smithi179 ceph-mon[36603]: pgmap v10: 65 pgs: 65 active+clean; 518 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 838 KiB/s rd, 51 MiB/s wr, 231 op/s 2024-04-02T04:47:43.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:43 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:43.324 INFO:tasks.workunit.client.0.smithi070.stdout:1/189: dwrite d0/d9/d1d/d22/f2d [0,4194304] 0 2024-04-02T04:47:43.325 INFO:tasks.workunit.client.0.smithi070.stdout:1/190: write d0/d9/d1d/f26 [5054699,78166] 0 2024-04-02T04:47:43.329 INFO:tasks.workunit.client.0.smithi070.stdout:7/216: symlink d7/dd/df/l3f 0 2024-04-02T04:47:43.332 INFO:tasks.workunit.client.0.smithi070.stdout:8/275: rmdir d2/d8/d29/d22/d4d 39 2024-04-02T04:47:43.335 INFO:tasks.workunit.client.1.smithi179.stdout:7/290: rename d1/d9/d35/f50 to d1/d9/f68 0 2024-04-02T04:47:43.335 INFO:tasks.workunit.client.1.smithi179.stdout:7/291: write d1/ff [3375723,26697] 0 2024-04-02T04:47:43.336 INFO:tasks.workunit.client.0.smithi070.stdout:6/153: getdents d0/d3 0 2024-04-02T04:47:43.336 INFO:tasks.workunit.client.0.smithi070.stdout:6/154: stat d0/d3/d19/l2b 0 2024-04-02T04:47:43.341 INFO:tasks.workunit.client.0.smithi070.stdout:6/155: dread d0/d20/f23 [0,4194304] 0 2024-04-02T04:47:43.341 INFO:tasks.workunit.client.0.smithi070.stdout:6/156: dread - d0/d3/d19/f2f zero size 2024-04-02T04:47:43.342 INFO:tasks.workunit.client.1.smithi179.stdout:2/130: symlink dc/d16/l28 0 2024-04-02T04:47:43.343 INFO:tasks.workunit.client.0.smithi070.stdout:4/193: mknod d1/d3/c37 0 2024-04-02T04:47:43.346 INFO:tasks.workunit.client.0.smithi070.stdout:7/217: mknod d7/dd/df/c40 0 2024-04-02T04:47:43.348 INFO:tasks.workunit.client.1.smithi179.stdout:7/292: dread d1/f45 [0,4194304] 0 2024-04-02T04:47:43.348 INFO:tasks.workunit.client.1.smithi179.stdout:2/131: dwrite dc/f12 [4194304,4194304] 0 2024-04-02T04:47:43.349 INFO:tasks.workunit.client.1.smithi179.stdout:9/256: truncate d0/d1/f4 4014124 0 2024-04-02T04:47:43.350 INFO:tasks.workunit.client.0.smithi070.stdout:2/318: getdents d2 0 2024-04-02T04:47:43.352 INFO:tasks.workunit.client.0.smithi070.stdout:6/157: symlink d0/d20/l32 0 2024-04-02T04:47:43.352 INFO:tasks.workunit.client.1.smithi179.stdout:3/186: getdents d8 0 2024-04-02T04:47:43.354 INFO:tasks.workunit.client.1.smithi179.stdout:6/223: truncate d3/d12/d23/d26/f37 1511807 0 2024-04-02T04:47:43.354 INFO:tasks.workunit.client.1.smithi179.stdout:6/224: chown d3/ff 2935 1 2024-04-02T04:47:43.355 INFO:tasks.workunit.client.0.smithi070.stdout:2/319: dread d2/f32 [0,4194304] 0 2024-04-02T04:47:43.355 INFO:tasks.workunit.client.0.smithi070.stdout:4/194: symlink d1/d3/d9/l38 0 2024-04-02T04:47:43.355 INFO:tasks.workunit.client.1.smithi179.stdout:6/225: stat d3/d2d 0 2024-04-02T04:47:43.356 INFO:tasks.workunit.client.0.smithi070.stdout:4/195: write d1/d3/d9/f13 [5924568,30923] 0 2024-04-02T04:47:43.356 INFO:tasks.workunit.client.1.smithi179.stdout:6/226: dread - d3/d12/d23/d26/f4f zero size 2024-04-02T04:47:43.359 INFO:tasks.workunit.client.1.smithi179.stdout:3/187: dwrite d8/f1b [0,4194304] 0 2024-04-02T04:47:43.360 INFO:tasks.workunit.client.0.smithi070.stdout:7/218: fsync d7/dd/df/f32 0 2024-04-02T04:47:43.362 INFO:tasks.workunit.client.1.smithi179.stdout:9/257: creat d0/d19/f4e x:0 0 0 2024-04-02T04:47:43.363 INFO:tasks.workunit.client.1.smithi179.stdout:2/132: dread f2 [0,4194304] 0 2024-04-02T04:47:43.364 INFO:tasks.workunit.client.0.smithi070.stdout:8/276: rename d2/d8/d29/db/c1e to d2/d3/d35/d3a/c5c 0 2024-04-02T04:47:43.369 INFO:tasks.workunit.client.1.smithi179.stdout:2/133: dwrite dc/f1a [4194304,4194304] 0 2024-04-02T04:47:43.372 INFO:tasks.workunit.client.1.smithi179.stdout:6/227: rmdir d3/d12/d23/d2f 39 2024-04-02T04:47:43.372 INFO:tasks.workunit.client.1.smithi179.stdout:2/134: dread dc/f17 [0,4194304] 0 2024-04-02T04:47:43.373 INFO:tasks.workunit.client.0.smithi070.stdout:2/320: truncate d2/fb 4167392 0 2024-04-02T04:47:43.373 INFO:tasks.workunit.client.1.smithi179.stdout:6/228: dread - d3/d2d/d43/f4b zero size 2024-04-02T04:47:43.373 INFO:tasks.workunit.client.0.smithi070.stdout:2/321: readlink d2/d12/d48/l56 0 2024-04-02T04:47:43.374 INFO:tasks.workunit.client.1.smithi179.stdout:2/135: write dc/d16/d1e/f23 [1040412,57141] 0 2024-04-02T04:47:43.375 INFO:tasks.workunit.client.1.smithi179.stdout:3/188: rename d8/c30 to d8/d26/d36/c3d 0 2024-04-02T04:47:43.377 INFO:tasks.workunit.client.0.smithi070.stdout:1/191: rename d0/d9/f31 to d0/d9/f3e 0 2024-04-02T04:47:43.383 INFO:tasks.workunit.client.1.smithi179.stdout:7/293: link d1/dd/d16/f62 d1/d9/d27/f69 0 2024-04-02T04:47:43.385 INFO:tasks.workunit.client.0.smithi070.stdout:6/158: rename d0/c10 to d0/c33 0 2024-04-02T04:47:43.386 INFO:tasks.workunit.client.0.smithi070.stdout:6/159: read - d0/d3/d19/f1f zero size 2024-04-02T04:47:43.390 INFO:tasks.workunit.client.1.smithi179.stdout:9/258: dread d0/d1/d12/f20 [0,4194304] 0 2024-04-02T04:47:43.390 INFO:tasks.workunit.client.0.smithi070.stdout:7/219: symlink d7/l41 0 2024-04-02T04:47:43.391 INFO:tasks.workunit.client.1.smithi179.stdout:9/259: truncate d0/d1/f36 1257085 0 2024-04-02T04:47:43.395 INFO:tasks.workunit.client.0.smithi070.stdout:6/160: fsync d0/d3/fb 0 2024-04-02T04:47:43.395 INFO:tasks.workunit.client.0.smithi070.stdout:6/161: chown d0/d20/f28 3 1 2024-04-02T04:47:43.397 INFO:tasks.workunit.client.0.smithi070.stdout:1/192: mknod d0/d7/c3f 0 2024-04-02T04:47:43.401 INFO:tasks.workunit.client.0.smithi070.stdout:4/196: rename d1/d3/f2b to d1/d3/f39 0 2024-04-02T04:47:43.401 INFO:tasks.workunit.client.0.smithi070.stdout:4/197: fdatasync d1/d3/d9/f17 0 2024-04-02T04:47:43.405 INFO:tasks.workunit.client.1.smithi179.stdout:3/189: write d8/d26/f33 [1135499,36904] 0 2024-04-02T04:47:43.408 INFO:tasks.workunit.client.0.smithi070.stdout:1/193: creat d0/d9/d1d/f40 x:0 0 0 2024-04-02T04:47:43.410 INFO:tasks.workunit.client.1.smithi179.stdout:9/260: rename d0/d1/d12/c1a to d0/d19/d1c/c4f 0 2024-04-02T04:47:43.410 INFO:tasks.workunit.client.0.smithi070.stdout:4/198: dread d1/fb [0,4194304] 0 2024-04-02T04:47:43.410 INFO:tasks.workunit.client.1.smithi179.stdout:9/261: readlink d0/d1/d11/l46 0 2024-04-02T04:47:43.412 INFO:tasks.workunit.client.1.smithi179.stdout:5/163: sync 2024-04-02T04:47:43.413 INFO:tasks.workunit.client.0.smithi070.stdout:7/220: symlink d7/l42 0 2024-04-02T04:47:43.414 INFO:tasks.workunit.client.0.smithi070.stdout:7/221: read d7/d9/f1b [1159524,126899] 0 2024-04-02T04:47:43.414 INFO:tasks.workunit.client.0.smithi070.stdout:8/277: getdents d2/d8/d29/db 0 2024-04-02T04:47:43.415 INFO:tasks.workunit.client.0.smithi070.stdout:7/222: fdatasync d7/dd/f2e 0 2024-04-02T04:47:43.415 INFO:tasks.workunit.client.0.smithi070.stdout:2/322: truncate d2/f4 3206024 0 2024-04-02T04:47:43.416 INFO:tasks.workunit.client.0.smithi070.stdout:2/323: write d2/f32 [4749418,40668] 0 2024-04-02T04:47:43.417 INFO:tasks.workunit.client.1.smithi179.stdout:2/136: truncate dc/d16/f24 7790319 0 2024-04-02T04:47:43.418 INFO:tasks.workunit.client.0.smithi070.stdout:2/324: write d2/d24/f37 [1358665,39773] 0 2024-04-02T04:47:43.418 INFO:tasks.workunit.client.1.smithi179.stdout:7/294: dwrite d1/dd/d23/f24 [0,4194304] 0 2024-04-02T04:47:43.422 INFO:tasks.workunit.client.0.smithi070.stdout:8/278: dwrite d2/f1f [0,4194304] 0 2024-04-02T04:47:43.423 INFO:tasks.workunit.client.0.smithi070.stdout:8/279: stat d2/d3/d35 0 2024-04-02T04:47:43.424 INFO:tasks.workunit.client.1.smithi179.stdout:9/262: dwrite d0/d19/f4e [0,4194304] 0 2024-04-02T04:47:43.428 INFO:tasks.workunit.client.1.smithi179.stdout:6/229: dwrite d3/fb [0,4194304] 0 2024-04-02T04:47:43.428 INFO:tasks.workunit.client.1.smithi179.stdout:9/263: readlink d0/d19/d1c/l22 0 2024-04-02T04:47:43.431 INFO:tasks.workunit.client.0.smithi070.stdout:6/162: dwrite d0/d20/f23 [0,4194304] 0 2024-04-02T04:47:43.431 INFO:tasks.workunit.client.1.smithi179.stdout:5/164: dwrite f1 [0,4194304] 0 2024-04-02T04:47:43.432 INFO:tasks.workunit.client.0.smithi070.stdout:6/163: stat d0/c2c 0 2024-04-02T04:47:43.432 INFO:tasks.workunit.client.1.smithi179.stdout:7/295: mknod d1/dd/d16/c6a 0 2024-04-02T04:47:43.434 INFO:tasks.workunit.client.1.smithi179.stdout:5/165: chown d8/f10 31124 1 2024-04-02T04:47:43.444 INFO:tasks.workunit.client.1.smithi179.stdout:4/313: sync 2024-04-02T04:47:43.444 INFO:tasks.workunit.client.1.smithi179.stdout:0/286: sync 2024-04-02T04:47:43.444 INFO:tasks.workunit.client.1.smithi179.stdout:8/214: sync 2024-04-02T04:47:43.444 INFO:tasks.workunit.client.1.smithi179.stdout:1/255: sync 2024-04-02T04:47:43.445 INFO:tasks.workunit.client.0.smithi070.stdout:1/194: write d0/d9/d1d/f13 [58448,95530] 0 2024-04-02T04:47:43.448 INFO:tasks.workunit.client.0.smithi070.stdout:2/325: unlink d2/d10/c41 0 2024-04-02T04:47:43.450 INFO:tasks.workunit.client.1.smithi179.stdout:3/190: dwrite d8/f1c [0,4194304] 0 2024-04-02T04:47:43.451 INFO:tasks.workunit.client.1.smithi179.stdout:9/264: dwrite d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:43.452 INFO:tasks.workunit.client.1.smithi179.stdout:5/166: symlink d8/d1b/d23/d32/l35 0 2024-04-02T04:47:43.452 INFO:tasks.workunit.client.1.smithi179.stdout:4/314: chown d1/d3/dc/f18 528327617 1 2024-04-02T04:47:43.453 INFO:tasks.workunit.client.1.smithi179.stdout:7/296: mkdir d1/d9/d35/d6b 0 2024-04-02T04:47:43.453 INFO:tasks.workunit.client.1.smithi179.stdout:2/137: write f2 [1108203,16689] 0 2024-04-02T04:47:43.460 INFO:tasks.workunit.client.1.smithi179.stdout:6/230: truncate d3/d12/f22 1994591 0 2024-04-02T04:47:43.461 INFO:tasks.workunit.client.0.smithi070.stdout:8/280: dread d2/d3/fd [0,4194304] 0 2024-04-02T04:47:43.462 INFO:tasks.workunit.client.0.smithi070.stdout:8/281: chown d2/d8/d29/f15 2941 1 2024-04-02T04:47:43.463 INFO:tasks.workunit.client.0.smithi070.stdout:7/223: mknod d7/c43 0 2024-04-02T04:47:43.465 INFO:tasks.workunit.client.1.smithi179.stdout:9/265: stat d0/d1/d12/l1b 0 2024-04-02T04:47:43.465 INFO:tasks.workunit.client.1.smithi179.stdout:2/138: write dc/d16/d1e/f23 [540746,78019] 0 2024-04-02T04:47:43.465 INFO:tasks.workunit.client.1.smithi179.stdout:0/287: rmdir d1/da/d28 39 2024-04-02T04:47:43.465 INFO:tasks.workunit.client.0.smithi070.stdout:6/164: creat d0/d20/f34 x:0 0 0 2024-04-02T04:47:43.466 INFO:tasks.workunit.client.1.smithi179.stdout:4/315: creat d1/f65 x:0 0 0 2024-04-02T04:47:43.468 INFO:tasks.workunit.client.1.smithi179.stdout:1/256: dwrite d3/d5/de/f32 [0,4194304] 0 2024-04-02T04:47:43.469 INFO:tasks.workunit.client.1.smithi179.stdout:8/215: dwrite d3/d32/d21/d22/d33/f34 [0,4194304] 0 2024-04-02T04:47:43.469 INFO:tasks.workunit.client.1.smithi179.stdout:9/266: write d0/d1/d12/f25 [563741,20841] 0 2024-04-02T04:47:43.470 INFO:tasks.workunit.client.0.smithi070.stdout:4/199: rmdir d1/d3/d36 0 2024-04-02T04:47:43.471 INFO:tasks.workunit.client.1.smithi179.stdout:8/216: fdatasync d3/d32/d21/d22/d33/d3c/f44 0 2024-04-02T04:47:43.472 INFO:tasks.workunit.client.0.smithi070.stdout:8/282: dwrite d2/d8/d23/d24/f5b [0,4194304] 0 2024-04-02T04:47:43.472 INFO:tasks.workunit.client.0.smithi070.stdout:6/165: dread d0/d20/f28 [0,4194304] 0 2024-04-02T04:47:43.472 INFO:tasks.workunit.client.1.smithi179.stdout:3/191: rename d8/d26/d36/c3b to d8/d26/d36/c3e 0 2024-04-02T04:47:43.474 INFO:tasks.workunit.client.0.smithi070.stdout:7/224: mknod d7/d9/d19/c44 0 2024-04-02T04:47:43.475 INFO:tasks.workunit.client.1.smithi179.stdout:7/297: creat d1/dd/d16/d4d/f6c x:0 0 0 2024-04-02T04:47:43.477 INFO:tasks.workunit.client.1.smithi179.stdout:6/231: creat d3/d42/f54 x:0 0 0 2024-04-02T04:47:43.478 INFO:tasks.workunit.client.0.smithi070.stdout:9/217: sync 2024-04-02T04:47:43.478 INFO:tasks.workunit.client.0.smithi070.stdout:5/245: sync 2024-04-02T04:47:43.478 INFO:tasks.workunit.client.0.smithi070.stdout:0/347: sync 2024-04-02T04:47:43.481 INFO:tasks.workunit.client.0.smithi070.stdout:4/200: dwrite d1/f32 [0,4194304] 0 2024-04-02T04:47:43.481 INFO:tasks.workunit.client.0.smithi070.stdout:5/246: write d5/d9/d3f/f41 [66282,83781] 0 2024-04-02T04:47:43.483 INFO:tasks.workunit.client.0.smithi070.stdout:9/218: write d4/d13/d16/f1d [894968,96264] 0 2024-04-02T04:47:43.490 INFO:tasks.workunit.client.0.smithi070.stdout:6/166: symlink d0/l35 0 2024-04-02T04:47:43.491 INFO:tasks.workunit.client.1.smithi179.stdout:0/288: dread d1/f3b [0,4194304] 0 2024-04-02T04:47:43.492 INFO:tasks.workunit.client.0.smithi070.stdout:6/167: dread d0/d3/f13 [0,4194304] 0 2024-04-02T04:47:43.494 INFO:tasks.workunit.client.0.smithi070.stdout:7/225: write d7/d9/f27 [1254142,31044] 0 2024-04-02T04:47:43.495 INFO:tasks.workunit.client.0.smithi070.stdout:7/226: chown d7/d2f/c3d 2067 1 2024-04-02T04:47:43.496 INFO:tasks.workunit.client.0.smithi070.stdout:7/227: chown d7/dd/f38 27 1 2024-04-02T04:47:43.500 INFO:tasks.workunit.client.0.smithi070.stdout:6/168: dwrite d0/d20/f23 [0,4194304] 0 2024-04-02T04:47:43.505 INFO:tasks.workunit.client.0.smithi070.stdout:5/247: creat d5/d9/d23/f49 x:0 0 0 2024-04-02T04:47:43.510 INFO:tasks.workunit.client.0.smithi070.stdout:1/195: getdents d0 0 2024-04-02T04:47:43.510 INFO:tasks.workunit.client.1.smithi179.stdout:6/232: chown d3/d12/d23/d2f/f3c 200 1 2024-04-02T04:47:43.511 INFO:tasks.workunit.client.0.smithi070.stdout:1/196: write d0/d9/d1d/f16 [1391068,80239] 0 2024-04-02T04:47:43.512 INFO:tasks.workunit.client.0.smithi070.stdout:8/283: mknod d2/d8/d29/d22/c5d 0 2024-04-02T04:47:43.514 INFO:tasks.workunit.client.1.smithi179.stdout:2/139: mkdir dc/d29 0 2024-04-02T04:47:43.514 INFO:tasks.workunit.client.0.smithi070.stdout:7/228: unlink d7/dd/df/f10 0 2024-04-02T04:47:43.514 INFO:tasks.workunit.client.0.smithi070.stdout:2/326: getdents d2/d10/d21/d2b 0 2024-04-02T04:47:43.517 INFO:tasks.workunit.client.1.smithi179.stdout:5/167: truncate d8/f1a 3155563 0 2024-04-02T04:47:43.519 INFO:tasks.workunit.client.1.smithi179.stdout:6/233: dwrite d3/d12/d23/f2b [0,4194304] 0 2024-04-02T04:47:43.519 INFO:tasks.workunit.client.1.smithi179.stdout:3/192: write d8/d26/f2c [2041324,61768] 0 2024-04-02T04:47:43.520 INFO:tasks.workunit.client.0.smithi070.stdout:5/248: mknod d5/d9/d23/c4a 0 2024-04-02T04:47:43.521 INFO:tasks.workunit.client.0.smithi070.stdout:4/201: mknod d1/d3/c3a 0 2024-04-02T04:47:43.526 INFO:tasks.workunit.client.1.smithi179.stdout:0/289: dread d1/f2a [0,4194304] 0 2024-04-02T04:47:43.527 INFO:tasks.workunit.client.1.smithi179.stdout:7/298: creat d1/d9/d35/d6b/f6d x:0 0 0 2024-04-02T04:47:43.528 INFO:tasks.workunit.client.1.smithi179.stdout:7/299: stat d1/dd/d23/f24 0 2024-04-02T04:47:43.529 INFO:tasks.workunit.client.1.smithi179.stdout:4/316: write d1/d3/f7 [3562001,7990] 0 2024-04-02T04:47:43.530 INFO:tasks.workunit.client.0.smithi070.stdout:1/197: write d0/d9/f3e [2667013,23750] 0 2024-04-02T04:47:43.530 INFO:tasks.workunit.client.1.smithi179.stdout:5/168: mkdir d8/d1b/d23/d36 0 2024-04-02T04:47:43.532 INFO:tasks.workunit.client.0.smithi070.stdout:2/327: creat d2/d10/d21/d2b/f67 x:0 0 0 2024-04-02T04:47:43.533 INFO:tasks.workunit.client.1.smithi179.stdout:6/234: chown d3/d12/d23/c3e 1463433 1 2024-04-02T04:47:43.534 INFO:tasks.workunit.client.1.smithi179.stdout:6/235: fsync d3/d42/f54 0 2024-04-02T04:47:43.535 INFO:tasks.workunit.client.1.smithi179.stdout:3/193: creat d8/d26/f3f x:0 0 0 2024-04-02T04:47:43.536 INFO:tasks.workunit.client.1.smithi179.stdout:3/194: truncate d8/f2d 378146 0 2024-04-02T04:47:43.536 INFO:tasks.workunit.client.0.smithi070.stdout:6/169: creat d0/d3/d19/d31/f36 x:0 0 0 2024-04-02T04:47:43.538 INFO:tasks.workunit.client.1.smithi179.stdout:6/236: dread d3/d12/d23/f2b [0,4194304] 0 2024-04-02T04:47:43.538 INFO:tasks.workunit.client.0.smithi070.stdout:9/219: write d4/fc [1697867,71958] 0 2024-04-02T04:47:43.539 INFO:tasks.workunit.client.1.smithi179.stdout:6/237: write d3/d2d/d43/f4b [141143,95156] 0 2024-04-02T04:47:43.539 INFO:tasks.workunit.client.1.smithi179.stdout:1/257: fdatasync d3/d5/d2f/d36/f44 0 2024-04-02T04:47:43.539 INFO:tasks.workunit.client.0.smithi070.stdout:0/348: dwrite d4/d5/f14 [0,4194304] 0 2024-04-02T04:47:43.541 INFO:tasks.workunit.client.0.smithi070.stdout:5/249: truncate d5/d17/d27/d2c/d35/f38 1164172 0 2024-04-02T04:47:43.542 INFO:tasks.workunit.client.1.smithi179.stdout:8/217: link d3/d32/l1a d3/d32/d21/d22/l4f 0 2024-04-02T04:47:43.544 INFO:tasks.workunit.client.1.smithi179.stdout:0/290: truncate d1/da/d28/d37/f3f 8978875 0 2024-04-02T04:47:43.544 INFO:tasks.workunit.client.0.smithi070.stdout:6/170: dread d0/d20/f27 [0,4194304] 0 2024-04-02T04:47:43.545 INFO:tasks.workunit.client.0.smithi070.stdout:1/198: mknod d0/d9/d1d/d22/d2c/c41 0 2024-04-02T04:47:43.546 INFO:tasks.workunit.client.0.smithi070.stdout:1/199: readlink d0/d9/lf 0 2024-04-02T04:47:43.546 INFO:tasks.workunit.client.0.smithi070.stdout:1/200: fdatasync d0/d9/fc 0 2024-04-02T04:47:43.547 INFO:tasks.workunit.client.1.smithi179.stdout:4/317: write d1/f42 [1605109,120756] 0 2024-04-02T04:47:43.547 INFO:tasks.workunit.client.0.smithi070.stdout:7/229: mknod d7/d9/c45 0 2024-04-02T04:47:43.548 INFO:tasks.workunit.client.0.smithi070.stdout:1/201: chown d0/d7/c15 1136 1 2024-04-02T04:47:43.548 INFO:tasks.workunit.client.1.smithi179.stdout:5/169: creat d8/d1b/f37 x:0 0 0 2024-04-02T04:47:43.550 INFO:tasks.workunit.client.1.smithi179.stdout:5/170: chown d8/d1b/c1c 232307343 1 2024-04-02T04:47:43.550 INFO:tasks.workunit.client.0.smithi070.stdout:4/202: mknod d1/d3/c3b 0 2024-04-02T04:47:43.552 INFO:tasks.workunit.client.1.smithi179.stdout:9/267: getdents d0/d1/d11/d29/d40 0 2024-04-02T04:47:43.552 INFO:tasks.workunit.client.0.smithi070.stdout:9/220: creat d4/d10/d12/d27/d32/f45 x:0 0 0 2024-04-02T04:47:43.553 INFO:tasks.workunit.client.1.smithi179.stdout:7/300: dwrite d1/d9/f68 [0,4194304] 0 2024-04-02T04:47:43.553 INFO:tasks.workunit.client.0.smithi070.stdout:4/203: write d1/d3/f30 [2416879,45538] 0 2024-04-02T04:47:43.555 INFO:tasks.workunit.client.1.smithi179.stdout:9/268: write d0/d1/f36 [1699981,87504] 0 2024-04-02T04:47:43.556 INFO:tasks.workunit.client.0.smithi070.stdout:1/202: dwrite d0/d7/f8 [4194304,4194304] 0 2024-04-02T04:47:43.558 INFO:tasks.workunit.client.0.smithi070.stdout:8/284: creat d2/d8/f5e x:0 0 0 2024-04-02T04:47:43.560 INFO:tasks.workunit.client.0.smithi070.stdout:8/285: readlink d2/d8/d23/d24/l45 0 2024-04-02T04:47:43.560 INFO:tasks.workunit.client.1.smithi179.stdout:1/258: rmdir d3/d5/de 39 2024-04-02T04:47:43.560 INFO:tasks.workunit.client.0.smithi070.stdout:8/286: chown d2/d8/d23 50528083 1 2024-04-02T04:47:43.561 INFO:tasks.workunit.client.1.smithi179.stdout:4/318: dwrite d1/d15/f56 [0,4194304] 0 2024-04-02T04:47:43.562 INFO:tasks.workunit.client.0.smithi070.stdout:6/171: unlink d0/d3/fb 0 2024-04-02T04:47:43.563 INFO:tasks.workunit.client.1.smithi179.stdout:1/259: read d3/d5/d2f/f47 [2603379,55531] 0 2024-04-02T04:47:43.566 INFO:tasks.workunit.client.1.smithi179.stdout:9/269: dwrite d0/d1/d11/f32 [0,4194304] 0 2024-04-02T04:47:43.568 INFO:tasks.workunit.client.0.smithi070.stdout:7/230: mkdir d7/d2f/d46 0 2024-04-02T04:47:43.568 INFO:tasks.workunit.client.0.smithi070.stdout:1/203: dwrite d0/d9/d1d/f16 [0,4194304] 0 2024-04-02T04:47:43.575 INFO:tasks.workunit.client.0.smithi070.stdout:8/287: dwrite d2/d8/d23/d24/f5b [4194304,4194304] 0 2024-04-02T04:47:43.577 INFO:tasks.workunit.client.1.smithi179.stdout:5/171: creat d8/d1b/d23/d32/f38 x:0 0 0 2024-04-02T04:47:43.577 INFO:tasks.workunit.client.0.smithi070.stdout:6/172: dwrite d0/d20/f23 [0,4194304] 0 2024-04-02T04:47:43.578 INFO:tasks.workunit.client.1.smithi179.stdout:5/172: dread - d8/d1b/d23/f2f zero size 2024-04-02T04:47:43.579 INFO:tasks.workunit.client.1.smithi179.stdout:5/173: truncate d8/d1b/f24 577726 0 2024-04-02T04:47:43.580 INFO:tasks.workunit.client.1.smithi179.stdout:5/174: chown l3 10183 1 2024-04-02T04:47:43.581 INFO:tasks.workunit.client.0.smithi070.stdout:9/221: symlink d4/d13/l46 0 2024-04-02T04:47:43.583 INFO:tasks.workunit.client.1.smithi179.stdout:3/195: rename d8/d26/d36/c3e to d8/d26/c40 0 2024-04-02T04:47:43.590 INFO:tasks.workunit.client.0.smithi070.stdout:5/250: dwrite d5/ff [4194304,4194304] 0 2024-04-02T04:47:43.592 INFO:tasks.workunit.client.1.smithi179.stdout:3/196: dwrite d8/f32 [0,4194304] 0 2024-04-02T04:47:43.593 INFO:tasks.workunit.client.1.smithi179.stdout:6/238: truncate d3/d12/f22 236799 0 2024-04-02T04:47:43.597 INFO:tasks.workunit.client.0.smithi070.stdout:5/251: dwrite d5/f6 [0,4194304] 0 2024-04-02T04:47:43.599 INFO:tasks.workunit.client.0.smithi070.stdout:7/231: symlink d7/dd/df/d34/l47 0 2024-04-02T04:47:43.602 INFO:tasks.workunit.client.1.smithi179.stdout:7/301: write d1/dd/d16/f48 [2546057,25604] 0 2024-04-02T04:47:43.608 INFO:tasks.workunit.client.1.smithi179.stdout:4/319: dwrite d1/d3/d19/f2c [4194304,4194304] 0 2024-04-02T04:47:43.609 INFO:tasks.workunit.client.0.smithi070.stdout:1/204: creat d0/d9/d1d/f42 x:0 0 0 2024-04-02T04:47:43.610 INFO:tasks.workunit.client.1.smithi179.stdout:8/218: mkdir d3/d32/d21/d50 0 2024-04-02T04:47:43.611 INFO:tasks.workunit.client.0.smithi070.stdout:8/288: rename d2/d8/d23/d36 to d2/d8/d23/d24/d5f 0 2024-04-02T04:47:43.611 INFO:tasks.workunit.client.0.smithi070.stdout:5/252: dwrite d5/d17/d27/d2e/f3b [0,4194304] 0 2024-04-02T04:47:43.613 INFO:tasks.workunit.client.1.smithi179.stdout:9/270: creat d0/d1/d11/d29/f50 x:0 0 0 2024-04-02T04:47:43.615 INFO:tasks.workunit.client.1.smithi179.stdout:5/175: rename d8/c2a to d8/d1b/d23/c39 0 2024-04-02T04:47:43.616 INFO:tasks.workunit.client.0.smithi070.stdout:9/222: symlink d4/d13/d16/d19/l47 0 2024-04-02T04:47:43.617 INFO:tasks.workunit.client.0.smithi070.stdout:9/223: write d4/d10/d12/d27/d32/f45 [431212,63724] 0 2024-04-02T04:47:43.620 INFO:tasks.workunit.client.1.smithi179.stdout:3/197: creat d8/d26/d36/f41 x:0 0 0 2024-04-02T04:47:43.621 INFO:tasks.workunit.client.0.smithi070.stdout:7/232: creat d7/d2f/f48 x:0 0 0 2024-04-02T04:47:43.622 INFO:tasks.workunit.client.0.smithi070.stdout:7/233: chown d7/dd/df/l3f 111243 1 2024-04-02T04:47:43.623 INFO:tasks.workunit.client.1.smithi179.stdout:5/176: dwrite d8/d1b/d23/d32/f33 [0,4194304] 0 2024-04-02T04:47:43.623 INFO:tasks.workunit.client.1.smithi179.stdout:5/177: stat f6 0 2024-04-02T04:47:43.625 INFO:tasks.workunit.client.1.smithi179.stdout:1/260: creat d3/d5/de/f57 x:0 0 0 2024-04-02T04:47:43.627 INFO:tasks.workunit.client.1.smithi179.stdout:4/320: symlink d1/l66 0 2024-04-02T04:47:43.628 INFO:tasks.workunit.client.0.smithi070.stdout:5/253: creat d5/d17/d27/f4b x:0 0 0 2024-04-02T04:47:43.628 INFO:tasks.workunit.client.1.smithi179.stdout:0/291: creat d1/f57 x:0 0 0 2024-04-02T04:47:43.629 INFO:tasks.workunit.client.0.smithi070.stdout:0/349: rmdir d4/d5/d13/d23/d55 0 2024-04-02T04:47:43.629 INFO:tasks.workunit.client.1.smithi179.stdout:0/292: chown d1/db/f2f 57753185 1 2024-04-02T04:47:43.634 INFO:tasks.workunit.client.1.smithi179.stdout:7/302: rename d1/d9/d35/d6b/f6d to d1/d9/f6e 0 2024-04-02T04:47:43.636 INFO:tasks.workunit.client.0.smithi070.stdout:5/254: dwrite d5/d17/d27/d2c/f37 [0,4194304] 0 2024-04-02T04:47:43.637 INFO:tasks.workunit.client.1.smithi179.stdout:9/271: dwrite d0/d1/d12/f20 [4194304,4194304] 0 2024-04-02T04:47:43.638 INFO:tasks.workunit.client.1.smithi179.stdout:3/198: symlink d8/d26/d36/l42 0 2024-04-02T04:47:43.639 INFO:tasks.workunit.client.0.smithi070.stdout:0/350: creat d4/d5/d13/d2a/d29/f5d x:0 0 0 2024-04-02T04:47:43.641 INFO:tasks.workunit.client.0.smithi070.stdout:0/351: truncate d4/d5/d13/d2a/d29/f5d 377142 0 2024-04-02T04:47:43.642 INFO:tasks.workunit.client.1.smithi179.stdout:4/321: mknod d1/d3/df/d10/d3a/c67 0 2024-04-02T04:47:43.642 INFO:tasks.workunit.client.0.smithi070.stdout:0/352: stat d4/d5/d13/d23/l34 0 2024-04-02T04:47:43.642 INFO:tasks.workunit.client.0.smithi070.stdout:7/234: mknod d7/d9/c49 0 2024-04-02T04:47:43.646 INFO:tasks.workunit.client.1.smithi179.stdout:9/272: dwrite d0/d1/d12/f25 [0,4194304] 0 2024-04-02T04:47:43.646 INFO:tasks.workunit.client.0.smithi070.stdout:1/205: creat d0/d7/f43 x:0 0 0 2024-04-02T04:47:43.647 INFO:tasks.workunit.client.0.smithi070.stdout:8/289: creat d2/d8/d29/d22/f60 x:0 0 0 2024-04-02T04:47:43.648 INFO:tasks.workunit.client.0.smithi070.stdout:8/290: chown d2/d8/d29 812868229 1 2024-04-02T04:47:43.650 INFO:tasks.workunit.client.0.smithi070.stdout:5/255: rename d5/d9/l1c to d5/d17/d2b/l4c 0 2024-04-02T04:47:43.654 INFO:tasks.workunit.client.1.smithi179.stdout:1/261: rename d3/d5/d2f/c3b to d3/c58 0 2024-04-02T04:47:43.656 INFO:tasks.workunit.client.0.smithi070.stdout:5/256: dread d5/d9/d23/f48 [0,4194304] 0 2024-04-02T04:47:43.657 INFO:tasks.workunit.client.1.smithi179.stdout:1/262: write d3/d5/d2f/d4f/f37 [2773848,108025] 0 2024-04-02T04:47:43.657 INFO:tasks.workunit.client.1.smithi179.stdout:9/273: dwrite d0/d1/d11/d29/d40/f49 [0,4194304] 0 2024-04-02T04:47:43.657 INFO:tasks.workunit.client.0.smithi070.stdout:7/235: creat d7/dd/df/d34/f4a x:0 0 0 2024-04-02T04:47:43.658 INFO:tasks.workunit.client.0.smithi070.stdout:5/257: write d5/d17/d27/f28 [3422926,24723] 0 2024-04-02T04:47:43.658 INFO:tasks.workunit.client.0.smithi070.stdout:7/236: stat d7/dd/f38 0 2024-04-02T04:47:43.658 INFO:tasks.workunit.client.1.smithi179.stdout:9/274: fsync d0/d19/f4e 0 2024-04-02T04:47:43.659 INFO:tasks.workunit.client.0.smithi070.stdout:5/258: chown d5/d17/d27/d2c/d35 99930 1 2024-04-02T04:47:43.659 INFO:tasks.workunit.client.0.smithi070.stdout:5/259: chown d5 7270222 1 2024-04-02T04:47:43.659 INFO:tasks.workunit.client.1.smithi179.stdout:6/239: creat d3/d12/d23/f55 x:0 0 0 2024-04-02T04:47:43.661 INFO:tasks.workunit.client.0.smithi070.stdout:1/206: symlink d0/l44 0 2024-04-02T04:47:43.664 INFO:tasks.workunit.client.0.smithi070.stdout:8/291: unlink d2/d3/d35/d3a/d4f/c58 0 2024-04-02T04:47:43.665 INFO:tasks.workunit.client.0.smithi070.stdout:7/237: dwrite d7/d2f/f2d [0,4194304] 0 2024-04-02T04:47:43.665 INFO:tasks.workunit.client.0.smithi070.stdout:7/238: readlink d7/l2c 0 2024-04-02T04:47:43.667 INFO:tasks.workunit.client.0.smithi070.stdout:6/173: write d0/f15 [510179,2750] 0 2024-04-02T04:47:43.669 INFO:tasks.workunit.client.1.smithi179.stdout:3/199: mknod d8/d26/d36/c43 0 2024-04-02T04:47:43.670 INFO:tasks.workunit.client.1.smithi179.stdout:3/200: write d8/f22 [5968631,5223] 0 2024-04-02T04:47:43.673 INFO:tasks.workunit.client.1.smithi179.stdout:0/293: mknod d1/da/d52/c58 0 2024-04-02T04:47:43.675 INFO:tasks.workunit.client.1.smithi179.stdout:3/201: dread d8/f1c [0,4194304] 0 2024-04-02T04:47:43.676 INFO:tasks.workunit.client.1.smithi179.stdout:4/322: symlink d1/d3/df/d10/l68 0 2024-04-02T04:47:43.676 INFO:tasks.workunit.client.0.smithi070.stdout:9/224: write d4/d13/d16/d19/f2e [262064,113815] 0 2024-04-02T04:47:43.678 INFO:tasks.workunit.client.0.smithi070.stdout:5/260: symlink d5/d9/d3f/l4d 0 2024-04-02T04:47:43.678 INFO:tasks.workunit.client.0.smithi070.stdout:9/225: fsync d4/d10/d12/d27/f30 0 2024-04-02T04:47:43.678 INFO:tasks.workunit.client.0.smithi070.stdout:9/226: readlink d4/d13/l46 0 2024-04-02T04:47:43.679 INFO:tasks.workunit.client.0.smithi070.stdout:9/227: write d4/d10/f2c [51389,3191] 0 2024-04-02T04:47:43.684 INFO:tasks.workunit.client.1.smithi179.stdout:5/178: write d8/f27 [607536,95177] 0 2024-04-02T04:47:43.684 INFO:tasks.workunit.client.1.smithi179.stdout:5/179: readlink d8/l11 0 2024-04-02T04:47:43.686 INFO:tasks.workunit.client.1.smithi179.stdout:2/140: sync 2024-04-02T04:47:43.687 INFO:tasks.workunit.client.1.smithi179.stdout:2/141: chown dc/fe 7341 1 2024-04-02T04:47:43.688 INFO:tasks.workunit.client.1.smithi179.stdout:1/263: symlink d3/d5/de/l59 0 2024-04-02T04:47:43.688 INFO:tasks.workunit.client.1.smithi179.stdout:1/264: read - d3/f1f zero size 2024-04-02T04:47:43.689 INFO:tasks.workunit.client.1.smithi179.stdout:9/275: creat d0/d1/d11/f51 x:0 0 0 2024-04-02T04:47:43.690 INFO:tasks.workunit.client.0.smithi070.stdout:5/261: rename d5/d17/d27/l29 to d5/d17/d27/d2e/l4e 0 2024-04-02T04:47:43.691 INFO:tasks.workunit.client.1.smithi179.stdout:8/219: truncate d3/d1b/d2a/d41/f42 3565598 0 2024-04-02T04:47:43.694 INFO:tasks.workunit.client.0.smithi070.stdout:8/292: truncate d2/d8/d29/f3b 231102 0 2024-04-02T04:47:43.695 INFO:tasks.workunit.client.1.smithi179.stdout:9/276: dwrite d0/d1/d12/fe [0,4194304] 0 2024-04-02T04:47:43.697 INFO:tasks.workunit.client.1.smithi179.stdout:9/277: fsync d0/d1/d11/f32 0 2024-04-02T04:47:43.697 INFO:tasks.workunit.client.1.smithi179.stdout:0/294: chown d1/f36 0 1 2024-04-02T04:47:43.698 INFO:tasks.workunit.client.1.smithi179.stdout:6/240: dwrite d3/d12/f25 [4194304,4194304] 0 2024-04-02T04:47:43.699 INFO:tasks.workunit.client.1.smithi179.stdout:0/295: dread d1/f3b [0,4194304] 0 2024-04-02T04:47:43.700 INFO:tasks.workunit.client.1.smithi179.stdout:3/202: unlink d8/f2d 0 2024-04-02T04:47:43.701 INFO:tasks.workunit.client.1.smithi179.stdout:3/203: chown d8/f1b 1770803 1 2024-04-02T04:47:43.703 INFO:tasks.workunit.client.0.smithi070.stdout:0/353: rename d4/d5/f15 to d4/d5/f5e 0 2024-04-02T04:47:43.705 INFO:tasks.workunit.client.1.smithi179.stdout:4/323: creat d1/d3/dc/d27/d62/f69 x:0 0 0 2024-04-02T04:47:43.707 INFO:tasks.workunit.client.1.smithi179.stdout:0/296: dwrite d1/f2 [4194304,4194304] 0 2024-04-02T04:47:43.714 INFO:tasks.workunit.client.0.smithi070.stdout:8/293: rename d2/d8/d23/f31 to d2/d3/f61 0 2024-04-02T04:47:43.716 INFO:tasks.workunit.client.1.smithi179.stdout:7/303: truncate d1/d9/f11 953556 0 2024-04-02T04:47:43.716 INFO:tasks.workunit.client.1.smithi179.stdout:7/304: readlink d1/l2 0 2024-04-02T04:47:43.717 INFO:tasks.workunit.client.1.smithi179.stdout:7/305: chown d1/dd/f20 4474 1 2024-04-02T04:47:43.717 INFO:tasks.workunit.client.1.smithi179.stdout:7/306: readlink l0 0 2024-04-02T04:47:43.717 INFO:tasks.workunit.client.0.smithi070.stdout:6/174: mkdir d0/d3/d19/d31/d37 0 2024-04-02T04:47:43.723 INFO:tasks.workunit.client.0.smithi070.stdout:5/262: link d5/d17/d2b/f3a d5/d17/d27/d2c/d35/f4f 0 2024-04-02T04:47:43.724 INFO:tasks.workunit.client.0.smithi070.stdout:7/239: dwrite d7/dd/df/f32 [0,4194304] 0 2024-04-02T04:47:43.724 INFO:tasks.workunit.client.1.smithi179.stdout:9/278: creat d0/d19/d1c/f52 x:0 0 0 2024-04-02T04:47:43.725 INFO:tasks.workunit.client.0.smithi070.stdout:5/263: chown d5/d9/d23/d30 63 1 2024-04-02T04:47:43.725 INFO:tasks.workunit.client.0.smithi070.stdout:3/228: sync 2024-04-02T04:47:43.728 INFO:tasks.workunit.client.1.smithi179.stdout:6/241: unlink d3/c5 0 2024-04-02T04:47:43.729 INFO:tasks.workunit.client.0.smithi070.stdout:8/294: mkdir d2/d3/d52/d62 0 2024-04-02T04:47:43.731 INFO:tasks.workunit.client.0.smithi070.stdout:1/207: symlink d0/d9/d1d/l45 0 2024-04-02T04:47:43.733 INFO:tasks.workunit.client.1.smithi179.stdout:0/297: symlink d1/da/df/l59 0 2024-04-02T04:47:43.734 INFO:tasks.workunit.client.1.smithi179.stdout:0/298: chown d1/f22 135 1 2024-04-02T04:47:43.736 INFO:tasks.workunit.client.1.smithi179.stdout:0/299: read d1/db/f2f [4504784,34995] 0 2024-04-02T04:47:43.736 INFO:tasks.workunit.client.0.smithi070.stdout:8/295: dwrite d2/d8/d29/f15 [4194304,4194304] 0 2024-04-02T04:47:43.736 INFO:tasks.workunit.client.0.smithi070.stdout:0/354: rename d4/d5/d13/d23/l2e to d4/d5/d21/l5f 0 2024-04-02T04:47:43.738 INFO:tasks.workunit.client.0.smithi070.stdout:7/240: dread d7/fc [0,4194304] 0 2024-04-02T04:47:43.739 INFO:tasks.workunit.client.1.smithi179.stdout:1/265: mkdir d3/d5/d2f/d5a 0 2024-04-02T04:47:43.740 INFO:tasks.workunit.client.1.smithi179.stdout:1/266: chown d3/d5/d2f/d4f/d42/d55 20578 1 2024-04-02T04:47:43.741 INFO:tasks.workunit.client.0.smithi070.stdout:9/228: getdents d4/d13/d16/d19 0 2024-04-02T04:47:43.742 INFO:tasks.workunit.client.1.smithi179.stdout:8/220: mkdir d3/d32/d21/d51 0 2024-04-02T04:47:43.744 INFO:tasks.workunit.client.1.smithi179.stdout:9/279: creat d0/d1/d11/d29/d40/f53 x:0 0 0 2024-04-02T04:47:43.746 INFO:tasks.workunit.client.1.smithi179.stdout:6/242: readlink d3/d12/d44/l51 0 2024-04-02T04:47:43.746 INFO:tasks.workunit.client.1.smithi179.stdout:8/221: read d3/f18 [3545795,102775] 0 2024-04-02T04:47:43.748 INFO:tasks.workunit.client.0.smithi070.stdout:8/296: truncate d2/d3/fd 5174252 0 2024-04-02T04:47:43.751 INFO:tasks.workunit.client.0.smithi070.stdout:3/229: symlink d10/d32/d4b/l50 0 2024-04-02T04:47:43.752 INFO:tasks.workunit.client.0.smithi070.stdout:1/208: dwrite d0/f2a [0,4194304] 0 2024-04-02T04:47:43.754 INFO:tasks.workunit.client.1.smithi179.stdout:2/142: rename dc/c10 to dc/c2a 0 2024-04-02T04:47:43.755 INFO:tasks.workunit.client.1.smithi179.stdout:2/143: write dc/f12 [6958696,38574] 0 2024-04-02T04:47:43.757 INFO:tasks.workunit.client.0.smithi070.stdout:1/209: dread - d0/d9/d1d/d22/f25 zero size 2024-04-02T04:47:43.757 INFO:tasks.workunit.client.0.smithi070.stdout:8/297: dread d2/d3/d52/f3c [0,4194304] 0 2024-04-02T04:47:43.757 INFO:tasks.workunit.client.0.smithi070.stdout:5/264: mkdir d5/d17/d50 0 2024-04-02T04:47:43.757 INFO:tasks.workunit.client.0.smithi070.stdout:3/230: truncate d10/d32/d3f/f3b 93623 0 2024-04-02T04:47:43.760 INFO:tasks.workunit.client.1.smithi179.stdout:7/307: creat d1/dd/d23/d5e/f6f x:0 0 0 2024-04-02T04:47:43.761 INFO:tasks.workunit.client.0.smithi070.stdout:7/241: read d7/d9/f1b [2229805,130331] 0 2024-04-02T04:47:43.762 INFO:tasks.workunit.client.1.smithi179.stdout:0/300: dread d1/da/d28/d34/f35 [0,4194304] 0 2024-04-02T04:47:43.763 INFO:tasks.workunit.client.0.smithi070.stdout:9/229: dread d4/d10/f18 [4194304,4194304] 0 2024-04-02T04:47:43.763 INFO:tasks.workunit.client.1.smithi179.stdout:0/301: dread - d1/da/f4a zero size 2024-04-02T04:47:43.765 INFO:tasks.workunit.client.0.smithi070.stdout:9/230: chown d4/d10/d12/c38 2467 1 2024-04-02T04:47:43.767 INFO:tasks.workunit.client.0.smithi070.stdout:1/210: dwrite d0/d9/d1d/d22/f2d [0,4194304] 0 2024-04-02T04:47:43.769 INFO:tasks.workunit.client.0.smithi070.stdout:5/265: dwrite d5/d9/d23/f42 [0,4194304] 0 2024-04-02T04:47:43.769 INFO:tasks.workunit.client.1.smithi179.stdout:6/243: creat d3/d12/d23/d2f/f56 x:0 0 0 2024-04-02T04:47:43.771 INFO:tasks.workunit.client.1.smithi179.stdout:0/302: dwrite d1/da/d28/d37/f49 [0,4194304] 0 2024-04-02T04:47:43.772 INFO:tasks.workunit.client.0.smithi070.stdout:5/266: truncate d5/d9/d3f/f41 163552 0 2024-04-02T04:47:43.773 INFO:tasks.workunit.client.1.smithi179.stdout:5/180: dwrite d8/f13 [0,4194304] 0 2024-04-02T04:47:43.773 INFO:tasks.workunit.client.0.smithi070.stdout:5/267: write d5/d17/f32 [4456716,16891] 0 2024-04-02T04:47:43.775 INFO:tasks.workunit.client.1.smithi179.stdout:5/181: stat d8/f1d 0 2024-04-02T04:47:43.778 INFO:tasks.workunit.client.1.smithi179.stdout:6/244: write d3/d12/d23/d26/f27 [3043426,28301] 0 2024-04-02T04:47:43.782 INFO:tasks.workunit.client.1.smithi179.stdout:5/182: dread d8/d1b/f24 [0,4194304] 0 2024-04-02T04:47:43.783 INFO:tasks.workunit.client.1.smithi179.stdout:5/183: chown d8/d1b/f24 814784 1 2024-04-02T04:47:43.784 INFO:tasks.workunit.client.1.smithi179.stdout:5/184: chown d8/l11 513177 1 2024-04-02T04:47:43.784 INFO:tasks.workunit.client.0.smithi070.stdout:0/355: mkdir d4/d5/d58/d60 0 2024-04-02T04:47:43.785 INFO:tasks.workunit.client.1.smithi179.stdout:8/222: dwrite d3/d32/d21/f25 [0,4194304] 0 2024-04-02T04:47:43.787 INFO:tasks.workunit.client.1.smithi179.stdout:1/267: mkdir d3/d5/d2f/d4f/d42/d53/d5b 0 2024-04-02T04:47:43.788 INFO:tasks.workunit.client.0.smithi070.stdout:7/242: truncate d7/d9/f1b 2668718 0 2024-04-02T04:47:43.791 INFO:tasks.workunit.client.0.smithi070.stdout:9/231: creat d4/d13/d16/d19/f48 x:0 0 0 2024-04-02T04:47:43.797 INFO:tasks.workunit.client.0.smithi070.stdout:1/211: dread d0/d7/d1a/f1b [0,4194304] 0 2024-04-02T04:47:43.798 INFO:tasks.workunit.client.0.smithi070.stdout:1/212: write d0/d9/d1d/f3d [3449209,71577] 0 2024-04-02T04:47:43.801 INFO:tasks.workunit.client.1.smithi179.stdout:0/303: rename d1/da/df/f4f to d1/da/d28/d37/f5a 0 2024-04-02T04:47:43.804 INFO:tasks.workunit.client.0.smithi070.stdout:5/268: rename d5/d17/d2b/d3e to d5/d9/d10/d51 0 2024-04-02T04:47:43.806 INFO:tasks.workunit.client.0.smithi070.stdout:6/175: dwrite d0/d3/f13 [0,4194304] 0 2024-04-02T04:47:43.806 INFO:tasks.workunit.client.0.smithi070.stdout:8/298: mknod d2/d8/d29/d22/d5a/c63 0 2024-04-02T04:47:43.808 INFO:tasks.workunit.client.1.smithi179.stdout:6/245: mkdir d3/d12/d23/d26/d57 0 2024-04-02T04:47:43.810 INFO:tasks.workunit.client.0.smithi070.stdout:0/356: mknod d4/d5/d13/d17/c61 0 2024-04-02T04:47:43.810 INFO:tasks.workunit.client.0.smithi070.stdout:0/357: readlink d4/d3c/l47 0 2024-04-02T04:47:43.811 INFO:tasks.workunit.client.0.smithi070.stdout:7/243: mknod d7/dd/d35/c4b 0 2024-04-02T04:47:43.813 INFO:tasks.workunit.client.1.smithi179.stdout:5/185: symlink d8/d1b/d23/d30/l3a 0 2024-04-02T04:47:43.816 INFO:tasks.workunit.client.1.smithi179.stdout:8/223: creat d3/d32/d21/d49/f52 x:0 0 0 2024-04-02T04:47:43.817 INFO:tasks.workunit.client.1.smithi179.stdout:8/224: fdatasync d3/d32/d21/d22/f2d 0 2024-04-02T04:47:43.818 INFO:tasks.workunit.client.0.smithi070.stdout:7/244: dwrite d7/d2f/f3b [0,4194304] 0 2024-04-02T04:47:43.822 INFO:tasks.workunit.client.0.smithi070.stdout:3/231: write d10/d17/f2b [1278685,65056] 0 2024-04-02T04:47:43.825 INFO:tasks.workunit.client.1.smithi179.stdout:2/144: creat dc/d16/f2b x:0 0 0 2024-04-02T04:47:43.826 INFO:tasks.workunit.client.0.smithi070.stdout:0/358: rmdir d4/d3c 39 2024-04-02T04:47:43.827 INFO:tasks.workunit.client.0.smithi070.stdout:0/359: fdatasync d4/d5/d13/d2a/d29/f59 0 2024-04-02T04:47:43.832 INFO:tasks.workunit.client.1.smithi179.stdout:1/268: symlink d3/d5/d2f/d5a/l5c 0 2024-04-02T04:47:43.836 INFO:tasks.workunit.client.1.smithi179.stdout:0/304: mkdir d1/d5b 0 2024-04-02T04:47:43.836 INFO:tasks.workunit.client.0.smithi070.stdout:5/269: dread d5/f2f [0,4194304] 0 2024-04-02T04:47:43.836 INFO:tasks.workunit.client.0.smithi070.stdout:5/270: chown d5/d9/c1f 77 1 2024-04-02T04:47:43.837 INFO:tasks.workunit.client.1.smithi179.stdout:0/305: stat d1/da/f3c 0 2024-04-02T04:47:43.838 INFO:tasks.workunit.client.1.smithi179.stdout:9/280: dwrite d0/d1/d11/f42 [0,4194304] 0 2024-04-02T04:47:43.840 INFO:tasks.workunit.client.0.smithi070.stdout:8/299: mknod d2/c64 0 2024-04-02T04:47:43.843 INFO:tasks.workunit.client.0.smithi070.stdout:6/176: unlink d0/d3/c2a 0 2024-04-02T04:47:43.845 INFO:tasks.workunit.client.1.smithi179.stdout:0/306: dwrite d1/da/d28/f3a [0,4194304] 0 2024-04-02T04:47:43.845 INFO:tasks.workunit.client.0.smithi070.stdout:0/360: creat d4/d5/d13/d17/f62 x:0 0 0 2024-04-02T04:47:43.847 INFO:tasks.workunit.client.1.smithi179.stdout:5/186: rename d8/d1b/f25 to d8/d1b/f3b 0 2024-04-02T04:47:43.852 INFO:tasks.workunit.client.0.smithi070.stdout:3/232: mkdir d10/d17/d51 0 2024-04-02T04:47:43.853 INFO:tasks.workunit.client.0.smithi070.stdout:3/233: chown d10/d17/l2f 700 1 2024-04-02T04:47:43.854 INFO:tasks.workunit.client.0.smithi070.stdout:3/234: write d10/f29 [2413635,95784] 0 2024-04-02T04:47:43.859 INFO:tasks.workunit.client.1.smithi179.stdout:9/281: symlink d0/d1/d11/l54 0 2024-04-02T04:47:43.860 INFO:tasks.workunit.client.0.smithi070.stdout:1/213: dwrite d0/d7/f20 [0,4194304] 0 2024-04-02T04:47:43.861 INFO:tasks.workunit.client.0.smithi070.stdout:1/214: write d0/d9/d1d/f42 [841415,65300] 0 2024-04-02T04:47:43.864 INFO:tasks.workunit.client.0.smithi070.stdout:7/245: rename d7/d9/d19/l3e to d7/d9/l4c 0 2024-04-02T04:47:43.865 INFO:tasks.workunit.client.1.smithi179.stdout:9/282: dwrite d0/d1/f44 [0,4194304] 0 2024-04-02T04:47:43.865 INFO:tasks.workunit.client.1.smithi179.stdout:0/307: mkdir d1/da/d48/d5c 0 2024-04-02T04:47:43.867 INFO:tasks.workunit.client.1.smithi179.stdout:0/308: truncate d1/db/fc 2508254 0 2024-04-02T04:47:43.868 INFO:tasks.workunit.client.0.smithi070.stdout:7/246: dread d7/d2f/f2d [0,4194304] 0 2024-04-02T04:47:43.869 INFO:tasks.workunit.client.0.smithi070.stdout:7/247: write d7/dd/f38 [3939488,10369] 0 2024-04-02T04:47:43.869 INFO:tasks.workunit.client.0.smithi070.stdout:8/300: creat d2/d3/d35/d3a/d4f/d59/f65 x:0 0 0 2024-04-02T04:47:43.870 INFO:tasks.workunit.client.1.smithi179.stdout:6/246: truncate d3/ff 137779 0 2024-04-02T04:47:43.870 INFO:tasks.workunit.client.0.smithi070.stdout:7/248: write d7/dd/df/d34/f4a [245038,17563] 0 2024-04-02T04:47:43.873 INFO:tasks.workunit.client.0.smithi070.stdout:3/235: creat d10/d32/f52 x:0 0 0 2024-04-02T04:47:43.875 INFO:tasks.workunit.client.0.smithi070.stdout:7/249: dread d7/d2f/f2d [0,4194304] 0 2024-04-02T04:47:43.876 INFO:tasks.workunit.client.1.smithi179.stdout:6/247: dwrite d3/d12/d23/d26/f27 [0,4194304] 0 2024-04-02T04:47:43.876 INFO:tasks.workunit.client.0.smithi070.stdout:7/250: chown d7/dd/df/d34 87276 1 2024-04-02T04:47:43.876 INFO:tasks.workunit.client.0.smithi070.stdout:6/177: rmdir d0/d3/d19/d31 39 2024-04-02T04:47:43.877 INFO:tasks.workunit.client.0.smithi070.stdout:6/178: fdatasync d0/f15 0 2024-04-02T04:47:43.879 INFO:tasks.workunit.client.0.smithi070.stdout:7/251: dread d7/d9/f27 [0,4194304] 0 2024-04-02T04:47:43.881 INFO:tasks.workunit.client.1.smithi179.stdout:5/187: mknod d8/d1b/d23/d36/c3c 0 2024-04-02T04:47:43.882 INFO:tasks.workunit.client.1.smithi179.stdout:8/225: truncate d3/d32/d21/d22/d33/f34 2770 0 2024-04-02T04:47:43.888 INFO:tasks.workunit.client.0.smithi070.stdout:5/271: dwrite d5/f22 [0,4194304] 0 2024-04-02T04:47:43.890 INFO:tasks.workunit.client.0.smithi070.stdout:1/215: creat d0/d9/d1d/d22/f46 x:0 0 0 2024-04-02T04:47:43.891 INFO:tasks.workunit.client.0.smithi070.stdout:1/216: fdatasync d0/d9/d1d/f38 0 2024-04-02T04:47:43.897 INFO:tasks.workunit.client.0.smithi070.stdout:8/301: read d2/d3/d52/f43 [1993935,72554] 0 2024-04-02T04:47:43.901 INFO:tasks.workunit.client.0.smithi070.stdout:7/252: fdatasync d7/fc 0 2024-04-02T04:47:43.904 INFO:tasks.workunit.client.0.smithi070.stdout:5/272: rename d5/d9/d3f/l46 to d5/l52 0 2024-04-02T04:47:43.906 INFO:tasks.workunit.client.0.smithi070.stdout:1/217: rename d0/d9/d1d/l1f to d0/d9/d1d/l47 0 2024-04-02T04:47:43.907 INFO:tasks.workunit.client.0.smithi070.stdout:1/218: fsync d0/d9/d1d/f16 0 2024-04-02T04:47:43.907 INFO:tasks.workunit.client.0.smithi070.stdout:1/219: readlink d0/d7/l3b 0 2024-04-02T04:47:43.913 INFO:tasks.workunit.client.0.smithi070.stdout:7/253: creat d7/dd/df/f4d x:0 0 0 2024-04-02T04:47:43.914 INFO:tasks.workunit.client.1.smithi179.stdout:0/309: creat d1/da/d52/f5d x:0 0 0 2024-04-02T04:47:43.914 INFO:tasks.workunit.client.0.smithi070.stdout:0/361: creat d4/d5/da/f63 x:0 0 0 2024-04-02T04:47:43.915 INFO:tasks.workunit.client.1.smithi179.stdout:2/145: getdents dc 0 2024-04-02T04:47:43.915 INFO:tasks.workunit.client.1.smithi179.stdout:6/248: rmdir d3/d12/d23/d2f 39 2024-04-02T04:47:43.919 INFO:tasks.workunit.client.0.smithi070.stdout:0/362: dread d4/d5/d13/f1f [4194304,4194304] 0 2024-04-02T04:47:43.922 INFO:tasks.workunit.client.0.smithi070.stdout:8/302: dread d2/d8/d29/d22/d4d/f4e [0,4194304] 0 2024-04-02T04:47:43.924 INFO:tasks.workunit.client.0.smithi070.stdout:1/220: dread d0/d9/d1d/f13 [0,4194304] 0 2024-04-02T04:47:43.927 INFO:tasks.workunit.client.0.smithi070.stdout:3/236: truncate f1 1406026 0 2024-04-02T04:47:43.927 INFO:tasks.workunit.client.0.smithi070.stdout:3/237: stat d10/d17/f48 0 2024-04-02T04:47:43.927 INFO:tasks.workunit.client.0.smithi070.stdout:0/363: rmdir d4/d5/d13/d2a/d3a 39 2024-04-02T04:47:43.934 INFO:tasks.workunit.client.0.smithi070.stdout:8/303: mkdir d2/d8/d23/d24/d66 0 2024-04-02T04:47:43.936 INFO:tasks.workunit.client.0.smithi070.stdout:1/221: rmdir d0/d9 39 2024-04-02T04:47:43.936 INFO:tasks.workunit.client.0.smithi070.stdout:3/238: dwrite d10/d32/d3f/f3a [0,4194304] 0 2024-04-02T04:47:43.939 INFO:tasks.workunit.client.0.smithi070.stdout:5/273: dwrite d5/ff [0,4194304] 0 2024-04-02T04:47:43.939 INFO:tasks.workunit.client.0.smithi070.stdout:7/254: mkdir d7/d4e 0 2024-04-02T04:47:43.945 INFO:tasks.workunit.client.0.smithi070.stdout:8/304: creat d2/d8/d23/d24/f67 x:0 0 0 2024-04-02T04:47:43.950 INFO:tasks.workunit.client.0.smithi070.stdout:3/239: mknod d10/d17/c53 0 2024-04-02T04:47:43.954 INFO:tasks.workunit.client.0.smithi070.stdout:7/255: creat d7/d4e/f4f x:0 0 0 2024-04-02T04:47:43.954 INFO:tasks.workunit.client.0.smithi070.stdout:8/305: read d2/d3/d52/f16 [74930,16622] 0 2024-04-02T04:47:43.955 INFO:tasks.workunit.client.0.smithi070.stdout:1/222: dwrite d0/d9/d1d/f38 [0,4194304] 0 2024-04-02T04:47:43.956 INFO:tasks.workunit.client.0.smithi070.stdout:0/364: link d4/d5/f3f d4/d5/d21/f64 0 2024-04-02T04:47:43.957 INFO:tasks.workunit.client.0.smithi070.stdout:8/306: dread - d2/d3/d35/d3a/d4f/d59/f65 zero size 2024-04-02T04:47:43.958 INFO:tasks.workunit.client.0.smithi070.stdout:3/240: chown d10/d32/d3f/l26 1058796335 1 2024-04-02T04:47:43.958 INFO:tasks.workunit.client.0.smithi070.stdout:3/241: readlink d10/d32/d3f/l26 0 2024-04-02T04:47:43.960 INFO:tasks.workunit.client.0.smithi070.stdout:7/256: truncate d7/d9/f27 121782 0 2024-04-02T04:47:43.965 INFO:tasks.workunit.client.0.smithi070.stdout:5/274: creat d5/d17/f53 x:0 0 0 2024-04-02T04:47:43.965 INFO:tasks.workunit.client.0.smithi070.stdout:8/307: read d2/d3/fd [899537,123777] 0 2024-04-02T04:47:43.965 INFO:tasks.workunit.client.0.smithi070.stdout:3/242: creat d10/d32/d3f/f54 x:0 0 0 2024-04-02T04:47:43.971 INFO:tasks.workunit.client.0.smithi070.stdout:7/257: dwrite d7/d2f/f31 [0,4194304] 0 2024-04-02T04:47:43.971 INFO:tasks.workunit.client.0.smithi070.stdout:5/275: chown d5/d9/d10/c1b 400 1 2024-04-02T04:47:43.974 INFO:tasks.workunit.client.0.smithi070.stdout:3/243: creat d10/d32/f55 x:0 0 0 2024-04-02T04:47:43.975 INFO:tasks.workunit.client.0.smithi070.stdout:0/365: dwrite d4/d5/f3f [0,4194304] 0 2024-04-02T04:47:43.976 INFO:tasks.workunit.client.0.smithi070.stdout:1/223: creat d0/d7/f48 x:0 0 0 2024-04-02T04:47:43.978 INFO:tasks.workunit.client.0.smithi070.stdout:8/308: symlink d2/d8/d29/l68 0 2024-04-02T04:47:43.979 INFO:tasks.workunit.client.0.smithi070.stdout:0/366: dread d4/d5/d13/d2a/d29/f5d [0,4194304] 0 2024-04-02T04:47:43.980 INFO:tasks.workunit.client.0.smithi070.stdout:7/258: symlink d7/d4e/l50 0 2024-04-02T04:47:43.981 INFO:tasks.workunit.client.0.smithi070.stdout:0/367: fdatasync d4/d5/d13/d17/f62 0 2024-04-02T04:47:43.983 INFO:tasks.workunit.client.0.smithi070.stdout:5/276: dread d5/f6 [0,4194304] 0 2024-04-02T04:47:43.984 INFO:tasks.workunit.client.0.smithi070.stdout:1/224: symlink d0/d7/d1a/l49 0 2024-04-02T04:47:43.985 INFO:tasks.workunit.client.1.smithi179.stdout:3/204: sync 2024-04-02T04:47:43.986 INFO:tasks.workunit.client.0.smithi070.stdout:8/309: mknod d2/d3/d35/d3a/d4f/c69 0 2024-04-02T04:47:43.987 INFO:tasks.workunit.client.1.smithi179.stdout:2/146: rmdir dc 39 2024-04-02T04:47:43.987 INFO:tasks.workunit.client.1.smithi179.stdout:6/249: symlink d3/d12/l58 0 2024-04-02T04:47:43.988 INFO:tasks.workunit.client.0.smithi070.stdout:3/244: unlink d10/d32/d3f/f54 0 2024-04-02T04:47:43.989 INFO:tasks.workunit.client.0.smithi070.stdout:8/310: dread d2/d8/d29/db/f13 [4194304,4194304] 0 2024-04-02T04:47:43.992 INFO:tasks.workunit.client.0.smithi070.stdout:1/225: fsync d0/d9/d1d/f35 0 2024-04-02T04:47:43.993 INFO:tasks.workunit.client.1.smithi179.stdout:8/226: dwrite d3/f27 [0,4194304] 0 2024-04-02T04:47:43.993 INFO:tasks.workunit.client.0.smithi070.stdout:7/259: creat d7/d2f/d46/f51 x:0 0 0 2024-04-02T04:47:43.994 INFO:tasks.workunit.client.1.smithi179.stdout:3/205: chown d8/d26/d36/c37 3 1 2024-04-02T04:47:43.994 INFO:tasks.workunit.client.1.smithi179.stdout:1/269: dwrite d3/d5/d2f/f47 [0,4194304] 0 2024-04-02T04:47:43.995 INFO:tasks.workunit.client.0.smithi070.stdout:7/260: readlink d7/dd/df/d34/l33 0 2024-04-02T04:47:43.996 INFO:tasks.workunit.client.0.smithi070.stdout:3/245: creat d10/d32/f56 x:0 0 0 2024-04-02T04:47:43.996 INFO:tasks.workunit.client.1.smithi179.stdout:1/270: stat d3/fa 0 2024-04-02T04:47:43.998 INFO:tasks.workunit.client.1.smithi179.stdout:0/310: dwrite d1/da/f46 [0,4194304] 0 2024-04-02T04:47:43.998 INFO:tasks.workunit.client.1.smithi179.stdout:2/147: rmdir dc/d16/d1e 39 2024-04-02T04:47:44.000 INFO:tasks.workunit.client.1.smithi179.stdout:2/148: truncate dc/d16/f22 946791 0 2024-04-02T04:47:44.001 INFO:tasks.workunit.client.0.smithi070.stdout:1/226: dwrite d0/d9/d1d/f40 [0,4194304] 0 2024-04-02T04:47:44.004 INFO:tasks.workunit.client.1.smithi179.stdout:8/227: mknod d3/d31/c53 0 2024-04-02T04:47:44.004 INFO:tasks.workunit.client.0.smithi070.stdout:5/277: rename d5/d17/d27/d2c/f37 to d5/d14/f54 0 2024-04-02T04:47:44.013 INFO:tasks.workunit.client.1.smithi179.stdout:1/271: dread d3/d5/f13 [0,4194304] 0 2024-04-02T04:47:44.015 INFO:tasks.workunit.client.0.smithi070.stdout:8/311: fsync d2/d8/d23/d24/f37 0 2024-04-02T04:47:44.019 INFO:tasks.workunit.client.0.smithi070.stdout:0/368: write d4/d5/d13/d2a/d29/f46 [4908547,122231] 0 2024-04-02T04:47:44.021 INFO:tasks.workunit.client.0.smithi070.stdout:8/312: dread d2/d3/f61 [0,4194304] 0 2024-04-02T04:47:44.021 INFO:tasks.workunit.client.1.smithi179.stdout:8/228: symlink d3/d31/l54 0 2024-04-02T04:47:44.022 INFO:tasks.workunit.client.0.smithi070.stdout:1/227: rmdir d0/d7/d28 39 2024-04-02T04:47:44.022 INFO:tasks.workunit.client.1.smithi179.stdout:8/229: write d3/d32/d21/d22/d33/d3c/f40 [2914273,109482] 0 2024-04-02T04:47:44.025 INFO:tasks.workunit.client.0.smithi070.stdout:5/278: rmdir d5/d17/d27/d2c 39 2024-04-02T04:47:44.026 INFO:tasks.workunit.client.0.smithi070.stdout:0/369: dwrite d4/d5/f14 [8388608,4194304] 0 2024-04-02T04:47:44.027 INFO:tasks.workunit.client.0.smithi070.stdout:7/261: symlink d7/dd/l52 0 2024-04-02T04:47:44.029 INFO:tasks.workunit.client.0.smithi070.stdout:4/204: sync 2024-04-02T04:47:44.029 INFO:tasks.workunit.client.0.smithi070.stdout:2/328: sync 2024-04-02T04:47:44.029 INFO:tasks.workunit.client.0.smithi070.stdout:9/232: sync 2024-04-02T04:47:44.030 INFO:tasks.workunit.client.0.smithi070.stdout:3/246: symlink d10/d17/l57 0 2024-04-02T04:47:44.031 INFO:tasks.workunit.client.0.smithi070.stdout:3/247: chown f0 13496605 1 2024-04-02T04:47:44.032 INFO:tasks.workunit.client.0.smithi070.stdout:7/262: dread d7/dd/df/f32 [0,4194304] 0 2024-04-02T04:47:44.033 INFO:tasks.workunit.client.0.smithi070.stdout:7/263: chown d7/l41 236974261 1 2024-04-02T04:47:44.035 INFO:tasks.workunit.client.0.smithi070.stdout:8/313: write d2/d3/f61 [741493,86414] 0 2024-04-02T04:47:44.036 INFO:tasks.workunit.client.0.smithi070.stdout:5/279: creat d5/d17/d2b/f55 x:0 0 0 2024-04-02T04:47:44.037 INFO:tasks.workunit.client.0.smithi070.stdout:5/280: fsync d5/ff 0 2024-04-02T04:47:44.037 INFO:tasks.workunit.client.0.smithi070.stdout:0/370: creat d4/d5/d13/d2a/d29/f65 x:0 0 0 2024-04-02T04:47:44.039 INFO:tasks.workunit.client.0.smithi070.stdout:9/233: mknod d4/d13/c49 0 2024-04-02T04:47:44.040 INFO:tasks.workunit.client.0.smithi070.stdout:2/329: mkdir d2/d12/d3e/d68 0 2024-04-02T04:47:44.041 INFO:tasks.workunit.client.0.smithi070.stdout:4/205: mknod d1/d3/d9/c3c 0 2024-04-02T04:47:44.041 INFO:tasks.workunit.client.0.smithi070.stdout:3/248: symlink d10/d32/l58 0 2024-04-02T04:47:44.042 INFO:tasks.workunit.client.0.smithi070.stdout:7/264: creat d7/d2f/d46/f53 x:0 0 0 2024-04-02T04:47:44.044 INFO:tasks.workunit.client.0.smithi070.stdout:8/314: unlink d2/d8/d29/f15 0 2024-04-02T04:47:44.045 INFO:tasks.workunit.client.0.smithi070.stdout:5/281: truncate d5/d9/d23/f48 3745590 0 2024-04-02T04:47:44.048 INFO:tasks.workunit.client.0.smithi070.stdout:9/234: rename d4/d13/d16/d19/d2f/d39/f3d to d4/d34/f4a 0 2024-04-02T04:47:44.048 INFO:tasks.workunit.client.1.smithi179.stdout:6/250: creat d3/d12/d23/f59 x:0 0 0 2024-04-02T04:47:44.049 INFO:tasks.workunit.client.1.smithi179.stdout:1/272: creat d3/d5/d2f/d5a/f5d x:0 0 0 2024-04-02T04:47:44.049 INFO:tasks.workunit.client.0.smithi070.stdout:2/330: mkdir d2/d12/d59/d69 0 2024-04-02T04:47:44.050 INFO:tasks.workunit.client.1.smithi179.stdout:2/149: mknod dc/d16/d1e/c2c 0 2024-04-02T04:47:44.050 INFO:tasks.workunit.client.1.smithi179.stdout:6/251: chown d3/ld 43188985 1 2024-04-02T04:47:44.050 INFO:tasks.workunit.client.1.smithi179.stdout:2/150: truncate dc/fe 4211313 0 2024-04-02T04:47:44.050 INFO:tasks.workunit.client.1.smithi179.stdout:3/206: link d8/c21 d8/d26/d36/c44 0 2024-04-02T04:47:44.051 INFO:tasks.workunit.client.0.smithi070.stdout:3/249: creat d10/d32/f59 x:0 0 0 2024-04-02T04:47:44.051 INFO:tasks.workunit.client.1.smithi179.stdout:8/230: creat d3/d32/d21/d51/f55 x:0 0 0 2024-04-02T04:47:44.052 INFO:tasks.workunit.client.1.smithi179.stdout:8/231: write d3/d32/d21/f25 [2738612,75530] 0 2024-04-02T04:47:44.053 INFO:tasks.workunit.client.1.smithi179.stdout:1/273: chown d3/c58 35098 1 2024-04-02T04:47:44.055 INFO:tasks.workunit.client.0.smithi070.stdout:7/265: mkdir d7/d9/d19/d54 0 2024-04-02T04:47:44.055 INFO:tasks.workunit.client.0.smithi070.stdout:4/206: dread d1/f26 [0,4194304] 0 2024-04-02T04:47:44.055 INFO:tasks.workunit.client.0.smithi070.stdout:5/282: dwrite d5/d9/d23/f49 [0,4194304] 0 2024-04-02T04:47:44.056 INFO:tasks.workunit.client.0.smithi070.stdout:5/283: dread - d5/d17/f53 zero size 2024-04-02T04:47:44.057 INFO:tasks.workunit.client.1.smithi179.stdout:2/151: creat dc/d16/d1e/f2d x:0 0 0 2024-04-02T04:47:44.061 INFO:tasks.workunit.client.1.smithi179.stdout:1/274: creat d3/d5/d2f/d4f/d42/d53/f5e x:0 0 0 2024-04-02T04:47:44.062 INFO:tasks.workunit.client.0.smithi070.stdout:2/331: symlink d2/d12/d48/l6a 0 2024-04-02T04:47:44.062 INFO:tasks.workunit.client.1.smithi179.stdout:1/275: dread - d3/d5/de/f57 zero size 2024-04-02T04:47:44.063 INFO:tasks.workunit.client.0.smithi070.stdout:9/235: dread d4/d13/d16/d19/f2e [0,4194304] 0 2024-04-02T04:47:44.063 INFO:tasks.workunit.client.0.smithi070.stdout:3/250: mknod d10/d32/d4b/c5a 0 2024-04-02T04:47:44.065 INFO:tasks.workunit.client.0.smithi070.stdout:9/236: write d4/d10/d11/f42 [1705117,72262] 0 2024-04-02T04:47:44.065 INFO:tasks.workunit.client.0.smithi070.stdout:8/315: creat d2/d8/d23/d24/d66/f6a x:0 0 0 2024-04-02T04:47:44.067 INFO:tasks.workunit.client.0.smithi070.stdout:9/237: fdatasync d4/d10/d11/f21 0 2024-04-02T04:47:44.071 INFO:tasks.workunit.client.0.smithi070.stdout:7/266: mkdir d7/d4e/d55 0 2024-04-02T04:47:44.073 INFO:tasks.workunit.client.1.smithi179.stdout:1/276: unlink d3/d5/d2f/d36/f44 0 2024-04-02T04:47:44.073 INFO:tasks.workunit.client.1.smithi179.stdout:8/232: dread d3/f16 [0,4194304] 0 2024-04-02T04:47:44.074 INFO:tasks.workunit.client.0.smithi070.stdout:5/284: rmdir d5/d9/d3f 39 2024-04-02T04:47:44.075 INFO:tasks.workunit.client.1.smithi179.stdout:2/152: mknod dc/d16/c2e 0 2024-04-02T04:47:44.076 INFO:tasks.workunit.client.1.smithi179.stdout:3/207: link d8/d26/d36/l42 d8/d26/d36/l45 0 2024-04-02T04:47:44.077 INFO:tasks.workunit.client.0.smithi070.stdout:2/332: creat d2/f6b x:0 0 0 2024-04-02T04:47:44.078 INFO:tasks.workunit.client.1.smithi179.stdout:6/252: link d3/d12/l58 d3/l5a 0 2024-04-02T04:47:44.080 INFO:tasks.workunit.client.1.smithi179.stdout:6/253: write d3/f18 [3085652,7496] 0 2024-04-02T04:47:44.080 INFO:tasks.workunit.client.1.smithi179.stdout:8/233: dread d3/d32/d21/f25 [0,4194304] 0 2024-04-02T04:47:44.081 INFO:tasks.workunit.client.1.smithi179.stdout:1/277: dwrite d3/d5/de/f3c [0,4194304] 0 2024-04-02T04:47:44.082 INFO:tasks.workunit.client.1.smithi179.stdout:8/234: chown d3/d32/d21/d49 7550 1 2024-04-02T04:47:44.082 INFO:tasks.workunit.client.1.smithi179.stdout:6/254: read d3/d12/d23/d26/f37 [2307449,91965] 0 2024-04-02T04:47:44.091 INFO:tasks.workunit.client.0.smithi070.stdout:4/207: mknod d1/c3d 0 2024-04-02T04:47:44.092 INFO:tasks.workunit.client.0.smithi070.stdout:9/238: rmdir d4/d13/d16/d19 39 2024-04-02T04:47:44.092 INFO:tasks.workunit.client.0.smithi070.stdout:5/285: dread d5/d9/d10/f11 [0,4194304] 0 2024-04-02T04:47:44.094 INFO:tasks.workunit.client.1.smithi179.stdout:3/208: dread d8/d26/f2c [0,4194304] 0 2024-04-02T04:47:44.095 INFO:tasks.workunit.client.0.smithi070.stdout:7/267: unlink d7/d4e/l50 0 2024-04-02T04:47:44.096 INFO:tasks.workunit.client.1.smithi179.stdout:8/235: stat l2 0 2024-04-02T04:47:44.098 INFO:tasks.workunit.client.1.smithi179.stdout:4/324: sync 2024-04-02T04:47:44.098 INFO:tasks.workunit.client.1.smithi179.stdout:7/308: sync 2024-04-02T04:47:44.098 INFO:tasks.workunit.client.1.smithi179.stdout:6/255: creat d3/d2c/f5b x:0 0 0 2024-04-02T04:47:44.099 INFO:tasks.workunit.client.1.smithi179.stdout:6/256: chown d3/d12 1580785043 1 2024-04-02T04:47:44.102 INFO:tasks.workunit.client.1.smithi179.stdout:1/278: symlink d3/d5/d2f/d4f/d42/d53/l5f 0 2024-04-02T04:47:44.103 INFO:tasks.workunit.client.0.smithi070.stdout:8/316: mknod d2/d8/d29/d22/d30/c6b 0 2024-04-02T04:47:44.103 INFO:tasks.workunit.client.0.smithi070.stdout:5/286: dread d5/d17/d27/f28 [0,4194304] 0 2024-04-02T04:47:44.105 INFO:tasks.workunit.client.1.smithi179.stdout:6/257: dwrite d3/d12/d23/f55 [0,4194304] 0 2024-04-02T04:47:44.107 INFO:tasks.workunit.client.1.smithi179.stdout:3/209: rename d8/f28 to d8/d26/f46 0 2024-04-02T04:47:44.108 INFO:tasks.workunit.client.0.smithi070.stdout:9/239: mknod d4/d34/c4b 0 2024-04-02T04:47:44.110 INFO:tasks.workunit.client.1.smithi179.stdout:8/236: fdatasync d3/f16 0 2024-04-02T04:47:44.111 INFO:tasks.workunit.client.1.smithi179.stdout:8/237: write d3/d32/f19 [4286899,39654] 0 2024-04-02T04:47:44.113 INFO:tasks.workunit.client.0.smithi070.stdout:7/268: mknod d7/dd/df/c56 0 2024-04-02T04:47:44.117 INFO:tasks.workunit.client.1.smithi179.stdout:6/258: stat d3/d12/d23/d2f/f56 0 2024-04-02T04:47:44.118 INFO:tasks.workunit.client.1.smithi179.stdout:6/259: stat d3/fa 0 2024-04-02T04:47:44.119 INFO:tasks.workunit.client.0.smithi070.stdout:8/317: truncate d2/d3/f7 4202732 0 2024-04-02T04:47:44.120 INFO:tasks.workunit.client.1.smithi179.stdout:3/210: write d8/d26/f2c [4707510,98057] 0 2024-04-02T04:47:44.120 INFO:tasks.workunit.client.0.smithi070.stdout:8/318: read d2/f48 [2694212,9668] 0 2024-04-02T04:47:44.122 INFO:tasks.workunit.client.0.smithi070.stdout:8/319: write d2/d3/d35/d3a/d4f/d59/f65 [492035,118192] 0 2024-04-02T04:47:44.123 INFO:tasks.workunit.client.0.smithi070.stdout:9/240: mknod d4/d10/d11/c4c 0 2024-04-02T04:47:44.126 INFO:tasks.workunit.client.0.smithi070.stdout:7/269: mkdir d7/dd/d35/d57 0 2024-04-02T04:47:44.127 INFO:tasks.workunit.client.1.smithi179.stdout:8/238: dread d3/d32/d21/f37 [0,4194304] 0 2024-04-02T04:47:44.129 INFO:tasks.workunit.client.0.smithi070.stdout:3/251: getdents d10 0 2024-04-02T04:47:44.132 INFO:tasks.workunit.client.0.smithi070.stdout:7/270: dread d7/d9/f17 [0,4194304] 0 2024-04-02T04:47:44.132 INFO:tasks.workunit.client.1.smithi179.stdout:6/260: symlink d3/d12/d23/d39/d3f/l5c 0 2024-04-02T04:47:44.133 INFO:tasks.workunit.client.0.smithi070.stdout:4/208: creat d1/f3e x:0 0 0 2024-04-02T04:47:44.134 INFO:tasks.workunit.client.1.smithi179.stdout:6/261: dread d3/f10 [0,4194304] 0 2024-04-02T04:47:44.140 INFO:tasks.workunit.client.1.smithi179.stdout:4/325: link d1/d3/d19/c61 d1/d3/df/d10/d3a/c6a 0 2024-04-02T04:47:44.142 INFO:tasks.workunit.client.0.smithi070.stdout:3/252: mknod d10/d32/d3f/c5b 0 2024-04-02T04:47:44.143 INFO:tasks.workunit.client.1.smithi179.stdout:1/279: rename d3/d5/d2f/f34 to d3/d5/f60 0 2024-04-02T04:47:44.145 INFO:tasks.workunit.client.1.smithi179.stdout:4/326: dwrite d1/d3/df/d10/d3e/f55 [0,4194304] 0 2024-04-02T04:47:44.146 INFO:tasks.workunit.client.0.smithi070.stdout:5/287: creat d5/f56 x:0 0 0 2024-04-02T04:47:44.146 INFO:tasks.workunit.client.1.smithi179.stdout:4/327: chown d1/c30 366973297 1 2024-04-02T04:47:44.147 INFO:tasks.workunit.client.1.smithi179.stdout:6/262: mkdir d3/d2d/d38/d5d 0 2024-04-02T04:47:44.147 INFO:tasks.workunit.client.1.smithi179.stdout:4/328: rename d1/d3/df/d10 to d1/d3/df/d10/d2f/d6b 22 2024-04-02T04:47:44.148 INFO:tasks.workunit.client.1.smithi179.stdout:7/309: link d1/d9/l59 d1/l70 0 2024-04-02T04:47:44.149 INFO:tasks.workunit.client.1.smithi179.stdout:8/239: mkdir d3/d32/d21/d50/d56 0 2024-04-02T04:47:44.150 INFO:tasks.workunit.client.1.smithi179.stdout:8/240: write d3/d32/d21/d22/d33/d3c/f44 [944496,75212] 0 2024-04-02T04:47:44.151 INFO:tasks.workunit.client.1.smithi179.stdout:8/241: readlink d3/d32/d21/l28 0 2024-04-02T04:47:44.152 INFO:tasks.workunit.client.0.smithi070.stdout:9/241: mkdir d4/d13/d16/d19/d2f/d4d 0 2024-04-02T04:47:44.153 INFO:tasks.workunit.client.0.smithi070.stdout:9/242: dread - d4/d13/d16/f2b zero size 2024-04-02T04:47:44.155 INFO:tasks.workunit.client.0.smithi070.stdout:4/209: dwrite d1/d3/f39 [0,4194304] 0 2024-04-02T04:47:44.156 INFO:tasks.workunit.client.1.smithi179.stdout:6/263: creat d3/d12/d44/f5e x:0 0 0 2024-04-02T04:47:44.158 INFO:tasks.workunit.client.1.smithi179.stdout:7/310: creat d1/dd/d23/f71 x:0 0 0 2024-04-02T04:47:44.161 INFO:tasks.workunit.client.0.smithi070.stdout:5/288: read d5/d9/d23/f42 [3088670,52983] 0 2024-04-02T04:47:44.163 INFO:tasks.workunit.client.0.smithi070.stdout:9/243: dwrite d4/d13/f26 [0,4194304] 0 2024-04-02T04:47:44.163 INFO:tasks.workunit.client.1.smithi179.stdout:4/329: rename d1/d3/f13 to d1/d3/df/d10/d2f/d60/f6c 0 2024-04-02T04:47:44.163 INFO:tasks.workunit.client.1.smithi179.stdout:4/330: chown d1/d3/f7 43 1 2024-04-02T04:47:44.164 INFO:tasks.workunit.client.0.smithi070.stdout:7/271: symlink d7/l58 0 2024-04-02T04:47:44.166 INFO:tasks.workunit.client.1.smithi179.stdout:3/211: getdents d8/d26/d36 0 2024-04-02T04:47:44.168 INFO:tasks.workunit.client.1.smithi179.stdout:1/280: creat d3/d5/d2f/d4f/d42/d53/d5b/f61 x:0 0 0 2024-04-02T04:47:44.170 INFO:tasks.workunit.client.0.smithi070.stdout:3/253: symlink d10/d17/d4c/l5c 0 2024-04-02T04:47:44.170 INFO:tasks.workunit.client.0.smithi070.stdout:3/254: fdatasync fb 0 2024-04-02T04:47:44.172 INFO:tasks.workunit.client.0.smithi070.stdout:3/255: read f0 [6366509,111127] 0 2024-04-02T04:47:44.173 INFO:tasks.workunit.client.1.smithi179.stdout:8/242: rename d3/l15 to d3/d32/d21/d50/d56/l57 0 2024-04-02T04:47:44.175 INFO:tasks.workunit.client.1.smithi179.stdout:4/331: creat d1/d3/dc/d26/f6d x:0 0 0 2024-04-02T04:47:44.175 INFO:tasks.workunit.client.1.smithi179.stdout:3/212: dwrite d8/d26/d36/f41 [0,4194304] 0 2024-04-02T04:47:44.177 INFO:tasks.workunit.client.1.smithi179.stdout:6/264: creat d3/d12/d23/d39/d4e/f5f x:0 0 0 2024-04-02T04:47:44.179 INFO:tasks.workunit.client.0.smithi070.stdout:3/256: dwrite d10/d32/d3f/f1f [0,4194304] 0 2024-04-02T04:47:44.179 INFO:tasks.workunit.client.1.smithi179.stdout:6/265: chown d3/d12/l17 89375 1 2024-04-02T04:47:44.179 INFO:tasks.workunit.client.0.smithi070.stdout:3/257: readlink d10/l18 0 2024-04-02T04:47:44.183 INFO:tasks.workunit.client.1.smithi179.stdout:8/243: dwrite d3/d32/f47 [4194304,4194304] 0 2024-04-02T04:47:44.183 INFO:tasks.workunit.client.0.smithi070.stdout:9/244: creat d4/d10/d11/f4e x:0 0 0 2024-04-02T04:47:44.185 INFO:tasks.workunit.client.0.smithi070.stdout:8/320: getdents d2/d8/d23/d24/d66 0 2024-04-02T04:47:44.186 INFO:tasks.workunit.client.0.smithi070.stdout:8/321: write d2/d8/d29/d22/f46 [851613,82479] 0 2024-04-02T04:47:44.187 INFO:tasks.workunit.client.1.smithi179.stdout:4/332: mkdir d1/d15/d6e 0 2024-04-02T04:47:44.187 INFO:tasks.workunit.client.0.smithi070.stdout:8/322: stat d2/d8/l2e 0 2024-04-02T04:47:44.190 INFO:tasks.workunit.client.0.smithi070.stdout:2/333: getdents d2/d10/d53 0 2024-04-02T04:47:44.192 INFO:tasks.workunit.client.1.smithi179.stdout:3/213: dwrite d8/d26/f2c [4194304,4194304] 0 2024-04-02T04:47:44.194 INFO:tasks.workunit.client.1.smithi179.stdout:7/311: dread d1/d9/f12 [0,4194304] 0 2024-04-02T04:47:44.194 INFO:tasks.workunit.client.1.smithi179.stdout:1/281: link d3/d5/d2f/d36/l38 d3/d5/de/l62 0 2024-04-02T04:47:44.195 INFO:tasks.workunit.client.0.smithi070.stdout:3/258: unlink d10/f41 0 2024-04-02T04:47:44.196 INFO:tasks.workunit.client.0.smithi070.stdout:5/289: rmdir d5/d14 39 2024-04-02T04:47:44.198 INFO:tasks.workunit.client.1.smithi179.stdout:1/282: write d3/d5/d2f/d4f/d42/d53/f5e [543841,10222] 0 2024-04-02T04:47:44.198 INFO:tasks.workunit.client.0.smithi070.stdout:7/272: link d7/d2f/d46/f53 d7/dd/d35/f59 0 2024-04-02T04:47:44.200 INFO:tasks.workunit.client.1.smithi179.stdout:3/214: creat d8/d26/d36/f47 x:0 0 0 2024-04-02T04:47:44.200 INFO:tasks.workunit.client.1.smithi179.stdout:1/283: readlink d3/d5/d2f/d4f/d42/d53/l5f 0 2024-04-02T04:47:44.200 INFO:tasks.workunit.client.1.smithi179.stdout:8/244: dwrite d3/d32/f19 [0,4194304] 0 2024-04-02T04:47:44.201 INFO:tasks.workunit.client.0.smithi070.stdout:9/245: mkdir d4/d13/d16/d19/d2f/d39/d4f 0 2024-04-02T04:47:44.206 INFO:tasks.workunit.client.1.smithi179.stdout:7/312: rmdir d1 39 2024-04-02T04:47:44.207 INFO:tasks.workunit.client.1.smithi179.stdout:1/284: dwrite d3/d5/f27 [0,4194304] 0 2024-04-02T04:47:44.209 INFO:tasks.workunit.client.0.smithi070.stdout:9/246: dwrite d4/d13/f26 [0,4194304] 0 2024-04-02T04:47:44.210 INFO:tasks.workunit.client.0.smithi070.stdout:9/247: read - d4/d34/f4a zero size 2024-04-02T04:47:44.210 INFO:tasks.workunit.client.0.smithi070.stdout:4/210: creat d1/f3f x:0 0 0 2024-04-02T04:47:44.212 INFO:tasks.workunit.client.0.smithi070.stdout:4/211: write d1/d3/f30 [1132875,75512] 0 2024-04-02T04:47:44.215 INFO:tasks.workunit.client.1.smithi179.stdout:1/285: dwrite d3/d5/d2f/d4f/f37 [0,4194304] 0 2024-04-02T04:47:44.218 INFO:tasks.workunit.client.1.smithi179.stdout:8/245: rename d3/d32/d30/c48 to d3/d32/d21/d50/d56/c58 0 2024-04-02T04:47:44.219 INFO:tasks.workunit.client.0.smithi070.stdout:5/290: chown d5/d17/d27/f28 457 1 2024-04-02T04:47:44.220 INFO:tasks.workunit.client.0.smithi070.stdout:4/212: dread d1/d3/d9/f22 [0,4194304] 0 2024-04-02T04:47:44.220 INFO:tasks.workunit.client.0.smithi070.stdout:5/291: fsync d5/d17/d27/f45 0 2024-04-02T04:47:44.220 INFO:tasks.workunit.client.0.smithi070.stdout:6/179: sync 2024-04-02T04:47:44.221 INFO:tasks.workunit.client.1.smithi179.stdout:9/283: sync 2024-04-02T04:47:44.221 INFO:tasks.workunit.client.1.smithi179.stdout:5/188: sync 2024-04-02T04:47:44.221 INFO:tasks.workunit.client.0.smithi070.stdout:6/180: chown d0/d20/l32 1042937 1 2024-04-02T04:47:44.221 INFO:tasks.workunit.client.0.smithi070.stdout:7/273: fdatasync d7/dd/d35/f59 0 2024-04-02T04:47:44.222 INFO:tasks.workunit.client.0.smithi070.stdout:6/181: write d0/d3/d19/f2f [629456,84501] 0 2024-04-02T04:47:44.223 INFO:tasks.workunit.client.1.smithi179.stdout:5/189: readlink d8/l16 0 2024-04-02T04:47:44.225 INFO:tasks.workunit.client.1.smithi179.stdout:6/266: getdents d3/d2c 0 2024-04-02T04:47:44.228 INFO:tasks.workunit.client.1.smithi179.stdout:4/333: getdents d1/d3/dc/d27/d62 0 2024-04-02T04:47:44.228 INFO:tasks.workunit.client.1.smithi179.stdout:1/286: dread d3/d5/d2f/d4f/f37 [0,4194304] 0 2024-04-02T04:47:44.229 INFO:tasks.workunit.client.1.smithi179.stdout:6/267: truncate d3/d2d/d43/f47 736392 0 2024-04-02T04:47:44.229 INFO:tasks.workunit.client.0.smithi070.stdout:9/248: creat d4/d13/d16/d19/f50 x:0 0 0 2024-04-02T04:47:44.231 INFO:tasks.workunit.client.1.smithi179.stdout:4/334: stat d1/d3/df/f51 0 2024-04-02T04:47:44.234 INFO:tasks.workunit.client.1.smithi179.stdout:5/190: dwrite d8/f27 [0,4194304] 0 2024-04-02T04:47:44.236 INFO:tasks.workunit.client.1.smithi179.stdout:7/313: mknod d1/dd/d16/c72 0 2024-04-02T04:47:44.237 INFO:tasks.workunit.client.1.smithi179.stdout:9/284: mknod d0/d1/d11/d29/d40/c55 0 2024-04-02T04:47:44.237 INFO:tasks.workunit.client.1.smithi179.stdout:3/215: dread d8/d26/f46 [0,4194304] 0 2024-04-02T04:47:44.245 INFO:tasks.workunit.client.0.smithi070.stdout:3/259: link d10/d17/f48 d10/d17/f5d 0 2024-04-02T04:47:44.245 INFO:tasks.workunit.client.0.smithi070.stdout:3/260: stat d10/d32/f55 0 2024-04-02T04:47:44.246 INFO:tasks.workunit.client.0.smithi070.stdout:3/261: readlink d10/d17/l57 0 2024-04-02T04:47:44.246 INFO:tasks.workunit.client.0.smithi070.stdout:4/213: mknod d1/c40 0 2024-04-02T04:47:44.247 INFO:tasks.workunit.client.1.smithi179.stdout:9/285: dwrite d0/d1/d11/d29/f50 [0,4194304] 0 2024-04-02T04:47:44.247 INFO:tasks.workunit.client.0.smithi070.stdout:9/249: dwrite d4/f8 [0,4194304] 0 2024-04-02T04:47:44.247 INFO:tasks.workunit.client.0.smithi070.stdout:4/214: dread - d1/d3/d9/f29 zero size 2024-04-02T04:47:44.248 INFO:tasks.workunit.client.1.smithi179.stdout:3/216: creat d8/f48 x:0 0 0 2024-04-02T04:47:44.249 INFO:tasks.workunit.client.1.smithi179.stdout:9/286: fsync d0/d1/d11/f51 0 2024-04-02T04:47:44.249 INFO:tasks.workunit.client.1.smithi179.stdout:3/217: fdatasync d8/f22 0 2024-04-02T04:47:44.249 INFO:tasks.workunit.client.0.smithi070.stdout:5/292: fdatasync d5/f1d 0 2024-04-02T04:47:44.249 INFO:tasks.workunit.client.0.smithi070.stdout:2/334: dread d2/d10/d21/f31 [0,4194304] 0 2024-04-02T04:47:44.251 INFO:tasks.workunit.client.1.smithi179.stdout:7/314: rename d1/ff to d1/d9/d35/d6b/f73 0 2024-04-02T04:47:44.252 INFO:tasks.workunit.client.1.smithi179.stdout:7/315: chown d1/d9/l26 1 1 2024-04-02T04:47:44.253 INFO:tasks.workunit.client.0.smithi070.stdout:8/323: dwrite d2/d8/d29/db/f13 [0,4194304] 0 2024-04-02T04:47:44.254 INFO:tasks.workunit.client.1.smithi179.stdout:1/287: symlink d3/d5/d2f/l63 0 2024-04-02T04:47:44.254 INFO:tasks.workunit.client.0.smithi070.stdout:8/324: readlink d2/d8/l50 0 2024-04-02T04:47:44.257 INFO:tasks.workunit.client.0.smithi070.stdout:6/182: symlink d0/d3/d19/d31/l38 0 2024-04-02T04:47:44.258 INFO:tasks.workunit.client.0.smithi070.stdout:6/183: chown d0/l1a 562779260 1 2024-04-02T04:47:44.260 INFO:tasks.workunit.client.1.smithi179.stdout:6/268: mknod d3/d12/d23/d26/d57/c60 0 2024-04-02T04:47:44.261 INFO:tasks.workunit.client.1.smithi179.stdout:7/316: dwrite d1/dd/f20 [0,4194304] 0 2024-04-02T04:47:44.263 INFO:tasks.workunit.client.1.smithi179.stdout:1/288: read d3/d5/de/f16 [3560623,43969] 0 2024-04-02T04:47:44.263 INFO:tasks.workunit.client.1.smithi179.stdout:2/153: sync 2024-04-02T04:47:44.263 INFO:tasks.workunit.client.1.smithi179.stdout:0/311: sync 2024-04-02T04:47:44.263 INFO:tasks.workunit.client.1.smithi179.stdout:8/246: dread d3/d32/d21/d22/d33/d3c/f40 [0,4194304] 0 2024-04-02T04:47:44.264 INFO:tasks.workunit.client.0.smithi070.stdout:4/215: mkdir d1/d3/d9/d41 0 2024-04-02T04:47:44.264 INFO:tasks.workunit.client.1.smithi179.stdout:2/154: rename dc to dc/d2f 22 2024-04-02T04:47:44.264 INFO:tasks.workunit.client.0.smithi070.stdout:4/216: chown d1/d3/d9/f22 26054 1 2024-04-02T04:47:44.265 INFO:tasks.workunit.client.1.smithi179.stdout:8/247: chown d3/d32/l2c 1 1 2024-04-02T04:47:44.268 INFO:tasks.workunit.client.0.smithi070.stdout:2/335: chown d2/l15 215942 1 2024-04-02T04:47:44.269 INFO:tasks.workunit.client.0.smithi070.stdout:4/217: dread d1/d3/f39 [0,4194304] 0 2024-04-02T04:47:44.273 INFO:tasks.workunit.client.0.smithi070.stdout:9/250: rename d4/d10/d12/d27/d32/f33 to d4/d13/f51 0 2024-04-02T04:47:44.277 INFO:tasks.workunit.client.1.smithi179.stdout:9/287: unlink d0/d1/d11/d29/d40/f53 0 2024-04-02T04:47:44.279 INFO:tasks.workunit.client.1.smithi179.stdout:3/218: creat d8/d26/d36/f49 x:0 0 0 2024-04-02T04:47:44.280 INFO:tasks.workunit.client.0.smithi070.stdout:7/274: dwrite d7/fc [8388608,4194304] 0 2024-04-02T04:47:44.282 INFO:tasks.workunit.client.0.smithi070.stdout:5/293: chown d5/d17/d27/d2c/d35/c47 14 1 2024-04-02T04:47:44.283 INFO:tasks.workunit.client.1.smithi179.stdout:9/288: dwrite d0/f4c [0,4194304] 0 2024-04-02T04:47:44.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/336: truncate d2/d10/d21/d3d/f47 940073 0 2024-04-02T04:47:44.289 INFO:tasks.workunit.client.1.smithi179.stdout:7/317: mkdir d1/dd/d16/d74 0 2024-04-02T04:47:44.291 INFO:tasks.workunit.client.0.smithi070.stdout:9/251: symlink d4/d10/d12/d27/l52 0 2024-04-02T04:47:44.292 INFO:tasks.workunit.client.1.smithi179.stdout:4/335: creat d1/d3/df/f6f x:0 0 0 2024-04-02T04:47:44.292 INFO:tasks.workunit.client.0.smithi070.stdout:9/252: write d4/d10/d12/d27/d32/f45 [811565,105573] 0 2024-04-02T04:47:44.293 INFO:tasks.workunit.client.1.smithi179.stdout:4/336: write d1/d3/df/d10/d3e/f4a [1769966,114592] 0 2024-04-02T04:47:44.294 INFO:tasks.workunit.client.1.smithi179.stdout:5/191: truncate d8/fb 2621770 0 2024-04-02T04:47:44.295 INFO:tasks.workunit.client.1.smithi179.stdout:0/312: mkdir d1/da/d28/d5e 0 2024-04-02T04:47:44.296 INFO:tasks.workunit.client.1.smithi179.stdout:0/313: fdatasync d1/da/f4a 0 2024-04-02T04:47:44.298 INFO:tasks.workunit.client.1.smithi179.stdout:1/289: mknod d3/d5/d2f/d4f/d42/d53/d5b/c64 0 2024-04-02T04:47:44.300 INFO:tasks.workunit.client.1.smithi179.stdout:8/248: truncate d3/f9 1344345 0 2024-04-02T04:47:44.302 INFO:tasks.workunit.client.1.smithi179.stdout:5/192: dwrite d8/f1d [0,4194304] 0 2024-04-02T04:47:44.303 INFO:tasks.workunit.client.1.smithi179.stdout:3/219: creat d8/d26/d36/f4a x:0 0 0 2024-04-02T04:47:44.303 INFO:tasks.workunit.client.0.smithi070.stdout:4/218: truncate d1/f1e 1819782 0 2024-04-02T04:47:44.307 INFO:tasks.workunit.client.0.smithi070.stdout:3/262: getdents d10/d17 0 2024-04-02T04:47:44.307 INFO:tasks.workunit.client.0.smithi070.stdout:9/253: mkdir d4/d13/d16/d19/d53 0 2024-04-02T04:47:44.308 INFO:tasks.workunit.client.0.smithi070.stdout:9/254: readlink d4/d13/l3e 0 2024-04-02T04:47:44.309 INFO:tasks.workunit.client.1.smithi179.stdout:7/318: mkdir d1/dd/d23/d5e/d75 0 2024-04-02T04:47:44.311 INFO:tasks.workunit.client.1.smithi179.stdout:6/269: dwrite d3/d12/d23/d26/f37 [4194304,4194304] 0 2024-04-02T04:47:44.311 INFO:tasks.workunit.client.0.smithi070.stdout:7/275: creat d7/d4e/d55/f5a x:0 0 0 2024-04-02T04:47:44.312 INFO:tasks.workunit.client.1.smithi179.stdout:7/319: chown d1/d9/d55 41 1 2024-04-02T04:47:44.312 INFO:tasks.workunit.client.0.smithi070.stdout:7/276: stat d7/dd/df/d34/l47 0 2024-04-02T04:47:44.314 INFO:tasks.workunit.client.0.smithi070.stdout:5/294: mkdir d5/d57 0 2024-04-02T04:47:44.314 INFO:tasks.workunit.client.1.smithi179.stdout:1/290: dwrite d3/d5/de/f4e [0,4194304] 0 2024-04-02T04:47:44.315 INFO:tasks.workunit.client.1.smithi179.stdout:6/270: write d3/d12/d23/d2f/f3a [365220,49771] 0 2024-04-02T04:47:44.318 INFO:tasks.workunit.client.1.smithi179.stdout:4/337: symlink d1/d3/df/d10/d2f/d60/l70 0 2024-04-02T04:47:44.319 INFO:tasks.workunit.client.1.smithi179.stdout:2/155: symlink dc/d29/l30 0 2024-04-02T04:47:44.320 INFO:tasks.workunit.client.1.smithi179.stdout:6/271: write d3/d2d/d43/f4b [1112705,31459] 0 2024-04-02T04:47:44.320 INFO:tasks.workunit.client.0.smithi070.stdout:2/337: fsync d2/f1c 0 2024-04-02T04:47:44.322 INFO:tasks.workunit.client.0.smithi070.stdout:3/263: unlink d10/d17/c53 0 2024-04-02T04:47:44.325 INFO:tasks.workunit.client.1.smithi179.stdout:5/193: rename d8/f12 to d8/d1b/d23/d32/f3d 0 2024-04-02T04:47:44.325 INFO:tasks.workunit.client.0.smithi070.stdout:6/184: getdents d0/d20 0 2024-04-02T04:47:44.326 INFO:tasks.workunit.client.0.smithi070.stdout:9/255: creat d4/d13/d16/d19/f54 x:0 0 0 2024-04-02T04:47:44.329 INFO:tasks.workunit.client.1.smithi179.stdout:7/320: mkdir d1/dd/d76 0 2024-04-02T04:47:44.329 INFO:tasks.workunit.client.1.smithi179.stdout:4/338: dwrite d1/f2b [0,4194304] 0 2024-04-02T04:47:44.331 INFO:tasks.workunit.client.0.smithi070.stdout:4/219: mknod d1/d3/c42 0 2024-04-02T04:47:44.337 INFO:tasks.workunit.client.1.smithi179.stdout:2/156: mkdir dc/d29/d31 0 2024-04-02T04:47:44.338 INFO:tasks.workunit.client.0.smithi070.stdout:9/256: write f3 [1422170,66212] 0 2024-04-02T04:47:44.340 INFO:tasks.workunit.client.1.smithi179.stdout:6/272: creat d3/d12/d23/d39/d3f/f61 x:0 0 0 2024-04-02T04:47:44.343 INFO:tasks.workunit.client.0.smithi070.stdout:2/338: dwrite d2/d12/d59/f5e [0,4194304] 0 2024-04-02T04:47:44.345 INFO:tasks.workunit.client.0.smithi070.stdout:5/295: creat d5/d17/d50/f58 x:0 0 0 2024-04-02T04:47:44.345 INFO:tasks.workunit.client.1.smithi179.stdout:5/194: rmdir d8/d1b/d23 39 2024-04-02T04:47:44.350 INFO:tasks.workunit.client.0.smithi070.stdout:4/220: creat d1/d3/d9/f43 x:0 0 0 2024-04-02T04:47:44.352 INFO:tasks.workunit.client.0.smithi070.stdout:6/185: mkdir d0/d3/d19/d39 0 2024-04-02T04:47:44.353 INFO:tasks.workunit.client.0.smithi070.stdout:5/296: dwrite d5/d17/d27/f45 [0,4194304] 0 2024-04-02T04:47:44.353 INFO:tasks.workunit.client.1.smithi179.stdout:3/220: write d8/d26/f46 [3418421,12283] 0 2024-04-02T04:47:44.353 INFO:tasks.workunit.client.1.smithi179.stdout:3/221: stat f5 0 2024-04-02T04:47:44.353 INFO:tasks.workunit.client.0.smithi070.stdout:6/186: fdatasync d0/d3/d19/f1f 0 2024-04-02T04:47:44.354 INFO:tasks.workunit.client.1.smithi179.stdout:1/291: mkdir d3/d5/d2f/d65 0 2024-04-02T04:47:44.360 INFO:tasks.workunit.client.0.smithi070.stdout:5/297: dwrite d5/f56 [0,4194304] 0 2024-04-02T04:47:44.362 INFO:tasks.workunit.client.0.smithi070.stdout:9/257: mkdir d4/d10/d12/d55 0 2024-04-02T04:47:44.363 INFO:tasks.workunit.client.1.smithi179.stdout:0/314: link d1/da/df/l55 d1/da/df/l5f 0 2024-04-02T04:47:44.364 INFO:tasks.workunit.client.1.smithi179.stdout:0/315: write d1/da/d52/f5d [631536,54406] 0 2024-04-02T04:47:44.365 INFO:tasks.workunit.client.0.smithi070.stdout:7/277: creat d7/d9/f5b x:0 0 0 2024-04-02T04:47:44.366 INFO:tasks.workunit.client.1.smithi179.stdout:0/316: dread - d1/da/d28/d37/f3e zero size 2024-04-02T04:47:44.367 INFO:tasks.workunit.client.1.smithi179.stdout:2/157: mknod dc/d16/d1e/c32 0 2024-04-02T04:47:44.368 INFO:tasks.workunit.client.1.smithi179.stdout:2/158: fsync dc/f25 0 2024-04-02T04:47:44.369 INFO:tasks.workunit.client.1.smithi179.stdout:2/159: chown dc/d16/l28 4 1 2024-04-02T04:47:44.370 INFO:tasks.workunit.client.1.smithi179.stdout:6/273: creat d3/d12/f62 x:0 0 0 2024-04-02T04:47:44.371 INFO:tasks.workunit.client.1.smithi179.stdout:6/274: chown d3/d12/d23/f2b 1522481824 1 2024-04-02T04:47:44.373 INFO:tasks.workunit.client.0.smithi070.stdout:4/221: fdatasync d1/fb 0 2024-04-02T04:47:44.375 INFO:tasks.workunit.client.1.smithi179.stdout:2/160: dwrite dc/f12 [0,4194304] 0 2024-04-02T04:47:44.375 INFO:tasks.workunit.client.1.smithi179.stdout:5/195: creat d8/f3e x:0 0 0 2024-04-02T04:47:44.375 INFO:tasks.workunit.client.0.smithi070.stdout:6/187: rmdir d0 39 2024-04-02T04:47:44.379 INFO:tasks.workunit.client.0.smithi070.stdout:4/222: dwrite d1/fd [0,4194304] 0 2024-04-02T04:47:44.381 INFO:tasks.workunit.client.0.smithi070.stdout:3/264: dread - d10/d17/f4a zero size 2024-04-02T04:47:44.382 INFO:tasks.workunit.client.0.smithi070.stdout:3/265: write d10/d32/f55 [459729,64970] 0 2024-04-02T04:47:44.383 INFO:tasks.workunit.client.1.smithi179.stdout:1/292: mknod d3/d5/d2f/d4f/c66 0 2024-04-02T04:47:44.384 INFO:tasks.workunit.client.1.smithi179.stdout:2/161: dread dc/d16/d1e/f23 [0,4194304] 0 2024-04-02T04:47:44.384 INFO:tasks.workunit.client.1.smithi179.stdout:2/162: stat dc/d29 0 2024-04-02T04:47:44.385 INFO:tasks.workunit.client.0.smithi070.stdout:5/298: creat d5/d9/d10/d51/f59 x:0 0 0 2024-04-02T04:47:44.390 INFO:tasks.workunit.client.0.smithi070.stdout:9/258: rename d4/d10/d12/d27/d32 to d4/d10/d12/d56 0 2024-04-02T04:47:44.394 INFO:tasks.workunit.client.0.smithi070.stdout:2/339: creat d2/d24/f6c x:0 0 0 2024-04-02T04:47:44.395 INFO:tasks.workunit.client.1.smithi179.stdout:0/317: truncate d1/da/f17 1595163 0 2024-04-02T04:47:44.400 INFO:tasks.workunit.client.1.smithi179.stdout:9/289: link d0/d1/f4 d0/d19/f56 0 2024-04-02T04:47:44.400 INFO:tasks.workunit.client.0.smithi070.stdout:3/266: dread d10/d32/f36 [0,4194304] 0 2024-04-02T04:47:44.401 INFO:tasks.workunit.client.1.smithi179.stdout:6/275: rmdir d3/d2c 39 2024-04-02T04:47:44.401 INFO:tasks.workunit.client.0.smithi070.stdout:2/340: dwrite d2/d10/d21/d3d/f5d [0,4194304] 0 2024-04-02T04:47:44.405 INFO:tasks.workunit.client.1.smithi179.stdout:4/339: truncate d1/f42 838844 0 2024-04-02T04:47:44.406 INFO:tasks.workunit.client.1.smithi179.stdout:4/340: fsync d1/d3/d19/f2c 0 2024-04-02T04:47:44.407 INFO:tasks.workunit.client.0.smithi070.stdout:3/267: dwrite d10/d32/f36 [0,4194304] 0 2024-04-02T04:47:44.408 INFO:tasks.workunit.client.0.smithi070.stdout:3/268: chown d10/d32/f52 7519 1 2024-04-02T04:47:44.408 INFO:tasks.workunit.client.0.smithi070.stdout:9/259: unlink d4/d13/d16/d19/l47 0 2024-04-02T04:47:44.409 INFO:tasks.workunit.client.0.smithi070.stdout:5/299: dread d5/d9/d23/f42 [0,4194304] 0 2024-04-02T04:47:44.411 INFO:tasks.workunit.client.0.smithi070.stdout:0/371: sync 2024-04-02T04:47:44.411 INFO:tasks.workunit.client.0.smithi070.stdout:1/228: sync 2024-04-02T04:47:44.412 INFO:tasks.workunit.client.0.smithi070.stdout:1/229: stat d0/d9/d1d/f35 0 2024-04-02T04:47:44.413 INFO:tasks.workunit.client.0.smithi070.stdout:1/230: chown d0/d9/d1d/d22/d2c/c37 150699789 1 2024-04-02T04:47:44.415 INFO:tasks.workunit.client.0.smithi070.stdout:4/223: symlink d1/l44 0 2024-04-02T04:47:44.416 INFO:tasks.workunit.client.1.smithi179.stdout:6/276: unlink d3/d12/d23/d39/d4e/f5f 0 2024-04-02T04:47:44.416 INFO:tasks.workunit.client.1.smithi179.stdout:0/318: read d1/f23 [217947,126259] 0 2024-04-02T04:47:44.417 INFO:tasks.workunit.client.1.smithi179.stdout:6/277: dread - d3/d12/d44/f5e zero size 2024-04-02T04:47:44.419 INFO:tasks.workunit.client.0.smithi070.stdout:3/269: mkdir d10/d32/d4b/d5e 0 2024-04-02T04:47:44.420 INFO:tasks.workunit.client.0.smithi070.stdout:3/270: write d10/f29 [2855449,31690] 0 2024-04-02T04:47:44.421 INFO:tasks.workunit.client.0.smithi070.stdout:3/271: stat fd 0 2024-04-02T04:47:44.424 INFO:tasks.workunit.client.0.smithi070.stdout:9/260: mkdir d4/d13/d16/d19/d2f/d39/d57 0 2024-04-02T04:47:44.424 INFO:tasks.workunit.client.1.smithi179.stdout:4/341: mkdir d1/d3/df/d10/d2f/d60/d71 0 2024-04-02T04:47:44.426 INFO:tasks.workunit.client.1.smithi179.stdout:2/163: mknod dc/d29/d31/c33 0 2024-04-02T04:47:44.426 INFO:tasks.workunit.client.1.smithi179.stdout:2/164: truncate dc/f25 202227 0 2024-04-02T04:47:44.430 INFO:tasks.workunit.client.0.smithi070.stdout:4/224: rename d1/d3/f2c to d1/d3/d9/d41/f45 0 2024-04-02T04:47:44.431 INFO:tasks.workunit.client.0.smithi070.stdout:4/225: chown d1/d3/c3a 14136942 1 2024-04-02T04:47:44.435 INFO:tasks.workunit.client.0.smithi070.stdout:1/231: dread d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:44.435 INFO:tasks.workunit.client.0.smithi070.stdout:7/278: dwrite d7/dd/f2e [0,4194304] 0 2024-04-02T04:47:44.438 INFO:tasks.workunit.client.1.smithi179.stdout:5/196: truncate d8/d1b/f1f 586675 0 2024-04-02T04:47:44.439 INFO:tasks.workunit.client.0.smithi070.stdout:7/279: write d7/dd/df/f4d [261612,107402] 0 2024-04-02T04:47:44.440 INFO:tasks.workunit.client.0.smithi070.stdout:4/226: dwrite d1/d3/d9/f17 [0,4194304] 0 2024-04-02T04:47:44.442 INFO:tasks.workunit.client.0.smithi070.stdout:9/261: creat d4/d10/d11/f58 x:0 0 0 2024-04-02T04:47:44.443 INFO:tasks.workunit.client.0.smithi070.stdout:9/262: read d4/f8 [1249084,22849] 0 2024-04-02T04:47:44.443 INFO:tasks.workunit.client.1.smithi179.stdout:0/319: truncate d1/da/df/f20 1184012 0 2024-04-02T04:47:44.445 INFO:tasks.workunit.client.0.smithi070.stdout:6/188: creat d0/d3/f3a x:0 0 0 2024-04-02T04:47:44.446 INFO:tasks.workunit.client.0.smithi070.stdout:6/189: chown d0/f1c 24458703 1 2024-04-02T04:47:44.447 INFO:tasks.workunit.client.1.smithi179.stdout:6/278: symlink d3/d12/d23/d39/d3f/l63 0 2024-04-02T04:47:44.449 INFO:tasks.workunit.client.1.smithi179.stdout:6/279: chown d3/d12/d23/d2f/f4a 104 1 2024-04-02T04:47:44.449 INFO:tasks.workunit.client.1.smithi179.stdout:3/222: link d8/d26/d36/l42 d8/d26/l4b 0 2024-04-02T04:47:44.454 INFO:tasks.workunit.client.1.smithi179.stdout:9/290: truncate d0/d1/d11/d29/f34 541748 0 2024-04-02T04:47:44.454 INFO:tasks.workunit.client.1.smithi179.stdout:2/165: rmdir dc/d29 39 2024-04-02T04:47:44.455 INFO:tasks.workunit.client.0.smithi070.stdout:5/300: link d5/f1d d5/d17/d50/f5a 0 2024-04-02T04:47:44.455 INFO:tasks.workunit.client.1.smithi179.stdout:7/321: link d1/d9/ce d1/dd/d16/c77 0 2024-04-02T04:47:44.457 INFO:tasks.workunit.client.0.smithi070.stdout:2/341: dwrite d2/fb [4194304,4194304] 0 2024-04-02T04:47:44.457 INFO:tasks.workunit.client.1.smithi179.stdout:5/197: chown d8/d1b/d23/l31 3224678 1 2024-04-02T04:47:44.458 INFO:tasks.workunit.client.1.smithi179.stdout:5/198: chown d8/d1b/c29 1484082416 1 2024-04-02T04:47:44.459 INFO:tasks.workunit.client.1.smithi179.stdout:5/199: fsync d8/d1b/d23/f2b 0 2024-04-02T04:47:44.459 INFO:tasks.workunit.client.0.smithi070.stdout:1/232: creat d0/d9/d1d/d22/d2c/f4a x:0 0 0 2024-04-02T04:47:44.460 INFO:tasks.workunit.client.0.smithi070.stdout:5/301: dread d5/f2f [0,4194304] 0 2024-04-02T04:47:44.461 INFO:tasks.workunit.client.0.smithi070.stdout:3/272: mknod d10/d17/c5f 0 2024-04-02T04:47:44.464 INFO:tasks.workunit.client.0.smithi070.stdout:7/280: symlink d7/d4e/l5c 0 2024-04-02T04:47:44.466 INFO:tasks.workunit.client.0.smithi070.stdout:3/273: dread d10/f2c [0,4194304] 0 2024-04-02T04:47:44.466 INFO:tasks.workunit.client.1.smithi179.stdout:6/280: mkdir d3/d2d/d38/d64 0 2024-04-02T04:47:44.466 INFO:tasks.workunit.client.1.smithi179.stdout:6/281: chown d3/d12/f25 4895378 1 2024-04-02T04:47:44.468 INFO:tasks.workunit.client.1.smithi179.stdout:3/223: mknod d8/d26/d36/c4c 0 2024-04-02T04:47:44.469 INFO:tasks.workunit.client.1.smithi179.stdout:3/224: stat d8/d26/f2c 0 2024-04-02T04:47:44.471 INFO:tasks.workunit.client.0.smithi070.stdout:9/263: fdatasync d4/d10/f18 0 2024-04-02T04:47:44.472 INFO:tasks.workunit.client.0.smithi070.stdout:9/264: dread - d4/d10/d12/d27/f30 zero size 2024-04-02T04:47:44.477 INFO:tasks.workunit.client.1.smithi179.stdout:7/322: stat d1/dd/f20 0 2024-04-02T04:47:44.478 INFO:tasks.workunit.client.0.smithi070.stdout:4/227: write d1/fb [4609199,54019] 0 2024-04-02T04:47:44.481 INFO:tasks.workunit.client.1.smithi179.stdout:4/342: truncate d1/d3/dc/fe 3016984 0 2024-04-02T04:47:44.482 INFO:tasks.workunit.client.1.smithi179.stdout:4/343: truncate d1/d3/df/f57 1847639 0 2024-04-02T04:47:44.482 INFO:tasks.workunit.client.1.smithi179.stdout:5/200: unlink d8/l17 0 2024-04-02T04:47:44.483 INFO:tasks.workunit.client.1.smithi179.stdout:4/344: truncate d1/d3/fa 1321442 0 2024-04-02T04:47:44.483 INFO:tasks.workunit.client.0.smithi070.stdout:5/302: symlink d5/d17/d27/d2c/d35/l5b 0 2024-04-02T04:47:44.485 INFO:tasks.workunit.client.0.smithi070.stdout:7/281: dread d7/d9/f17 [0,4194304] 0 2024-04-02T04:47:44.485 INFO:tasks.workunit.client.1.smithi179.stdout:5/201: dread f6 [0,4194304] 0 2024-04-02T04:47:44.485 INFO:tasks.workunit.client.0.smithi070.stdout:3/274: symlink d10/d32/d4b/l60 0 2024-04-02T04:47:44.486 INFO:tasks.workunit.client.0.smithi070.stdout:7/282: write d7/dd/d35/f59 [706102,49907] 0 2024-04-02T04:47:44.486 INFO:tasks.workunit.client.1.smithi179.stdout:3/225: creat d8/d26/d36/f4d x:0 0 0 2024-04-02T04:47:44.487 INFO:tasks.workunit.client.0.smithi070.stdout:3/275: write d10/d32/f36 [1194257,103828] 0 2024-04-02T04:47:44.488 INFO:tasks.workunit.client.0.smithi070.stdout:3/276: truncate d10/f4f 235483 0 2024-04-02T04:47:44.488 INFO:tasks.workunit.client.0.smithi070.stdout:3/277: fdatasync d10/f4f 0 2024-04-02T04:47:44.489 INFO:tasks.workunit.client.0.smithi070.stdout:6/190: symlink d0/d3/d19/d31/d37/l3b 0 2024-04-02T04:47:44.490 INFO:tasks.workunit.client.0.smithi070.stdout:6/191: dread - d0/f2d zero size 2024-04-02T04:47:44.490 INFO:tasks.workunit.client.0.smithi070.stdout:3/278: chown d10/d32/d3f/f3a 11157 1 2024-04-02T04:47:44.490 INFO:tasks.workunit.client.0.smithi070.stdout:6/192: dread - d0/f2d zero size 2024-04-02T04:47:44.491 INFO:tasks.workunit.client.0.smithi070.stdout:9/265: rename d4/d13/d16/d19/f22 to d4/d13/d16/d19/d53/f59 0 2024-04-02T04:47:44.491 INFO:tasks.workunit.client.1.smithi179.stdout:2/166: write dc/d16/f2b [379912,60154] 0 2024-04-02T04:47:44.492 INFO:tasks.workunit.client.1.smithi179.stdout:6/282: dwrite d3/d2c/f5b [0,4194304] 0 2024-04-02T04:47:44.492 INFO:tasks.workunit.client.1.smithi179.stdout:5/202: dwrite d8/f3e [0,4194304] 0 2024-04-02T04:47:44.493 INFO:tasks.workunit.client.0.smithi070.stdout:9/266: write f3 [379588,61049] 0 2024-04-02T04:47:44.494 INFO:tasks.workunit.client.1.smithi179.stdout:6/283: write d3/d12/f34 [6243499,125995] 0 2024-04-02T04:47:44.496 INFO:tasks.workunit.client.0.smithi070.stdout:1/233: mkdir d0/d7/d28/d4b 0 2024-04-02T04:47:44.498 INFO:tasks.workunit.client.0.smithi070.stdout:1/234: readlink d0/d9/d1d/lb 0 2024-04-02T04:47:44.498 INFO:tasks.workunit.client.1.smithi179.stdout:4/345: mknod d1/d3/dc/d26/c72 0 2024-04-02T04:47:44.498 INFO:tasks.workunit.client.1.smithi179.stdout:4/346: readlink d1/l53 0 2024-04-02T04:47:44.499 INFO:tasks.workunit.client.0.smithi070.stdout:3/279: dwrite d10/d32/f55 [0,4194304] 0 2024-04-02T04:47:44.499 INFO:tasks.workunit.client.0.smithi070.stdout:1/235: write d0/d7/d1a/f2e [3873199,79977] 0 2024-04-02T04:47:44.500 INFO:tasks.workunit.client.1.smithi179.stdout:9/291: rename d0/d19/f4e to d0/f57 0 2024-04-02T04:47:44.503 INFO:tasks.workunit.client.1.smithi179.stdout:3/226: creat d8/d26/f4e x:0 0 0 2024-04-02T04:47:44.508 INFO:tasks.workunit.client.1.smithi179.stdout:0/320: link d1/db/c3d d1/da/c60 0 2024-04-02T04:47:44.510 INFO:tasks.workunit.client.0.smithi070.stdout:6/193: rmdir d0 39 2024-04-02T04:47:44.511 INFO:tasks.workunit.client.1.smithi179.stdout:0/321: write d1/f57 [1011298,117082] 0 2024-04-02T04:47:44.511 INFO:tasks.workunit.client.1.smithi179.stdout:5/203: chown d8/d1b/d23/c39 22 1 2024-04-02T04:47:44.512 INFO:tasks.workunit.client.1.smithi179.stdout:3/227: dwrite d8/d26/f33 [0,4194304] 0 2024-04-02T04:47:44.513 INFO:tasks.workunit.client.1.smithi179.stdout:5/204: fdatasync d8/f10 0 2024-04-02T04:47:44.513 INFO:tasks.workunit.client.1.smithi179.stdout:0/322: chown d1/f57 843 1 2024-04-02T04:47:44.514 INFO:tasks.workunit.client.1.smithi179.stdout:5/205: read - d8/d1b/d23/f2f zero size 2024-04-02T04:47:44.517 INFO:tasks.workunit.client.1.smithi179.stdout:7/323: rename d1/dd/d16/c54 to d1/d9/c78 0 2024-04-02T04:47:44.519 INFO:tasks.workunit.client.1.smithi179.stdout:7/324: write d1/dd/d16/f2c [1984404,127268] 0 2024-04-02T04:47:44.520 INFO:tasks.workunit.client.0.smithi070.stdout:9/267: creat d4/d13/d16/d19/d53/f5a x:0 0 0 2024-04-02T04:47:44.521 INFO:tasks.workunit.client.1.smithi179.stdout:9/292: write d0/f57 [452823,57583] 0 2024-04-02T04:47:44.522 INFO:tasks.workunit.client.1.smithi179.stdout:8/249: sync 2024-04-02T04:47:44.526 INFO:tasks.workunit.client.0.smithi070.stdout:9/268: dwrite d4/d13/f26 [0,4194304] 0 2024-04-02T04:47:44.527 INFO:tasks.workunit.client.0.smithi070.stdout:9/269: stat d4/d10/d12/d55 0 2024-04-02T04:47:44.530 INFO:tasks.workunit.client.0.smithi070.stdout:5/303: mknod d5/d14/c5c 0 2024-04-02T04:47:44.531 INFO:tasks.workunit.client.1.smithi179.stdout:3/228: unlink c1 0 2024-04-02T04:47:44.532 INFO:tasks.workunit.client.0.smithi070.stdout:3/280: dread f1 [0,4194304] 0 2024-04-02T04:47:44.534 INFO:tasks.workunit.client.0.smithi070.stdout:6/194: write d0/f15 [1192,46996] 0 2024-04-02T04:47:44.535 INFO:tasks.workunit.client.1.smithi179.stdout:5/206: symlink d8/d1b/d23/l3f 0 2024-04-02T04:47:44.535 INFO:tasks.workunit.client.0.smithi070.stdout:6/195: write d0/d20/f34 [112414,6621] 0 2024-04-02T04:47:44.536 INFO:tasks.workunit.client.0.smithi070.stdout:2/342: getdents d2/d10/d21/d3d 0 2024-04-02T04:47:44.538 INFO:tasks.workunit.client.0.smithi070.stdout:4/228: creat d1/f46 x:0 0 0 2024-04-02T04:47:44.540 INFO:tasks.workunit.client.1.smithi179.stdout:6/284: rename d3/d12/f34 to d3/d2d/f65 0 2024-04-02T04:47:44.542 INFO:tasks.workunit.client.1.smithi179.stdout:7/325: symlink d1/dd/d16/l79 0 2024-04-02T04:47:44.543 INFO:tasks.workunit.client.1.smithi179.stdout:2/167: write dc/d16/f15 [1722789,39272] 0 2024-04-02T04:47:44.544 INFO:tasks.workunit.client.1.smithi179.stdout:2/168: stat dc/d16/d1e/c20 0 2024-04-02T04:47:44.545 INFO:tasks.workunit.client.1.smithi179.stdout:9/293: mkdir d0/d1/d12/d58 0 2024-04-02T04:47:44.547 INFO:tasks.workunit.client.1.smithi179.stdout:3/229: creat d8/d26/d36/f4f x:0 0 0 2024-04-02T04:47:44.548 INFO:tasks.workunit.client.1.smithi179.stdout:3/230: chown d8/f22 2470654 1 2024-04-02T04:47:44.548 INFO:tasks.workunit.client.1.smithi179.stdout:8/250: dwrite d3/d32/f47 [4194304,4194304] 0 2024-04-02T04:47:44.549 INFO:tasks.workunit.client.1.smithi179.stdout:3/231: fsync d8/d26/f2c 0 2024-04-02T04:47:44.550 INFO:tasks.workunit.client.1.smithi179.stdout:3/232: fdatasync d8/d26/d36/f41 0 2024-04-02T04:47:44.552 INFO:tasks.workunit.client.0.smithi070.stdout:1/236: rmdir d0/d7/d28/d4b 0 2024-04-02T04:47:44.553 INFO:tasks.workunit.client.0.smithi070.stdout:1/237: chown d0/d9/d1d/f40 4 1 2024-04-02T04:47:44.554 INFO:tasks.workunit.client.0.smithi070.stdout:5/304: creat d5/d57/f5d x:0 0 0 2024-04-02T04:47:44.554 INFO:tasks.workunit.client.0.smithi070.stdout:5/305: stat d5/d14 0 2024-04-02T04:47:44.555 INFO:tasks.workunit.client.0.smithi070.stdout:5/306: chown d5/d9/d10/c1b 93 1 2024-04-02T04:47:44.555 INFO:tasks.workunit.client.1.smithi179.stdout:6/285: mkdir d3/d12/d23/d39/d4e/d66 0 2024-04-02T04:47:44.556 INFO:tasks.workunit.client.0.smithi070.stdout:7/283: getdents d7 0 2024-04-02T04:47:44.557 INFO:tasks.workunit.client.1.smithi179.stdout:3/233: dwrite d8/f11 [0,4194304] 0 2024-04-02T04:47:44.558 INFO:tasks.workunit.client.0.smithi070.stdout:3/281: rename d10/d17/f27 to d10/d32/d4b/f61 0 2024-04-02T04:47:44.561 INFO:tasks.workunit.client.0.smithi070.stdout:6/196: mknod d0/d30/c3c 0 2024-04-02T04:47:44.564 INFO:tasks.workunit.client.1.smithi179.stdout:4/347: getdents d1/d15 0 2024-04-02T04:47:44.564 INFO:tasks.workunit.client.1.smithi179.stdout:4/348: fsync d1/f65 0 2024-04-02T04:47:44.567 INFO:tasks.workunit.client.0.smithi070.stdout:7/284: mknod d7/dd/d35/c5d 0 2024-04-02T04:47:44.568 INFO:tasks.workunit.client.1.smithi179.stdout:5/207: write d8/ff [510334,2734] 0 2024-04-02T04:47:44.568 INFO:tasks.workunit.client.0.smithi070.stdout:7/285: readlink d7/l14 0 2024-04-02T04:47:44.569 INFO:tasks.workunit.client.1.smithi179.stdout:0/323: truncate d1/da/d28/d37/f49 1862188 0 2024-04-02T04:47:44.572 INFO:tasks.workunit.client.0.smithi070.stdout:4/229: rename d1/d3/c42 to d1/d3/d9/c47 0 2024-04-02T04:47:44.572 INFO:tasks.workunit.client.1.smithi179.stdout:6/286: readlink d3/d12/l58 0 2024-04-02T04:47:44.573 INFO:tasks.workunit.client.1.smithi179.stdout:1/293: sync 2024-04-02T04:47:44.574 INFO:tasks.workunit.client.1.smithi179.stdout:7/326: creat d1/dd/d16/d74/f7a x:0 0 0 2024-04-02T04:47:44.574 INFO:tasks.workunit.client.1.smithi179.stdout:6/287: write d3/d12/d23/d2f/f3c [254573,71245] 0 2024-04-02T04:47:44.574 INFO:tasks.workunit.client.0.smithi070.stdout:7/286: dwrite d7/d2f/f3b [0,4194304] 0 2024-04-02T04:47:44.574 INFO:tasks.workunit.client.0.smithi070.stdout:6/197: symlink d0/d30/l3d 0 2024-04-02T04:47:44.576 INFO:tasks.workunit.client.1.smithi179.stdout:9/294: dread d0/d19/f56 [0,4194304] 0 2024-04-02T04:47:44.576 INFO:tasks.workunit.client.1.smithi179.stdout:3/234: mknod d8/d26/c50 0 2024-04-02T04:47:44.579 INFO:tasks.workunit.client.0.smithi070.stdout:1/238: mkdir d0/d4c 0 2024-04-02T04:47:44.579 INFO:tasks.workunit.client.0.smithi070.stdout:1/239: dread - d0/d7/f43 zero size 2024-04-02T04:47:44.580 INFO:tasks.workunit.client.1.smithi179.stdout:2/169: creat dc/d29/f34 x:0 0 0 2024-04-02T04:47:44.581 INFO:tasks.workunit.client.0.smithi070.stdout:2/343: dwrite d2/d10/d21/f31 [0,4194304] 0 2024-04-02T04:47:44.583 INFO:tasks.workunit.client.1.smithi179.stdout:5/208: dread d8/ff [0,4194304] 0 2024-04-02T04:47:44.584 INFO:tasks.workunit.client.0.smithi070.stdout:2/344: write d2/d10/d21/d2b/f67 [272803,56599] 0 2024-04-02T04:47:44.584 INFO:tasks.workunit.client.1.smithi179.stdout:8/251: link d3/f27 d3/f59 0 2024-04-02T04:47:44.584 INFO:tasks.workunit.client.1.smithi179.stdout:5/209: chown d8/d1b/d23/l31 16399005 1 2024-04-02T04:47:44.585 INFO:tasks.workunit.client.1.smithi179.stdout:5/210: fsync d8/f3e 0 2024-04-02T04:47:44.585 INFO:tasks.workunit.client.1.smithi179.stdout:5/211: readlink d8/d1b/l21 0 2024-04-02T04:47:44.589 INFO:tasks.workunit.client.1.smithi179.stdout:4/349: unlink d1/f2b 0 2024-04-02T04:47:44.590 INFO:tasks.workunit.client.0.smithi070.stdout:4/230: creat d1/d3/d9/d41/f48 x:0 0 0 2024-04-02T04:47:44.591 INFO:tasks.workunit.client.0.smithi070.stdout:4/231: stat d1/d3/c3b 0 2024-04-02T04:47:44.595 INFO:tasks.workunit.client.0.smithi070.stdout:1/240: dread d0/d9/d1d/f3d [0,4194304] 0 2024-04-02T04:47:44.596 INFO:tasks.workunit.client.0.smithi070.stdout:1/241: write d0/d7/d1a/f2e [675819,6569] 0 2024-04-02T04:47:44.596 INFO:tasks.workunit.client.1.smithi179.stdout:6/288: symlink d3/d12/d44/l67 0 2024-04-02T04:47:44.597 INFO:tasks.workunit.client.1.smithi179.stdout:4/350: dwrite d1/d3/dc/d27/d62/f69 [0,4194304] 0 2024-04-02T04:47:44.598 INFO:tasks.workunit.client.0.smithi070.stdout:5/307: creat d5/d9/f5e x:0 0 0 2024-04-02T04:47:44.598 INFO:tasks.workunit.client.1.smithi179.stdout:6/289: chown d3/d12/d23/d39/d3f/l5c 4 1 2024-04-02T04:47:44.599 INFO:tasks.workunit.client.1.smithi179.stdout:1/294: creat d3/d5/d2f/d36/f67 x:0 0 0 2024-04-02T04:47:44.601 INFO:tasks.workunit.client.0.smithi070.stdout:2/345: symlink d2/d10/d53/l6d 0 2024-04-02T04:47:44.601 INFO:tasks.workunit.client.0.smithi070.stdout:2/346: chown d2/d24/f2d 204 1 2024-04-02T04:47:44.602 INFO:tasks.workunit.client.1.smithi179.stdout:3/235: mknod d8/d26/d36/c51 0 2024-04-02T04:47:44.604 INFO:tasks.workunit.client.1.smithi179.stdout:3/236: write d8/d26/d36/f47 [814437,42674] 0 2024-04-02T04:47:44.604 INFO:tasks.workunit.client.0.smithi070.stdout:3/282: link d10/d32/f59 d10/d17/d4c/f62 0 2024-04-02T04:47:44.604 INFO:tasks.workunit.client.1.smithi179.stdout:3/237: fdatasync d8/d26/d36/f4d 0 2024-04-02T04:47:44.605 INFO:tasks.workunit.client.1.smithi179.stdout:2/170: mknod dc/d16/d1e/c35 0 2024-04-02T04:47:44.605 INFO:tasks.workunit.client.1.smithi179.stdout:3/238: fsync d8/d26/d36/f4f 0 2024-04-02T04:47:44.606 INFO:tasks.workunit.client.0.smithi070.stdout:4/232: symlink d1/d3/d9/l49 0 2024-04-02T04:47:44.606 INFO:tasks.workunit.client.1.smithi179.stdout:3/239: write d8/f1b [3522437,117844] 0 2024-04-02T04:47:44.607 INFO:tasks.workunit.client.0.smithi070.stdout:6/198: mkdir d0/d3/d19/d3e 0 2024-04-02T04:47:44.608 INFO:tasks.workunit.client.1.smithi179.stdout:8/252: mkdir d3/d32/d21/d45/d5a 0 2024-04-02T04:47:44.609 INFO:tasks.workunit.client.1.smithi179.stdout:8/253: chown d3/f59 8 1 2024-04-02T04:47:44.609 INFO:tasks.workunit.client.1.smithi179.stdout:2/171: dread dc/d16/f2b [0,4194304] 0 2024-04-02T04:47:44.613 INFO:tasks.workunit.client.0.smithi070.stdout:1/242: mkdir d0/d9/d1d/d22/d2c/d4d 0 2024-04-02T04:47:44.617 INFO:tasks.workunit.client.1.smithi179.stdout:0/324: write d1/f22 [56579,40026] 0 2024-04-02T04:47:44.620 INFO:tasks.workunit.client.0.smithi070.stdout:3/283: fdatasync f8 0 2024-04-02T04:47:44.621 INFO:tasks.workunit.client.1.smithi179.stdout:4/351: readlink d1/d3/l50 0 2024-04-02T04:47:44.622 INFO:tasks.workunit.client.0.smithi070.stdout:3/284: write d10/d32/f55 [2885346,107782] 0 2024-04-02T04:47:44.622 INFO:tasks.workunit.client.0.smithi070.stdout:8/325: sync 2024-04-02T04:47:44.623 INFO:tasks.workunit.client.0.smithi070.stdout:6/199: rmdir d0/d3/d19/d31/d37 39 2024-04-02T04:47:44.623 INFO:tasks.workunit.client.0.smithi070.stdout:3/285: chown d10/f29 225438092 1 2024-04-02T04:47:44.624 INFO:tasks.workunit.client.0.smithi070.stdout:2/347: dwrite d2/f8 [0,4194304] 0 2024-04-02T04:47:44.625 INFO:tasks.workunit.client.1.smithi179.stdout:1/295: mknod d3/d5/de/c68 0 2024-04-02T04:47:44.625 INFO:tasks.workunit.client.1.smithi179.stdout:1/296: fdatasync d3/d5/de/f4e 0 2024-04-02T04:47:44.626 INFO:tasks.workunit.client.0.smithi070.stdout:7/287: link d7/d9/f17 d7/d2f/f5e 0 2024-04-02T04:47:44.629 INFO:tasks.workunit.client.0.smithi070.stdout:1/243: creat d0/d9/d1d/f4e x:0 0 0 2024-04-02T04:47:44.633 INFO:tasks.workunit.client.1.smithi179.stdout:3/240: mkdir d8/d26/d52 0 2024-04-02T04:47:44.635 INFO:tasks.workunit.client.1.smithi179.stdout:3/241: write d8/f48 [308832,4875] 0 2024-04-02T04:47:44.639 INFO:tasks.workunit.client.0.smithi070.stdout:5/308: write d5/d9/d10/f11 [1117807,97089] 0 2024-04-02T04:47:44.639 INFO:tasks.workunit.client.0.smithi070.stdout:6/200: chown d0/d3/l29 0 1 2024-04-02T04:47:44.639 INFO:tasks.workunit.client.0.smithi070.stdout:5/309: fsync d5/d17/f53 0 2024-04-02T04:47:44.640 INFO:tasks.workunit.client.1.smithi179.stdout:5/212: dwrite f2 [4194304,4194304] 0 2024-04-02T04:47:44.640 INFO:tasks.workunit.client.1.smithi179.stdout:5/213: readlink l4 0 2024-04-02T04:47:44.641 INFO:tasks.workunit.client.1.smithi179.stdout:7/327: creat d1/f7b x:0 0 0 2024-04-02T04:47:44.642 INFO:tasks.workunit.client.0.smithi070.stdout:3/286: creat d10/d32/f63 x:0 0 0 2024-04-02T04:47:44.644 INFO:tasks.workunit.client.0.smithi070.stdout:2/348: creat d2/d10/d21/d2b/f6e x:0 0 0 2024-04-02T04:47:44.644 INFO:tasks.workunit.client.1.smithi179.stdout:9/295: write d0/d1/d11/d29/f34 [470370,107629] 0 2024-04-02T04:47:44.645 INFO:tasks.workunit.client.0.smithi070.stdout:2/349: write d2/d10/d21/d2b/f67 [207710,26844] 0 2024-04-02T04:47:44.646 INFO:tasks.workunit.client.0.smithi070.stdout:8/326: truncate d2/d8/d23/d24/f34 4998423 0 2024-04-02T04:47:44.647 INFO:tasks.workunit.client.1.smithi179.stdout:4/352: symlink d1/d3/dc/d27/d62/l73 0 2024-04-02T04:47:44.648 INFO:tasks.workunit.client.0.smithi070.stdout:1/244: mkdir d0/d9/d1d/d22/d2c/d4f 0 2024-04-02T04:47:44.649 INFO:tasks.workunit.client.0.smithi070.stdout:1/245: chown d0/d7/d1a 6842 1 2024-04-02T04:47:44.650 INFO:tasks.workunit.client.0.smithi070.stdout:1/246: write d0/d7/d1a/f2e [17292,115259] 0 2024-04-02T04:47:44.652 INFO:tasks.workunit.client.0.smithi070.stdout:5/310: creat d5/d17/d50/f5f x:0 0 0 2024-04-02T04:47:44.652 INFO:tasks.workunit.client.1.smithi179.stdout:9/296: dwrite d0/d1/d11/d29/f34 [0,4194304] 0 2024-04-02T04:47:44.653 INFO:tasks.workunit.client.1.smithi179.stdout:9/297: readlink d0/d1/d11/l54 0 2024-04-02T04:47:44.654 INFO:tasks.workunit.client.0.smithi070.stdout:4/233: truncate d1/f26 2290826 0 2024-04-02T04:47:44.655 INFO:tasks.workunit.client.0.smithi070.stdout:3/287: chown d10/d17/l31 1719084351 1 2024-04-02T04:47:44.656 INFO:tasks.workunit.client.1.smithi179.stdout:3/242: mknod d8/c53 0 2024-04-02T04:47:44.657 INFO:tasks.workunit.client.1.smithi179.stdout:3/243: write d8/f32 [2205485,113188] 0 2024-04-02T04:47:44.658 INFO:tasks.workunit.client.0.smithi070.stdout:2/350: read d2/f14 [269006,79991] 0 2024-04-02T04:47:44.660 INFO:tasks.workunit.client.1.smithi179.stdout:8/254: unlink d3/d1b/d2a/d41/f42 0 2024-04-02T04:47:44.660 INFO:tasks.workunit.client.0.smithi070.stdout:1/247: fsync d0/d7/d1a/f2e 0 2024-04-02T04:47:44.661 INFO:tasks.workunit.client.0.smithi070.stdout:7/288: mknod d7/dd/d35/d57/c5f 0 2024-04-02T04:47:44.661 INFO:tasks.workunit.client.1.smithi179.stdout:8/255: read d3/d32/d21/d22/d33/d3c/f44 [905949,9725] 0 2024-04-02T04:47:44.662 INFO:tasks.workunit.client.0.smithi070.stdout:3/288: dread d10/d32/f55 [0,4194304] 0 2024-04-02T04:47:44.662 INFO:tasks.workunit.client.1.smithi179.stdout:8/256: write d3/d32/d30/f3b [655922,52568] 0 2024-04-02T04:47:44.663 INFO:tasks.workunit.client.0.smithi070.stdout:8/327: truncate d2/d3/d52/f3c 112766 0 2024-04-02T04:47:44.666 INFO:tasks.workunit.client.1.smithi179.stdout:0/325: creat d1/d5b/f61 x:0 0 0 2024-04-02T04:47:44.669 INFO:tasks.workunit.client.0.smithi070.stdout:5/311: creat d5/d14/f60 x:0 0 0 2024-04-02T04:47:44.670 INFO:tasks.workunit.client.0.smithi070.stdout:4/234: rename d1 to d1/d4a 22 2024-04-02T04:47:44.672 INFO:tasks.workunit.client.0.smithi070.stdout:2/351: fsync d2/f1b 0 2024-04-02T04:47:44.673 INFO:tasks.workunit.client.1.smithi179.stdout:9/298: write d0/d19/d1c/f52 [975854,37486] 0 2024-04-02T04:47:44.676 INFO:tasks.workunit.client.0.smithi070.stdout:2/352: dread d2/d10/d21/d3d/f5d [0,4194304] 0 2024-04-02T04:47:44.677 INFO:tasks.workunit.client.1.smithi179.stdout:3/244: mknod d8/c54 0 2024-04-02T04:47:44.678 INFO:tasks.workunit.client.0.smithi070.stdout:8/328: mknod d2/d3/d35/d3a/d4f/d59/c6c 0 2024-04-02T04:47:44.681 INFO:tasks.workunit.client.0.smithi070.stdout:6/201: creat d0/d3/d19/f3f x:0 0 0 2024-04-02T04:47:44.683 INFO:tasks.workunit.client.1.smithi179.stdout:3/245: dread f5 [0,4194304] 0 2024-04-02T04:47:44.683 INFO:tasks.workunit.client.1.smithi179.stdout:8/257: mknod d3/d31/c5b 0 2024-04-02T04:47:44.684 INFO:tasks.workunit.client.1.smithi179.stdout:3/246: rename d8 to d8/d26/d52/d55 22 2024-04-02T04:47:44.684 INFO:tasks.workunit.client.0.smithi070.stdout:5/312: mknod d5/d57/c61 0 2024-04-02T04:47:44.689 INFO:tasks.workunit.client.1.smithi179.stdout:6/290: getdents d3/d12 0 2024-04-02T04:47:44.691 INFO:tasks.workunit.client.1.smithi179.stdout:4/353: creat d1/d3/df/d10/d2f/d60/d71/f74 x:0 0 0 2024-04-02T04:47:44.691 INFO:tasks.workunit.client.1.smithi179.stdout:6/291: write d3/d12/f46 [994020,28031] 0 2024-04-02T04:47:44.692 INFO:tasks.workunit.client.1.smithi179.stdout:1/297: dwrite d3/d5/f60 [0,4194304] 0 2024-04-02T04:47:44.693 INFO:tasks.workunit.client.1.smithi179.stdout:1/298: readlink d3/l46 0 2024-04-02T04:47:44.693 INFO:tasks.workunit.client.0.smithi070.stdout:1/248: creat d0/d9/d1d/d22/d2c/d4f/f50 x:0 0 0 2024-04-02T04:47:44.693 INFO:tasks.workunit.client.1.smithi179.stdout:1/299: stat d3/d5/d2f/d4f 0 2024-04-02T04:47:44.694 INFO:tasks.workunit.client.0.smithi070.stdout:1/249: fdatasync d0/d7/f8 0 2024-04-02T04:47:44.694 INFO:tasks.workunit.client.0.smithi070.stdout:1/250: chown d0/d7/c2f 1532613 1 2024-04-02T04:47:44.696 INFO:tasks.workunit.client.1.smithi179.stdout:1/300: fdatasync d3/d5/d2f/d4f/f4c 0 2024-04-02T04:47:44.696 INFO:tasks.workunit.client.1.smithi179.stdout:1/301: write d3/d5/d2f/d36/f67 [280503,75617] 0 2024-04-02T04:47:44.697 INFO:tasks.workunit.client.0.smithi070.stdout:2/353: mknod d2/d10/d21/d3d/d42/c6f 0 2024-04-02T04:47:44.700 INFO:tasks.workunit.client.1.smithi179.stdout:2/172: link dc/f17 dc/d16/d1e/f36 0 2024-04-02T04:47:44.700 INFO:tasks.workunit.client.0.smithi070.stdout:6/202: stat d0/d3/l8 0 2024-04-02T04:47:44.701 INFO:tasks.workunit.client.1.smithi179.stdout:5/214: write d8/d1b/f3b [388146,47502] 0 2024-04-02T04:47:44.702 INFO:tasks.workunit.client.1.smithi179.stdout:3/247: creat d8/d26/f56 x:0 0 0 2024-04-02T04:47:44.703 INFO:tasks.workunit.client.0.smithi070.stdout:5/313: mkdir d5/d9/d10/d62 0 2024-04-02T04:47:44.704 INFO:tasks.workunit.client.0.smithi070.stdout:4/235: mknod d1/d3/c4b 0 2024-04-02T04:47:44.706 INFO:tasks.workunit.client.0.smithi070.stdout:4/236: read d1/d3/d9/d41/f45 [2842638,3050] 0 2024-04-02T04:47:44.708 INFO:tasks.workunit.client.1.smithi179.stdout:9/299: write d0/d1/f4 [3402884,101117] 0 2024-04-02T04:47:44.709 INFO:tasks.workunit.client.1.smithi179.stdout:9/300: write d0/d1/d12/fe [4825012,104735] 0 2024-04-02T04:47:44.710 INFO:tasks.workunit.client.1.smithi179.stdout:4/354: stat d1/d3/f1b 0 2024-04-02T04:47:44.712 INFO:tasks.workunit.client.1.smithi179.stdout:9/301: read d0/d1/d12/f25 [2122378,87909] 0 2024-04-02T04:47:44.712 INFO:tasks.workunit.client.1.smithi179.stdout:6/292: mknod d3/d12/d23/d26/c68 0 2024-04-02T04:47:44.714 INFO:tasks.workunit.client.0.smithi070.stdout:3/289: creat d10/d17/d51/f64 x:0 0 0 2024-04-02T04:47:44.714 INFO:tasks.workunit.client.0.smithi070.stdout:4/237: dread d1/d3/d9/f25 [0,4194304] 0 2024-04-02T04:47:44.715 INFO:tasks.workunit.client.1.smithi179.stdout:1/302: creat d3/d5/d2f/d4f/d42/d53/d5b/f69 x:0 0 0 2024-04-02T04:47:44.715 INFO:tasks.workunit.client.0.smithi070.stdout:4/238: chown d1/d3/d9/f24 28 1 2024-04-02T04:47:44.716 INFO:tasks.workunit.client.1.smithi179.stdout:1/303: chown d3/d5/de/c29 22 1 2024-04-02T04:47:44.716 INFO:tasks.workunit.client.1.smithi179.stdout:1/304: fdatasync d3/d5/f25 0 2024-04-02T04:47:44.716 INFO:tasks.workunit.client.0.smithi070.stdout:4/239: fdatasync d1/d3/f30 0 2024-04-02T04:47:44.716 INFO:tasks.workunit.client.0.smithi070.stdout:1/251: rename d0/d9/d1d/d22/f46 to d0/d4c/f51 0 2024-04-02T04:47:44.717 INFO:tasks.workunit.client.1.smithi179.stdout:2/173: fsync dc/f25 0 2024-04-02T04:47:44.718 INFO:tasks.workunit.client.0.smithi070.stdout:4/240: read d1/d3/d9/f1d [1666227,12181] 0 2024-04-02T04:47:44.718 INFO:tasks.workunit.client.0.smithi070.stdout:2/354: creat d2/d12/d59/f70 x:0 0 0 2024-04-02T04:47:44.722 INFO:tasks.workunit.client.1.smithi179.stdout:0/326: creat d1/f62 x:0 0 0 2024-04-02T04:47:44.723 INFO:tasks.workunit.client.0.smithi070.stdout:2/355: dwrite d2/fb [4194304,4194304] 0 2024-04-02T04:47:44.723 INFO:tasks.workunit.client.1.smithi179.stdout:7/328: getdents d1/dd/d16/d74 0 2024-04-02T04:47:44.726 INFO:tasks.workunit.client.0.smithi070.stdout:5/314: read d5/ff [833382,14657] 0 2024-04-02T04:47:44.728 INFO:tasks.workunit.client.1.smithi179.stdout:3/248: write d8/f1c [2031744,21920] 0 2024-04-02T04:47:44.730 INFO:tasks.workunit.client.0.smithi070.stdout:5/315: dread d5/d9/d23/f49 [0,4194304] 0 2024-04-02T04:47:44.731 INFO:tasks.workunit.client.1.smithi179.stdout:4/355: creat d1/d3/d19/f75 x:0 0 0 2024-04-02T04:47:44.732 INFO:tasks.workunit.client.1.smithi179.stdout:9/302: symlink d0/d19/d1c/l59 0 2024-04-02T04:47:44.733 INFO:tasks.workunit.client.1.smithi179.stdout:6/293: creat d3/d12/d23/d26/f69 x:0 0 0 2024-04-02T04:47:44.738 INFO:tasks.workunit.client.0.smithi070.stdout:8/329: creat d2/d8/f6d x:0 0 0 2024-04-02T04:47:44.740 INFO:tasks.workunit.client.0.smithi070.stdout:8/330: write d2/d3/d35/d3a/d4f/d59/f65 [1047026,43800] 0 2024-04-02T04:47:44.741 INFO:tasks.workunit.client.0.smithi070.stdout:8/331: dread - d2/d8/d29/d22/f60 zero size 2024-04-02T04:47:44.741 INFO:tasks.workunit.client.0.smithi070.stdout:2/356: mkdir d2/d10/d21/d71 0 2024-04-02T04:47:44.746 INFO:tasks.workunit.client.0.smithi070.stdout:5/316: rename f2 to d5/d14/f63 0 2024-04-02T04:47:44.747 INFO:tasks.workunit.client.0.smithi070.stdout:1/252: symlink d0/d9/l52 0 2024-04-02T04:47:44.747 INFO:tasks.workunit.client.0.smithi070.stdout:1/253: chown d0/d9/lf 5 1 2024-04-02T04:47:44.748 INFO:tasks.workunit.client.0.smithi070.stdout:1/254: write d0/d7/f48 [724450,45279] 0 2024-04-02T04:47:44.749 INFO:tasks.workunit.client.0.smithi070.stdout:1/255: chown d0/d9/d1d/d22/d2c/d4f/f50 33501 1 2024-04-02T04:47:44.750 INFO:tasks.workunit.client.0.smithi070.stdout:6/203: rmdir d0/d3/d19/d3e 0 2024-04-02T04:47:44.750 INFO:tasks.workunit.client.0.smithi070.stdout:6/204: read - d0/f26 zero size 2024-04-02T04:47:44.751 INFO:tasks.workunit.client.0.smithi070.stdout:2/357: stat d2/l3a 0 2024-04-02T04:47:44.752 INFO:tasks.workunit.client.0.smithi070.stdout:2/358: fdatasync d2/f8 0 2024-04-02T04:47:44.753 INFO:tasks.workunit.client.0.smithi070.stdout:1/256: dread d0/d9/d1d/d22/f2d [0,4194304] 0 2024-04-02T04:47:44.754 INFO:tasks.workunit.client.0.smithi070.stdout:8/332: rename d2/d3/d52/f16 to d2/d3/d35/d3a/d4f/d59/f6e 0 2024-04-02T04:47:44.756 INFO:tasks.workunit.client.0.smithi070.stdout:4/241: creat d1/d3/f4c x:0 0 0 2024-04-02T04:47:44.759 INFO:tasks.workunit.client.0.smithi070.stdout:8/333: symlink d2/d3/d35/d3a/d4f/d59/l6f 0 2024-04-02T04:47:44.781 INFO:tasks.workunit.client.1.smithi179.stdout:1/305: mkdir d3/d5/d2f/d4f/d6a 0 2024-04-02T04:47:44.781 INFO:tasks.workunit.client.1.smithi179.stdout:5/215: symlink d8/d1b/l40 0 2024-04-02T04:47:44.782 INFO:tasks.workunit.client.1.smithi179.stdout:3/249: mkdir d8/d57 0 2024-04-02T04:47:44.782 INFO:tasks.workunit.client.1.smithi179.stdout:0/327: dread d1/da/df/f11 [0,4194304] 0 2024-04-02T04:47:44.783 INFO:tasks.workunit.client.1.smithi179.stdout:7/329: write d1/dd/d23/f71 [429520,70429] 0 2024-04-02T04:47:44.783 INFO:tasks.workunit.client.1.smithi179.stdout:5/216: write d8/d1b/d23/f2b [144713,35120] 0 2024-04-02T04:47:44.784 INFO:tasks.workunit.client.1.smithi179.stdout:6/294: mkdir d3/d12/d6a 0 2024-04-02T04:47:44.785 INFO:tasks.workunit.client.1.smithi179.stdout:3/250: mknod d8/d26/d36/c58 0 2024-04-02T04:47:44.787 INFO:tasks.workunit.client.1.smithi179.stdout:4/356: link d1/d3/dc/d27/d62/f4d d1/d3/dc/d27/d62/f76 0 2024-04-02T04:47:44.789 INFO:tasks.workunit.client.1.smithi179.stdout:4/357: readlink d1/d3/l5c 0 2024-04-02T04:47:44.790 INFO:tasks.workunit.client.1.smithi179.stdout:7/330: mknod d1/dd/d16/d4d/c7c 0 2024-04-02T04:47:44.790 INFO:tasks.workunit.client.1.smithi179.stdout:3/251: creat d8/d26/d36/f59 x:0 0 0 2024-04-02T04:47:44.790 INFO:tasks.workunit.client.1.smithi179.stdout:0/328: mknod d1/da/d28/d5e/c63 0 2024-04-02T04:47:44.792 INFO:tasks.workunit.client.1.smithi179.stdout:6/295: rename d3/d12/d23/d39/d3f/l63 to d3/d12/d23/d39/d4e/d66/l6b 0 2024-04-02T04:47:44.792 INFO:tasks.workunit.client.1.smithi179.stdout:0/329: fdatasync d1/da/d52/f5d 0 2024-04-02T04:47:44.793 INFO:tasks.workunit.client.1.smithi179.stdout:2/174: getdents dc/d16 0 2024-04-02T04:47:44.793 INFO:tasks.workunit.client.1.smithi179.stdout:0/330: fsync d1/f57 0 2024-04-02T04:47:44.794 INFO:tasks.workunit.client.1.smithi179.stdout:2/175: stat dc/f17 0 2024-04-02T04:47:44.794 INFO:tasks.workunit.client.1.smithi179.stdout:5/217: link f1 d8/d1b/d23/d36/f41 0 2024-04-02T04:47:44.795 INFO:tasks.workunit.client.1.smithi179.stdout:7/331: mknod d1/dd/d23/c7d 0 2024-04-02T04:47:44.795 INFO:tasks.workunit.client.1.smithi179.stdout:0/331: write d1/f1e [2471102,98174] 0 2024-04-02T04:47:44.796 INFO:tasks.workunit.client.1.smithi179.stdout:2/176: write dc/d16/f22 [958483,15458] 0 2024-04-02T04:47:44.798 INFO:tasks.workunit.client.1.smithi179.stdout:4/358: symlink d1/d3/df/d10/d2f/d60/l77 0 2024-04-02T04:47:44.798 INFO:tasks.workunit.client.1.smithi179.stdout:9/303: truncate d0/d1/d11/f42 2634441 0 2024-04-02T04:47:44.801 INFO:tasks.workunit.client.1.smithi179.stdout:5/218: mkdir d8/d1b/d23/d42 0 2024-04-02T04:47:44.804 INFO:tasks.workunit.client.1.smithi179.stdout:7/332: mkdir d1/d9/d7e 0 2024-04-02T04:47:44.806 INFO:tasks.workunit.client.1.smithi179.stdout:6/296: rename d3/d12/d23/d39/d3f to d3/d12/d23/d39/d4e/d6c 0 2024-04-02T04:47:44.806 INFO:tasks.workunit.client.1.smithi179.stdout:3/252: dwrite d8/d26/d36/f4f [0,4194304] 0 2024-04-02T04:47:44.808 INFO:tasks.workunit.client.1.smithi179.stdout:0/332: rmdir d1/da/d48 39 2024-04-02T04:47:44.810 INFO:tasks.workunit.client.1.smithi179.stdout:7/333: read d1/d9/f68 [1396142,61394] 0 2024-04-02T04:47:44.814 INFO:tasks.workunit.client.1.smithi179.stdout:3/253: write d8/d26/d36/f41 [1143239,49576] 0 2024-04-02T04:47:44.815 INFO:tasks.workunit.client.1.smithi179.stdout:7/334: chown d1/dd/d16/d4d 17 1 2024-04-02T04:47:44.816 INFO:tasks.workunit.client.1.smithi179.stdout:5/219: rename d8/d1b/d23/l31 to d8/d1b/d23/d32/l43 0 2024-04-02T04:47:44.816 INFO:tasks.workunit.client.1.smithi179.stdout:6/297: read d3/ff [55202,38160] 0 2024-04-02T04:47:44.817 INFO:tasks.workunit.client.1.smithi179.stdout:0/333: mkdir d1/db/d64 0 2024-04-02T04:47:44.817 INFO:tasks.workunit.client.1.smithi179.stdout:7/335: stat d1/dd/d16/d4d 0 2024-04-02T04:47:44.820 INFO:tasks.workunit.client.1.smithi179.stdout:0/334: write d1/d5b/f61 [92074,110690] 0 2024-04-02T04:47:44.820 INFO:tasks.workunit.client.1.smithi179.stdout:5/220: write d8/f2c [57149,109244] 0 2024-04-02T04:47:44.825 INFO:tasks.workunit.client.1.smithi179.stdout:6/298: truncate d3/d42/f54 1015453 0 2024-04-02T04:47:44.826 INFO:tasks.workunit.client.1.smithi179.stdout:3/254: write f5 [2900834,61939] 0 2024-04-02T04:47:44.828 INFO:tasks.workunit.client.1.smithi179.stdout:4/359: truncate d1/d3/df/d10/f11 3200571 0 2024-04-02T04:47:44.829 INFO:tasks.workunit.client.1.smithi179.stdout:3/255: chown d8/d57 1117541 1 2024-04-02T04:47:44.831 INFO:tasks.workunit.client.1.smithi179.stdout:9/304: dread d0/d1/d10/f2b [0,4194304] 0 2024-04-02T04:47:44.831 INFO:tasks.workunit.client.1.smithi179.stdout:4/360: stat d1/d3/df/f5a 0 2024-04-02T04:47:44.833 INFO:tasks.workunit.client.1.smithi179.stdout:2/177: dread f6 [0,4194304] 0 2024-04-02T04:47:44.834 INFO:tasks.workunit.client.1.smithi179.stdout:3/256: unlink d8/f32 0 2024-04-02T04:47:44.835 INFO:tasks.workunit.client.0.smithi070.stdout:1/257: symlink d0/d7/l53 0 2024-04-02T04:47:44.835 INFO:tasks.workunit.client.0.smithi070.stdout:2/359: truncate d2/d10/f20 1256633 0 2024-04-02T04:47:44.835 INFO:tasks.workunit.client.0.smithi070.stdout:8/334: creat d2/d8/d23/d24/d66/f70 x:0 0 0 2024-04-02T04:47:44.836 INFO:tasks.workunit.client.1.smithi179.stdout:0/335: getdents d1 0 2024-04-02T04:47:44.837 INFO:tasks.workunit.client.1.smithi179.stdout:5/221: dwrite d8/f27 [0,4194304] 0 2024-04-02T04:47:44.837 INFO:tasks.workunit.client.0.smithi070.stdout:8/335: chown d2/d3/d52/f33 84647962 1 2024-04-02T04:47:44.837 INFO:tasks.workunit.client.1.smithi179.stdout:3/257: write d8/f1c [4392704,75137] 0 2024-04-02T04:47:44.838 INFO:tasks.workunit.client.0.smithi070.stdout:5/317: getdents d5/d9/d23 0 2024-04-02T04:47:44.839 INFO:tasks.workunit.client.1.smithi179.stdout:3/258: stat d8/d26/f4e 0 2024-04-02T04:47:44.839 INFO:tasks.workunit.client.0.smithi070.stdout:8/336: read d2/d8/d23/d24/f37 [104932,76062] 0 2024-04-02T04:47:44.843 INFO:tasks.workunit.client.1.smithi179.stdout:3/259: readlink d8/d26/d36/l42 0 2024-04-02T04:47:44.844 INFO:tasks.workunit.client.1.smithi179.stdout:6/299: rename d3/l11 to d3/d12/d23/d39/d4e/d66/l6d 0 2024-04-02T04:47:44.844 INFO:tasks.workunit.client.0.smithi070.stdout:4/242: creat d1/d3/f4d x:0 0 0 2024-04-02T04:47:44.844 INFO:tasks.workunit.client.0.smithi070.stdout:5/318: dwrite d5/d17/f32 [0,4194304] 0 2024-04-02T04:47:44.845 INFO:tasks.workunit.client.0.smithi070.stdout:5/319: fsync f4 0 2024-04-02T04:47:44.845 INFO:tasks.workunit.client.0.smithi070.stdout:4/243: write d1/f32 [1646327,74147] 0 2024-04-02T04:47:44.846 INFO:tasks.workunit.client.0.smithi070.stdout:5/320: chown d5/d9/d10/f19 7 1 2024-04-02T04:47:44.846 INFO:tasks.workunit.client.0.smithi070.stdout:4/244: fdatasync d1/d3/f30 0 2024-04-02T04:47:44.846 INFO:tasks.workunit.client.1.smithi179.stdout:9/305: mknod d0/c5a 0 2024-04-02T04:47:44.847 INFO:tasks.workunit.client.0.smithi070.stdout:2/360: mknod d2/d10/d21/d3d/d42/c72 0 2024-04-02T04:47:44.847 INFO:tasks.workunit.client.0.smithi070.stdout:4/245: fdatasync d1/d3/f4d 0 2024-04-02T04:47:44.848 INFO:tasks.workunit.client.0.smithi070.stdout:6/205: link d0/d3/d19/l2b d0/d3/l40 0 2024-04-02T04:47:44.848 INFO:tasks.workunit.client.0.smithi070.stdout:2/361: fsync d2/d10/d21/d2b/f6e 0 2024-04-02T04:47:44.848 INFO:tasks.workunit.client.0.smithi070.stdout:4/246: fdatasync d1/d3/d9/f1a 0 2024-04-02T04:47:44.850 INFO:tasks.workunit.client.1.smithi179.stdout:8/258: sync 2024-04-02T04:47:44.850 INFO:tasks.workunit.client.1.smithi179.stdout:5/222: dread d8/f18 [0,4194304] 0 2024-04-02T04:47:44.851 INFO:tasks.workunit.client.0.smithi070.stdout:6/206: dread d0/d20/f34 [0,4194304] 0 2024-04-02T04:47:44.852 INFO:tasks.workunit.client.0.smithi070.stdout:1/258: mkdir d0/d54 0 2024-04-02T04:47:44.852 INFO:tasks.workunit.client.0.smithi070.stdout:8/337: dread d2/d8/d29/f3b [0,4194304] 0 2024-04-02T04:47:44.855 INFO:tasks.workunit.client.0.smithi070.stdout:5/321: symlink d5/d57/l64 0 2024-04-02T04:47:44.856 INFO:tasks.workunit.client.1.smithi179.stdout:8/259: chown d3/d32/d21/f25 0 1 2024-04-02T04:47:44.856 INFO:tasks.workunit.client.1.smithi179.stdout:5/223: chown d8/d1b/d23/d36/f41 20443 1 2024-04-02T04:47:44.857 INFO:tasks.workunit.client.1.smithi179.stdout:3/260: creat d8/d57/f5a x:0 0 0 2024-04-02T04:47:44.857 INFO:tasks.workunit.client.0.smithi070.stdout:4/247: unlink d1/d3/c3b 0 2024-04-02T04:47:44.858 INFO:tasks.workunit.client.1.smithi179.stdout:4/361: dwrite d1/d3/df/f59 [4194304,4194304] 0 2024-04-02T04:47:44.858 INFO:tasks.workunit.client.0.smithi070.stdout:5/322: dread d5/d17/d27/f45 [0,4194304] 0 2024-04-02T04:47:44.859 INFO:tasks.workunit.client.0.smithi070.stdout:5/323: rename d5/d17/d27/d2e to d5/d17/d27/d2e/d65 22 2024-04-02T04:47:44.860 INFO:tasks.workunit.client.1.smithi179.stdout:0/336: mkdir d1/da/d48/d65 0 2024-04-02T04:47:44.860 INFO:tasks.workunit.client.0.smithi070.stdout:2/362: symlink d2/d10/d21/d2b/l73 0 2024-04-02T04:47:44.860 INFO:tasks.workunit.client.1.smithi179.stdout:6/300: creat d3/d12/d6a/f6e x:0 0 0 2024-04-02T04:47:44.862 INFO:tasks.workunit.client.1.smithi179.stdout:4/362: dread - d1/d3/dc/d27/d62/f76 zero size 2024-04-02T04:47:44.862 INFO:tasks.workunit.client.1.smithi179.stdout:7/336: dread d1/d9/fb [0,4194304] 0 2024-04-02T04:47:44.862 INFO:tasks.workunit.client.0.smithi070.stdout:6/207: creat d0/d20/f41 x:0 0 0 2024-04-02T04:47:44.863 INFO:tasks.workunit.client.1.smithi179.stdout:6/301: truncate d3/d42/f54 1794321 0 2024-04-02T04:47:44.863 INFO:tasks.workunit.client.0.smithi070.stdout:6/208: chown d0/d20/l32 60170429 1 2024-04-02T04:47:44.864 INFO:tasks.workunit.client.0.smithi070.stdout:1/259: mknod d0/d7/d1a/c55 0 2024-04-02T04:47:44.864 INFO:tasks.workunit.client.0.smithi070.stdout:6/209: write d0/d3/d19/f3f [770794,47572] 0 2024-04-02T04:47:44.866 INFO:tasks.workunit.client.0.smithi070.stdout:8/338: mkdir d2/d8/d23/d24/d66/d71 0 2024-04-02T04:47:44.870 INFO:tasks.workunit.client.1.smithi179.stdout:5/224: symlink d8/d1b/d23/d32/l44 0 2024-04-02T04:47:44.875 INFO:tasks.workunit.client.0.smithi070.stdout:2/363: rename d2/d24/c57 to d2/d12/c74 0 2024-04-02T04:47:44.876 INFO:tasks.workunit.client.1.smithi179.stdout:2/178: getdents dc/d29 0 2024-04-02T04:47:44.877 INFO:tasks.workunit.client.0.smithi070.stdout:1/260: mkdir d0/d9/d1d/d22/d56 0 2024-04-02T04:47:44.877 INFO:tasks.workunit.client.0.smithi070.stdout:1/261: dread - d0/d4c/f51 zero size 2024-04-02T04:47:44.879 INFO:tasks.workunit.client.1.smithi179.stdout:5/225: dwrite d8/f2c [0,4194304] 0 2024-04-02T04:47:44.884 INFO:tasks.workunit.client.0.smithi070.stdout:4/248: symlink d1/d3/l4e 0 2024-04-02T04:47:44.885 INFO:tasks.workunit.client.1.smithi179.stdout:7/337: mknod d1/d9/d35/c7f 0 2024-04-02T04:47:44.889 INFO:tasks.workunit.client.0.smithi070.stdout:2/364: symlink d2/d10/d19/l75 0 2024-04-02T04:47:44.889 INFO:tasks.workunit.client.1.smithi179.stdout:6/302: rename d3/d2c/c33 to d3/d12/d23/d39/d4e/d6c/c6f 0 2024-04-02T04:47:44.890 INFO:tasks.workunit.client.1.smithi179.stdout:5/226: dwrite d8/d1b/d23/d32/f38 [0,4194304] 0 2024-04-02T04:47:44.892 INFO:tasks.workunit.client.0.smithi070.stdout:8/339: mkdir d2/d8/d29/d72 0 2024-04-02T04:47:44.893 INFO:tasks.workunit.client.1.smithi179.stdout:3/261: dread d8/f1b [0,4194304] 0 2024-04-02T04:47:44.893 INFO:tasks.workunit.client.1.smithi179.stdout:3/262: fdatasync d8/d26/d36/f38 0 2024-04-02T04:47:44.899 INFO:tasks.workunit.client.0.smithi070.stdout:2/365: unlink d2/d10/d21/d2b/f67 0 2024-04-02T04:47:44.900 INFO:tasks.workunit.client.0.smithi070.stdout:2/366: chown d2/d10/d21/d2b/l73 441377579 1 2024-04-02T04:47:44.900 INFO:tasks.workunit.client.1.smithi179.stdout:3/263: dwrite d8/f22 [0,4194304] 0 2024-04-02T04:47:44.903 INFO:tasks.workunit.client.1.smithi179.stdout:4/363: symlink d1/d3/df/l78 0 2024-04-02T04:47:44.904 INFO:tasks.workunit.client.1.smithi179.stdout:9/306: truncate d0/d1/f7 61055 0 2024-04-02T04:47:44.906 INFO:tasks.workunit.client.0.smithi070.stdout:4/249: symlink d1/l4f 0 2024-04-02T04:47:44.910 INFO:tasks.workunit.client.1.smithi179.stdout:8/260: creat d3/d32/f5c x:0 0 0 2024-04-02T04:47:44.911 INFO:tasks.workunit.client.0.smithi070.stdout:1/262: rename d0/d7/d1a/c55 to d0/d9/c57 0 2024-04-02T04:47:44.912 INFO:tasks.workunit.client.0.smithi070.stdout:1/263: chown d0/d7/f10 9439 1 2024-04-02T04:47:44.914 INFO:tasks.workunit.client.0.smithi070.stdout:1/264: truncate d0/d9/fc 1148826 0 2024-04-02T04:47:44.915 INFO:tasks.workunit.client.1.smithi179.stdout:5/227: mknod d8/d1b/d23/d32/c45 0 2024-04-02T04:47:44.918 INFO:tasks.workunit.client.0.smithi070.stdout:2/367: rename d2/f6b to d2/d10/d21/d71/f76 0 2024-04-02T04:47:44.919 INFO:tasks.workunit.client.1.smithi179.stdout:8/261: dwrite d3/d32/f5c [0,4194304] 0 2024-04-02T04:47:44.920 INFO:tasks.workunit.client.0.smithi070.stdout:1/265: dwrite d0/d9/d1d/f4e [0,4194304] 0 2024-04-02T04:47:44.921 INFO:tasks.workunit.client.0.smithi070.stdout:1/266: readlink d0/d7/l53 0 2024-04-02T04:47:44.923 INFO:tasks.workunit.client.1.smithi179.stdout:2/179: unlink dc/c19 0 2024-04-02T04:47:44.923 INFO:tasks.workunit.client.1.smithi179.stdout:2/180: readlink dc/d16/l28 0 2024-04-02T04:47:44.926 INFO:tasks.workunit.client.1.smithi179.stdout:3/264: creat d8/d26/f5b x:0 0 0 2024-04-02T04:47:44.927 INFO:tasks.workunit.client.1.smithi179.stdout:3/265: read - d8/d26/d36/f4d zero size 2024-04-02T04:47:44.928 INFO:tasks.workunit.client.0.smithi070.stdout:4/250: fsync d1/f1e 0 2024-04-02T04:47:44.932 INFO:tasks.workunit.client.0.smithi070.stdout:5/324: dwrite d5/f6 [0,4194304] 0 2024-04-02T04:47:44.932 INFO:tasks.workunit.client.1.smithi179.stdout:7/338: rename d1/d9/c78 to d1/dd/d16/d4d/c80 0 2024-04-02T04:47:44.934 INFO:tasks.workunit.client.1.smithi179.stdout:6/303: mknod d3/d2d/d38/d64/c70 0 2024-04-02T04:47:44.936 INFO:tasks.workunit.client.1.smithi179.stdout:6/304: write d3/d12/d23/d26/f4f [782898,53044] 0 2024-04-02T04:47:44.941 INFO:tasks.workunit.client.1.smithi179.stdout:0/337: dwrite d1/da/f17 [0,4194304] 0 2024-04-02T04:47:44.943 INFO:tasks.workunit.client.1.smithi179.stdout:0/338: stat d1/da/d28/f41 0 2024-04-02T04:47:44.946 INFO:tasks.workunit.client.1.smithi179.stdout:2/181: creat dc/d29/f37 x:0 0 0 2024-04-02T04:47:44.949 INFO:tasks.workunit.client.0.smithi070.stdout:8/340: write d2/d8/d23/d24/f34 [965750,87955] 0 2024-04-02T04:47:44.950 INFO:tasks.workunit.client.1.smithi179.stdout:3/266: creat d8/d26/d36/f5c x:0 0 0 2024-04-02T04:47:44.954 INFO:tasks.workunit.client.1.smithi179.stdout:9/307: unlink d0/d1/d11/f42 0 2024-04-02T04:47:44.956 INFO:tasks.workunit.client.0.smithi070.stdout:1/267: creat d0/d54/f58 x:0 0 0 2024-04-02T04:47:44.958 INFO:tasks.workunit.client.1.smithi179.stdout:3/267: dwrite d8/d26/d36/f4a [0,4194304] 0 2024-04-02T04:47:44.966 INFO:tasks.workunit.client.1.smithi179.stdout:0/339: mkdir d1/da/df/d56/d66 0 2024-04-02T04:47:44.966 INFO:tasks.workunit.client.1.smithi179.stdout:6/305: dread d3/d12/d23/d2f/f4a [0,4194304] 0 2024-04-02T04:47:44.967 INFO:tasks.workunit.client.1.smithi179.stdout:0/340: stat d1/da/d48/d5c 0 2024-04-02T04:47:44.968 INFO:tasks.workunit.client.1.smithi179.stdout:0/341: read - d1/f30 zero size 2024-04-02T04:47:44.970 INFO:tasks.workunit.client.0.smithi070.stdout:1/268: mkdir d0/d54/d59 0 2024-04-02T04:47:44.972 INFO:tasks.workunit.client.1.smithi179.stdout:2/182: truncate dc/d16/f24 7630079 0 2024-04-02T04:47:44.972 INFO:tasks.workunit.client.0.smithi070.stdout:1/269: read - d0/d9/d1d/d22/d2c/d4f/f50 zero size 2024-04-02T04:47:44.973 INFO:tasks.workunit.client.1.smithi179.stdout:6/306: dread d3/d12/d23/f55 [0,4194304] 0 2024-04-02T04:47:44.974 INFO:tasks.workunit.client.0.smithi070.stdout:2/368: dwrite d2/f14 [0,4194304] 0 2024-04-02T04:47:44.974 INFO:tasks.workunit.client.1.smithi179.stdout:5/228: write f6 [1882550,11699] 0 2024-04-02T04:47:44.976 INFO:tasks.workunit.client.0.smithi070.stdout:5/325: rename d5/d17/d27/d2c/d35 to d5/d66 0 2024-04-02T04:47:44.976 INFO:tasks.workunit.client.1.smithi179.stdout:6/307: stat d3/d12/d23/d26/f37 0 2024-04-02T04:47:44.977 INFO:tasks.workunit.client.1.smithi179.stdout:6/308: write d3/d12/d23/d26/f37 [592880,97320] 0 2024-04-02T04:47:44.979 INFO:tasks.workunit.client.1.smithi179.stdout:3/268: mknod d8/d26/d36/c5d 0 2024-04-02T04:47:44.980 INFO:tasks.workunit.client.1.smithi179.stdout:3/269: read - d8/d26/d36/f49 zero size 2024-04-02T04:47:44.980 INFO:tasks.workunit.client.0.smithi070.stdout:1/270: mkdir d0/d4c/d5a 0 2024-04-02T04:47:44.980 INFO:tasks.workunit.client.0.smithi070.stdout:1/271: chown d0/d9/lf 633 1 2024-04-02T04:47:44.981 INFO:tasks.workunit.client.1.smithi179.stdout:3/270: write d8/d26/f56 [770643,122911] 0 2024-04-02T04:47:44.981 INFO:tasks.workunit.client.1.smithi179.stdout:3/271: chown d8/d26/f4e 1 1 2024-04-02T04:47:44.982 INFO:tasks.workunit.client.1.smithi179.stdout:3/272: chown d8/d26/d36/l42 8 1 2024-04-02T04:47:44.983 INFO:tasks.workunit.client.0.smithi070.stdout:2/369: truncate d2/d12/d59/f70 40582 0 2024-04-02T04:47:44.984 INFO:tasks.workunit.client.0.smithi070.stdout:5/326: creat d5/d17/d2b/f67 x:0 0 0 2024-04-02T04:47:44.986 INFO:tasks.workunit.client.1.smithi179.stdout:1/306: sync 2024-04-02T04:47:44.986 INFO:tasks.workunit.client.1.smithi179.stdout:2/183: chown dc/c2a 0 1 2024-04-02T04:47:44.988 INFO:tasks.workunit.client.0.smithi070.stdout:2/370: write d2/d10/d21/f33 [22690,56314] 0 2024-04-02T04:47:44.990 INFO:tasks.workunit.client.1.smithi179.stdout:2/184: truncate dc/d16/d1e/f2d 863590 0 2024-04-02T04:47:44.990 INFO:tasks.workunit.client.1.smithi179.stdout:8/262: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:44.990 INFO:tasks.workunit.client.1.smithi179.stdout:2/185: readlink dc/d29/l30 0 2024-04-02T04:47:44.991 INFO:tasks.workunit.client.0.smithi070.stdout:1/272: symlink d0/d9/d1d/d22/d2c/d4d/l5b 0 2024-04-02T04:47:44.994 INFO:tasks.workunit.client.0.smithi070.stdout:9/270: sync 2024-04-02T04:47:44.994 INFO:tasks.workunit.client.0.smithi070.stdout:0/372: sync 2024-04-02T04:47:44.996 INFO:tasks.workunit.client.1.smithi179.stdout:6/309: creat d3/d12/d23/d39/f71 x:0 0 0 2024-04-02T04:47:44.998 INFO:tasks.workunit.client.1.smithi179.stdout:6/310: dread - d3/d12/d23/d39/d4e/d6c/f61 zero size 2024-04-02T04:47:44.999 INFO:tasks.workunit.client.1.smithi179.stdout:0/342: mkdir d1/da/d67 0 2024-04-02T04:47:45.000 INFO:tasks.workunit.client.0.smithi070.stdout:0/373: dwrite d4/d5/d21/f64 [0,4194304] 0 2024-04-02T04:47:45.000 INFO:tasks.workunit.client.0.smithi070.stdout:5/327: fdatasync d5/fa 0 2024-04-02T04:47:45.001 INFO:tasks.workunit.client.0.smithi070.stdout:0/374: fsync f0 0 2024-04-02T04:47:45.002 INFO:tasks.workunit.client.1.smithi179.stdout:3/273: write d8/d26/f2b [2119380,89996] 0 2024-04-02T04:47:45.003 INFO:tasks.workunit.client.0.smithi070.stdout:4/251: write d1/fa [2187747,73063] 0 2024-04-02T04:47:45.003 INFO:tasks.workunit.client.0.smithi070.stdout:5/328: chown d5/d9/d10/f11 0 1 2024-04-02T04:47:45.003 INFO:tasks.workunit.client.0.smithi070.stdout:1/273: mkdir d0/d7/d28/d5c 0 2024-04-02T04:47:45.003 INFO:tasks.workunit.client.0.smithi070.stdout:0/375: chown d4/d5/f5e 99655083 1 2024-04-02T04:47:45.004 INFO:tasks.workunit.client.1.smithi179.stdout:3/274: stat d8/d26/f4e 0 2024-04-02T04:47:45.004 INFO:tasks.workunit.client.0.smithi070.stdout:4/252: write d1/fb [3553189,35906] 0 2024-04-02T04:47:45.005 INFO:tasks.workunit.client.0.smithi070.stdout:8/341: write d2/d8/d23/f51 [308822,84230] 0 2024-04-02T04:47:45.006 INFO:tasks.workunit.client.1.smithi179.stdout:3/275: chown d8/d26/d36/f4d 382817305 1 2024-04-02T04:47:45.006 INFO:tasks.workunit.client.0.smithi070.stdout:9/271: chown d4/d10/d12/d56/c43 896 1 2024-04-02T04:47:45.007 INFO:tasks.workunit.client.0.smithi070.stdout:8/342: write d2/d8/d23/d24/f4b [571634,41187] 0 2024-04-02T04:47:45.008 INFO:tasks.workunit.client.1.smithi179.stdout:3/276: dread - d8/d26/d36/f38 zero size 2024-04-02T04:47:45.009 INFO:tasks.workunit.client.0.smithi070.stdout:8/343: fdatasync d2/d3/f49 0 2024-04-02T04:47:45.009 INFO:tasks.workunit.client.1.smithi179.stdout:5/229: dread f2 [0,4194304] 0 2024-04-02T04:47:45.009 INFO:tasks.workunit.client.1.smithi179.stdout:7/339: dwrite d1/dd/d16/f62 [0,4194304] 0 2024-04-02T04:47:45.014 INFO:tasks.workunit.client.1.smithi179.stdout:1/307: rmdir d3/d5/d2f/d4f/d42/d53/d5b 39 2024-04-02T04:47:45.015 INFO:tasks.workunit.client.0.smithi070.stdout:9/272: dwrite d4/d10/d11/f42 [0,4194304] 0 2024-04-02T04:47:45.020 INFO:tasks.workunit.client.1.smithi179.stdout:1/308: dwrite d3/d5/f21 [4194304,4194304] 0 2024-04-02T04:47:45.022 INFO:tasks.workunit.client.1.smithi179.stdout:2/186: rmdir dc/d29 39 2024-04-02T04:47:45.022 INFO:tasks.workunit.client.1.smithi179.stdout:1/309: write d3/d5/d2f/d36/f67 [459562,34640] 0 2024-04-02T04:47:45.027 INFO:tasks.workunit.client.0.smithi070.stdout:5/329: fsync d5/d14/f63 0 2024-04-02T04:47:45.028 INFO:tasks.workunit.client.0.smithi070.stdout:5/330: write d5/d17/d27/f4b [894787,116442] 0 2024-04-02T04:47:45.031 INFO:tasks.workunit.client.1.smithi179.stdout:8/263: rename d3/d31/c36 to d3/d32/d21/d22/c5d 0 2024-04-02T04:47:45.033 INFO:tasks.workunit.client.0.smithi070.stdout:1/274: creat d0/d9/d1d/d22/f5d x:0 0 0 2024-04-02T04:47:45.033 INFO:tasks.workunit.client.0.smithi070.stdout:1/275: fsync d0/d9/d1d/f40 0 2024-04-02T04:47:45.035 INFO:tasks.workunit.client.0.smithi070.stdout:1/276: write d0/d9/d1d/f42 [42990,113207] 0 2024-04-02T04:47:45.036 INFO:tasks.workunit.client.0.smithi070.stdout:2/371: write d2/d10/d19/f39 [1740896,56841] 0 2024-04-02T04:47:45.036 INFO:tasks.workunit.client.1.smithi179.stdout:0/343: truncate d1/f23 986055 0 2024-04-02T04:47:45.036 INFO:tasks.workunit.client.1.smithi179.stdout:4/364: sync 2024-04-02T04:47:45.037 INFO:tasks.workunit.client.1.smithi179.stdout:4/365: truncate d1/d3/dc/d26/f6d 79914 0 2024-04-02T04:47:45.038 INFO:tasks.workunit.client.0.smithi070.stdout:3/290: sync 2024-04-02T04:47:45.038 INFO:tasks.workunit.client.0.smithi070.stdout:7/289: sync 2024-04-02T04:47:45.041 INFO:tasks.workunit.client.1.smithi179.stdout:3/277: readlink d8/d26/l3a 0 2024-04-02T04:47:45.043 INFO:tasks.workunit.client.0.smithi070.stdout:3/291: dread d10/d32/f36 [0,4194304] 0 2024-04-02T04:47:45.044 INFO:tasks.workunit.client.0.smithi070.stdout:7/290: dwrite d7/d2f/f48 [0,4194304] 0 2024-04-02T04:47:45.045 INFO:tasks.workunit.client.1.smithi179.stdout:5/230: mkdir d8/d1b/d23/d32/d46 0 2024-04-02T04:47:45.046 INFO:tasks.workunit.client.0.smithi070.stdout:7/291: stat d7/dd/df/d34/l33 0 2024-04-02T04:47:45.046 INFO:tasks.workunit.client.0.smithi070.stdout:8/344: mknod d2/d3/d52/c73 0 2024-04-02T04:47:45.049 INFO:tasks.workunit.client.0.smithi070.stdout:9/273: dread d4/d13/f51 [0,4194304] 0 2024-04-02T04:47:45.050 INFO:tasks.workunit.client.0.smithi070.stdout:5/331: creat d5/d14/f68 x:0 0 0 2024-04-02T04:47:45.054 INFO:tasks.workunit.client.0.smithi070.stdout:8/345: dwrite d2/f48 [0,4194304] 0 2024-04-02T04:47:45.060 INFO:tasks.workunit.client.0.smithi070.stdout:5/332: dwrite d5/d17/d50/f5f [0,4194304] 0 2024-04-02T04:47:45.062 INFO:tasks.workunit.client.1.smithi179.stdout:8/264: read d3/d32/d21/f25 [1042,83719] 0 2024-04-02T04:47:45.065 INFO:tasks.workunit.client.0.smithi070.stdout:8/346: dread d2/d8/f44 [0,4194304] 0 2024-04-02T04:47:45.065 INFO:tasks.workunit.client.1.smithi179.stdout:8/265: chown d3/d32/f19 5213 1 2024-04-02T04:47:45.066 INFO:tasks.workunit.client.1.smithi179.stdout:9/308: dwrite d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:45.066 INFO:tasks.workunit.client.1.smithi179.stdout:8/266: chown d3/d32/d21 27705716 1 2024-04-02T04:47:45.068 INFO:tasks.workunit.client.1.smithi179.stdout:9/309: fsync d0/d1/f4 0 2024-04-02T04:47:45.068 INFO:tasks.workunit.client.0.smithi070.stdout:8/347: write d2/d3/d35/d3a/f56 [1965284,87300] 0 2024-04-02T04:47:45.068 INFO:tasks.workunit.client.0.smithi070.stdout:1/277: dwrite d0/f2a [4194304,4194304] 0 2024-04-02T04:47:45.070 INFO:tasks.workunit.client.0.smithi070.stdout:1/278: readlink d0/d7/l53 0 2024-04-02T04:47:45.071 INFO:tasks.workunit.client.0.smithi070.stdout:2/372: write d2/d10/d19/f1e [1454806,123761] 0 2024-04-02T04:47:45.072 INFO:tasks.workunit.client.1.smithi179.stdout:4/366: truncate d1/d3/f49 501628 0 2024-04-02T04:47:45.072 INFO:tasks.workunit.client.1.smithi179.stdout:3/278: rename d8/d26/d36/f4f to d8/d26/d36/f5e 0 2024-04-02T04:47:45.074 INFO:tasks.workunit.client.0.smithi070.stdout:3/292: rename d10/d32/d3f/f3a to d10/d32/d3f/f65 0 2024-04-02T04:47:45.074 INFO:tasks.workunit.client.0.smithi070.stdout:3/293: stat d10/d32/d4b/f61 0 2024-04-02T04:47:45.078 INFO:tasks.workunit.client.1.smithi179.stdout:4/367: dwrite d1/d3/f49 [0,4194304] 0 2024-04-02T04:47:45.079 INFO:tasks.workunit.client.0.smithi070.stdout:9/274: creat d4/d10/d12/d56/f5b x:0 0 0 2024-04-02T04:47:45.079 INFO:tasks.workunit.client.0.smithi070.stdout:4/253: creat d1/f50 x:0 0 0 2024-04-02T04:47:45.080 INFO:tasks.workunit.client.1.smithi179.stdout:7/340: creat d1/d9/d7e/f81 x:0 0 0 2024-04-02T04:47:45.082 INFO:tasks.workunit.client.1.smithi179.stdout:4/368: write d1/d3/df/f51 [283731,51806] 0 2024-04-02T04:47:45.084 INFO:tasks.workunit.client.1.smithi179.stdout:2/187: mknod dc/d29/c38 0 2024-04-02T04:47:45.086 INFO:tasks.workunit.client.1.smithi179.stdout:1/310: creat d3/d5/d2f/d65/f6b x:0 0 0 2024-04-02T04:47:45.088 INFO:tasks.workunit.client.0.smithi070.stdout:0/376: link d4/d5/d13/d43/f53 d4/d5/da/d2d/f66 0 2024-04-02T04:47:45.088 INFO:tasks.workunit.client.1.smithi179.stdout:6/311: creat d3/f72 x:0 0 0 2024-04-02T04:47:45.089 INFO:tasks.workunit.client.1.smithi179.stdout:2/188: write dc/d16/d1e/f2d [869259,45171] 0 2024-04-02T04:47:45.089 INFO:tasks.workunit.client.1.smithi179.stdout:3/279: dwrite d8/d26/f2b [0,4194304] 0 2024-04-02T04:47:45.089 INFO:tasks.workunit.client.0.smithi070.stdout:0/377: write d4/d5/da/d2d/f4c [578881,16919] 0 2024-04-02T04:47:45.089 INFO:tasks.workunit.client.1.smithi179.stdout:0/344: symlink d1/da/d48/d65/l68 0 2024-04-02T04:47:45.091 INFO:tasks.workunit.client.1.smithi179.stdout:0/345: fsync d1/da/f1b 0 2024-04-02T04:47:45.091 INFO:tasks.workunit.client.1.smithi179.stdout:3/280: dread - d8/d26/d36/f4d zero size 2024-04-02T04:47:45.092 INFO:tasks.workunit.client.1.smithi179.stdout:0/346: readlink d1/l45 0 2024-04-02T04:47:45.094 INFO:tasks.workunit.client.1.smithi179.stdout:3/281: read - d8/d26/d36/f4d zero size 2024-04-02T04:47:45.096 INFO:tasks.workunit.client.0.smithi070.stdout:1/279: unlink d0/d9/d1d/f3d 0 2024-04-02T04:47:45.097 INFO:tasks.workunit.client.0.smithi070.stdout:6/210: sync 2024-04-02T04:47:45.098 INFO:tasks.workunit.client.0.smithi070.stdout:6/211: fsync d0/d20/f28 0 2024-04-02T04:47:45.100 INFO:tasks.workunit.client.0.smithi070.stdout:2/373: creat d2/d10/d19/f77 x:0 0 0 2024-04-02T04:47:45.101 INFO:tasks.workunit.client.0.smithi070.stdout:3/294: mkdir d10/d66 0 2024-04-02T04:47:45.104 INFO:tasks.workunit.client.1.smithi179.stdout:5/231: truncate d8/f3e 3133511 0 2024-04-02T04:47:45.104 INFO:tasks.workunit.client.0.smithi070.stdout:6/212: dread d0/d20/f27 [0,4194304] 0 2024-04-02T04:47:45.104 INFO:tasks.workunit.client.0.smithi070.stdout:9/275: creat d4/d10/d12/d27/f5c x:0 0 0 2024-04-02T04:47:45.106 INFO:tasks.workunit.client.0.smithi070.stdout:4/254: mknod d1/d3/d9/d41/c51 0 2024-04-02T04:47:45.111 INFO:tasks.workunit.client.1.smithi179.stdout:4/369: rename d1/d3/df/f5a to d1/d15/f79 0 2024-04-02T04:47:45.111 INFO:tasks.workunit.client.0.smithi070.stdout:9/276: dwrite d4/d13/d16/d19/d2f/d39/f44 [0,4194304] 0 2024-04-02T04:47:45.118 INFO:tasks.workunit.client.0.smithi070.stdout:0/378: readlink d4/d5/d21/l5f 0 2024-04-02T04:47:45.118 INFO:tasks.workunit.client.1.smithi179.stdout:4/370: dwrite d1/d3/dc/d26/f6d [0,4194304] 0 2024-04-02T04:47:45.119 INFO:tasks.workunit.client.0.smithi070.stdout:3/295: dread d10/d32/d3f/f38 [0,4194304] 0 2024-04-02T04:47:45.119 INFO:tasks.workunit.client.0.smithi070.stdout:8/348: truncate d2/d3/f9 2545040 0 2024-04-02T04:47:45.120 INFO:tasks.workunit.client.0.smithi070.stdout:9/277: dwrite f3 [0,4194304] 0 2024-04-02T04:47:45.121 INFO:tasks.workunit.client.1.smithi179.stdout:8/267: symlink d3/d32/d21/l5e 0 2024-04-02T04:47:45.122 INFO:tasks.workunit.client.0.smithi070.stdout:5/333: write d5/d9/d23/f49 [3715324,105376] 0 2024-04-02T04:47:45.122 INFO:tasks.workunit.client.1.smithi179.stdout:8/268: readlink d3/d32/d21/l5e 0 2024-04-02T04:47:45.123 INFO:tasks.workunit.client.1.smithi179.stdout:7/341: dread d1/d9/f68 [0,4194304] 0 2024-04-02T04:47:45.123 INFO:tasks.workunit.client.1.smithi179.stdout:8/269: chown d3/d31/c5b 4011874 1 2024-04-02T04:47:45.123 INFO:tasks.workunit.client.0.smithi070.stdout:1/280: creat d0/d9/d1d/f5e x:0 0 0 2024-04-02T04:47:45.124 INFO:tasks.workunit.client.1.smithi179.stdout:8/270: read - d3/d32/d30/f4e zero size 2024-04-02T04:47:45.124 INFO:tasks.workunit.client.1.smithi179.stdout:7/342: write d1/d9/f43 [232333,46675] 0 2024-04-02T04:47:45.125 INFO:tasks.workunit.client.1.smithi179.stdout:8/271: stat d3/d32/d21/d22/d33/c39 0 2024-04-02T04:47:45.126 INFO:tasks.workunit.client.1.smithi179.stdout:7/343: dread d1/f15 [0,4194304] 0 2024-04-02T04:47:45.127 INFO:tasks.workunit.client.1.smithi179.stdout:7/344: fsync d1/dd/d16/f2c 0 2024-04-02T04:47:45.128 INFO:tasks.workunit.client.0.smithi070.stdout:2/374: unlink d2/d12/d3e/l51 0 2024-04-02T04:47:45.128 INFO:tasks.workunit.client.1.smithi179.stdout:2/189: mkdir dc/d29/d31/d39 0 2024-04-02T04:47:45.129 INFO:tasks.workunit.client.1.smithi179.stdout:7/345: write d1/dd/d16/f2c [5221907,126053] 0 2024-04-02T04:47:45.129 INFO:tasks.workunit.client.1.smithi179.stdout:7/346: fdatasync d1/dd/d16/f62 0 2024-04-02T04:47:45.133 INFO:tasks.workunit.client.0.smithi070.stdout:2/375: truncate d2/d10/d21/d2b/f6e 607308 0 2024-04-02T04:47:45.134 INFO:tasks.workunit.client.1.smithi179.stdout:9/310: mkdir d0/d1/d11/d3a/d5b 0 2024-04-02T04:47:45.135 INFO:tasks.workunit.client.1.smithi179.stdout:9/311: chown d0/d1/d12/f2d 308797 1 2024-04-02T04:47:45.135 INFO:tasks.workunit.client.0.smithi070.stdout:9/278: dwrite d4/d10/d12/d27/f5c [0,4194304] 0 2024-04-02T04:47:45.145 INFO:tasks.workunit.client.0.smithi070.stdout:3/296: mknod d10/d32/c67 0 2024-04-02T04:47:45.147 INFO:tasks.workunit.client.0.smithi070.stdout:5/334: symlink d5/d57/l69 0 2024-04-02T04:47:45.149 INFO:tasks.workunit.client.1.smithi179.stdout:1/311: mkdir d3/d5/d2f/d6c 0 2024-04-02T04:47:45.152 INFO:tasks.workunit.client.0.smithi070.stdout:3/297: dread d10/f29 [0,4194304] 0 2024-04-02T04:47:45.155 INFO:tasks.workunit.client.1.smithi179.stdout:5/232: write d8/d1b/d23/d36/f41 [2569211,2213] 0 2024-04-02T04:47:45.156 INFO:tasks.workunit.client.0.smithi070.stdout:7/292: getdents d7/dd/df/d34 0 2024-04-02T04:47:45.158 INFO:tasks.workunit.client.0.smithi070.stdout:2/376: creat d2/d10/f78 x:0 0 0 2024-04-02T04:47:45.159 INFO:tasks.workunit.client.0.smithi070.stdout:0/379: write d4/d5/d13/f2b [2123859,103888] 0 2024-04-02T04:47:45.161 INFO:tasks.workunit.client.1.smithi179.stdout:7/347: creat d1/dd/d23/f82 x:0 0 0 2024-04-02T04:47:45.163 INFO:tasks.workunit.client.1.smithi179.stdout:2/190: dwrite f6 [0,4194304] 0 2024-04-02T04:47:45.163 INFO:tasks.workunit.client.1.smithi179.stdout:7/348: write d1/d9/d7e/f81 [904367,76372] 0 2024-04-02T04:47:45.164 INFO:tasks.workunit.client.0.smithi070.stdout:9/279: dread d4/f8 [0,4194304] 0 2024-04-02T04:47:45.164 INFO:tasks.workunit.client.0.smithi070.stdout:6/213: unlink d0/d3/l40 0 2024-04-02T04:47:45.165 INFO:tasks.workunit.client.0.smithi070.stdout:4/255: mknod d1/d3/c52 0 2024-04-02T04:47:45.165 INFO:tasks.workunit.client.0.smithi070.stdout:4/256: stat d1/f28 0 2024-04-02T04:47:45.167 INFO:tasks.workunit.client.0.smithi070.stdout:8/349: write d2/d3/d52/f33 [1298178,58858] 0 2024-04-02T04:47:45.169 INFO:tasks.workunit.client.0.smithi070.stdout:8/350: dread - d2/d8/d23/d24/d66/f70 zero size 2024-04-02T04:47:45.171 INFO:tasks.workunit.client.0.smithi070.stdout:5/335: mkdir d5/d9/d10/d6a 0 2024-04-02T04:47:45.173 INFO:tasks.workunit.client.1.smithi179.stdout:6/312: rmdir d3/d2d/d38/d5d 0 2024-04-02T04:47:45.173 INFO:tasks.workunit.client.0.smithi070.stdout:4/257: dwrite d1/d3/d9/f24 [0,4194304] 0 2024-04-02T04:47:45.175 INFO:tasks.workunit.client.1.smithi179.stdout:4/371: mkdir d1/d3/dc/d7a 0 2024-04-02T04:47:45.177 INFO:tasks.workunit.client.0.smithi070.stdout:3/298: symlink d10/d32/d4b/l68 0 2024-04-02T04:47:45.177 INFO:tasks.workunit.client.1.smithi179.stdout:4/372: read d1/d3/df/d10/f2d [125249,23486] 0 2024-04-02T04:47:45.178 INFO:tasks.workunit.client.1.smithi179.stdout:4/373: write d1/d3/df/f48 [741966,1496] 0 2024-04-02T04:47:45.181 INFO:tasks.workunit.client.0.smithi070.stdout:4/258: dwrite d1/fe [0,4194304] 0 2024-04-02T04:47:45.181 INFO:tasks.workunit.client.1.smithi179.stdout:2/191: creat dc/d29/d31/f3a x:0 0 0 2024-04-02T04:47:45.182 INFO:tasks.workunit.client.0.smithi070.stdout:9/280: mknod d4/d10/c5d 0 2024-04-02T04:47:45.184 INFO:tasks.workunit.client.1.smithi179.stdout:7/349: unlink d1/d9/d35/c7f 0 2024-04-02T04:47:45.184 INFO:tasks.workunit.client.0.smithi070.stdout:9/281: chown d4/d13/d16/l1f 10787 1 2024-04-02T04:47:45.188 INFO:tasks.workunit.client.0.smithi070.stdout:2/377: dwrite d2/d24/f37 [4194304,4194304] 0 2024-04-02T04:47:45.191 INFO:tasks.workunit.client.0.smithi070.stdout:5/336: mknod d5/d14/c6b 0 2024-04-02T04:47:45.192 INFO:tasks.workunit.client.1.smithi179.stdout:6/313: mknod d3/d12/d23/d39/d4e/d6c/c73 0 2024-04-02T04:47:45.193 INFO:tasks.workunit.client.0.smithi070.stdout:9/282: dwrite d4/d10/d11/f4e [0,4194304] 0 2024-04-02T04:47:45.198 INFO:tasks.workunit.client.0.smithi070.stdout:7/293: mknod d7/d9/c60 0 2024-04-02T04:47:45.200 INFO:tasks.workunit.client.1.smithi179.stdout:4/374: mkdir d1/d3/df/d10/d2f/d7b 0 2024-04-02T04:47:45.201 INFO:tasks.workunit.client.0.smithi070.stdout:4/259: fdatasync d1/d3/d9/d41/f45 0 2024-04-02T04:47:45.202 INFO:tasks.workunit.client.0.smithi070.stdout:4/260: dread - d1/d3/d9/d41/f48 zero size 2024-04-02T04:47:45.203 INFO:tasks.workunit.client.0.smithi070.stdout:4/261: fsync d1/d3/f4c 0 2024-04-02T04:47:45.204 INFO:tasks.workunit.client.1.smithi179.stdout:3/282: getdents d8/d26 0 2024-04-02T04:47:45.205 INFO:tasks.workunit.client.0.smithi070.stdout:2/378: symlink d2/d24/l79 0 2024-04-02T04:47:45.205 INFO:tasks.workunit.client.1.smithi179.stdout:3/283: truncate d8/d26/f4e 843015 0 2024-04-02T04:47:45.207 INFO:tasks.workunit.client.1.smithi179.stdout:2/192: dread dc/fe [0,4194304] 0 2024-04-02T04:47:45.208 INFO:tasks.workunit.client.1.smithi179.stdout:0/347: sync 2024-04-02T04:47:45.209 INFO:tasks.workunit.client.1.smithi179.stdout:6/314: fsync d3/fa 0 2024-04-02T04:47:45.212 INFO:tasks.workunit.client.0.smithi070.stdout:5/337: rmdir d5/d17/d2b 39 2024-04-02T04:47:45.213 INFO:tasks.workunit.client.1.smithi179.stdout:8/272: getdents d3/d32/d21/d22/d33 0 2024-04-02T04:47:45.214 INFO:tasks.workunit.client.0.smithi070.stdout:1/281: getdents d0/d9 0 2024-04-02T04:47:45.217 INFO:tasks.workunit.client.1.smithi179.stdout:4/375: mkdir d1/d15/d7c 0 2024-04-02T04:47:45.220 INFO:tasks.workunit.client.1.smithi179.stdout:1/312: truncate d3/f28 5972743 0 2024-04-02T04:47:45.223 INFO:tasks.workunit.client.0.smithi070.stdout:6/214: rename d0/f16 to d0/d3/d19/d31/f42 0 2024-04-02T04:47:45.223 INFO:tasks.workunit.client.1.smithi179.stdout:4/376: dwrite d1/d3/dc/d27/d62/f5e [0,4194304] 0 2024-04-02T04:47:45.225 INFO:tasks.workunit.client.1.smithi179.stdout:3/284: creat d8/d26/f5f x:0 0 0 2024-04-02T04:47:45.225 INFO:tasks.workunit.client.1.smithi179.stdout:5/233: truncate d8/d1b/d23/d32/f3d 3745198 0 2024-04-02T04:47:45.229 INFO:tasks.workunit.client.0.smithi070.stdout:8/351: link d2/d3/d52/l11 d2/d8/d29/db/l74 0 2024-04-02T04:47:45.230 INFO:tasks.workunit.client.1.smithi179.stdout:0/348: rename d1/da/c1a to d1/db/d64/c69 0 2024-04-02T04:47:45.230 INFO:tasks.workunit.client.1.smithi179.stdout:0/349: chown d1/db/fc 63572560 1 2024-04-02T04:47:45.231 INFO:tasks.workunit.client.0.smithi070.stdout:4/262: dread d1/f2d [0,4194304] 0 2024-04-02T04:47:45.231 INFO:tasks.workunit.client.1.smithi179.stdout:0/350: dread d1/d5b/f61 [0,4194304] 0 2024-04-02T04:47:45.232 INFO:tasks.workunit.client.0.smithi070.stdout:4/263: fdatasync d1/f3e 0 2024-04-02T04:47:45.233 INFO:tasks.workunit.client.1.smithi179.stdout:6/315: mknod d3/d12/d44/c74 0 2024-04-02T04:47:45.235 INFO:tasks.workunit.client.1.smithi179.stdout:8/273: truncate d3/d32/d21/f25 2846525 0 2024-04-02T04:47:45.238 INFO:tasks.workunit.client.0.smithi070.stdout:3/299: write d10/d17/f5d [213578,117929] 0 2024-04-02T04:47:45.239 INFO:tasks.workunit.client.1.smithi179.stdout:7/350: truncate d1/dd/d23/f24 976230 0 2024-04-02T04:47:45.239 INFO:tasks.workunit.client.0.smithi070.stdout:4/264: dwrite d1/f50 [0,4194304] 0 2024-04-02T04:47:45.240 INFO:tasks.workunit.client.0.smithi070.stdout:7/294: symlink d7/d36/l61 0 2024-04-02T04:47:45.242 INFO:tasks.workunit.client.1.smithi179.stdout:6/316: dwrite d3/d12/d23/d2f/f3a [0,4194304] 0 2024-04-02T04:47:45.245 INFO:tasks.workunit.client.1.smithi179.stdout:4/377: read d1/d3/df/d10/f2d [413776,39645] 0 2024-04-02T04:47:45.246 INFO:tasks.workunit.client.0.smithi070.stdout:6/215: fsync d0/d20/f34 0 2024-04-02T04:47:45.249 INFO:tasks.workunit.client.0.smithi070.stdout:5/338: write d5/d17/d27/f28 [735386,25326] 0 2024-04-02T04:47:45.249 INFO:tasks.workunit.client.1.smithi179.stdout:9/312: getdents d0/d19/d27 0 2024-04-02T04:47:45.250 INFO:tasks.workunit.client.1.smithi179.stdout:0/351: dread d1/db/fc [0,4194304] 0 2024-04-02T04:47:45.251 INFO:tasks.workunit.client.1.smithi179.stdout:9/313: chown d0/d1/d11/d29/f34 377 1 2024-04-02T04:47:45.251 INFO:tasks.workunit.client.0.smithi070.stdout:2/379: unlink d2/d10/f20 0 2024-04-02T04:47:45.251 INFO:tasks.workunit.client.1.smithi179.stdout:0/352: fdatasync d1/f22 0 2024-04-02T04:47:45.253 INFO:tasks.workunit.client.0.smithi070.stdout:6/216: read d0/d3/d19/d31/f42 [2824779,44260] 0 2024-04-02T04:47:45.253 INFO:tasks.workunit.client.1.smithi179.stdout:0/353: truncate d1/f30 794248 0 2024-04-02T04:47:45.253 INFO:tasks.workunit.client.1.smithi179.stdout:3/285: mknod d8/d26/c60 0 2024-04-02T04:47:45.255 INFO:tasks.workunit.client.0.smithi070.stdout:8/352: symlink d2/d8/d23/d24/d66/l75 0 2024-04-02T04:47:45.261 INFO:tasks.workunit.client.0.smithi070.stdout:1/282: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:45.261 INFO:tasks.workunit.client.1.smithi179.stdout:5/234: read d8/d1b/f1f [446515,25933] 0 2024-04-02T04:47:45.262 INFO:tasks.workunit.client.0.smithi070.stdout:6/217: dwrite d0/d20/f28 [0,4194304] 0 2024-04-02T04:47:45.268 INFO:tasks.workunit.client.0.smithi070.stdout:3/300: symlink d10/l69 0 2024-04-02T04:47:45.270 INFO:tasks.workunit.client.0.smithi070.stdout:7/295: rmdir d7/d36 39 2024-04-02T04:47:45.273 INFO:tasks.workunit.client.0.smithi070.stdout:4/265: creat d1/d3/d9/f53 x:0 0 0 2024-04-02T04:47:45.276 INFO:tasks.workunit.client.1.smithi179.stdout:6/317: rename d3/d12/d23/d39/d4e/d6c/f61 to d3/d12/d6a/f75 0 2024-04-02T04:47:45.278 INFO:tasks.workunit.client.1.smithi179.stdout:4/378: mknod d1/d3/df/d10/d3a/c7d 0 2024-04-02T04:47:45.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:45 smithi070 ceph-mon[29212]: pgmap v11: 65 pgs: 65 active+clean; 713 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 5.9 MiB/s rd, 78 MiB/s wr, 262 op/s 2024-04-02T04:47:45.280 INFO:tasks.workunit.client.1.smithi179.stdout:6/318: dread d3/d12/d23/d26/f4f [0,4194304] 0 2024-04-02T04:47:45.281 INFO:tasks.workunit.client.0.smithi070.stdout:9/283: getdents d4/d13/d16/d19/d53 0 2024-04-02T04:47:45.283 INFO:tasks.workunit.client.1.smithi179.stdout:0/354: creat d1/db/d64/f6a x:0 0 0 2024-04-02T04:47:45.283 INFO:tasks.workunit.client.0.smithi070.stdout:1/283: chown d0/f1c 110280297 1 2024-04-02T04:47:45.283 INFO:tasks.workunit.client.1.smithi179.stdout:3/286: stat d8/d26/d36/c44 0 2024-04-02T04:47:45.284 INFO:tasks.workunit.client.0.smithi070.stdout:6/218: rename d0/d30 to d0/d43 0 2024-04-02T04:47:45.285 INFO:tasks.workunit.client.1.smithi179.stdout:3/287: stat d8/d26/d36 0 2024-04-02T04:47:45.285 INFO:tasks.workunit.client.0.smithi070.stdout:6/219: chown d0/c2c 0 1 2024-04-02T04:47:45.286 INFO:tasks.workunit.client.1.smithi179.stdout:3/288: chown d8/d26/f34 1749 1 2024-04-02T04:47:45.286 INFO:tasks.workunit.client.0.smithi070.stdout:3/301: creat d10/d32/d3f/f6a x:0 0 0 2024-04-02T04:47:45.287 INFO:tasks.workunit.client.1.smithi179.stdout:8/274: dwrite d3/f59 [0,4194304] 0 2024-04-02T04:47:45.287 INFO:tasks.workunit.client.0.smithi070.stdout:6/220: write d0/d3/d19/d31/f36 [33129,87317] 0 2024-04-02T04:47:45.287 INFO:tasks.workunit.client.1.smithi179.stdout:3/289: truncate d8/d26/f46 4834934 0 2024-04-02T04:47:45.288 INFO:tasks.workunit.client.0.smithi070.stdout:3/302: write d10/d32/d3f/f6a [273846,79004] 0 2024-04-02T04:47:45.288 INFO:tasks.workunit.client.1.smithi179.stdout:8/275: dread - d3/d32/d21/d49/f52 zero size 2024-04-02T04:47:45.289 INFO:tasks.workunit.client.1.smithi179.stdout:8/276: stat d3/c8 0 2024-04-02T04:47:45.289 INFO:tasks.workunit.client.1.smithi179.stdout:8/277: stat d3/f18 0 2024-04-02T04:47:45.290 INFO:tasks.workunit.client.1.smithi179.stdout:8/278: chown d3/f18 1530718 1 2024-04-02T04:47:45.290 INFO:tasks.workunit.client.0.smithi070.stdout:3/303: write d10/d32/d3f/f3b [867502,33276] 0 2024-04-02T04:47:45.294 INFO:tasks.workunit.client.1.smithi179.stdout:3/290: dwrite d8/d26/d36/f47 [0,4194304] 0 2024-04-02T04:47:45.297 INFO:tasks.workunit.client.1.smithi179.stdout:1/313: creat d3/d5/f6d x:0 0 0 2024-04-02T04:47:45.299 INFO:tasks.workunit.client.0.smithi070.stdout:9/284: creat d4/d10/d12/f5e x:0 0 0 2024-04-02T04:47:45.301 INFO:tasks.workunit.client.0.smithi070.stdout:3/304: dwrite d10/d32/d3f/f4d [0,4194304] 0 2024-04-02T04:47:45.304 INFO:tasks.workunit.client.0.smithi070.stdout:7/296: fdatasync d7/fc 0 2024-04-02T04:47:45.306 INFO:tasks.workunit.client.1.smithi179.stdout:4/379: write d1/d3/dc/fd [81889,110095] 0 2024-04-02T04:47:45.307 INFO:tasks.workunit.client.1.smithi179.stdout:4/380: write d1/d3/f7 [2287396,106576] 0 2024-04-02T04:47:45.312 INFO:tasks.workunit.client.0.smithi070.stdout:8/353: link d2/c26 d2/d8/d29/d22/d30/c76 0 2024-04-02T04:47:45.314 INFO:tasks.workunit.client.1.smithi179.stdout:4/381: truncate d1/d3/df/d10/d2f/f4e 83034 0 2024-04-02T04:47:45.314 INFO:tasks.workunit.client.1.smithi179.stdout:6/319: creat d3/d12/d23/d26/f76 x:0 0 0 2024-04-02T04:47:45.314 INFO:tasks.workunit.client.0.smithi070.stdout:8/354: readlink d2/d3/l28 0 2024-04-02T04:47:45.314 INFO:tasks.workunit.client.0.smithi070.stdout:6/221: dread d0/f15 [0,4194304] 0 2024-04-02T04:47:45.315 INFO:tasks.workunit.client.0.smithi070.stdout:9/285: mknod d4/d34/c5f 0 2024-04-02T04:47:45.317 INFO:tasks.workunit.client.0.smithi070.stdout:1/284: write d0/d7/d1a/f1b [763184,84247] 0 2024-04-02T04:47:45.318 INFO:tasks.workunit.client.1.smithi179.stdout:8/279: truncate d3/f16 4469430 0 2024-04-02T04:47:45.319 INFO:tasks.workunit.client.0.smithi070.stdout:7/297: fsync d7/dd/df/d34/f2a 0 2024-04-02T04:47:45.320 INFO:tasks.workunit.client.1.smithi179.stdout:7/351: creat d1/f83 x:0 0 0 2024-04-02T04:47:45.320 INFO:tasks.workunit.client.1.smithi179.stdout:7/352: fsync d1/fc 0 2024-04-02T04:47:45.321 INFO:tasks.workunit.client.0.smithi070.stdout:4/266: creat d1/d3/f54 x:0 0 0 2024-04-02T04:47:45.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:45 smithi179 ceph-mon[36603]: pgmap v11: 65 pgs: 65 active+clean; 713 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 5.9 MiB/s rd, 78 MiB/s wr, 262 op/s 2024-04-02T04:47:45.322 INFO:tasks.workunit.client.1.smithi179.stdout:1/314: creat d3/d5/d2f/d4f/d35/f6e x:0 0 0 2024-04-02T04:47:45.322 INFO:tasks.workunit.client.0.smithi070.stdout:4/267: chown d1/d3/d9/c3c 772 1 2024-04-02T04:47:45.323 INFO:tasks.workunit.client.0.smithi070.stdout:5/339: rename d5/d9/d3f to d5/d17/d6c 0 2024-04-02T04:47:45.325 INFO:tasks.workunit.client.0.smithi070.stdout:6/222: mknod d0/d43/c44 0 2024-04-02T04:47:45.325 INFO:tasks.workunit.client.1.smithi179.stdout:9/314: getdents d0/d19/d27 0 2024-04-02T04:47:45.326 INFO:tasks.workunit.client.0.smithi070.stdout:9/286: mkdir d4/d10/d12/d27/d60 0 2024-04-02T04:47:45.326 INFO:tasks.workunit.client.0.smithi070.stdout:6/223: read d0/f1c [3373890,101741] 0 2024-04-02T04:47:45.327 INFO:tasks.workunit.client.0.smithi070.stdout:1/285: mknod d0/d4c/c5f 0 2024-04-02T04:47:45.328 INFO:tasks.workunit.client.0.smithi070.stdout:1/286: fdatasync d0/d7/f8 0 2024-04-02T04:47:45.329 INFO:tasks.workunit.client.0.smithi070.stdout:1/287: truncate d0/d7/d1a/f2e 4828469 0 2024-04-02T04:47:45.330 INFO:tasks.workunit.client.0.smithi070.stdout:1/288: read - d0/d7/f43 zero size 2024-04-02T04:47:45.330 INFO:tasks.workunit.client.0.smithi070.stdout:1/289: chown d0/d9/f3e 1273 1 2024-04-02T04:47:45.331 INFO:tasks.workunit.client.1.smithi179.stdout:4/382: mknod d1/d3/df/d10/d3e/c7e 0 2024-04-02T04:47:45.332 INFO:tasks.workunit.client.1.smithi179.stdout:5/235: truncate d8/f1d 1733257 0 2024-04-02T04:47:45.332 INFO:tasks.workunit.client.0.smithi070.stdout:1/290: truncate d0/d7/d1a/f1b 4933179 0 2024-04-02T04:47:45.332 INFO:tasks.workunit.client.1.smithi179.stdout:4/383: write d1/d3/df/f59 [7087059,70976] 0 2024-04-02T04:47:45.333 INFO:tasks.workunit.client.1.smithi179.stdout:6/320: symlink d3/d2d/d38/l77 0 2024-04-02T04:47:45.337 INFO:tasks.workunit.client.1.smithi179.stdout:3/291: mknod d8/d26/d52/c61 0 2024-04-02T04:47:45.338 INFO:tasks.workunit.client.1.smithi179.stdout:3/292: dread - d8/d26/d36/f38 zero size 2024-04-02T04:47:45.339 INFO:tasks.workunit.client.1.smithi179.stdout:6/321: dread d3/d12/d23/d2f/f4a [0,4194304] 0 2024-04-02T04:47:45.340 INFO:tasks.workunit.client.0.smithi070.stdout:2/380: rename d2/d12/d48 to d2/d10/d5c/d7a 0 2024-04-02T04:47:45.340 INFO:tasks.workunit.client.0.smithi070.stdout:7/298: dwrite d7/d2f/d46/f51 [0,4194304] 0 2024-04-02T04:47:45.340 INFO:tasks.workunit.client.1.smithi179.stdout:6/322: chown d3/d12/d23/d39/d4e/d6c/f50 6 1 2024-04-02T04:47:45.341 INFO:tasks.workunit.client.0.smithi070.stdout:1/291: dread d0/d7/f48 [0,4194304] 0 2024-04-02T04:47:45.342 INFO:tasks.workunit.client.1.smithi179.stdout:8/280: creat d3/d32/d21/d50/f5f x:0 0 0 2024-04-02T04:47:45.342 INFO:tasks.workunit.client.0.smithi070.stdout:1/292: chown d0/d7/f8 339654790 1 2024-04-02T04:47:45.342 INFO:tasks.workunit.client.0.smithi070.stdout:2/381: truncate d2/d10/d19/f1e 2060614 0 2024-04-02T04:47:45.343 INFO:tasks.workunit.client.0.smithi070.stdout:5/340: creat d5/d17/d6c/f6d x:0 0 0 2024-04-02T04:47:45.345 INFO:tasks.workunit.client.0.smithi070.stdout:5/341: chown d5/d17/d27 2635689 1 2024-04-02T04:47:45.346 INFO:tasks.workunit.client.0.smithi070.stdout:8/355: unlink d2/d3/c1b 0 2024-04-02T04:47:45.347 INFO:tasks.workunit.client.0.smithi070.stdout:6/224: dread d0/d3/d19/d31/f42 [0,4194304] 0 2024-04-02T04:47:45.350 INFO:tasks.workunit.client.0.smithi070.stdout:8/356: chown d2/d8/f5e 77090 1 2024-04-02T04:47:45.350 INFO:tasks.workunit.client.1.smithi179.stdout:2/193: sync 2024-04-02T04:47:45.352 INFO:tasks.workunit.client.0.smithi070.stdout:1/293: dwrite d0/d9/d1d/f42 [0,4194304] 0 2024-04-02T04:47:45.352 INFO:tasks.workunit.client.0.smithi070.stdout:2/382: dwrite d2/d12/f52 [0,4194304] 0 2024-04-02T04:47:45.354 INFO:tasks.workunit.client.0.smithi070.stdout:4/268: symlink d1/d3/l55 0 2024-04-02T04:47:45.355 INFO:tasks.workunit.client.0.smithi070.stdout:1/294: dread - d0/d9/d1d/d22/d2c/f4a zero size 2024-04-02T04:47:45.357 INFO:tasks.workunit.client.1.smithi179.stdout:0/355: rmdir d1/da/df/d56/d66 0 2024-04-02T04:47:45.358 INFO:tasks.workunit.client.0.smithi070.stdout:1/295: write d0/d7/d1a/f1b [2235949,67332] 0 2024-04-02T04:47:45.359 INFO:tasks.workunit.client.0.smithi070.stdout:1/296: stat d0/f2a 0 2024-04-02T04:47:45.361 INFO:tasks.workunit.client.0.smithi070.stdout:1/297: write d0/d9/fc [595152,24719] 0 2024-04-02T04:47:45.364 INFO:tasks.workunit.client.0.smithi070.stdout:4/269: dwrite d1/d3/f30 [0,4194304] 0 2024-04-02T04:47:45.364 INFO:tasks.workunit.client.0.smithi070.stdout:3/305: rename d10/d32/d4b/l60 to d10/l6b 0 2024-04-02T04:47:45.365 INFO:tasks.workunit.client.0.smithi070.stdout:1/298: read - d0/d9/d1d/d22/d2c/f4a zero size 2024-04-02T04:47:45.372 INFO:tasks.workunit.client.1.smithi179.stdout:3/293: creat d8/d26/f62 x:0 0 0 2024-04-02T04:47:45.374 INFO:tasks.workunit.client.0.smithi070.stdout:1/299: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:45.374 INFO:tasks.workunit.client.0.smithi070.stdout:1/300: stat d0/d54 0 2024-04-02T04:47:45.375 INFO:tasks.workunit.client.0.smithi070.stdout:5/342: creat d5/d9/d23/f6e x:0 0 0 2024-04-02T04:47:45.377 INFO:tasks.workunit.client.1.smithi179.stdout:8/281: creat d3/d32/d21/d22/d33/f60 x:0 0 0 2024-04-02T04:47:45.377 INFO:tasks.workunit.client.0.smithi070.stdout:6/225: creat d0/d20/f45 x:0 0 0 2024-04-02T04:47:45.378 INFO:tasks.workunit.client.1.smithi179.stdout:7/353: mkdir d1/d84 0 2024-04-02T04:47:45.383 INFO:tasks.workunit.client.1.smithi179.stdout:8/282: dread d3/d32/f5c [0,4194304] 0 2024-04-02T04:47:45.384 INFO:tasks.workunit.client.1.smithi179.stdout:9/315: symlink d0/d19/d27/d48/l5c 0 2024-04-02T04:47:45.385 INFO:tasks.workunit.client.0.smithi070.stdout:2/383: unlink d2/d10/d5c/c60 0 2024-04-02T04:47:45.386 INFO:tasks.workunit.client.1.smithi179.stdout:0/356: fdatasync d1/da/f1d 0 2024-04-02T04:47:45.386 INFO:tasks.workunit.client.0.smithi070.stdout:2/384: dread - d2/d10/d19/f77 zero size 2024-04-02T04:47:45.387 INFO:tasks.workunit.client.1.smithi179.stdout:5/236: dread d8/fb [0,4194304] 0 2024-04-02T04:47:45.387 INFO:tasks.workunit.client.1.smithi179.stdout:8/283: dread d3/f59 [0,4194304] 0 2024-04-02T04:47:45.390 INFO:tasks.workunit.client.1.smithi179.stdout:4/384: link d1/d3/f1b d1/d3/dc/d27/f7f 0 2024-04-02T04:47:45.390 INFO:tasks.workunit.client.0.smithi070.stdout:4/270: write d1/d3/d9/f1d [1874984,122871] 0 2024-04-02T04:47:45.392 INFO:tasks.workunit.client.1.smithi179.stdout:3/294: mkdir d8/d57/d63 0 2024-04-02T04:47:45.394 INFO:tasks.workunit.client.0.smithi070.stdout:7/299: creat d7/d36/f62 x:0 0 0 2024-04-02T04:47:45.395 INFO:tasks.workunit.client.1.smithi179.stdout:7/354: creat d1/d9/f85 x:0 0 0 2024-04-02T04:47:45.396 INFO:tasks.workunit.client.1.smithi179.stdout:7/355: dread - d1/dd/d16/f33 zero size 2024-04-02T04:47:45.397 INFO:tasks.workunit.client.1.smithi179.stdout:1/315: rename d3/d5/d2f/d36/c41 to d3/d5/d2f/d4f/d42/d53/c6f 0 2024-04-02T04:47:45.397 INFO:tasks.workunit.client.1.smithi179.stdout:7/356: write d1/d9/d27/f5f [1036598,96917] 0 2024-04-02T04:47:45.398 INFO:tasks.workunit.client.1.smithi179.stdout:1/316: dread - d3/d5/d2f/d4f/d35/f6e zero size 2024-04-02T04:47:45.398 INFO:tasks.workunit.client.0.smithi070.stdout:1/301: unlink d0/d9/d1d/d22/l33 0 2024-04-02T04:47:45.399 INFO:tasks.workunit.client.1.smithi179.stdout:1/317: dread - d3/d5/de/f24 zero size 2024-04-02T04:47:45.399 INFO:tasks.workunit.client.0.smithi070.stdout:1/302: chown d0/d7/l53 3780 1 2024-04-02T04:47:45.404 INFO:tasks.workunit.client.0.smithi070.stdout:4/271: symlink d1/d3/d9/d41/l56 0 2024-04-02T04:47:45.406 INFO:tasks.workunit.client.1.smithi179.stdout:8/284: creat d3/f61 x:0 0 0 2024-04-02T04:47:45.407 INFO:tasks.workunit.client.0.smithi070.stdout:7/300: symlink d7/dd/df/l63 0 2024-04-02T04:47:45.408 INFO:tasks.workunit.client.0.smithi070.stdout:7/301: write d7/d36/f62 [304082,115365] 0 2024-04-02T04:47:45.412 INFO:tasks.workunit.client.1.smithi179.stdout:6/323: truncate d3/d2d/f65 8150440 0 2024-04-02T04:47:45.413 INFO:tasks.workunit.client.1.smithi179.stdout:1/318: fsync d3/d5/d2f/d4f/f37 0 2024-04-02T04:47:45.413 INFO:tasks.workunit.client.1.smithi179.stdout:1/319: chown d3/d5/c17 98750770 1 2024-04-02T04:47:45.414 INFO:tasks.workunit.client.1.smithi179.stdout:1/320: write d3/d5/d2f/d5a/f5d [603527,91924] 0 2024-04-02T04:47:45.420 INFO:tasks.workunit.client.1.smithi179.stdout:1/321: dwrite d3/d5/d2f/d36/f67 [0,4194304] 0 2024-04-02T04:47:45.426 INFO:tasks.workunit.client.0.smithi070.stdout:6/226: mkdir d0/d3/d19/d31/d46 0 2024-04-02T04:47:45.427 INFO:tasks.workunit.client.0.smithi070.stdout:5/343: creat d5/d17/d2b/f6f x:0 0 0 2024-04-02T04:47:45.427 INFO:tasks.workunit.client.0.smithi070.stdout:6/227: readlink d0/l35 0 2024-04-02T04:47:45.428 INFO:tasks.workunit.client.0.smithi070.stdout:8/357: rmdir d2/d8/d23/d24/d66/d71 0 2024-04-02T04:47:45.429 INFO:tasks.workunit.client.0.smithi070.stdout:8/358: chown d2/d8/d23/d24/l45 27514925 1 2024-04-02T04:47:45.430 INFO:tasks.workunit.client.1.smithi179.stdout:2/194: rename dc/f25 to dc/f3b 0 2024-04-02T04:47:45.431 INFO:tasks.workunit.client.0.smithi070.stdout:4/272: unlink d1/d3/d9/c3c 0 2024-04-02T04:47:45.432 INFO:tasks.workunit.client.0.smithi070.stdout:4/273: write d1/d3/f4c [261180,6430] 0 2024-04-02T04:47:45.432 INFO:tasks.workunit.client.0.smithi070.stdout:3/306: creat d10/d17/f6c x:0 0 0 2024-04-02T04:47:45.435 INFO:tasks.workunit.client.1.smithi179.stdout:5/237: link f2 d8/d1b/d23/f47 0 2024-04-02T04:47:45.436 INFO:tasks.workunit.client.0.smithi070.stdout:1/303: link d0/d9/f14 d0/d9/d1d/f60 0 2024-04-02T04:47:45.439 INFO:tasks.workunit.client.0.smithi070.stdout:8/359: mkdir d2/d8/d29/d22/d5a/d77 0 2024-04-02T04:47:45.442 INFO:tasks.workunit.client.0.smithi070.stdout:4/274: mknod d1/d3/d9/c57 0 2024-04-02T04:47:45.444 INFO:tasks.workunit.client.1.smithi179.stdout:6/324: symlink d3/d12/l78 0 2024-04-02T04:47:45.444 INFO:tasks.workunit.client.0.smithi070.stdout:8/360: dwrite d2/d3/d52/f43 [0,4194304] 0 2024-04-02T04:47:45.445 INFO:tasks.workunit.client.0.smithi070.stdout:3/307: creat d10/d32/d3f/f6d x:0 0 0 2024-04-02T04:47:45.448 INFO:tasks.workunit.client.1.smithi179.stdout:1/322: symlink d3/d5/d2f/d4f/l70 0 2024-04-02T04:47:45.451 INFO:tasks.workunit.client.0.smithi070.stdout:1/304: creat d0/d9/d1d/d22/d2c/d4f/f61 x:0 0 0 2024-04-02T04:47:45.453 INFO:tasks.workunit.client.1.smithi179.stdout:2/195: write dc/f1a [2114775,82283] 0 2024-04-02T04:47:45.453 INFO:tasks.workunit.client.0.smithi070.stdout:5/344: rename d5/l18 to d5/d17/d50/l70 0 2024-04-02T04:47:45.454 INFO:tasks.workunit.client.1.smithi179.stdout:2/196: fsync dc/d29/d31/f3a 0 2024-04-02T04:47:45.459 INFO:tasks.workunit.client.0.smithi070.stdout:8/361: unlink d2/d8/d29/l68 0 2024-04-02T04:47:45.460 INFO:tasks.workunit.client.1.smithi179.stdout:5/238: dread d8/f3e [0,4194304] 0 2024-04-02T04:47:45.461 INFO:tasks.workunit.client.0.smithi070.stdout:1/305: symlink d0/d9/d1d/d22/d2c/l62 0 2024-04-02T04:47:45.461 INFO:tasks.workunit.client.0.smithi070.stdout:7/302: write d7/dd/f38 [4277105,4860] 0 2024-04-02T04:47:45.461 INFO:tasks.workunit.client.0.smithi070.stdout:8/362: read d2/d8/d23/d24/f37 [358851,97700] 0 2024-04-02T04:47:45.462 INFO:tasks.workunit.client.1.smithi179.stdout:3/295: link d8/c25 d8/d26/c64 0 2024-04-02T04:47:45.462 INFO:tasks.workunit.client.0.smithi070.stdout:7/303: chown d7/d2f/d46/f51 266807 1 2024-04-02T04:47:45.463 INFO:tasks.workunit.client.1.smithi179.stdout:5/239: dread f6 [0,4194304] 0 2024-04-02T04:47:45.464 INFO:tasks.workunit.client.1.smithi179.stdout:3/296: write d8/d26/f5b [625490,26309] 0 2024-04-02T04:47:45.464 INFO:tasks.workunit.client.0.smithi070.stdout:7/304: write d7/dd/df/d34/f2a [386706,48077] 0 2024-04-02T04:47:45.467 INFO:tasks.workunit.client.0.smithi070.stdout:3/308: dread d10/f22 [0,4194304] 0 2024-04-02T04:47:45.467 INFO:tasks.workunit.client.0.smithi070.stdout:5/345: unlink d5/d17/d2b/f3a 0 2024-04-02T04:47:45.468 INFO:tasks.workunit.client.0.smithi070.stdout:5/346: chown d5/d17 11220 1 2024-04-02T04:47:45.469 INFO:tasks.workunit.client.0.smithi070.stdout:5/347: dread - d5/d14/f60 zero size 2024-04-02T04:47:45.470 INFO:tasks.workunit.client.0.smithi070.stdout:7/305: dread d7/d2f/d46/f51 [0,4194304] 0 2024-04-02T04:47:45.470 INFO:tasks.workunit.client.0.smithi070.stdout:1/306: rmdir d0/d7/d1a 39 2024-04-02T04:47:45.471 INFO:tasks.workunit.client.1.smithi179.stdout:8/285: write d3/d32/d21/d22/d33/d3c/f44 [957306,99586] 0 2024-04-02T04:47:45.471 INFO:tasks.workunit.client.0.smithi070.stdout:1/307: fdatasync d0/d9/d1d/f26 0 2024-04-02T04:47:45.478 INFO:tasks.workunit.client.0.smithi070.stdout:1/308: dwrite d0/f1c [0,4194304] 0 2024-04-02T04:47:45.478 INFO:tasks.workunit.client.0.smithi070.stdout:3/309: chown d10/d17/l57 889 1 2024-04-02T04:47:45.479 INFO:tasks.workunit.client.1.smithi179.stdout:9/316: getdents d0/d1/d11/d29/d40 0 2024-04-02T04:47:45.479 INFO:tasks.workunit.client.0.smithi070.stdout:5/348: dread d5/d17/d27/f28 [0,4194304] 0 2024-04-02T04:47:45.483 INFO:tasks.workunit.client.0.smithi070.stdout:7/306: write d7/d9/f1b [4403596,46042] 0 2024-04-02T04:47:45.488 INFO:tasks.workunit.client.1.smithi179.stdout:3/297: rename d8/d26/d36/c44 to d8/c65 0 2024-04-02T04:47:45.490 INFO:tasks.workunit.client.1.smithi179.stdout:7/357: getdents d1/dd/d23 0 2024-04-02T04:47:45.495 INFO:tasks.workunit.client.1.smithi179.stdout:3/298: dwrite d8/d26/f33 [0,4194304] 0 2024-04-02T04:47:45.497 INFO:tasks.workunit.client.1.smithi179.stdout:2/197: mkdir dc/d3c 0 2024-04-02T04:47:45.498 INFO:tasks.workunit.client.0.smithi070.stdout:3/310: symlink d10/d32/d4b/l6e 0 2024-04-02T04:47:45.498 INFO:tasks.workunit.client.1.smithi179.stdout:7/358: dwrite d1/d9/f1d [0,4194304] 0 2024-04-02T04:47:45.498 INFO:tasks.workunit.client.0.smithi070.stdout:3/311: readlink d10/d17/l31 0 2024-04-02T04:47:45.500 INFO:tasks.workunit.client.1.smithi179.stdout:9/317: creat d0/d1/d11/d29/d40/f5d x:0 0 0 2024-04-02T04:47:45.502 INFO:tasks.workunit.client.1.smithi179.stdout:6/325: truncate d3/d12/d23/f2b 1783481 0 2024-04-02T04:47:45.502 INFO:tasks.workunit.client.1.smithi179.stdout:0/357: sync 2024-04-02T04:47:45.509 INFO:tasks.workunit.client.0.smithi070.stdout:1/309: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:45.509 INFO:tasks.workunit.client.0.smithi070.stdout:6/228: rename d0/d3/f1e to d0/d3/d19/f47 0 2024-04-02T04:47:45.512 INFO:tasks.workunit.client.0.smithi070.stdout:1/310: dread - d0/d9/d1d/d22/f5d zero size 2024-04-02T04:47:45.512 INFO:tasks.workunit.client.1.smithi179.stdout:8/286: rename d3/d32/f47 to d3/d32/d21/d22/f62 0 2024-04-02T04:47:45.512 INFO:tasks.workunit.client.0.smithi070.stdout:1/311: readlink d0/d7/l53 0 2024-04-02T04:47:45.513 INFO:tasks.workunit.client.0.smithi070.stdout:1/312: fsync d0/d9/d1d/f16 0 2024-04-02T04:47:45.514 INFO:tasks.workunit.client.0.smithi070.stdout:1/313: chown d0/d9/d1d/f16 13423 1 2024-04-02T04:47:45.514 INFO:tasks.workunit.client.1.smithi179.stdout:3/299: dread d8/f1c [0,4194304] 0 2024-04-02T04:47:45.515 INFO:tasks.workunit.client.0.smithi070.stdout:1/314: truncate d0/d9/fc 1231350 0 2024-04-02T04:47:45.520 INFO:tasks.workunit.client.1.smithi179.stdout:5/240: truncate d8/f27 47876 0 2024-04-02T04:47:45.521 INFO:tasks.workunit.client.1.smithi179.stdout:5/241: stat d8/d1b/d23/d32/c45 0 2024-04-02T04:47:45.522 INFO:tasks.workunit.client.0.smithi070.stdout:5/349: mknod d5/d9/d23/d33/c71 0 2024-04-02T04:47:45.524 INFO:tasks.workunit.client.1.smithi179.stdout:3/300: dwrite d8/d26/d36/f5c [0,4194304] 0 2024-04-02T04:47:45.526 INFO:tasks.workunit.client.1.smithi179.stdout:3/301: write d8/d57/f5a [756022,21419] 0 2024-04-02T04:47:45.528 INFO:tasks.workunit.client.1.smithi179.stdout:3/302: fdatasync d8/d26/f4e 0 2024-04-02T04:47:45.530 INFO:tasks.workunit.client.1.smithi179.stdout:3/303: write d8/d26/d36/f47 [1325293,107529] 0 2024-04-02T04:47:45.531 INFO:tasks.workunit.client.0.smithi070.stdout:8/363: rename d2/d3/d35/d3a/d4f/d59/f65 to d2/d3/d35/f78 0 2024-04-02T04:47:45.531 INFO:tasks.workunit.client.1.smithi179.stdout:5/242: dwrite f1 [0,4194304] 0 2024-04-02T04:47:45.533 INFO:tasks.workunit.client.1.smithi179.stdout:5/243: truncate d8/d1b/d23/f47 8950544 0 2024-04-02T04:47:45.535 INFO:tasks.workunit.client.1.smithi179.stdout:6/326: mknod d3/d42/c79 0 2024-04-02T04:47:45.536 INFO:tasks.workunit.client.1.smithi179.stdout:6/327: chown f2 498251 1 2024-04-02T04:47:45.537 INFO:tasks.workunit.client.0.smithi070.stdout:1/315: creat d0/d54/f63 x:0 0 0 2024-04-02T04:47:45.539 INFO:tasks.workunit.client.0.smithi070.stdout:3/312: getdents d10/d32/d4b/d5e 0 2024-04-02T04:47:45.540 INFO:tasks.workunit.client.0.smithi070.stdout:3/313: stat d10/l3d 0 2024-04-02T04:47:45.541 INFO:tasks.workunit.client.0.smithi070.stdout:6/229: dwrite d0/d3/d19/f47 [4194304,4194304] 0 2024-04-02T04:47:45.543 INFO:tasks.workunit.client.0.smithi070.stdout:7/307: truncate d7/d36/f62 79838 0 2024-04-02T04:47:45.544 INFO:tasks.workunit.client.1.smithi179.stdout:1/323: link d3/lc d3/d5/d2f/d4f/d42/l71 0 2024-04-02T04:47:45.549 INFO:tasks.workunit.client.1.smithi179.stdout:1/324: read d3/d5/de/f32 [891708,83745] 0 2024-04-02T04:47:45.549 INFO:tasks.workunit.client.0.smithi070.stdout:6/230: dwrite d0/d20/f41 [0,4194304] 0 2024-04-02T04:47:45.550 INFO:tasks.workunit.client.1.smithi179.stdout:1/325: chown d3/d5/c17 4514842 1 2024-04-02T04:47:45.552 INFO:tasks.workunit.client.1.smithi179.stdout:7/359: dwrite d1/d9/d27/f49 [0,4194304] 0 2024-04-02T04:47:45.553 INFO:tasks.workunit.client.0.smithi070.stdout:6/231: dread d0/d3/d19/d31/f36 [0,4194304] 0 2024-04-02T04:47:45.553 INFO:tasks.workunit.client.0.smithi070.stdout:6/232: chown d0/d20 14995759 1 2024-04-02T04:47:45.556 INFO:tasks.workunit.client.0.smithi070.stdout:1/316: mkdir d0/d54/d64 0 2024-04-02T04:47:45.559 INFO:tasks.workunit.client.1.smithi179.stdout:2/198: mknod dc/c3d 0 2024-04-02T04:47:45.559 INFO:tasks.workunit.client.0.smithi070.stdout:3/314: dread d10/d32/d3f/f65 [0,4194304] 0 2024-04-02T04:47:45.561 INFO:tasks.workunit.client.0.smithi070.stdout:3/315: write d10/d17/f6c [332726,128375] 0 2024-04-02T04:47:45.563 INFO:tasks.workunit.client.0.smithi070.stdout:3/316: read fd [4006815,32884] 0 2024-04-02T04:47:45.564 INFO:tasks.workunit.client.0.smithi070.stdout:7/308: fsync d7/d4e/d55/f5a 0 2024-04-02T04:47:45.564 INFO:tasks.workunit.client.1.smithi179.stdout:6/328: dread d3/d12/d23/d26/f27 [0,4194304] 0 2024-04-02T04:47:45.566 INFO:tasks.workunit.client.1.smithi179.stdout:4/385: sync 2024-04-02T04:47:45.570 INFO:tasks.workunit.client.1.smithi179.stdout:9/318: symlink d0/d19/l5e 0 2024-04-02T04:47:45.571 INFO:tasks.workunit.client.0.smithi070.stdout:6/233: rmdir d0/d20 39 2024-04-02T04:47:45.573 INFO:tasks.workunit.client.1.smithi179.stdout:0/358: mknod d1/da/c6b 0 2024-04-02T04:47:45.573 INFO:tasks.workunit.client.0.smithi070.stdout:8/364: dwrite d2/d8/f44 [0,4194304] 0 2024-04-02T04:47:45.574 INFO:tasks.workunit.client.0.smithi070.stdout:8/365: readlink d2/l1a 0 2024-04-02T04:47:45.574 INFO:tasks.workunit.client.1.smithi179.stdout:7/360: dread d1/dd/f20 [0,4194304] 0 2024-04-02T04:47:45.582 INFO:tasks.workunit.client.1.smithi179.stdout:7/361: dwrite d1/dd/d23/f82 [0,4194304] 0 2024-04-02T04:47:45.611 INFO:tasks.workunit.client.1.smithi179.stdout:3/304: mknod d8/d26/d36/c66 0 2024-04-02T04:47:45.612 INFO:tasks.workunit.client.1.smithi179.stdout:3/305: write d8/d26/d36/f59 [285295,74371] 0 2024-04-02T04:47:45.613 INFO:tasks.workunit.client.1.smithi179.stdout:5/244: rename l3 to d8/l48 0 2024-04-02T04:47:45.613 INFO:tasks.workunit.client.1.smithi179.stdout:3/306: fdatasync d8/d26/d36/f4d 0 2024-04-02T04:47:45.614 INFO:tasks.workunit.client.0.smithi070.stdout:7/309: readlink d7/l58 0 2024-04-02T04:47:45.615 INFO:tasks.workunit.client.1.smithi179.stdout:3/307: readlink d8/d26/l3a 0 2024-04-02T04:47:45.615 INFO:tasks.workunit.client.1.smithi179.stdout:3/308: stat d8/d26 0 2024-04-02T04:47:45.615 INFO:tasks.workunit.client.0.smithi070.stdout:5/350: dwrite d5/d17/d50/f5a [0,4194304] 0 2024-04-02T04:47:45.618 INFO:tasks.workunit.client.0.smithi070.stdout:5/351: read d5/ff [5115539,56892] 0 2024-04-02T04:47:45.633 INFO:tasks.workunit.client.0.smithi070.stdout:8/366: rename d2/d8/d29/d22/d4d to d2/d8/d23/d24/d79 0 2024-04-02T04:47:45.634 INFO:tasks.workunit.client.0.smithi070.stdout:1/317: write d0/d7/f48 [1111773,74947] 0 2024-04-02T04:47:45.634 INFO:tasks.workunit.client.0.smithi070.stdout:1/318: stat d0/d7/f20 0 2024-04-02T04:47:45.639 INFO:tasks.workunit.client.0.smithi070.stdout:7/310: rmdir d7/dd/d35 39 2024-04-02T04:47:45.639 INFO:tasks.workunit.client.1.smithi179.stdout:1/326: write d3/d5/d2f/d4f/d42/d53/d5b/f61 [281762,44518] 0 2024-04-02T04:47:45.640 INFO:tasks.workunit.client.1.smithi179.stdout:1/327: chown d3/c58 14 1 2024-04-02T04:47:45.640 INFO:tasks.workunit.client.1.smithi179.stdout:8/287: truncate d3/d32/d21/d22/d33/d3c/f44 678598 0 2024-04-02T04:47:45.644 INFO:tasks.workunit.client.0.smithi070.stdout:5/352: dread d5/f2f [0,4194304] 0 2024-04-02T04:47:45.644 INFO:tasks.workunit.client.0.smithi070.stdout:1/319: fdatasync d0/d4c/f51 0 2024-04-02T04:47:45.645 INFO:tasks.workunit.client.1.smithi179.stdout:2/199: mknod dc/d16/d1e/c3e 0 2024-04-02T04:47:45.648 INFO:tasks.workunit.client.1.smithi179.stdout:6/329: rmdir d3/d12 39 2024-04-02T04:47:45.650 INFO:tasks.workunit.client.0.smithi070.stdout:1/320: rmdir d0/d7/d28 39 2024-04-02T04:47:45.652 INFO:tasks.workunit.client.0.smithi070.stdout:1/321: read d0/d9/d1d/f38 [1305544,44690] 0 2024-04-02T04:47:45.653 INFO:tasks.workunit.client.1.smithi179.stdout:0/359: fdatasync d1/db/f2c 0 2024-04-02T04:47:45.655 INFO:tasks.workunit.client.1.smithi179.stdout:0/360: dread d1/da/df/f21 [0,4194304] 0 2024-04-02T04:47:45.655 INFO:tasks.workunit.client.0.smithi070.stdout:8/367: rename d2/d3/d52/f43 to d2/d8/d29/d22/f7a 0 2024-04-02T04:47:45.657 INFO:tasks.workunit.client.0.smithi070.stdout:8/368: chown d2/f1f 4687 1 2024-04-02T04:47:45.658 INFO:tasks.workunit.client.0.smithi070.stdout:5/353: dread d5/d9/d23/f42 [0,4194304] 0 2024-04-02T04:47:45.659 INFO:tasks.workunit.client.0.smithi070.stdout:1/322: dread d0/d9/fc [0,4194304] 0 2024-04-02T04:47:45.660 INFO:tasks.workunit.client.1.smithi179.stdout:7/362: rename d1/dd/d23/f71 to d1/dd/d76/f86 0 2024-04-02T04:47:45.665 INFO:tasks.workunit.client.0.smithi070.stdout:5/354: dwrite d5/f6 [4194304,4194304] 0 2024-04-02T04:47:45.668 INFO:tasks.workunit.client.1.smithi179.stdout:3/309: symlink d8/d26/l67 0 2024-04-02T04:47:45.670 INFO:tasks.workunit.client.0.smithi070.stdout:7/311: getdents d7/d36 0 2024-04-02T04:47:45.672 INFO:tasks.workunit.client.1.smithi179.stdout:2/200: creat dc/d16/f3f x:0 0 0 2024-04-02T04:47:45.674 INFO:tasks.workunit.client.0.smithi070.stdout:5/355: rmdir d5/d17/d27/d2e 39 2024-04-02T04:47:45.676 INFO:tasks.workunit.client.1.smithi179.stdout:6/330: write d3/d12/d23/d2f/f56 [102385,103785] 0 2024-04-02T04:47:45.678 INFO:tasks.workunit.client.1.smithi179.stdout:4/386: creat d1/d15/d6e/f80 x:0 0 0 2024-04-02T04:47:45.679 INFO:tasks.workunit.client.1.smithi179.stdout:9/319: symlink d0/l5f 0 2024-04-02T04:47:45.681 INFO:tasks.workunit.client.0.smithi070.stdout:7/312: creat d7/d4e/d55/f64 x:0 0 0 2024-04-02T04:47:45.683 INFO:tasks.workunit.client.0.smithi070.stdout:1/323: truncate d0/d9/d1d/f60 4145222 0 2024-04-02T04:47:45.683 INFO:tasks.workunit.client.1.smithi179.stdout:4/387: dwrite d1/d3/df/f59 [4194304,4194304] 0 2024-04-02T04:47:45.684 INFO:tasks.workunit.client.1.smithi179.stdout:4/388: read - d1/d15/d6e/f80 zero size 2024-04-02T04:47:45.685 INFO:tasks.workunit.client.1.smithi179.stdout:4/389: chown d1/d15/d6e/f80 85416351 1 2024-04-02T04:47:45.687 INFO:tasks.workunit.client.1.smithi179.stdout:4/390: read - d1/d3/df/f6f zero size 2024-04-02T04:47:45.689 INFO:tasks.workunit.client.1.smithi179.stdout:7/363: rmdir d1/d9 39 2024-04-02T04:47:45.689 INFO:tasks.workunit.client.1.smithi179.stdout:0/361: dread d1/f23 [0,4194304] 0 2024-04-02T04:47:45.691 INFO:tasks.workunit.client.1.smithi179.stdout:0/362: read d1/da/df/f33 [2736421,91819] 0 2024-04-02T04:47:45.693 INFO:tasks.workunit.client.0.smithi070.stdout:1/324: mknod d0/d9/d1d/d22/d56/c65 0 2024-04-02T04:47:45.695 INFO:tasks.workunit.client.1.smithi179.stdout:5/245: truncate d8/f10 119497 0 2024-04-02T04:47:45.697 INFO:tasks.workunit.client.1.smithi179.stdout:1/328: mkdir d3/d5/d2f/d4f/d42/d53/d72 0 2024-04-02T04:47:45.699 INFO:tasks.workunit.client.0.smithi070.stdout:7/313: dread d7/dd/df/d34/f2a [0,4194304] 0 2024-04-02T04:47:45.699 INFO:tasks.workunit.client.1.smithi179.stdout:3/310: rename d8/c53 to d8/d57/c68 0 2024-04-02T04:47:45.703 INFO:tasks.workunit.client.1.smithi179.stdout:8/288: symlink d3/d32/d21/d45/d5a/l63 0 2024-04-02T04:47:45.706 INFO:tasks.workunit.client.1.smithi179.stdout:3/311: dwrite d8/d26/f34 [0,4194304] 0 2024-04-02T04:47:45.706 INFO:tasks.workunit.client.1.smithi179.stdout:8/289: write d3/d32/d21/d51/f55 [1012131,86913] 0 2024-04-02T04:47:45.708 INFO:tasks.workunit.client.1.smithi179.stdout:3/312: write d8/d26/d36/f4d [526132,70460] 0 2024-04-02T04:47:45.708 INFO:tasks.workunit.client.1.smithi179.stdout:6/331: mknod d3/d12/d23/d26/d57/c7a 0 2024-04-02T04:47:45.710 INFO:tasks.workunit.client.1.smithi179.stdout:3/313: fdatasync d8/d26/f4e 0 2024-04-02T04:47:45.711 INFO:tasks.workunit.client.1.smithi179.stdout:6/332: write d3/d12/d23/d2f/f3a [802542,70894] 0 2024-04-02T04:47:45.714 INFO:tasks.workunit.client.1.smithi179.stdout:7/364: dread d1/dd/d23/f25 [0,4194304] 0 2024-04-02T04:47:45.715 INFO:tasks.workunit.client.1.smithi179.stdout:7/365: read d1/f32 [611166,53104] 0 2024-04-02T04:47:45.716 INFO:tasks.workunit.client.0.smithi070.stdout:1/325: mkdir d0/d7/d1a/d66 0 2024-04-02T04:47:45.718 INFO:tasks.workunit.client.1.smithi179.stdout:6/333: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:45.718 INFO:tasks.workunit.client.0.smithi070.stdout:7/314: truncate d7/d2f/f5e 252885 0 2024-04-02T04:47:45.722 INFO:tasks.workunit.client.1.smithi179.stdout:7/366: dwrite d1/f4e [0,4194304] 0 2024-04-02T04:47:45.723 INFO:tasks.workunit.client.0.smithi070.stdout:7/315: mkdir d7/d2f/d46/d65 0 2024-04-02T04:47:45.724 INFO:tasks.workunit.client.0.smithi070.stdout:1/326: mknod d0/d7/d28/c67 0 2024-04-02T04:47:45.725 INFO:tasks.workunit.client.0.smithi070.stdout:1/327: chown d0/l44 4494382 1 2024-04-02T04:47:45.726 INFO:tasks.workunit.client.1.smithi179.stdout:6/334: dwrite d3/fb [0,4194304] 0 2024-04-02T04:47:45.728 INFO:tasks.workunit.client.0.smithi070.stdout:1/328: dread - d0/d9/d1d/f5e zero size 2024-04-02T04:47:45.731 INFO:tasks.workunit.client.0.smithi070.stdout:1/329: truncate d0/d54/f63 1030775 0 2024-04-02T04:47:45.731 INFO:tasks.workunit.client.1.smithi179.stdout:0/363: creat d1/d5b/f6c x:0 0 0 2024-04-02T04:47:45.733 INFO:tasks.workunit.client.1.smithi179.stdout:1/329: mknod d3/d5/d2f/d4f/c73 0 2024-04-02T04:47:45.735 INFO:tasks.workunit.client.0.smithi070.stdout:1/330: mknod d0/d9/d1d/d22/d56/c68 0 2024-04-02T04:47:45.736 INFO:tasks.workunit.client.1.smithi179.stdout:6/335: dwrite d3/d12/d23/d39/d4e/d6c/f50 [0,4194304] 0 2024-04-02T04:47:45.737 INFO:tasks.workunit.client.1.smithi179.stdout:6/336: write d3/fb [1634249,66541] 0 2024-04-02T04:47:45.739 INFO:tasks.workunit.client.1.smithi179.stdout:1/330: dread d3/d5/d2f/d4f/d42/d53/d5b/f61 [0,4194304] 0 2024-04-02T04:47:45.740 INFO:tasks.workunit.client.1.smithi179.stdout:8/290: rmdir d3/d32/d21/d51 39 2024-04-02T04:47:45.746 INFO:tasks.workunit.client.0.smithi070.stdout:7/316: rename d7/dd/d35 to d7/dd/d66 0 2024-04-02T04:47:45.751 INFO:tasks.workunit.client.1.smithi179.stdout:0/364: dread d1/da/f27 [0,4194304] 0 2024-04-02T04:47:45.751 INFO:tasks.workunit.client.0.smithi070.stdout:7/317: creat d7/d2f/d46/d65/f67 x:0 0 0 2024-04-02T04:47:45.751 INFO:tasks.workunit.client.1.smithi179.stdout:0/365: readlink d1/da/l19 0 2024-04-02T04:47:45.752 INFO:tasks.workunit.client.0.smithi070.stdout:7/318: dread - d7/d4e/d55/f64 zero size 2024-04-02T04:47:45.752 INFO:tasks.workunit.client.1.smithi179.stdout:0/366: stat d1/l45 0 2024-04-02T04:47:45.753 INFO:tasks.workunit.client.0.smithi070.stdout:7/319: write d7/d2f/d46/f53 [933207,34600] 0 2024-04-02T04:47:45.759 INFO:tasks.workunit.client.0.smithi070.stdout:7/320: unlink d7/d2f/f2d 0 2024-04-02T04:47:45.759 INFO:tasks.workunit.client.0.smithi070.stdout:7/321: chown d7/d9/c3a 4 1 2024-04-02T04:47:45.760 INFO:tasks.workunit.client.1.smithi179.stdout:5/246: fsync d8/f1d 0 2024-04-02T04:47:45.761 INFO:tasks.workunit.client.1.smithi179.stdout:5/247: write d8/d1b/d23/d36/f41 [3363186,106185] 0 2024-04-02T04:47:45.763 INFO:tasks.workunit.client.0.smithi070.stdout:7/322: mkdir d7/dd/df/d34/d68 0 2024-04-02T04:47:45.764 INFO:tasks.workunit.client.0.smithi070.stdout:7/323: write d7/d2f/f3b [1279611,57550] 0 2024-04-02T04:47:45.765 INFO:tasks.workunit.client.0.smithi070.stdout:7/324: dread - d7/d4e/d55/f5a zero size 2024-04-02T04:47:45.767 INFO:tasks.workunit.client.0.smithi070.stdout:7/325: write d7/d2f/d46/d65/f67 [297658,51649] 0 2024-04-02T04:47:45.770 INFO:tasks.workunit.client.0.smithi070.stdout:1/331: dread d0/d7/d1a/f2e [0,4194304] 0 2024-04-02T04:47:45.774 INFO:tasks.workunit.client.1.smithi179.stdout:0/367: mkdir d1/da/d28/d6d 0 2024-04-02T04:47:45.774 INFO:tasks.workunit.client.0.smithi070.stdout:7/326: dwrite d7/d4e/f4f [0,4194304] 0 2024-04-02T04:47:45.775 INFO:tasks.workunit.client.1.smithi179.stdout:0/368: write d1/da/d28/f3a [676674,7010] 0 2024-04-02T04:47:45.778 INFO:tasks.workunit.client.1.smithi179.stdout:3/314: truncate d8/f11 1136733 0 2024-04-02T04:47:45.779 INFO:tasks.workunit.client.1.smithi179.stdout:7/367: mkdir d1/d9/d27/d87 0 2024-04-02T04:47:45.780 INFO:tasks.workunit.client.0.smithi070.stdout:1/332: dwrite d0/d9/f3e [0,4194304] 0 2024-04-02T04:47:45.781 INFO:tasks.workunit.client.1.smithi179.stdout:1/331: dread d3/d5/f1d [0,4194304] 0 2024-04-02T04:47:45.782 INFO:tasks.workunit.client.0.smithi070.stdout:7/327: write d7/dd/df/d34/f2a [386917,42036] 0 2024-04-02T04:47:45.783 INFO:tasks.workunit.client.0.smithi070.stdout:7/328: read d7/dd/df/d34/f2a [3665356,60136] 0 2024-04-02T04:47:45.786 INFO:tasks.workunit.client.0.smithi070.stdout:1/333: rename d0/d7/f48 to d0/d9/d1d/d22/d56/f69 0 2024-04-02T04:47:45.787 INFO:tasks.workunit.client.0.smithi070.stdout:1/334: dread - d0/d9/d1d/f5e zero size 2024-04-02T04:47:45.787 INFO:tasks.workunit.client.1.smithi179.stdout:5/248: fdatasync f6 0 2024-04-02T04:47:45.790 INFO:tasks.workunit.client.0.smithi070.stdout:1/335: dread d0/d54/f63 [0,4194304] 0 2024-04-02T04:47:45.793 INFO:tasks.workunit.client.1.smithi179.stdout:2/201: rename dc/d16/f2b to dc/d16/d1e/f40 0 2024-04-02T04:47:45.796 INFO:tasks.workunit.client.0.smithi070.stdout:1/336: dread d0/d9/d1d/f42 [0,4194304] 0 2024-04-02T04:47:45.797 INFO:tasks.workunit.client.0.smithi070.stdout:1/337: stat d0/d7/d1a/l49 0 2024-04-02T04:47:45.798 INFO:tasks.workunit.client.0.smithi070.stdout:1/338: chown d0/d9/d1d/d22/c39 5080238 1 2024-04-02T04:47:45.798 INFO:tasks.workunit.client.1.smithi179.stdout:9/320: link d0/d1/d12/l39 d0/d19/d1c/d45/l60 0 2024-04-02T04:47:45.799 INFO:tasks.workunit.client.1.smithi179.stdout:7/368: dread d1/dd/d23/f2f [0,4194304] 0 2024-04-02T04:47:45.799 INFO:tasks.workunit.client.0.smithi070.stdout:7/329: unlink d7/d9/f17 0 2024-04-02T04:47:45.800 INFO:tasks.workunit.client.1.smithi179.stdout:7/369: chown d1/d9/ce 35811458 1 2024-04-02T04:47:45.804 INFO:tasks.workunit.client.1.smithi179.stdout:4/391: getdents d1 0 2024-04-02T04:47:45.806 INFO:tasks.workunit.client.0.smithi070.stdout:7/330: rename d7/fc to d7/d2f/d46/d65/f69 0 2024-04-02T04:47:45.807 INFO:tasks.workunit.client.1.smithi179.stdout:3/315: creat d8/d26/f69 x:0 0 0 2024-04-02T04:47:45.807 INFO:tasks.workunit.client.0.smithi070.stdout:7/331: write d7/dd/f38 [4004726,19145] 0 2024-04-02T04:47:45.808 INFO:tasks.workunit.client.1.smithi179.stdout:3/316: chown d8/c17 1924 1 2024-04-02T04:47:45.814 INFO:tasks.workunit.client.1.smithi179.stdout:1/332: mknod d3/d5/de/d33/c74 0 2024-04-02T04:47:45.815 INFO:tasks.workunit.client.0.smithi070.stdout:7/332: dwrite d7/dd/df/d34/f4a [0,4194304] 0 2024-04-02T04:47:45.816 INFO:tasks.workunit.client.1.smithi179.stdout:1/333: readlink d3/d5/d2f/d4f/d35/l3f 0 2024-04-02T04:47:45.818 INFO:tasks.workunit.client.1.smithi179.stdout:0/369: dwrite d1/f23 [0,4194304] 0 2024-04-02T04:47:45.823 INFO:tasks.workunit.client.0.smithi070.stdout:1/339: dread d0/d9/d1d/f38 [0,4194304] 0 2024-04-02T04:47:45.823 INFO:tasks.workunit.client.1.smithi179.stdout:2/202: chown dc/d16/f3f 512264693 1 2024-04-02T04:47:45.824 INFO:tasks.workunit.client.1.smithi179.stdout:9/321: symlink d0/d1/d11/d29/d40/l61 0 2024-04-02T04:47:45.826 INFO:tasks.workunit.client.1.smithi179.stdout:7/370: fdatasync d1/d9/d35/d6b/f73 0 2024-04-02T04:47:45.831 INFO:tasks.workunit.client.0.smithi070.stdout:7/333: write d7/d2f/f3b [2882064,45325] 0 2024-04-02T04:47:45.832 INFO:tasks.workunit.client.0.smithi070.stdout:1/340: read d0/d7/d1a/f24 [933595,80483] 0 2024-04-02T04:47:45.833 INFO:tasks.workunit.client.0.smithi070.stdout:1/341: chown d0/d7/d28 1782518 1 2024-04-02T04:47:45.833 INFO:tasks.workunit.client.1.smithi179.stdout:3/317: rename d8/d26/c60 to d8/d26/c6a 0 2024-04-02T04:47:45.835 INFO:tasks.workunit.client.0.smithi070.stdout:7/334: symlink d7/l6a 0 2024-04-02T04:47:45.836 INFO:tasks.workunit.client.1.smithi179.stdout:0/370: fdatasync d1/da/f3c 0 2024-04-02T04:47:45.837 INFO:tasks.workunit.client.1.smithi179.stdout:0/371: chown d1/da/d48 10 1 2024-04-02T04:47:45.840 INFO:tasks.workunit.client.0.smithi070.stdout:7/335: chown d7/d2f/d46/d65/f67 10251 1 2024-04-02T04:47:45.840 INFO:tasks.workunit.client.1.smithi179.stdout:6/337: getdents d3 0 2024-04-02T04:47:45.840 INFO:tasks.workunit.client.1.smithi179.stdout:2/203: creat dc/d29/d31/f41 x:0 0 0 2024-04-02T04:47:45.840 INFO:tasks.workunit.client.1.smithi179.stdout:6/338: chown d3/f1c 125653 1 2024-04-02T04:47:45.841 INFO:tasks.workunit.client.0.smithi070.stdout:1/342: creat d0/d7/f6a x:0 0 0 2024-04-02T04:47:45.841 INFO:tasks.workunit.client.0.smithi070.stdout:7/336: write d7/dd/df/f32 [3890621,126353] 0 2024-04-02T04:47:45.841 INFO:tasks.workunit.client.1.smithi179.stdout:8/291: sync 2024-04-02T04:47:45.843 INFO:tasks.workunit.client.1.smithi179.stdout:7/371: unlink d1/l70 0 2024-04-02T04:47:45.844 INFO:tasks.workunit.client.1.smithi179.stdout:7/372: readlink d1/dd/d16/d4d/l52 0 2024-04-02T04:47:45.845 INFO:tasks.workunit.client.1.smithi179.stdout:4/392: getdents d1/d15/d7c 0 2024-04-02T04:47:45.845 INFO:tasks.workunit.client.1.smithi179.stdout:7/373: write d1/f83 [31581,102534] 0 2024-04-02T04:47:45.846 INFO:tasks.workunit.client.1.smithi179.stdout:6/339: dread d3/d12/d23/d2f/f3c [0,4194304] 0 2024-04-02T04:47:45.847 INFO:tasks.workunit.client.1.smithi179.stdout:7/374: write d1/dd/d16/d74/f7a [579081,48051] 0 2024-04-02T04:47:45.848 INFO:tasks.workunit.client.0.smithi070.stdout:1/343: dwrite d0/d4c/f51 [0,4194304] 0 2024-04-02T04:47:45.850 INFO:tasks.workunit.client.0.smithi070.stdout:7/337: symlink d7/d9/d19/d54/l6b 0 2024-04-02T04:47:45.851 INFO:tasks.workunit.client.1.smithi179.stdout:5/249: creat d8/d1b/f49 x:0 0 0 2024-04-02T04:47:45.851 INFO:tasks.workunit.client.1.smithi179.stdout:0/372: fsync d1/da/d28/d37/f5a 0 2024-04-02T04:47:45.853 INFO:tasks.workunit.client.1.smithi179.stdout:2/204: fdatasync dc/d16/d1e/f2d 0 2024-04-02T04:47:45.853 INFO:tasks.workunit.client.1.smithi179.stdout:0/373: fdatasync d1/da/d28/f3a 0 2024-04-02T04:47:45.853 INFO:tasks.workunit.client.1.smithi179.stdout:1/334: dread d3/d5/de/f1a [0,4194304] 0 2024-04-02T04:47:45.854 INFO:tasks.workunit.client.0.smithi070.stdout:1/344: truncate d0/d7/f10 3201054 0 2024-04-02T04:47:45.857 INFO:tasks.workunit.client.0.smithi070.stdout:7/338: mkdir d7/d4e/d55/d6c 0 2024-04-02T04:47:45.857 INFO:tasks.workunit.client.0.smithi070.stdout:7/339: stat d7 0 2024-04-02T04:47:45.858 INFO:tasks.workunit.client.0.smithi070.stdout:7/340: readlink d7/dd/l52 0 2024-04-02T04:47:45.859 INFO:tasks.workunit.client.0.smithi070.stdout:1/345: rmdir d0/d9/d1d/d22/d56 39 2024-04-02T04:47:45.861 INFO:tasks.workunit.client.0.smithi070.stdout:7/341: read d7/d2f/d46/f51 [2394740,88899] 0 2024-04-02T04:47:45.862 INFO:tasks.workunit.client.1.smithi179.stdout:3/318: creat d8/d57/d63/f6b x:0 0 0 2024-04-02T04:47:45.862 INFO:tasks.workunit.client.1.smithi179.stdout:8/292: dread d3/d32/f19 [0,4194304] 0 2024-04-02T04:47:45.862 INFO:tasks.workunit.client.0.smithi070.stdout:1/346: mknod d0/d7/d1a/c6b 0 2024-04-02T04:47:45.864 INFO:tasks.workunit.client.1.smithi179.stdout:6/340: symlink d3/d42/l7b 0 2024-04-02T04:47:45.865 INFO:tasks.workunit.client.1.smithi179.stdout:6/341: write d3/d12/d23/d26/f69 [327780,88886] 0 2024-04-02T04:47:45.867 INFO:tasks.workunit.client.1.smithi179.stdout:7/375: write d1/d9/f6e [763941,18393] 0 2024-04-02T04:47:45.867 INFO:tasks.workunit.client.0.smithi070.stdout:7/342: dread d7/d9/f21 [0,4194304] 0 2024-04-02T04:47:45.869 INFO:tasks.workunit.client.0.smithi070.stdout:7/343: write d7/d2f/d46/f53 [463111,9265] 0 2024-04-02T04:47:45.870 INFO:tasks.workunit.client.0.smithi070.stdout:7/344: readlink d7/dd/df/l3f 0 2024-04-02T04:47:45.870 INFO:tasks.workunit.client.1.smithi179.stdout:9/322: truncate d0/d1/d12/f25 2904043 0 2024-04-02T04:47:45.871 INFO:tasks.workunit.client.1.smithi179.stdout:9/323: read d0/d19/d1c/f31 [949029,16089] 0 2024-04-02T04:47:45.876 INFO:tasks.workunit.client.0.smithi070.stdout:7/345: rename d7/d4e/f4f to d7/d4e/d55/d6c/f6d 0 2024-04-02T04:47:45.876 INFO:tasks.workunit.client.1.smithi179.stdout:5/250: write d8/f1d [894953,5422] 0 2024-04-02T04:47:45.877 INFO:tasks.workunit.client.0.smithi070.stdout:1/347: creat d0/d7/f6c x:0 0 0 2024-04-02T04:47:45.878 INFO:tasks.workunit.client.1.smithi179.stdout:1/335: symlink d3/d5/d2f/d4f/d35/d49/l75 0 2024-04-02T04:47:45.881 INFO:tasks.workunit.client.0.smithi070.stdout:0/380: sync 2024-04-02T04:47:45.881 INFO:tasks.workunit.client.0.smithi070.stdout:2/385: sync 2024-04-02T04:47:45.881 INFO:tasks.workunit.client.0.smithi070.stdout:9/287: sync 2024-04-02T04:47:45.883 INFO:tasks.workunit.client.0.smithi070.stdout:9/288: stat d4/d10/d11/f4e 0 2024-04-02T04:47:45.884 INFO:tasks.workunit.client.0.smithi070.stdout:1/348: rename d0/d54/d59 to d0/d54/d64/d6d 0 2024-04-02T04:47:45.887 INFO:tasks.workunit.client.1.smithi179.stdout:8/293: symlink d3/d32/d21/d45/l64 0 2024-04-02T04:47:45.889 INFO:tasks.workunit.client.1.smithi179.stdout:2/205: dread dc/d16/f24 [4194304,4194304] 0 2024-04-02T04:47:45.889 INFO:tasks.workunit.client.0.smithi070.stdout:2/386: dwrite d2/f14 [4194304,4194304] 0 2024-04-02T04:47:45.890 INFO:tasks.workunit.client.1.smithi179.stdout:6/342: mkdir d3/d12/d23/d39/d4e/d66/d7c 0 2024-04-02T04:47:45.891 INFO:tasks.workunit.client.1.smithi179.stdout:1/336: dread d3/fa [0,4194304] 0 2024-04-02T04:47:45.893 INFO:tasks.workunit.client.1.smithi179.stdout:7/376: read d1/dd/f14 [1303006,112527] 0 2024-04-02T04:47:45.895 INFO:tasks.workunit.client.0.smithi070.stdout:1/349: dwrite d0/d9/d1d/d22/d2c/d4f/f61 [0,4194304] 0 2024-04-02T04:47:45.896 INFO:tasks.workunit.client.0.smithi070.stdout:1/350: chown d0/d7/c15 62944 1 2024-04-02T04:47:45.898 INFO:tasks.workunit.client.0.smithi070.stdout:1/351: stat d0/d4c/d5a 0 2024-04-02T04:47:45.900 INFO:tasks.workunit.client.1.smithi179.stdout:6/343: dread f2 [0,4194304] 0 2024-04-02T04:47:45.902 INFO:tasks.workunit.client.1.smithi179.stdout:9/324: rmdir d0 39 2024-04-02T04:47:45.907 INFO:tasks.workunit.client.1.smithi179.stdout:4/393: dwrite d1/d3/dc/fe [0,4194304] 0 2024-04-02T04:47:45.909 INFO:tasks.workunit.client.0.smithi070.stdout:0/381: dread d4/f9 [0,4194304] 0 2024-04-02T04:47:45.911 INFO:tasks.workunit.client.0.smithi070.stdout:7/346: truncate d7/d2f/f3b 2680051 0 2024-04-02T04:47:45.912 INFO:tasks.workunit.client.1.smithi179.stdout:0/374: symlink d1/da/d28/d6d/l6e 0 2024-04-02T04:47:45.913 INFO:tasks.workunit.client.0.smithi070.stdout:2/387: creat d2/d10/d19/f7b x:0 0 0 2024-04-02T04:47:45.914 INFO:tasks.workunit.client.1.smithi179.stdout:0/375: stat d1/da/d52/c58 0 2024-04-02T04:47:45.914 INFO:tasks.workunit.client.0.smithi070.stdout:7/347: dread d7/dd/df/f4d [0,4194304] 0 2024-04-02T04:47:45.915 INFO:tasks.workunit.client.1.smithi179.stdout:4/394: dwrite d1/d3/dc/d27/d62/f5e [0,4194304] 0 2024-04-02T04:47:45.918 INFO:tasks.workunit.client.1.smithi179.stdout:5/251: write d8/d1b/d23/f47 [8610750,55281] 0 2024-04-02T04:47:45.919 INFO:tasks.workunit.client.1.smithi179.stdout:8/294: mkdir d3/d32/d21/d50/d65 0 2024-04-02T04:47:45.928 INFO:tasks.workunit.client.0.smithi070.stdout:2/388: unlink d2/d24/f37 0 2024-04-02T04:47:45.930 INFO:tasks.workunit.client.0.smithi070.stdout:7/348: mknod d7/d4e/d55/d6c/c6e 0 2024-04-02T04:47:45.932 INFO:tasks.workunit.client.0.smithi070.stdout:6/234: sync 2024-04-02T04:47:45.932 INFO:tasks.workunit.client.0.smithi070.stdout:4/275: sync 2024-04-02T04:47:45.932 INFO:tasks.workunit.client.0.smithi070.stdout:5/356: sync 2024-04-02T04:47:45.933 INFO:tasks.workunit.client.0.smithi070.stdout:3/317: sync 2024-04-02T04:47:45.933 INFO:tasks.workunit.client.0.smithi070.stdout:8/369: sync 2024-04-02T04:47:45.934 INFO:tasks.workunit.client.0.smithi070.stdout:9/289: creat d4/d13/d16/f61 x:0 0 0 2024-04-02T04:47:45.935 INFO:tasks.workunit.client.0.smithi070.stdout:7/349: write d7/d4e/d55/f5a [865203,77575] 0 2024-04-02T04:47:45.935 INFO:tasks.workunit.client.0.smithi070.stdout:2/389: dread d2/d10/d21/d3d/f47 [0,4194304] 0 2024-04-02T04:47:45.936 INFO:tasks.workunit.client.0.smithi070.stdout:0/382: creat d4/d5/d13/d2a/d3a/f67 x:0 0 0 2024-04-02T04:47:45.937 INFO:tasks.workunit.client.0.smithi070.stdout:7/350: readlink d7/dd/df/l3f 0 2024-04-02T04:47:45.938 INFO:tasks.workunit.client.0.smithi070.stdout:1/352: dwrite d0/d7/d1a/f24 [0,4194304] 0 2024-04-02T04:47:45.941 INFO:tasks.workunit.client.1.smithi179.stdout:9/325: dread d0/d1/d11/d29/d40/f49 [0,4194304] 0 2024-04-02T04:47:45.945 INFO:tasks.workunit.client.0.smithi070.stdout:8/370: dwrite d2/d3/f61 [0,4194304] 0 2024-04-02T04:47:45.956 INFO:tasks.workunit.client.1.smithi179.stdout:6/344: write d3/d12/d23/d26/f4f [1845232,46261] 0 2024-04-02T04:47:45.958 INFO:tasks.workunit.client.1.smithi179.stdout:5/252: mknod d8/d1b/d23/c4a 0 2024-04-02T04:47:45.959 INFO:tasks.workunit.client.0.smithi070.stdout:4/276: symlink d1/d3/d9/l58 0 2024-04-02T04:47:45.960 INFO:tasks.workunit.client.1.smithi179.stdout:0/376: dwrite d1/da/d28/d34/f35 [0,4194304] 0 2024-04-02T04:47:45.961 INFO:tasks.workunit.client.0.smithi070.stdout:3/318: symlink d10/d17/d51/l6f 0 2024-04-02T04:47:45.965 INFO:tasks.workunit.client.1.smithi179.stdout:5/253: dwrite d8/d1b/d23/f47 [0,4194304] 0 2024-04-02T04:47:45.968 INFO:tasks.workunit.client.0.smithi070.stdout:7/351: mknod d7/dd/df/c6f 0 2024-04-02T04:47:45.969 INFO:tasks.workunit.client.0.smithi070.stdout:7/352: write d7/dd/f38 [4822307,8161] 0 2024-04-02T04:47:45.969 INFO:tasks.workunit.client.1.smithi179.stdout:7/377: symlink d1/dd/l88 0 2024-04-02T04:47:45.971 INFO:tasks.workunit.client.0.smithi070.stdout:7/353: read d7/d4e/d55/d6c/f6d [1850381,126970] 0 2024-04-02T04:47:45.972 INFO:tasks.workunit.client.0.smithi070.stdout:7/354: readlink d7/dd/df/l3f 0 2024-04-02T04:47:45.972 INFO:tasks.workunit.client.1.smithi179.stdout:5/254: dread f1 [0,4194304] 0 2024-04-02T04:47:45.975 INFO:tasks.workunit.client.1.smithi179.stdout:5/255: dread d8/d1b/d23/f2b [0,4194304] 0 2024-04-02T04:47:45.976 INFO:tasks.workunit.client.0.smithi070.stdout:6/235: truncate d0/d20/f27 428834 0 2024-04-02T04:47:45.977 INFO:tasks.workunit.client.1.smithi179.stdout:5/256: chown d8/d1b/d23/d32/f33 132362822 1 2024-04-02T04:47:45.977 INFO:tasks.workunit.client.1.smithi179.stdout:5/257: chown d8/f2c 45894 1 2024-04-02T04:47:45.977 INFO:tasks.workunit.client.1.smithi179.stdout:5/258: fsync d8/d1b/f37 0 2024-04-02T04:47:45.978 INFO:tasks.workunit.client.1.smithi179.stdout:3/319: getdents d8/d26 0 2024-04-02T04:47:45.979 INFO:tasks.workunit.client.1.smithi179.stdout:3/320: fdatasync d8/d26/f62 0 2024-04-02T04:47:45.979 INFO:tasks.workunit.client.0.smithi070.stdout:5/357: creat d5/d9/d10/d62/f72 x:0 0 0 2024-04-02T04:47:45.980 INFO:tasks.workunit.client.0.smithi070.stdout:5/358: chown d5/d9/d10/d51/f59 1454 1 2024-04-02T04:47:45.982 INFO:tasks.workunit.client.1.smithi179.stdout:4/395: dwrite d1/f3c [0,4194304] 0 2024-04-02T04:47:45.984 INFO:tasks.workunit.client.0.smithi070.stdout:3/319: mknod d10/c70 0 2024-04-02T04:47:45.987 INFO:tasks.workunit.client.0.smithi070.stdout:2/390: write d2/d10/d21/d3d/f47 [1029910,15103] 0 2024-04-02T04:47:45.988 INFO:tasks.workunit.client.1.smithi179.stdout:0/377: symlink d1/da/d28/d37/l6f 0 2024-04-02T04:47:45.991 INFO:tasks.workunit.client.1.smithi179.stdout:2/206: dwrite dc/d16/d1e/f40 [0,4194304] 0 2024-04-02T04:47:45.991 INFO:tasks.workunit.client.0.smithi070.stdout:1/353: mkdir d0/d7/d28/d5c/d6e 0 2024-04-02T04:47:45.992 INFO:tasks.workunit.client.1.smithi179.stdout:1/337: link d3/d5/d2f/d5a/l5c d3/d5/d2f/d4f/d42/d55/l76 0 2024-04-02T04:47:45.992 INFO:tasks.workunit.client.1.smithi179.stdout:6/345: write f2 [3658363,100345] 0 2024-04-02T04:47:45.998 INFO:tasks.workunit.client.0.smithi070.stdout:4/277: mknod d1/d3/c59 0 2024-04-02T04:47:45.998 INFO:tasks.workunit.client.1.smithi179.stdout:7/378: read - d1/dd/d23/d5e/f6f zero size 2024-04-02T04:47:45.999 INFO:tasks.workunit.client.0.smithi070.stdout:4/278: readlink d1/d3/d9/l38 0 2024-04-02T04:47:45.999 INFO:tasks.workunit.client.1.smithi179.stdout:1/338: dread d3/d5/d2f/d36/f67 [0,4194304] 0 2024-04-02T04:47:46.001 INFO:tasks.workunit.client.1.smithi179.stdout:5/259: fsync d8/f3e 0 2024-04-02T04:47:46.003 INFO:tasks.workunit.client.0.smithi070.stdout:3/320: creat d10/d32/f71 x:0 0 0 2024-04-02T04:47:46.004 INFO:tasks.workunit.client.0.smithi070.stdout:6/236: dwrite d0/d3/f13 [4194304,4194304] 0 2024-04-02T04:47:46.006 INFO:tasks.workunit.client.0.smithi070.stdout:0/383: creat d4/d5/d13/d2a/f68 x:0 0 0 2024-04-02T04:47:46.007 INFO:tasks.workunit.client.0.smithi070.stdout:6/237: write d0/d20/f41 [5114515,50132] 0 2024-04-02T04:47:46.008 INFO:tasks.workunit.client.0.smithi070.stdout:4/279: dread d1/f2d [0,4194304] 0 2024-04-02T04:47:46.009 INFO:tasks.workunit.client.0.smithi070.stdout:1/354: rename d0/d9/d1d/d22/f5d to d0/d7/d28/d5c/d6e/f6f 0 2024-04-02T04:47:46.010 INFO:tasks.workunit.client.1.smithi179.stdout:8/295: creat d3/d32/d21/f66 x:0 0 0 2024-04-02T04:47:46.014 INFO:tasks.workunit.client.1.smithi179.stdout:0/378: creat d1/db/d64/f70 x:0 0 0 2024-04-02T04:47:46.016 INFO:tasks.workunit.client.1.smithi179.stdout:2/207: truncate dc/d16/f24 5319321 0 2024-04-02T04:47:46.024 INFO:tasks.workunit.client.0.smithi070.stdout:2/391: symlink d2/d12/d3e/d68/l7c 0 2024-04-02T04:47:46.028 INFO:tasks.workunit.client.1.smithi179.stdout:9/326: rename d0/d19/d1c/l22 to d0/d1/l62 0 2024-04-02T04:47:46.029 INFO:tasks.workunit.client.1.smithi179.stdout:7/379: fdatasync d1/d9/d35/f3b 0 2024-04-02T04:47:46.032 INFO:tasks.workunit.client.0.smithi070.stdout:4/280: rename d1/d3/d9/l58 to d1/d3/d9/l5a 0 2024-04-02T04:47:46.033 INFO:tasks.workunit.client.0.smithi070.stdout:1/355: dread d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:46.035 INFO:tasks.workunit.client.0.smithi070.stdout:8/371: getdents d2/d3/d52 0 2024-04-02T04:47:46.035 INFO:tasks.workunit.client.0.smithi070.stdout:2/392: dread d2/d12/d59/f5e [0,4194304] 0 2024-04-02T04:47:46.036 INFO:tasks.workunit.client.0.smithi070.stdout:2/393: readlink d2/d24/l79 0 2024-04-02T04:47:46.040 INFO:tasks.workunit.client.1.smithi179.stdout:0/379: symlink d1/da/d28/l71 0 2024-04-02T04:47:46.042 INFO:tasks.workunit.client.1.smithi179.stdout:2/208: symlink dc/d29/l42 0 2024-04-02T04:47:46.043 INFO:tasks.workunit.client.0.smithi070.stdout:7/355: truncate d7/d2f/f31 2482090 0 2024-04-02T04:47:46.043 INFO:tasks.workunit.client.1.smithi179.stdout:2/209: read - dc/d16/f3f zero size 2024-04-02T04:47:46.043 INFO:tasks.workunit.client.1.smithi179.stdout:5/260: write d8/f2c [4459766,53794] 0 2024-04-02T04:47:46.044 INFO:tasks.workunit.client.1.smithi179.stdout:4/396: truncate d1/d3/df/f59 5361253 0 2024-04-02T04:47:46.045 INFO:tasks.workunit.client.0.smithi070.stdout:6/238: symlink d0/d3/d19/l48 0 2024-04-02T04:47:46.047 INFO:tasks.workunit.client.1.smithi179.stdout:6/346: rename d3/d12/d23 to d3/d12/d23/d7d 22 2024-04-02T04:47:46.047 INFO:tasks.workunit.client.0.smithi070.stdout:1/356: creat d0/d4c/f70 x:0 0 0 2024-04-02T04:47:46.048 INFO:tasks.workunit.client.1.smithi179.stdout:9/327: creat d0/d1/d11/d29/d40/f63 x:0 0 0 2024-04-02T04:47:46.049 INFO:tasks.workunit.client.0.smithi070.stdout:8/372: rename d2/d8/d29/c2b to d2/d8/d23/d24/d5f/c7b 0 2024-04-02T04:47:46.050 INFO:tasks.workunit.client.1.smithi179.stdout:9/328: truncate d0/d19/d1c/f41 1530171 0 2024-04-02T04:47:46.050 INFO:tasks.workunit.client.0.smithi070.stdout:8/373: write d2/d3/d52/f55 [765844,31219] 0 2024-04-02T04:47:46.051 INFO:tasks.workunit.client.0.smithi070.stdout:5/359: getdents d5/d9/d10 0 2024-04-02T04:47:46.053 INFO:tasks.workunit.client.0.smithi070.stdout:2/394: creat d2/d10/d21/d2b/f7d x:0 0 0 2024-04-02T04:47:46.054 INFO:tasks.workunit.client.1.smithi179.stdout:1/339: truncate d3/d5/d2f/d4f/f4c 3565630 0 2024-04-02T04:47:46.054 INFO:tasks.workunit.client.1.smithi179.stdout:7/380: dread d1/dd/d16/d74/f7a [0,4194304] 0 2024-04-02T04:47:46.054 INFO:tasks.workunit.client.0.smithi070.stdout:3/321: rmdir d10/d32/d4b/d5e 0 2024-04-02T04:47:46.055 INFO:tasks.workunit.client.0.smithi070.stdout:2/395: dread d2/d12/d59/f70 [0,4194304] 0 2024-04-02T04:47:46.057 INFO:tasks.workunit.client.1.smithi179.stdout:9/329: dwrite d0/d19/d1c/f41 [0,4194304] 0 2024-04-02T04:47:46.062 INFO:tasks.workunit.client.1.smithi179.stdout:0/380: unlink d1/da/c6b 0 2024-04-02T04:47:46.062 INFO:tasks.workunit.client.1.smithi179.stdout:0/381: chown d1/da/d48/f54 10548 1 2024-04-02T04:47:46.063 INFO:tasks.workunit.client.0.smithi070.stdout:3/322: dwrite d10/d32/f71 [0,4194304] 0 2024-04-02T04:47:46.065 INFO:tasks.workunit.client.0.smithi070.stdout:7/356: unlink d7/d36/l61 0 2024-04-02T04:47:46.066 INFO:tasks.workunit.client.1.smithi179.stdout:2/210: mkdir dc/d16/d1e/d43 0 2024-04-02T04:47:46.067 INFO:tasks.workunit.client.1.smithi179.stdout:2/211: dread - dc/d29/f37 zero size 2024-04-02T04:47:46.073 INFO:tasks.workunit.client.1.smithi179.stdout:4/397: symlink d1/d3/dc/d27/d62/l81 0 2024-04-02T04:47:46.073 INFO:tasks.workunit.client.1.smithi179.stdout:4/398: chown d1 273932960 1 2024-04-02T04:47:46.074 INFO:tasks.workunit.client.0.smithi070.stdout:4/281: truncate d1/f2d 806412 0 2024-04-02T04:47:46.074 INFO:tasks.workunit.client.0.smithi070.stdout:4/282: read - d1/f3e zero size 2024-04-02T04:47:46.077 INFO:tasks.workunit.client.0.smithi070.stdout:1/357: mknod d0/d54/d64/d6d/c71 0 2024-04-02T04:47:46.077 INFO:tasks.workunit.client.0.smithi070.stdout:1/358: write d0/f2a [5786699,58188] 0 2024-04-02T04:47:46.078 INFO:tasks.workunit.client.0.smithi070.stdout:8/374: rmdir d2/d8/d29/d22/d5a 39 2024-04-02T04:47:46.079 INFO:tasks.workunit.client.1.smithi179.stdout:4/399: dwrite d1/d3/df/d10/d2f/d60/d71/f74 [0,4194304] 0 2024-04-02T04:47:46.080 INFO:tasks.workunit.client.0.smithi070.stdout:5/360: fdatasync d5/f2f 0 2024-04-02T04:47:46.080 INFO:tasks.workunit.client.1.smithi179.stdout:4/400: chown d1/d3/d19/c1e 1811 1 2024-04-02T04:47:46.081 INFO:tasks.workunit.client.1.smithi179.stdout:4/401: chown d1/d3/df/d10/d3e 261577848 1 2024-04-02T04:47:46.082 INFO:tasks.workunit.client.0.smithi070.stdout:5/361: fsync d5/d9/d23/f6e 0 2024-04-02T04:47:46.082 INFO:tasks.workunit.client.0.smithi070.stdout:1/359: dread d0/f1c [0,4194304] 0 2024-04-02T04:47:46.084 INFO:tasks.workunit.client.1.smithi179.stdout:5/261: write f1 [2951983,95613] 0 2024-04-02T04:47:46.085 INFO:tasks.workunit.client.1.smithi179.stdout:3/321: sync 2024-04-02T04:47:46.086 INFO:tasks.workunit.client.1.smithi179.stdout:3/322: write d8/d26/d36/f38 [345019,34706] 0 2024-04-02T04:47:46.087 INFO:tasks.workunit.client.0.smithi070.stdout:0/384: getdents d4/d5/d21 0 2024-04-02T04:47:46.087 INFO:tasks.workunit.client.1.smithi179.stdout:3/323: write d8/d26/f34 [805207,63020] 0 2024-04-02T04:47:46.088 INFO:tasks.workunit.client.1.smithi179.stdout:3/324: chown d8/d26/d36/f59 313 1 2024-04-02T04:47:46.089 INFO:tasks.workunit.client.0.smithi070.stdout:7/357: rmdir d7/dd/df 39 2024-04-02T04:47:46.091 INFO:tasks.workunit.client.0.smithi070.stdout:7/358: write d7/d9/f1b [2511687,29611] 0 2024-04-02T04:47:46.092 INFO:tasks.workunit.client.0.smithi070.stdout:6/239: symlink d0/d3/l49 0 2024-04-02T04:47:46.092 INFO:tasks.workunit.client.1.smithi179.stdout:0/382: write d1/da/df/f33 [3786730,5492] 0 2024-04-02T04:47:46.093 INFO:tasks.workunit.client.0.smithi070.stdout:4/283: fsync d1/f12 0 2024-04-02T04:47:46.104 INFO:tasks.workunit.client.0.smithi070.stdout:5/362: readlink d5/d17/d2b/l4c 0 2024-04-02T04:47:46.104 INFO:tasks.workunit.client.0.smithi070.stdout:2/396: dread d2/f32 [0,4194304] 0 2024-04-02T04:47:46.105 INFO:tasks.workunit.client.1.smithi179.stdout:2/212: write f2 [5130998,4824] 0 2024-04-02T04:47:46.109 INFO:tasks.workunit.client.1.smithi179.stdout:6/347: truncate f2 1568709 0 2024-04-02T04:47:46.110 INFO:tasks.workunit.client.1.smithi179.stdout:6/348: write d3/d12/d23/d2f/f4a [2759322,18383] 0 2024-04-02T04:47:46.110 INFO:tasks.workunit.client.1.smithi179.stdout:7/381: rename d1/c40 to d1/dd/d16/d4d/c89 0 2024-04-02T04:47:46.111 INFO:tasks.workunit.client.0.smithi070.stdout:8/375: write d2/d3/d52/f3c [466880,69785] 0 2024-04-02T04:47:46.112 INFO:tasks.workunit.client.0.smithi070.stdout:2/397: dwrite d2/d10/d53/f55 [0,4194304] 0 2024-04-02T04:47:46.112 INFO:tasks.workunit.client.1.smithi179.stdout:6/349: chown d3/d12/d23/d39/d4e/d6c 58395 1 2024-04-02T04:47:46.113 INFO:tasks.workunit.client.1.smithi179.stdout:5/262: mkdir d8/d1b/d23/d32/d4b 0 2024-04-02T04:47:46.114 INFO:tasks.workunit.client.1.smithi179.stdout:5/263: chown d8/d1b/d23/f2f 90500825 1 2024-04-02T04:47:46.115 INFO:tasks.workunit.client.1.smithi179.stdout:1/340: mkdir d3/d5/d2f/d4f/d42/d53/d77 0 2024-04-02T04:47:46.118 INFO:tasks.workunit.client.0.smithi070.stdout:7/359: read d7/d2f/f48 [3264864,77217] 0 2024-04-02T04:47:46.119 INFO:tasks.workunit.client.1.smithi179.stdout:8/296: getdents d3/d32/d21/d22/d33 0 2024-04-02T04:47:46.119 INFO:tasks.workunit.client.0.smithi070.stdout:6/240: fdatasync d0/d3/d19/d31/f36 0 2024-04-02T04:47:46.120 INFO:tasks.workunit.client.1.smithi179.stdout:5/264: dwrite d8/f2c [4194304,4194304] 0 2024-04-02T04:47:46.121 INFO:tasks.workunit.client.1.smithi179.stdout:3/325: creat d8/d57/d63/f6c x:0 0 0 2024-04-02T04:47:46.124 INFO:tasks.workunit.client.1.smithi179.stdout:0/383: creat d1/da/d28/d37/f72 x:0 0 0 2024-04-02T04:47:46.125 INFO:tasks.workunit.client.1.smithi179.stdout:0/384: dread - d1/db/d64/f6a zero size 2024-04-02T04:47:46.130 INFO:tasks.workunit.client.1.smithi179.stdout:4/402: rename d1/d3/dc/fe to d1/d3/df/d10/d3e/f82 0 2024-04-02T04:47:46.131 INFO:tasks.workunit.client.1.smithi179.stdout:6/350: creat d3/d42/f7e x:0 0 0 2024-04-02T04:47:46.132 INFO:tasks.workunit.client.1.smithi179.stdout:4/403: write d1/d3/dc/d27/d62/f5e [1525387,130961] 0 2024-04-02T04:47:46.134 INFO:tasks.workunit.client.1.smithi179.stdout:4/404: read d1/d3/df/f51 [207927,47780] 0 2024-04-02T04:47:46.134 INFO:tasks.workunit.client.1.smithi179.stdout:1/341: rmdir d3/d5/de/d33 39 2024-04-02T04:47:46.136 INFO:tasks.workunit.client.1.smithi179.stdout:8/297: creat d3/f67 x:0 0 0 2024-04-02T04:47:46.137 INFO:tasks.workunit.client.0.smithi070.stdout:8/376: rmdir d2/d8/d29 39 2024-04-02T04:47:46.139 INFO:tasks.workunit.client.0.smithi070.stdout:8/377: write d2/d8/d23/d24/f67 [898303,42555] 0 2024-04-02T04:47:46.139 INFO:tasks.workunit.client.0.smithi070.stdout:2/398: creat d2/d24/f7e x:0 0 0 2024-04-02T04:47:46.140 INFO:tasks.workunit.client.0.smithi070.stdout:0/385: creat d4/d5/d13/d2a/d29/d52/f69 x:0 0 0 2024-04-02T04:47:46.141 INFO:tasks.workunit.client.1.smithi179.stdout:0/385: rmdir d1/da/d52 39 2024-04-02T04:47:46.142 INFO:tasks.workunit.client.1.smithi179.stdout:0/386: stat d1/da/l43 0 2024-04-02T04:47:46.151 INFO:tasks.workunit.client.0.smithi070.stdout:6/241: creat d0/d43/f4a x:0 0 0 2024-04-02T04:47:46.152 INFO:tasks.workunit.client.0.smithi070.stdout:0/386: dread d4/d5/d13/d23/f36 [0,4194304] 0 2024-04-02T04:47:46.153 INFO:tasks.workunit.client.0.smithi070.stdout:0/387: write f0 [1388440,112221] 0 2024-04-02T04:47:46.154 INFO:tasks.workunit.client.0.smithi070.stdout:4/284: mknod d1/d3/c5b 0 2024-04-02T04:47:46.154 INFO:tasks.workunit.client.0.smithi070.stdout:0/388: readlink d4/d5/d13/d23/l4e 0 2024-04-02T04:47:46.155 INFO:tasks.workunit.client.1.smithi179.stdout:9/330: getdents d0/d19/d1c 0 2024-04-02T04:47:46.157 INFO:tasks.workunit.client.1.smithi179.stdout:5/265: symlink d8/d1b/d23/d32/d4b/l4c 0 2024-04-02T04:47:46.158 INFO:tasks.workunit.client.1.smithi179.stdout:0/387: rmdir d1/db/d64 39 2024-04-02T04:47:46.162 INFO:tasks.workunit.client.0.smithi070.stdout:2/399: dwrite d2/d10/d21/d71/f76 [0,4194304] 0 2024-04-02T04:47:46.165 INFO:tasks.workunit.client.1.smithi179.stdout:4/405: symlink d1/d3/dc/l83 0 2024-04-02T04:47:46.168 INFO:tasks.workunit.client.0.smithi070.stdout:6/242: unlink d0/d20/f28 0 2024-04-02T04:47:46.172 INFO:tasks.workunit.client.1.smithi179.stdout:9/331: mkdir d0/d19/d1c/d64 0 2024-04-02T04:47:46.172 INFO:tasks.workunit.client.1.smithi179.stdout:8/298: truncate d3/f18 442485 0 2024-04-02T04:47:46.173 INFO:tasks.workunit.client.1.smithi179.stdout:8/299: chown d3/d31/c53 3 1 2024-04-02T04:47:46.174 INFO:tasks.workunit.client.0.smithi070.stdout:4/285: rename d1/d3/d9/l38 to d1/d3/d9/d41/l5c 0 2024-04-02T04:47:46.179 INFO:tasks.workunit.client.0.smithi070.stdout:8/378: mkdir d2/d3/d52/d62/d7c 0 2024-04-02T04:47:46.180 INFO:tasks.workunit.client.0.smithi070.stdout:0/389: dwrite d4/d5/d13/d43/f53 [0,4194304] 0 2024-04-02T04:47:46.181 INFO:tasks.workunit.client.1.smithi179.stdout:2/213: rename dc/d29/f37 to dc/f44 0 2024-04-02T04:47:46.181 INFO:tasks.workunit.client.0.smithi070.stdout:8/379: truncate d2/d3/d35/f78 1213428 0 2024-04-02T04:47:46.186 INFO:tasks.workunit.client.1.smithi179.stdout:8/300: write d3/d32/d21/d22/d33/d3c/f40 [1394428,58447] 0 2024-04-02T04:47:46.187 INFO:tasks.workunit.client.0.smithi070.stdout:2/400: write d2/f8 [2329071,2840] 0 2024-04-02T04:47:46.188 INFO:tasks.workunit.client.1.smithi179.stdout:3/326: getdents d8/d57/d63 0 2024-04-02T04:47:46.189 INFO:tasks.workunit.client.1.smithi179.stdout:8/301: chown d3/d32/d21/d49/c4b 5 1 2024-04-02T04:47:46.189 INFO:tasks.workunit.client.0.smithi070.stdout:7/360: creat d7/dd/df/d34/d68/f70 x:0 0 0 2024-04-02T04:47:46.190 INFO:tasks.workunit.client.1.smithi179.stdout:3/327: write d8/d26/d36/f4d [66107,124022] 0 2024-04-02T04:47:46.194 INFO:tasks.workunit.client.1.smithi179.stdout:2/214: readlink dc/l13 0 2024-04-02T04:47:46.194 INFO:tasks.workunit.client.0.smithi070.stdout:4/286: fdatasync d1/d3/d9/f22 0 2024-04-02T04:47:46.196 INFO:tasks.workunit.client.1.smithi179.stdout:7/382: rename d1/dd/d16/d4d/f6c to d1/d9/d7e/f8a 0 2024-04-02T04:47:46.197 INFO:tasks.workunit.client.1.smithi179.stdout:8/302: dwrite d3/f67 [0,4194304] 0 2024-04-02T04:47:46.197 INFO:tasks.workunit.client.0.smithi070.stdout:5/363: link d5/d17/d6c/f41 d5/d66/f73 0 2024-04-02T04:47:46.199 INFO:tasks.workunit.client.0.smithi070.stdout:1/360: getdents d0/d9/d1d/d22/d56 0 2024-04-02T04:47:46.200 INFO:tasks.workunit.client.1.smithi179.stdout:2/215: dread dc/fe [0,4194304] 0 2024-04-02T04:47:46.204 INFO:tasks.workunit.client.1.smithi179.stdout:5/266: truncate d8/f18 658953 0 2024-04-02T04:47:46.206 INFO:tasks.workunit.client.0.smithi070.stdout:8/380: unlink d2/d3/d35/d3a/f56 0 2024-04-02T04:47:46.207 INFO:tasks.workunit.client.1.smithi179.stdout:3/328: dread d8/d26/f2c [0,4194304] 0 2024-04-02T04:47:46.375 INFO:tasks.workunit.client.1.smithi179.stdout:3/329: chown d8/d26/f4e 8 1 2024-04-02T04:47:46.375 INFO:tasks.workunit.client.1.smithi179.stdout:3/330: write d8/d26/f34 [435983,63634] 0 2024-04-02T04:47:46.375 INFO:tasks.workunit.client.1.smithi179.stdout:3/331: dread d8/f1b [0,4194304] 0 2024-04-02T04:47:46.375 INFO:tasks.workunit.client.1.smithi179.stdout:9/332: creat d0/d1/f65 x:0 0 0 2024-04-02T04:47:46.375 INFO:tasks.workunit.client.1.smithi179.stdout:7/383: write d1/dd/d23/f25 [4582692,122829] 0 2024-04-02T04:47:46.375 INFO:tasks.workunit.client.1.smithi179.stdout:8/303: mknod d3/d32/d21/d50/d56/c68 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:2/216: rmdir dc/d29 39 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:5/267: truncate f6 361711 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:3/332: mkdir d8/d6d 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:3/333: read d8/f22 [5595930,111067] 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:3/334: dread d8/d26/d36/f5c [0,4194304] 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:1/342: sync 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:6/351: sync 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:7/384: rename d1/dd/d23/d5e to d1/d9/d27/d87/d8b 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:8/304: write d3/d32/f19 [5072732,29842] 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:3/335: dwrite d8/d26/f56 [0,4194304] 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:2/217: write dc/d29/d31/f3a [84096,12907] 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:0/388: getdents d1/db/d64 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:3/336: dread d8/d26/f33 [0,4194304] 0 2024-04-02T04:47:46.376 INFO:tasks.workunit.client.1.smithi179.stdout:9/333: dwrite d0/d19/d1c/f41 [4194304,4194304] 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:9/334: write d0/d1/d10/f18 [6193743,11641] 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:1/343: chown d3/d5/d2f/d4f/f4c 7540432 1 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:6/352: write d3/d12/d6a/f75 [433959,45484] 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:7/385: write d1/dd/d23/f2f [2062481,91222] 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:2/218: readlink dc/d16/l21 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:0/389: rmdir d1/da/df 39 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:3/337: fdatasync d8/d26/d36/f5e 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:6/353: read d3/d2d/f65 [5932280,73399] 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:7/386: rmdir d1/d9/d35 39 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:8/305: rename d3/f9 to d3/d31/f69 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:0/390: rmdir d1 39 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:3/338: unlink d8/d26/c40 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:5/268: getdents d8/d1b/d23/d36 0 2024-04-02T04:47:46.377 INFO:tasks.workunit.client.1.smithi179.stdout:7/387: dread d1/dd/d16/d74/f7a [0,4194304] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:1/344: link d3/d5/de/c68 d3/c78 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:5/269: dread d8/f13 [0,4194304] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:7/388: dread d1/dd/d23/f24 [0,4194304] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:1/345: dwrite d3/d5/de/f24 [0,4194304] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:0/391: creat d1/da/d52/f73 x:0 0 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:7/389: dwrite d1/d9/d55/f64 [0,4194304] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:0/392: read - d1/db/d64/f6a zero size 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:2/219: rename dc/d16/d1e/f2d to dc/d16/f45 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:1/346: dwrite d3/d5/de/f57 [0,4194304] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:8/306: write d3/d31/f69 [729502,112319] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:6/354: getdents d3/d2d/d38 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:0/393: dwrite d1/da/d52/f5d [0,4194304] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:2/220: write dc/d16/f15 [1931518,17373] 0 2024-04-02T04:47:46.378 INFO:tasks.workunit.client.1.smithi179.stdout:3/339: rename d8/c54 to d8/d57/c6e 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:2/221: chown dc/f3b 785636112 1 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:3/340: read d8/d26/f34 [452423,9190] 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:6/355: rmdir d3/d2d/d38/d64 39 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:8/307: creat d3/d32/d21/d50/d56/f6a x:0 0 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:0/394: dwrite d1/da/f17 [0,4194304] 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:4/406: sync 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:3/341: dwrite d8/d26/f69 [0,4194304] 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:5/270: rename d8/d1b/d23/c4a to d8/c4d 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:1/347: dread d3/d5/f60 [0,4194304] 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:3/342: dread d8/d26/d36/f4d [0,4194304] 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:3/343: chown d8/d26/l67 74807280 1 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:8/308: mkdir d3/d32/d21/d22/d6b 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:8/309: readlink d3/d32/l2c 0 2024-04-02T04:47:46.379 INFO:tasks.workunit.client.1.smithi179.stdout:9/335: sync 2024-04-02T04:47:46.380 INFO:tasks.workunit.client.0.smithi070.stdout:2/401: stat d2/d10/d19/c4e 0 2024-04-02T04:47:46.380 INFO:tasks.workunit.client.0.smithi070.stdout:7/361: chown d7/c26 7476 1 2024-04-02T04:47:46.380 INFO:tasks.workunit.client.0.smithi070.stdout:7/362: chown d7/dd/l52 4884558 1 2024-04-02T04:47:46.380 INFO:tasks.workunit.client.0.smithi070.stdout:6/243: creat d0/d3/d19/d39/f4b x:0 0 0 2024-04-02T04:47:46.380 INFO:tasks.workunit.client.0.smithi070.stdout:5/364: rename d5/d14/f63 to d5/d9/d23/f74 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:2/402: dread d2/d10/d21/d3d/f63 [0,4194304] 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:2/403: dread d2/d10/d21/d2b/f6e [0,4194304] 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:0/390: creat d4/d3c/f6a x:0 0 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:0/391: fsync d4/d5/d13/d2a/d29/f46 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:8/381: mknod d2/d3/c7d 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:2/404: dread d2/d10/d19/f1e [0,4194304] 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:8/382: truncate d2/d8/d23/d24/d66/f70 164823 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:7/363: creat d7/d2f/d46/d65/f71 x:0 0 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:2/405: stat d2/d12/f52 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:7/364: truncate d7/d2f/f48 4731893 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:4/287: mknod d1/d3/c5d 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:4/288: write d1/fe [4113852,14119] 0 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:9/290: sync 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:4/289: dread - d1/f3e zero size 2024-04-02T04:47:46.381 INFO:tasks.workunit.client.0.smithi070.stdout:9/291: read - d4/d13/d16/d19/d53/f5a zero size 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:4/290: fsync d1/fe 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:0/392: mkdir d4/d5/d21/d6b 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:8/383: read - d2/d8/d29/d22/f60 zero size 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:2/406: creat d2/d10/d5c/f7f x:0 0 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:6/244: mknod d0/c4c 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:9/292: creat d4/d10/d12/f62 x:0 0 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:4/291: unlink d1/d3/c16 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:2/407: read d2/f4 [2775617,71013] 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:6/245: unlink d0/d20/f41 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:9/293: write d4/d10/f25 [893669,33893] 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:4/292: mkdir d1/d3/d9/d5e 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:9/294: write d4/d13/f26 [1646321,82272] 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:1/361: getdents d0/d9/d1d/d22/d2c 0 2024-04-02T04:47:46.382 INFO:tasks.workunit.client.0.smithi070.stdout:3/323: sync 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:3/324: write d10/d32/f56 [933335,84741] 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:3/325: readlink d10/l11 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:2/408: mknod d2/d12/d3e/c80 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:0/393: truncate d4/d5/da/d2d/f4c 29459 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:8/384: truncate d2/d8/d29/d22/f7a 2788125 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:0/394: dwrite d4/d5/d13/d2a/d29/f46 [0,4194304] 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:5/365: rename d5/d9/d10/d62 to d5/d66/d75 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:1/362: creat d0/d4c/f72 x:0 0 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:7/365: getdents d7/d9 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:7/366: fdatasync d7/d2f/d46/f53 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:2/409: creat d2/d10/d5c/f81 x:0 0 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:8/385: rmdir d2/d3/d52 39 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:9/295: mknod d4/d10/d12/d55/c63 0 2024-04-02T04:47:46.383 INFO:tasks.workunit.client.0.smithi070.stdout:9/296: chown d4/d10/f18 1481 1 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:5/366: symlink d5/d9/d23/l76 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:7/367: creat d7/d36/f72 x:0 0 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:1/363: creat d0/d4c/d5a/f73 x:0 0 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:1/364: dread - d0/d54/f58 zero size 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:1/365: truncate d0/d7/f8 9427169 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:1/366: readlink d0/d9/d1d/lb 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:3/326: link d10/c14 d10/d32/c72 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:3/327: chown d10/d17 173 1 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:7/368: symlink d7/d9/d19/l73 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:8/386: fsync d2/d3/f7 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:8/387: write d2/f1f [3679025,15834] 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:0/395: creat d4/d5/d13/d2a/f6c x:0 0 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:4/293: getdents d1/d3/d9/d41 0 2024-04-02T04:47:46.384 INFO:tasks.workunit.client.0.smithi070.stdout:0/396: write d4/d5/d13/d23/f5b [1921318,15263] 0 2024-04-02T04:47:46.385 INFO:tasks.workunit.client.0.smithi070.stdout:6/246: rename d0/d3/d19/d31/l38 to d0/l4d 0 2024-04-02T04:47:46.385 INFO:tasks.workunit.client.0.smithi070.stdout:8/388: dwrite d2/d8/d23/d24/d66/f6a [0,4194304] 0 2024-04-02T04:47:46.385 INFO:tasks.workunit.client.0.smithi070.stdout:3/328: mkdir d10/d17/d51/d73 0 2024-04-02T04:47:46.385 INFO:tasks.workunit.client.0.smithi070.stdout:7/369: creat d7/d4e/d55/d6c/f74 x:0 0 0 2024-04-02T04:47:46.386 INFO:tasks.workunit.client.1.smithi179.stdout:7/390: unlink d1/d9/d35/l41 0 2024-04-02T04:47:46.386 INFO:tasks.workunit.client.1.smithi179.stdout:0/395: write d1/da/df/f33 [5040058,33025] 0 2024-04-02T04:47:46.394 INFO:tasks.workunit.client.0.smithi070.stdout:4/294: rename d1/d3/d9/f17 to d1/d3/d9/f5f 0 2024-04-02T04:47:46.396 INFO:tasks.workunit.client.1.smithi179.stdout:6/356: link d3/d12/d23/d26/f69 d3/f7f 0 2024-04-02T04:47:46.397 INFO:tasks.workunit.client.0.smithi070.stdout:2/410: rename d2/f14 to d2/d10/d21/d2b/f82 0 2024-04-02T04:47:46.398 INFO:tasks.workunit.client.1.smithi179.stdout:8/310: mkdir d3/d6c 0 2024-04-02T04:47:46.399 INFO:tasks.workunit.client.0.smithi070.stdout:5/367: mkdir d5/d66/d77 0 2024-04-02T04:47:46.401 INFO:tasks.workunit.client.1.smithi179.stdout:9/336: unlink d0/d1/d10/f13 0 2024-04-02T04:47:46.402 INFO:tasks.workunit.client.1.smithi179.stdout:9/337: stat d0/d1/d11/d29/d40 0 2024-04-02T04:47:46.406 INFO:tasks.workunit.client.1.smithi179.stdout:0/396: symlink d1/da/d48/l74 0 2024-04-02T04:47:46.407 INFO:tasks.workunit.client.0.smithi070.stdout:0/397: dwrite d4/d5/d13/d2a/f20 [0,4194304] 0 2024-04-02T04:47:46.407 INFO:tasks.workunit.client.1.smithi179.stdout:4/407: creat d1/d15/d7c/f84 x:0 0 0 2024-04-02T04:47:46.410 INFO:tasks.workunit.client.0.smithi070.stdout:0/398: dread d4/d5/d13/d23/f36 [0,4194304] 0 2024-04-02T04:47:46.412 INFO:tasks.workunit.client.0.smithi070.stdout:0/399: fsync d4/d5/d13/d2a/f68 0 2024-04-02T04:47:46.415 INFO:tasks.workunit.client.0.smithi070.stdout:4/295: dread d1/d3/d9/f25 [0,4194304] 0 2024-04-02T04:47:46.417 INFO:tasks.workunit.client.1.smithi179.stdout:2/222: creat dc/f46 x:0 0 0 2024-04-02T04:47:46.419 INFO:tasks.workunit.client.0.smithi070.stdout:1/367: rename d0/d9/d1d/f26 to d0/d7/d28/f74 0 2024-04-02T04:47:46.420 INFO:tasks.workunit.client.0.smithi070.stdout:1/368: dread - d0/d9/d1d/f5e zero size 2024-04-02T04:47:46.422 INFO:tasks.workunit.client.0.smithi070.stdout:5/368: mkdir d5/d9/d10/d51/d78 0 2024-04-02T04:47:46.424 INFO:tasks.workunit.client.1.smithi179.stdout:8/311: mkdir d3/d32/d21/d22/d6d 0 2024-04-02T04:47:46.427 INFO:tasks.workunit.client.0.smithi070.stdout:2/411: readlink d2/d10/d53/l6d 0 2024-04-02T04:47:46.429 INFO:tasks.workunit.client.0.smithi070.stdout:4/296: mknod d1/d3/d9/c60 0 2024-04-02T04:47:46.430 INFO:tasks.workunit.client.1.smithi179.stdout:0/397: mkdir d1/da/d52/d75 0 2024-04-02T04:47:46.434 INFO:tasks.workunit.client.1.smithi179.stdout:2/223: unlink dc/d16/d1e/c32 0 2024-04-02T04:47:46.436 INFO:tasks.workunit.client.1.smithi179.stdout:1/348: rmdir d3/d5/d2f/d6c 0 2024-04-02T04:47:46.437 INFO:tasks.workunit.client.0.smithi070.stdout:6/247: rename d0/d3/d19/d31/f42 to d0/d3/d19/d31/f4e 0 2024-04-02T04:47:46.437 INFO:tasks.workunit.client.0.smithi070.stdout:3/329: creat d10/d17/f74 x:0 0 0 2024-04-02T04:47:46.439 INFO:tasks.workunit.client.0.smithi070.stdout:2/412: creat d2/d10/d5c/f83 x:0 0 0 2024-04-02T04:47:46.439 INFO:tasks.workunit.client.1.smithi179.stdout:4/408: dwrite d1/f42 [0,4194304] 0 2024-04-02T04:47:46.439 INFO:tasks.workunit.client.0.smithi070.stdout:2/413: readlink d2/d10/d21/d2b/l73 0 2024-04-02T04:47:46.441 INFO:tasks.workunit.client.1.smithi179.stdout:7/391: rename d1/d9/d55/f56 to d1/dd/f8c 0 2024-04-02T04:47:46.443 INFO:tasks.workunit.client.1.smithi179.stdout:5/271: getdents d8/d1b/d23/d30 0 2024-04-02T04:47:46.445 INFO:tasks.workunit.client.0.smithi070.stdout:4/297: mknod d1/d3/c61 0 2024-04-02T04:47:46.445 INFO:tasks.workunit.client.0.smithi070.stdout:2/414: truncate d2/d10/d21/f34 2289217 0 2024-04-02T04:47:46.447 INFO:tasks.workunit.client.1.smithi179.stdout:9/338: rmdir d0/d1/d11/d3a/d5b 0 2024-04-02T04:47:46.448 INFO:tasks.workunit.client.1.smithi179.stdout:9/339: stat d0/d1/d11/d3a/c4a 0 2024-04-02T04:47:46.449 INFO:tasks.workunit.client.1.smithi179.stdout:9/340: write d0/d1/d11/d29/d40/f63 [257624,30262] 0 2024-04-02T04:47:46.449 INFO:tasks.workunit.client.0.smithi070.stdout:0/400: rename d4/d5/d13/f1f to d4/d5/da/f6d 0 2024-04-02T04:47:46.453 INFO:tasks.workunit.client.1.smithi179.stdout:4/409: creat d1/d3/df/d10/d2f/d60/d71/f85 x:0 0 0 2024-04-02T04:47:46.453 INFO:tasks.workunit.client.1.smithi179.stdout:4/410: stat d1/d15/d7c/f84 0 2024-04-02T04:47:46.454 INFO:tasks.workunit.client.1.smithi179.stdout:4/411: readlink d1/d3/dc/l37 0 2024-04-02T04:47:46.454 INFO:tasks.workunit.client.0.smithi070.stdout:1/369: truncate d0/d7/d28/f74 4993599 0 2024-04-02T04:47:46.455 INFO:tasks.workunit.client.1.smithi179.stdout:0/398: mkdir d1/da/d67/d76 0 2024-04-02T04:47:46.458 INFO:tasks.workunit.client.0.smithi070.stdout:1/370: read d0/d9/d1d/d22/d56/f69 [346413,56562] 0 2024-04-02T04:47:46.458 INFO:tasks.workunit.client.1.smithi179.stdout:2/224: rename dc/f3b to dc/d16/d1e/d43/f47 0 2024-04-02T04:47:46.459 INFO:tasks.workunit.client.1.smithi179.stdout:4/412: dwrite d1/d3/d19/f2c [0,4194304] 0 2024-04-02T04:47:46.460 INFO:tasks.workunit.client.1.smithi179.stdout:4/413: fsync d1/d3/df/f57 0 2024-04-02T04:47:46.464 INFO:tasks.workunit.client.0.smithi070.stdout:3/330: rename d10/d32/c46 to d10/d66/c75 0 2024-04-02T04:47:46.465 INFO:tasks.workunit.client.0.smithi070.stdout:3/331: truncate d10/d17/f74 756270 0 2024-04-02T04:47:46.466 INFO:tasks.workunit.client.1.smithi179.stdout:5/272: mkdir d8/d1b/d23/d32/d4e 0 2024-04-02T04:47:46.467 INFO:tasks.workunit.client.0.smithi070.stdout:0/401: mkdir d4/d5/d13/d17/d6e 0 2024-04-02T04:47:46.468 INFO:tasks.workunit.client.0.smithi070.stdout:4/298: dread d1/d3/f39 [0,4194304] 0 2024-04-02T04:47:46.469 INFO:tasks.workunit.client.1.smithi179.stdout:8/312: creat d3/d32/f6e x:0 0 0 2024-04-02T04:47:46.469 INFO:tasks.workunit.client.1.smithi179.stdout:8/313: stat d3/d1b/f23 0 2024-04-02T04:47:46.470 INFO:tasks.workunit.client.0.smithi070.stdout:5/369: getdents d5/d9 0 2024-04-02T04:47:46.471 INFO:tasks.workunit.client.0.smithi070.stdout:5/370: fsync d5/ff 0 2024-04-02T04:47:46.475 INFO:tasks.workunit.client.1.smithi179.stdout:0/399: rename d1/da/d67/d76 to d1/da/d28/d34/d39/d77 0 2024-04-02T04:47:46.477 INFO:tasks.workunit.client.1.smithi179.stdout:7/392: mkdir d1/d84/d8d 0 2024-04-02T04:47:46.480 INFO:tasks.workunit.client.1.smithi179.stdout:5/273: rmdir d8/d1b/d23 39 2024-04-02T04:47:46.480 INFO:tasks.workunit.client.1.smithi179.stdout:5/274: chown d8/d1b/c29 969 1 2024-04-02T04:47:46.490 INFO:tasks.workunit.client.1.smithi179.stdout:3/344: sync 2024-04-02T04:47:46.501 INFO:tasks.workunit.client.1.smithi179.stdout:8/314: mknod d3/d32/d21/d49/c6f 0 2024-04-02T04:47:46.502 INFO:tasks.workunit.client.0.smithi070.stdout:3/332: creat d10/d17/d51/f76 x:0 0 0 2024-04-02T04:47:46.503 INFO:tasks.workunit.client.0.smithi070.stdout:0/402: creat d4/d5/d13/d43/f6f x:0 0 0 2024-04-02T04:47:46.503 INFO:tasks.workunit.client.0.smithi070.stdout:6/248: getdents d0/d3/d19 0 2024-04-02T04:47:46.506 INFO:tasks.workunit.client.0.smithi070.stdout:5/371: creat d5/d9/d23/d30/f79 x:0 0 0 2024-04-02T04:47:46.506 INFO:tasks.workunit.client.0.smithi070.stdout:1/371: dread d0/d9/fc [0,4194304] 0 2024-04-02T04:47:46.507 INFO:tasks.workunit.client.0.smithi070.stdout:4/299: dwrite d1/d3/d9/f29 [0,4194304] 0 2024-04-02T04:47:46.508 INFO:tasks.workunit.client.0.smithi070.stdout:3/333: write d10/f22 [2134458,124120] 0 2024-04-02T04:47:46.510 INFO:tasks.workunit.client.0.smithi070.stdout:0/403: mknod d4/d3c/c70 0 2024-04-02T04:47:46.511 INFO:tasks.workunit.client.0.smithi070.stdout:2/415: dwrite d2/d10/d21/f31 [0,4194304] 0 2024-04-02T04:47:46.511 INFO:tasks.workunit.client.0.smithi070.stdout:6/249: write d0/d43/f4a [784886,116740] 0 2024-04-02T04:47:46.512 INFO:tasks.workunit.client.1.smithi179.stdout:4/414: creat d1/d3/df/d10/d2f/d7b/f86 x:0 0 0 2024-04-02T04:47:46.512 INFO:tasks.workunit.client.1.smithi179.stdout:9/341: truncate d0/d19/d1c/f52 18030 0 2024-04-02T04:47:46.513 INFO:tasks.workunit.client.0.smithi070.stdout:1/372: stat d0/d9/d1d/d22/d2c/d4d 0 2024-04-02T04:47:46.516 INFO:tasks.workunit.client.1.smithi179.stdout:7/393: dwrite d1/f45 [0,4194304] 0 2024-04-02T04:47:46.518 INFO:tasks.workunit.client.0.smithi070.stdout:5/372: dread d5/d17/d50/f5a [0,4194304] 0 2024-04-02T04:47:46.518 INFO:tasks.workunit.client.0.smithi070.stdout:2/416: mknod d2/d10/d5c/c84 0 2024-04-02T04:47:46.519 INFO:tasks.workunit.client.0.smithi070.stdout:6/250: read d0/d3/d19/f3f [735225,70457] 0 2024-04-02T04:47:46.524 INFO:tasks.workunit.client.1.smithi179.stdout:7/394: write d1/dd/d23/f25 [441005,99630] 0 2024-04-02T04:47:46.524 INFO:tasks.workunit.client.1.smithi179.stdout:0/400: creat d1/da/d48/d5c/f78 x:0 0 0 2024-04-02T04:47:46.524 INFO:tasks.workunit.client.1.smithi179.stdout:4/415: truncate d1/d3/df/f51 1002723 0 2024-04-02T04:47:46.528 INFO:tasks.workunit.client.1.smithi179.stdout:2/225: rename dc/d16/d1e/f23 to dc/d16/f48 0 2024-04-02T04:47:46.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:46 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:46.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:46 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:46.530 INFO:tasks.workunit.client.1.smithi179.stdout:8/315: mkdir d3/d32/d21/d51/d70 0 2024-04-02T04:47:46.532 INFO:tasks.workunit.client.1.smithi179.stdout:2/226: fdatasync dc/d29/f34 0 2024-04-02T04:47:46.532 INFO:tasks.workunit.client.1.smithi179.stdout:9/342: creat d0/d19/d1c/d45/f66 x:0 0 0 2024-04-02T04:47:46.532 INFO:tasks.workunit.client.1.smithi179.stdout:7/395: mknod d1/d9/d55/c8e 0 2024-04-02T04:47:46.533 INFO:tasks.workunit.client.1.smithi179.stdout:9/343: readlink d0/l3 0 2024-04-02T04:47:46.533 INFO:tasks.workunit.client.1.smithi179.stdout:7/396: dread - d1/d9/f85 zero size 2024-04-02T04:47:46.533 INFO:tasks.workunit.client.0.smithi070.stdout:6/251: chown d0/d3/d19/d31/d37/l3b 25545 1 2024-04-02T04:47:46.534 INFO:tasks.workunit.client.1.smithi179.stdout:7/397: chown d1/d9/d55/f64 3266 1 2024-04-02T04:47:46.536 INFO:tasks.workunit.client.1.smithi179.stdout:3/345: rename d8/d26/d36/f41 to d8/f6f 0 2024-04-02T04:47:46.538 INFO:tasks.workunit.client.1.smithi179.stdout:7/398: dread d1/d9/d55/f64 [0,4194304] 0 2024-04-02T04:47:46.539 INFO:tasks.workunit.client.1.smithi179.stdout:4/416: truncate d1/d3/f1b 479151 0 2024-04-02T04:47:46.539 INFO:tasks.workunit.client.1.smithi179.stdout:7/399: dread - d1/d9/f85 zero size 2024-04-02T04:47:46.540 INFO:tasks.workunit.client.0.smithi070.stdout:1/373: link d0/d7/d28/d5c/d6e/f6f d0/d7/d1a/f75 0 2024-04-02T04:47:46.540 INFO:tasks.workunit.client.1.smithi179.stdout:5/275: link d8/d1b/d23/c39 d8/d1b/d23/d42/c4f 0 2024-04-02T04:47:46.541 INFO:tasks.workunit.client.1.smithi179.stdout:5/276: chown d8/d1b/c1c 210 1 2024-04-02T04:47:46.541 INFO:tasks.workunit.client.0.smithi070.stdout:1/374: write d0/d4c/f70 [392360,58606] 0 2024-04-02T04:47:46.545 INFO:tasks.workunit.client.0.smithi070.stdout:5/373: dread d5/d9/d10/f11 [0,4194304] 0 2024-04-02T04:47:46.545 INFO:tasks.workunit.client.0.smithi070.stdout:6/252: write d0/d3/d19/f3f [121416,95131] 0 2024-04-02T04:47:46.546 INFO:tasks.workunit.client.0.smithi070.stdout:6/253: rename d0/d3 to d0/d3/d19/d31/d46/d4f 22 2024-04-02T04:47:46.546 INFO:tasks.workunit.client.1.smithi179.stdout:0/401: rename d1/da/f27 to d1/db/d64/f79 0 2024-04-02T04:47:46.547 INFO:tasks.workunit.client.1.smithi179.stdout:5/277: dwrite d8/d1b/d23/d32/f33 [0,4194304] 0 2024-04-02T04:47:46.548 INFO:tasks.workunit.client.1.smithi179.stdout:5/278: chown d8/d1b/d23/d32/d4e 11363222 1 2024-04-02T04:47:46.550 INFO:tasks.workunit.client.0.smithi070.stdout:4/300: getdents d1 0 2024-04-02T04:47:46.551 INFO:tasks.workunit.client.0.smithi070.stdout:3/334: getdents d10/d17 0 2024-04-02T04:47:46.551 INFO:tasks.workunit.client.1.smithi179.stdout:3/346: symlink d8/d26/d36/l70 0 2024-04-02T04:47:46.552 INFO:tasks.workunit.client.0.smithi070.stdout:1/375: write d0/d7/d1a/f2e [3498171,98570] 0 2024-04-02T04:47:46.554 INFO:tasks.workunit.client.1.smithi179.stdout:4/417: unlink d1/d3/df/f6f 0 2024-04-02T04:47:46.555 INFO:tasks.workunit.client.0.smithi070.stdout:0/404: getdents d4/d5/d13/d23 0 2024-04-02T04:47:46.557 INFO:tasks.workunit.client.0.smithi070.stdout:6/254: mkdir d0/d3/d19/d31/d37/d50 0 2024-04-02T04:47:46.560 INFO:tasks.workunit.client.0.smithi070.stdout:4/301: dread d1/d3/d9/f2a [0,4194304] 0 2024-04-02T04:47:46.562 INFO:tasks.workunit.client.0.smithi070.stdout:3/335: mknod d10/d17/d51/c77 0 2024-04-02T04:47:46.563 INFO:tasks.workunit.client.0.smithi070.stdout:5/374: mknod d5/d17/c7a 0 2024-04-02T04:47:46.565 INFO:tasks.workunit.client.0.smithi070.stdout:6/255: dwrite d0/d43/f4a [0,4194304] 0 2024-04-02T04:47:46.567 INFO:tasks.workunit.client.0.smithi070.stdout:0/405: creat d4/d5/d13/d2a/d29/f71 x:0 0 0 2024-04-02T04:47:46.568 INFO:tasks.workunit.client.1.smithi179.stdout:2/227: rmdir dc/d3c 0 2024-04-02T04:47:46.570 INFO:tasks.workunit.client.0.smithi070.stdout:0/406: read d4/f9 [2278330,82153] 0 2024-04-02T04:47:46.570 INFO:tasks.workunit.client.0.smithi070.stdout:4/302: dwrite d1/d3/d9/f1a [0,4194304] 0 2024-04-02T04:47:46.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:46 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:46.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:46 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:46.572 INFO:tasks.workunit.client.0.smithi070.stdout:0/407: dread - d4/d5/d13/d2a/d29/f71 zero size 2024-04-02T04:47:46.575 INFO:tasks.workunit.client.1.smithi179.stdout:3/347: dread d8/d26/f2b [0,4194304] 0 2024-04-02T04:47:46.579 INFO:tasks.workunit.client.0.smithi070.stdout:1/376: link d0/d9/d1d/d22/d2c/f4a d0/d9/d1d/d22/d2c/f76 0 2024-04-02T04:47:46.581 INFO:tasks.workunit.client.1.smithi179.stdout:7/400: creat d1/d84/d8d/f8f x:0 0 0 2024-04-02T04:47:46.581 INFO:tasks.workunit.client.0.smithi070.stdout:5/375: mkdir d5/d17/d2b/d7b 0 2024-04-02T04:47:46.581 INFO:tasks.workunit.client.1.smithi179.stdout:6/357: sync 2024-04-02T04:47:46.581 INFO:tasks.workunit.client.1.smithi179.stdout:1/349: sync 2024-04-02T04:47:46.582 INFO:tasks.workunit.client.1.smithi179.stdout:1/350: read d3/d5/f1d [655999,116557] 0 2024-04-02T04:47:46.582 INFO:tasks.workunit.client.0.smithi070.stdout:2/417: truncate d2/d10/d21/d3d/f63 2246949 0 2024-04-02T04:47:46.583 INFO:tasks.workunit.client.1.smithi179.stdout:1/351: stat d3/d5/d2f/d4f/f4c 0 2024-04-02T04:47:46.583 INFO:tasks.workunit.client.1.smithi179.stdout:1/352: chown d3/d5 184 1 2024-04-02T04:47:46.584 INFO:tasks.workunit.client.1.smithi179.stdout:4/418: rename d1/d3/df/d10/d2f/d60/l70 to d1/d3/dc/d7a/l87 0 2024-04-02T04:47:46.584 INFO:tasks.workunit.client.1.smithi179.stdout:1/353: write d3/d5/de/f57 [1875883,119883] 0 2024-04-02T04:47:46.590 INFO:tasks.workunit.client.1.smithi179.stdout:8/316: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:46.591 INFO:tasks.workunit.client.0.smithi070.stdout:5/376: dwrite d5/d17/d50/f5f [0,4194304] 0 2024-04-02T04:47:46.594 INFO:tasks.workunit.client.0.smithi070.stdout:5/377: chown d5/d9/d10/l13 0 1 2024-04-02T04:47:46.594 INFO:tasks.workunit.client.1.smithi179.stdout:5/279: mkdir d8/d1b/d23/d32/d4e/d50 0 2024-04-02T04:47:46.595 INFO:tasks.workunit.client.0.smithi070.stdout:6/256: dwrite d0/f1c [4194304,4194304] 0 2024-04-02T04:47:46.595 INFO:tasks.workunit.client.1.smithi179.stdout:5/280: readlink d8/d1b/d23/d30/l3a 0 2024-04-02T04:47:46.596 INFO:tasks.workunit.client.0.smithi070.stdout:5/378: write d5/f56 [2628794,18736] 0 2024-04-02T04:47:46.596 INFO:tasks.workunit.client.1.smithi179.stdout:5/281: read d8/d1b/d23/d36/f41 [4128553,81183] 0 2024-04-02T04:47:46.596 INFO:tasks.workunit.client.0.smithi070.stdout:6/257: truncate d0/f25 445064 0 2024-04-02T04:47:46.598 INFO:tasks.workunit.client.0.smithi070.stdout:5/379: truncate d5/d17/d2b/f67 849579 0 2024-04-02T04:47:46.599 INFO:tasks.workunit.client.0.smithi070.stdout:6/258: truncate d0/d3/d19/d31/f4e 5075320 0 2024-04-02T04:47:46.601 INFO:tasks.workunit.client.1.smithi179.stdout:9/344: sync 2024-04-02T04:47:46.602 INFO:tasks.workunit.client.1.smithi179.stdout:2/228: symlink dc/d29/l49 0 2024-04-02T04:47:46.603 INFO:tasks.workunit.client.0.smithi070.stdout:2/418: dread d2/d10/d19/f39 [0,4194304] 0 2024-04-02T04:47:46.605 INFO:tasks.workunit.client.0.smithi070.stdout:2/419: chown d2/d10/d21/d3d/d42/c72 60082724 1 2024-04-02T04:47:46.605 INFO:tasks.workunit.client.1.smithi179.stdout:3/348: symlink d8/d26/d36/l71 0 2024-04-02T04:47:46.610 INFO:tasks.workunit.client.0.smithi070.stdout:5/380: dwrite d5/d9/f5e [0,4194304] 0 2024-04-02T04:47:46.611 INFO:tasks.workunit.client.0.smithi070.stdout:5/381: chown d5/d9/d10/l2a 0 1 2024-04-02T04:47:46.614 INFO:tasks.workunit.client.1.smithi179.stdout:4/419: mknod d1/d3/dc/d26/c88 0 2024-04-02T04:47:46.618 INFO:tasks.workunit.client.1.smithi179.stdout:0/402: link d1/f36 d1/d5b/f7a 0 2024-04-02T04:47:46.619 INFO:tasks.workunit.client.1.smithi179.stdout:0/403: chown d1/db/d64/f70 290031 1 2024-04-02T04:47:46.619 INFO:tasks.workunit.client.0.smithi070.stdout:6/259: creat d0/d43/f51 x:0 0 0 2024-04-02T04:47:46.620 INFO:tasks.workunit.client.1.smithi179.stdout:0/404: write d1/f23 [1017061,130065] 0 2024-04-02T04:47:46.621 INFO:tasks.workunit.client.0.smithi070.stdout:2/420: chown d2/f4 10166 1 2024-04-02T04:47:46.623 INFO:tasks.workunit.client.0.smithi070.stdout:6/260: dread d0/d3/d19/f3f [0,4194304] 0 2024-04-02T04:47:46.651 INFO:tasks.workunit.client.1.smithi179.stdout:5/282: mkdir d8/d1b/d23/d32/d4e/d51 0 2024-04-02T04:47:46.656 INFO:tasks.workunit.client.1.smithi179.stdout:9/345: creat d0/d1/d11/d29/d40/f67 x:0 0 0 2024-04-02T04:47:46.656 INFO:tasks.workunit.client.0.smithi070.stdout:4/303: creat d1/d3/f62 x:0 0 0 2024-04-02T04:47:46.657 INFO:tasks.workunit.client.1.smithi179.stdout:9/346: dread - d0/d1/d11/f51 zero size 2024-04-02T04:47:46.657 INFO:tasks.workunit.client.1.smithi179.stdout:8/317: sync 2024-04-02T04:47:46.658 INFO:tasks.workunit.client.1.smithi179.stdout:8/318: chown d3/d31/c5b 29729527 1 2024-04-02T04:47:46.662 INFO:tasks.workunit.client.0.smithi070.stdout:3/336: getdents d10/d66 0 2024-04-02T04:47:46.662 INFO:tasks.workunit.client.0.smithi070.stdout:0/408: truncate d4/d5/d13/f19 8076123 0 2024-04-02T04:47:46.663 INFO:tasks.workunit.client.1.smithi179.stdout:7/401: fdatasync d1/d9/f11 0 2024-04-02T04:47:46.663 INFO:tasks.workunit.client.1.smithi179.stdout:8/319: truncate d3/d32/d21/d22/f2d 845361 0 2024-04-02T04:47:46.666 INFO:tasks.workunit.client.0.smithi070.stdout:1/377: write d0/d7/d28/f74 [1058940,57092] 0 2024-04-02T04:47:46.666 INFO:tasks.workunit.client.0.smithi070.stdout:3/337: dwrite d10/d32/d3f/f4d [0,4194304] 0 2024-04-02T04:47:46.669 INFO:tasks.workunit.client.0.smithi070.stdout:1/378: write d0/d7/f8 [8015588,45360] 0 2024-04-02T04:47:46.670 INFO:tasks.workunit.client.0.smithi070.stdout:4/304: mknod d1/c63 0 2024-04-02T04:47:46.670 INFO:tasks.workunit.client.0.smithi070.stdout:3/338: dread - d10/d17/d51/f64 zero size 2024-04-02T04:47:46.672 INFO:tasks.workunit.client.0.smithi070.stdout:0/409: dwrite d4/d5/d13/d23/f4d [0,4194304] 0 2024-04-02T04:47:46.673 INFO:tasks.workunit.client.0.smithi070.stdout:3/339: write d10/d32/d3f/f4d [3227936,96018] 0 2024-04-02T04:47:46.678 INFO:tasks.workunit.client.0.smithi070.stdout:2/421: mkdir d2/d12/d85 0 2024-04-02T04:47:46.679 INFO:tasks.workunit.client.1.smithi179.stdout:2/229: dwrite dc/fe [0,4194304] 0 2024-04-02T04:47:46.681 INFO:tasks.workunit.client.0.smithi070.stdout:6/261: mkdir d0/d3/d19/d31/d46/d52 0 2024-04-02T04:47:46.687 INFO:tasks.workunit.client.1.smithi179.stdout:5/283: write d8/ff [507256,112422] 0 2024-04-02T04:47:46.690 INFO:tasks.workunit.client.1.smithi179.stdout:6/358: mkdir d3/d12/d23/d39/d80 0 2024-04-02T04:47:46.691 INFO:tasks.workunit.client.1.smithi179.stdout:3/349: truncate d8/f22 1493216 0 2024-04-02T04:47:46.692 INFO:tasks.workunit.client.1.smithi179.stdout:3/350: chown d8/d26/d36/c3d 559749774 1 2024-04-02T04:47:46.693 INFO:tasks.workunit.client.1.smithi179.stdout:6/359: truncate d3/d12/d44/f52 141876 0 2024-04-02T04:47:46.694 INFO:tasks.workunit.client.1.smithi179.stdout:4/420: creat d1/d3/df/d10/d3e/d5d/f89 x:0 0 0 2024-04-02T04:47:46.695 INFO:tasks.workunit.client.1.smithi179.stdout:4/421: fdatasync d1/d3/df/d10/d2f/f4e 0 2024-04-02T04:47:46.695 INFO:tasks.workunit.client.1.smithi179.stdout:4/422: chown d1/d15/f31 1801762 1 2024-04-02T04:47:46.700 INFO:tasks.workunit.client.1.smithi179.stdout:2/230: rename dc/d16 to dc/d29/d4a 0 2024-04-02T04:47:46.701 INFO:tasks.workunit.client.1.smithi179.stdout:9/347: link d0/d1/f7 d0/d19/d27/f68 0 2024-04-02T04:47:46.703 INFO:tasks.workunit.client.1.smithi179.stdout:8/320: truncate d3/d32/d21/f25 2309822 0 2024-04-02T04:47:46.703 INFO:tasks.workunit.client.1.smithi179.stdout:8/321: readlink d3/l2e 0 2024-04-02T04:47:46.705 INFO:tasks.workunit.client.1.smithi179.stdout:1/354: getdents d3/d5/de 0 2024-04-02T04:47:46.705 INFO:tasks.workunit.client.1.smithi179.stdout:3/351: rename d8/d26/f5b to d8/d26/f72 0 2024-04-02T04:47:46.706 INFO:tasks.workunit.client.1.smithi179.stdout:8/322: write d3/d32/d21/d50/f5f [984781,26269] 0 2024-04-02T04:47:46.706 INFO:tasks.workunit.client.1.smithi179.stdout:6/360: mknod d3/d12/d23/d39/c81 0 2024-04-02T04:47:46.709 INFO:tasks.workunit.client.1.smithi179.stdout:6/361: write d3/d12/d23/d2f/f3a [5082991,7123] 0 2024-04-02T04:47:46.712 INFO:tasks.workunit.client.1.smithi179.stdout:5/284: dread d8/f1d [0,4194304] 0 2024-04-02T04:47:46.712 INFO:tasks.workunit.client.1.smithi179.stdout:7/402: creat d1/dd/f90 x:0 0 0 2024-04-02T04:47:46.714 INFO:tasks.workunit.client.1.smithi179.stdout:4/423: rename d1/d3/df/d10/d3e to d1/d3/dc/d26/d8a 0 2024-04-02T04:47:46.715 INFO:tasks.workunit.client.1.smithi179.stdout:0/405: truncate d1/f30 333086 0 2024-04-02T04:47:46.715 INFO:tasks.workunit.client.1.smithi179.stdout:1/355: dwrite d3/d5/f6d [0,4194304] 0 2024-04-02T04:47:46.717 INFO:tasks.workunit.client.1.smithi179.stdout:8/323: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:46.719 INFO:tasks.workunit.client.1.smithi179.stdout:3/352: stat d8/d57/c6e 0 2024-04-02T04:47:46.721 INFO:tasks.workunit.client.1.smithi179.stdout:3/353: read d8/f1b [2063418,61348] 0 2024-04-02T04:47:46.725 INFO:tasks.workunit.client.1.smithi179.stdout:2/231: mkdir dc/d29/d4a/d4b 0 2024-04-02T04:47:46.728 INFO:tasks.workunit.client.1.smithi179.stdout:7/403: rmdir d1/dd/d76 39 2024-04-02T04:47:46.730 INFO:tasks.workunit.client.1.smithi179.stdout:9/348: creat d0/d19/d1c/d64/f69 x:0 0 0 2024-04-02T04:47:46.731 INFO:tasks.workunit.client.1.smithi179.stdout:4/424: mknod d1/d3/df/d10/d2f/c8b 0 2024-04-02T04:47:46.734 INFO:tasks.workunit.client.1.smithi179.stdout:8/324: rename d3/d32/d21/l4a to d3/d32/d21/d51/d70/l71 0 2024-04-02T04:47:46.738 INFO:tasks.workunit.client.1.smithi179.stdout:7/404: creat d1/d9/d35/f91 x:0 0 0 2024-04-02T04:47:46.740 INFO:tasks.workunit.client.1.smithi179.stdout:1/356: mknod d3/d5/d2f/c79 0 2024-04-02T04:47:46.740 INFO:tasks.workunit.client.1.smithi179.stdout:1/357: chown d3/d5/d2f/d4f 105 1 2024-04-02T04:47:46.741 INFO:tasks.workunit.client.1.smithi179.stdout:4/425: symlink d1/d3/df/d10/d2f/d60/l8c 0 2024-04-02T04:47:46.741 INFO:tasks.workunit.client.1.smithi179.stdout:1/358: stat d3/d5/de/l59 0 2024-04-02T04:47:46.745 INFO:tasks.workunit.client.1.smithi179.stdout:2/232: unlink dc/c18 0 2024-04-02T04:47:46.746 INFO:tasks.workunit.client.1.smithi179.stdout:8/325: creat d3/d32/d21/d49/f72 x:0 0 0 2024-04-02T04:47:46.750 INFO:tasks.workunit.client.0.smithi070.stdout:0/410: mkdir d4/d5/d13/d17/d72 0 2024-04-02T04:47:46.751 INFO:tasks.workunit.client.1.smithi179.stdout:1/359: symlink d3/d5/d2f/d4f/l7a 0 2024-04-02T04:47:46.751 INFO:tasks.workunit.client.1.smithi179.stdout:1/360: read d3/d5/f1d [1082498,56434] 0 2024-04-02T04:47:46.753 INFO:tasks.workunit.client.0.smithi070.stdout:2/422: unlink d2/d10/d19/c4e 0 2024-04-02T04:47:46.754 INFO:tasks.workunit.client.1.smithi179.stdout:0/406: truncate d1/da/d28/d34/f35 763516 0 2024-04-02T04:47:46.754 INFO:tasks.workunit.client.1.smithi179.stdout:4/426: dwrite d1/d3/dc/d26/d8a/f82 [0,4194304] 0 2024-04-02T04:47:46.756 INFO:tasks.workunit.client.1.smithi179.stdout:5/285: link d8/l48 d8/d1b/d23/d32/l52 0 2024-04-02T04:47:46.758 INFO:tasks.workunit.client.1.smithi179.stdout:0/407: write d1/da/d28/d37/f3e [816824,9753] 0 2024-04-02T04:47:46.759 INFO:tasks.workunit.client.0.smithi070.stdout:5/382: rename d5/d66/f4f to d5/d17/d27/f7c 0 2024-04-02T04:47:46.761 INFO:tasks.workunit.client.1.smithi179.stdout:1/361: dwrite d3/d5/de/f3c [0,4194304] 0 2024-04-02T04:47:46.763 INFO:tasks.workunit.client.0.smithi070.stdout:3/340: rename d10/d32/d3f/f47 to d10/d17/f78 0 2024-04-02T04:47:46.764 INFO:tasks.workunit.client.1.smithi179.stdout:6/362: rename d3/d42/c79 to d3/d12/d23/c82 0 2024-04-02T04:47:46.766 INFO:tasks.workunit.client.0.smithi070.stdout:2/423: symlink d2/l86 0 2024-04-02T04:47:46.766 INFO:tasks.workunit.client.1.smithi179.stdout:6/363: dread - d3/d12/d23/d26/f76 zero size 2024-04-02T04:47:46.767 INFO:tasks.workunit.client.1.smithi179.stdout:8/326: dread d3/d32/d21/d22/f62 [0,4194304] 0 2024-04-02T04:47:46.768 INFO:tasks.workunit.client.1.smithi179.stdout:5/286: unlink d8/fb 0 2024-04-02T04:47:46.770 INFO:tasks.workunit.client.0.smithi070.stdout:6/262: rename d0/f25 to d0/d3/d19/d39/f53 0 2024-04-02T04:47:46.771 INFO:tasks.workunit.client.0.smithi070.stdout:6/263: truncate d0/d20/f45 496203 0 2024-04-02T04:47:46.772 INFO:tasks.workunit.client.1.smithi179.stdout:0/408: unlink d1/da/f1d 0 2024-04-02T04:47:46.775 INFO:tasks.workunit.client.1.smithi179.stdout:9/349: rename d0/d1/d10/c33 to d0/d1/d11/c6a 0 2024-04-02T04:47:46.777 INFO:tasks.workunit.client.1.smithi179.stdout:4/427: symlink d1/d3/dc/l8d 0 2024-04-02T04:47:46.778 INFO:tasks.workunit.client.1.smithi179.stdout:0/409: dwrite d1/da/d28/d37/f3e [0,4194304] 0 2024-04-02T04:47:46.779 INFO:tasks.workunit.client.1.smithi179.stdout:5/287: mkdir d8/d1b/d23/d36/d53 0 2024-04-02T04:47:46.782 INFO:tasks.workunit.client.0.smithi070.stdout:5/383: rmdir d5/d9/d10/d51/d78 0 2024-04-02T04:47:46.787 INFO:tasks.workunit.client.1.smithi179.stdout:4/428: dwrite d1/d3/dc/d26/d8a/f45 [4194304,4194304] 0 2024-04-02T04:47:46.787 INFO:tasks.workunit.client.0.smithi070.stdout:4/305: rename d1/f50 to d1/d3/d9/f64 0 2024-04-02T04:47:46.787 INFO:tasks.workunit.client.1.smithi179.stdout:0/410: dwrite d1/f57 [0,4194304] 0 2024-04-02T04:47:46.789 INFO:tasks.workunit.client.1.smithi179.stdout:2/233: getdents dc/d29 0 2024-04-02T04:47:46.790 INFO:tasks.workunit.client.1.smithi179.stdout:0/411: chown d1/d5b/f6c 0 1 2024-04-02T04:47:46.790 INFO:tasks.workunit.client.1.smithi179.stdout:7/405: rename d1/d9/d35/f91 to d1/d84/f92 0 2024-04-02T04:47:46.791 INFO:tasks.workunit.client.1.smithi179.stdout:3/354: sync 2024-04-02T04:47:46.796 INFO:tasks.workunit.client.1.smithi179.stdout:9/350: creat d0/d19/d1c/f6b x:0 0 0 2024-04-02T04:47:46.798 INFO:tasks.workunit.client.1.smithi179.stdout:2/234: dwrite dc/d29/d31/f41 [0,4194304] 0 2024-04-02T04:47:46.801 INFO:tasks.workunit.client.0.smithi070.stdout:0/411: rename d4/d5/d13/d43/c50 to d4/d5/d13/d2a/d29/d52/c73 0 2024-04-02T04:47:46.805 INFO:tasks.workunit.client.0.smithi070.stdout:5/384: mkdir d5/d9/d10/d6a/d7d 0 2024-04-02T04:47:46.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/385: truncate d5/d17/f53 418793 0 2024-04-02T04:47:46.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/386: stat d5/d57/l69 0 2024-04-02T04:47:46.809 INFO:tasks.workunit.client.0.smithi070.stdout:0/412: unlink d4/d5/f14 0 2024-04-02T04:47:46.810 INFO:tasks.workunit.client.0.smithi070.stdout:0/413: chown d4/d5/d13/f2b 0 1 2024-04-02T04:47:46.810 INFO:tasks.workunit.client.0.smithi070.stdout:4/306: dread d1/d3/d9/f64 [0,4194304] 0 2024-04-02T04:47:46.815 INFO:tasks.workunit.client.0.smithi070.stdout:0/414: readlink d4/d5/l38 0 2024-04-02T04:47:46.816 INFO:tasks.workunit.client.1.smithi179.stdout:0/412: mkdir d1/da/d48/d7b 0 2024-04-02T04:47:46.817 INFO:tasks.workunit.client.1.smithi179.stdout:0/413: chown d1/da/d28/l71 3731 1 2024-04-02T04:47:46.817 INFO:tasks.workunit.client.0.smithi070.stdout:4/307: mkdir d1/d3/d9/d41/d65 0 2024-04-02T04:47:46.820 INFO:tasks.workunit.client.0.smithi070.stdout:5/387: truncate d5/d17/d27/f7c 2107734 0 2024-04-02T04:47:46.820 INFO:tasks.workunit.client.1.smithi179.stdout:5/288: dread d8/f1a [0,4194304] 0 2024-04-02T04:47:46.821 INFO:tasks.workunit.client.0.smithi070.stdout:3/341: dwrite d10/d32/d3f/f38 [0,4194304] 0 2024-04-02T04:47:46.823 INFO:tasks.workunit.client.1.smithi179.stdout:3/355: mknod d8/d26/d52/c73 0 2024-04-02T04:47:46.824 INFO:tasks.workunit.client.0.smithi070.stdout:3/342: write d10/d32/f52 [492384,105411] 0 2024-04-02T04:47:46.825 INFO:tasks.workunit.client.0.smithi070.stdout:5/388: mkdir d5/d14/d7e 0 2024-04-02T04:47:46.828 INFO:tasks.workunit.client.0.smithi070.stdout:4/308: dwrite d1/d3/d9/f1a [0,4194304] 0 2024-04-02T04:47:46.829 INFO:tasks.workunit.client.1.smithi179.stdout:9/351: fsync d0/d1/d10/f2b 0 2024-04-02T04:47:46.830 INFO:tasks.workunit.client.1.smithi179.stdout:3/356: dwrite d8/d57/d63/f6c [0,4194304] 0 2024-04-02T04:47:46.831 INFO:tasks.workunit.client.0.smithi070.stdout:6/264: dwrite d0/d3/d19/d31/f36 [0,4194304] 0 2024-04-02T04:47:46.834 INFO:tasks.workunit.client.0.smithi070.stdout:5/389: mknod d5/d9/d10/c7f 0 2024-04-02T04:47:46.840 INFO:tasks.workunit.client.1.smithi179.stdout:1/362: getdents d3/d5/d2f/d4f/d42 0 2024-04-02T04:47:46.840 INFO:tasks.workunit.client.1.smithi179.stdout:4/429: truncate d1/d3/df/f51 1778831 0 2024-04-02T04:47:46.841 INFO:tasks.workunit.client.0.smithi070.stdout:4/309: creat d1/d3/d9/d41/f66 x:0 0 0 2024-04-02T04:47:46.846 INFO:tasks.workunit.client.1.smithi179.stdout:6/364: rename d3/ld to d3/d12/d23/d26/l83 0 2024-04-02T04:47:46.846 INFO:tasks.workunit.client.1.smithi179.stdout:1/363: dwrite d3/d5/de/f4e [0,4194304] 0 2024-04-02T04:47:46.848 INFO:tasks.workunit.client.1.smithi179.stdout:7/406: write d1/dd/d76/f86 [1023000,19207] 0 2024-04-02T04:47:46.851 INFO:tasks.workunit.client.1.smithi179.stdout:5/289: creat d8/d1b/d23/d32/d4e/f54 x:0 0 0 2024-04-02T04:47:46.855 INFO:tasks.workunit.client.0.smithi070.stdout:5/390: creat d5/d66/d75/f80 x:0 0 0 2024-04-02T04:47:46.856 INFO:tasks.workunit.client.0.smithi070.stdout:5/391: fsync d5/d9/d23/d30/f79 0 2024-04-02T04:47:46.858 INFO:tasks.workunit.client.0.smithi070.stdout:4/310: creat d1/d3/f67 x:0 0 0 2024-04-02T04:47:46.859 INFO:tasks.workunit.client.0.smithi070.stdout:4/311: write d1/d3/d9/d41/f45 [1252890,114180] 0 2024-04-02T04:47:46.861 INFO:tasks.workunit.client.1.smithi179.stdout:8/327: rename d3/d32/l1a to d3/d32/d21/d22/l73 0 2024-04-02T04:47:46.861 INFO:tasks.workunit.client.0.smithi070.stdout:3/343: link d10/c2e d10/d17/d51/c79 0 2024-04-02T04:47:46.867 INFO:tasks.workunit.client.1.smithi179.stdout:5/290: dread f6 [0,4194304] 0 2024-04-02T04:47:46.868 INFO:tasks.workunit.client.1.smithi179.stdout:0/414: creat d1/db/f7c x:0 0 0 2024-04-02T04:47:46.871 INFO:tasks.workunit.client.1.smithi179.stdout:3/357: rename d8/d26/d36/c37 to d8/d26/c74 0 2024-04-02T04:47:46.872 INFO:tasks.workunit.client.1.smithi179.stdout:7/407: dread d1/dd/f1c [0,4194304] 0 2024-04-02T04:47:46.873 INFO:tasks.workunit.client.1.smithi179.stdout:6/365: dread d3/fa [0,4194304] 0 2024-04-02T04:47:46.873 INFO:tasks.workunit.client.1.smithi179.stdout:5/291: mknod d8/d1b/c55 0 2024-04-02T04:47:46.874 INFO:tasks.workunit.client.1.smithi179.stdout:8/328: link d3/f4 d3/d32/d21/d50/d56/f74 0 2024-04-02T04:47:46.874 INFO:tasks.workunit.client.1.smithi179.stdout:5/292: dread - d8/d1b/f49 zero size 2024-04-02T04:47:46.875 INFO:tasks.workunit.client.1.smithi179.stdout:7/408: mknod d1/d9/d35/d6b/c93 0 2024-04-02T04:47:46.875 INFO:tasks.workunit.client.1.smithi179.stdout:6/366: truncate d3/d12/f22 394891 0 2024-04-02T04:47:46.875 INFO:tasks.workunit.client.1.smithi179.stdout:5/293: chown d8/d1b/d23/f47 7 1 2024-04-02T04:47:46.876 INFO:tasks.workunit.client.1.smithi179.stdout:7/409: write d1/d9/f85 [352870,12242] 0 2024-04-02T04:47:46.879 INFO:tasks.workunit.client.1.smithi179.stdout:6/367: unlink d3/d2d/d38/l49 0 2024-04-02T04:47:46.881 INFO:tasks.workunit.client.1.smithi179.stdout:8/329: dwrite d3/d32/d21/d22/f62 [4194304,4194304] 0 2024-04-02T04:47:46.881 INFO:tasks.workunit.client.1.smithi179.stdout:5/294: link d8/f3e d8/d1b/f56 0 2024-04-02T04:47:46.882 INFO:tasks.workunit.client.1.smithi179.stdout:5/295: dread - d8/d1b/f49 zero size 2024-04-02T04:47:46.884 INFO:tasks.workunit.client.1.smithi179.stdout:6/368: link d3/l4c d3/d12/d44/l84 0 2024-04-02T04:47:46.886 INFO:tasks.workunit.client.1.smithi179.stdout:8/330: link d3/d31/l54 d3/d32/d21/d51/l75 0 2024-04-02T04:47:46.889 INFO:tasks.workunit.client.1.smithi179.stdout:8/331: write d3/d32/d21/d49/f72 [360893,6239] 0 2024-04-02T04:47:46.891 INFO:tasks.workunit.client.1.smithi179.stdout:6/369: dwrite d3/f1c [0,4194304] 0 2024-04-02T04:47:46.892 INFO:tasks.workunit.client.1.smithi179.stdout:6/370: chown d3/d42/f54 1874040959 1 2024-04-02T04:47:46.893 INFO:tasks.workunit.client.1.smithi179.stdout:6/371: chown d3/d12/d23/d39 26457 1 2024-04-02T04:47:46.896 INFO:tasks.workunit.client.1.smithi179.stdout:6/372: truncate d3/d12/d23/d26/f4f 2234962 0 2024-04-02T04:47:46.896 INFO:tasks.workunit.client.1.smithi179.stdout:6/373: dread - d3/d12/d23/d39/f71 zero size 2024-04-02T04:47:46.897 INFO:tasks.workunit.client.0.smithi070.stdout:9/297: sync 2024-04-02T04:47:46.897 INFO:tasks.workunit.client.1.smithi179.stdout:6/374: dread - d3/d12/d6a/f6e zero size 2024-04-02T04:47:46.900 INFO:tasks.workunit.client.0.smithi070.stdout:5/392: rmdir d5/d66/d77 0 2024-04-02T04:47:46.901 INFO:tasks.workunit.client.1.smithi179.stdout:8/332: rmdir d3/d32/d21/d22/d6d 0 2024-04-02T04:47:46.902 INFO:tasks.workunit.client.1.smithi179.stdout:8/333: stat d3/d32/f6e 0 2024-04-02T04:47:46.902 INFO:tasks.workunit.client.0.smithi070.stdout:4/312: link d1/d3/d9/c60 d1/d3/d9/d41/d65/c68 0 2024-04-02T04:47:46.902 INFO:tasks.workunit.client.1.smithi179.stdout:5/296: dread d8/d1b/d23/d32/f3d [0,4194304] 0 2024-04-02T04:47:46.903 INFO:tasks.workunit.client.1.smithi179.stdout:5/297: stat d8/d1b/d23/d32/d4e/d51 0 2024-04-02T04:47:46.904 INFO:tasks.workunit.client.0.smithi070.stdout:9/298: rmdir d4/d10/d12 39 2024-04-02T04:47:46.904 INFO:tasks.workunit.client.1.smithi179.stdout:3/358: dread d8/d26/d36/f38 [0,4194304] 0 2024-04-02T04:47:46.905 INFO:tasks.workunit.client.1.smithi179.stdout:3/359: fdatasync d8/d26/d36/f47 0 2024-04-02T04:47:46.905 INFO:tasks.workunit.client.1.smithi179.stdout:6/375: mkdir d3/d2c/d85 0 2024-04-02T04:47:46.906 INFO:tasks.workunit.client.1.smithi179.stdout:6/376: write d3/f1c [2478781,98255] 0 2024-04-02T04:47:46.907 INFO:tasks.workunit.client.0.smithi070.stdout:3/344: dwrite d10/d32/d4b/f61 [0,4194304] 0 2024-04-02T04:47:46.909 INFO:tasks.workunit.client.0.smithi070.stdout:3/345: truncate d10/d32/d3f/f6a 653939 0 2024-04-02T04:47:46.912 INFO:tasks.workunit.client.0.smithi070.stdout:5/393: creat d5/d9/d10/d6a/f81 x:0 0 0 2024-04-02T04:47:46.916 INFO:tasks.workunit.client.0.smithi070.stdout:6/265: dwrite d0/d3/d19/d31/f36 [4194304,4194304] 0 2024-04-02T04:47:46.919 INFO:tasks.workunit.client.0.smithi070.stdout:9/299: rmdir d4/d13/d16/d19/d2f 39 2024-04-02T04:47:46.922 INFO:tasks.workunit.client.0.smithi070.stdout:4/313: mknod d1/d3/c69 0 2024-04-02T04:47:46.924 INFO:tasks.workunit.client.1.smithi179.stdout:6/377: mknod d3/d12/d23/d26/d57/c86 0 2024-04-02T04:47:46.927 INFO:tasks.workunit.client.1.smithi179.stdout:5/298: symlink d8/d1b/d23/d32/d4e/d50/l57 0 2024-04-02T04:47:46.928 INFO:tasks.workunit.client.0.smithi070.stdout:5/394: symlink d5/d14/l82 0 2024-04-02T04:47:46.929 INFO:tasks.workunit.client.1.smithi179.stdout:6/378: dread d3/fa [0,4194304] 0 2024-04-02T04:47:46.930 INFO:tasks.workunit.client.0.smithi070.stdout:6/266: symlink d0/d43/l54 0 2024-04-02T04:47:46.931 INFO:tasks.workunit.client.1.smithi179.stdout:1/364: write d3/d5/d2f/d4f/f4c [3842943,19923] 0 2024-04-02T04:47:46.934 INFO:tasks.workunit.client.1.smithi179.stdout:2/235: dwrite dc/d29/d4a/f24 [4194304,4194304] 0 2024-04-02T04:47:46.934 INFO:tasks.workunit.client.1.smithi179.stdout:9/352: truncate d0/d1/d11/d29/f37 3881725 0 2024-04-02T04:47:46.937 INFO:tasks.workunit.client.0.smithi070.stdout:4/314: write d1/fb [4021971,61280] 0 2024-04-02T04:47:46.938 INFO:tasks.workunit.client.0.smithi070.stdout:4/315: chown d1/d3/f62 1 1 2024-04-02T04:47:46.939 INFO:tasks.workunit.client.1.smithi179.stdout:4/430: truncate d1/d3/df/d10/d2f/d36/f58 1148205 0 2024-04-02T04:47:46.942 INFO:tasks.workunit.client.0.smithi070.stdout:3/346: rename d10/d17/c1e to d10/c7a 0 2024-04-02T04:47:46.943 INFO:tasks.workunit.client.0.smithi070.stdout:3/347: chown d10/d32/d4b/l6e 98173997 1 2024-04-02T04:47:46.947 INFO:tasks.workunit.client.0.smithi070.stdout:3/348: dread f1 [0,4194304] 0 2024-04-02T04:47:46.948 INFO:tasks.workunit.client.0.smithi070.stdout:3/349: rename d10/d17 to d10/d17/d51/d73/d7b 22 2024-04-02T04:47:46.952 INFO:tasks.workunit.client.1.smithi179.stdout:9/353: mkdir d0/d19/d27/d48/d6c 0 2024-04-02T04:47:46.952 INFO:tasks.workunit.client.1.smithi179.stdout:7/410: write d1/dd/d16/d74/f7a [121632,50097] 0 2024-04-02T04:47:46.958 INFO:tasks.workunit.client.0.smithi070.stdout:4/316: dread d1/d3/d9/f13 [4194304,4194304] 0 2024-04-02T04:47:46.960 INFO:tasks.workunit.client.0.smithi070.stdout:4/317: truncate d1/d3/f54 228593 0 2024-04-02T04:47:46.962 INFO:tasks.workunit.client.0.smithi070.stdout:5/395: creat d5/d17/d27/d2e/f83 x:0 0 0 2024-04-02T04:47:46.963 INFO:tasks.workunit.client.0.smithi070.stdout:5/396: read d5/d17/d2b/f67 [60912,77855] 0 2024-04-02T04:47:46.963 INFO:tasks.workunit.client.1.smithi179.stdout:6/379: truncate d3/fa 3457423 0 2024-04-02T04:47:46.964 INFO:tasks.workunit.client.0.smithi070.stdout:6/267: mkdir d0/d55 0 2024-04-02T04:47:46.970 INFO:tasks.workunit.client.1.smithi179.stdout:2/236: mknod dc/c4c 0 2024-04-02T04:47:46.972 INFO:tasks.workunit.client.1.smithi179.stdout:3/360: dwrite d8/d26/d36/f4d [0,4194304] 0 2024-04-02T04:47:46.973 INFO:tasks.workunit.client.1.smithi179.stdout:3/361: fdatasync d8/d26/f4e 0 2024-04-02T04:47:46.974 INFO:tasks.workunit.client.1.smithi179.stdout:7/411: rmdir d1/dd/d16/d4d 39 2024-04-02T04:47:46.975 INFO:tasks.workunit.client.1.smithi179.stdout:3/362: dread d8/d26/d36/f38 [0,4194304] 0 2024-04-02T04:47:46.975 INFO:tasks.workunit.client.1.smithi179.stdout:9/354: rename d0/d1/d12 to d0/d1/d10/d6d 0 2024-04-02T04:47:46.975 INFO:tasks.workunit.client.0.smithi070.stdout:3/350: truncate d10/d17/f33 500444 0 2024-04-02T04:47:46.977 INFO:tasks.workunit.client.1.smithi179.stdout:0/415: write d1/da/d28/d34/f35 [1199670,52202] 0 2024-04-02T04:47:46.978 INFO:tasks.workunit.client.1.smithi179.stdout:4/431: dread d1/d3/f28 [0,4194304] 0 2024-04-02T04:47:46.983 INFO:tasks.workunit.client.0.smithi070.stdout:3/351: dwrite d10/f22 [0,4194304] 0 2024-04-02T04:47:46.984 INFO:tasks.workunit.client.1.smithi179.stdout:8/334: sync 2024-04-02T04:47:46.985 INFO:tasks.workunit.client.1.smithi179.stdout:8/335: readlink d3/l3a 0 2024-04-02T04:47:46.988 INFO:tasks.workunit.client.1.smithi179.stdout:8/336: write d3/d32/d21/d22/d33/d3c/f40 [2769994,22612] 0 2024-04-02T04:47:46.993 INFO:tasks.workunit.client.0.smithi070.stdout:3/352: symlink d10/d32/d3f/l7c 0 2024-04-02T04:47:46.993 INFO:tasks.workunit.client.1.smithi179.stdout:3/363: mknod d8/c75 0 2024-04-02T04:47:46.994 INFO:tasks.workunit.client.0.smithi070.stdout:3/353: read d10/f22 [2439523,6716] 0 2024-04-02T04:47:46.994 INFO:tasks.workunit.client.1.smithi179.stdout:9/355: symlink d0/d19/d27/d48/l6e 0 2024-04-02T04:47:46.995 INFO:tasks.workunit.client.0.smithi070.stdout:5/397: link d5/d9/d23/d30/c44 d5/d9/d23/d30/c84 0 2024-04-02T04:47:46.995 INFO:tasks.workunit.client.0.smithi070.stdout:5/398: dread - d5/d57/f5d zero size 2024-04-02T04:47:46.996 INFO:tasks.workunit.client.1.smithi179.stdout:5/299: getdents d8/d1b/d23/d42 0 2024-04-02T04:47:46.997 INFO:tasks.workunit.client.0.smithi070.stdout:5/399: read d5/d9/f5e [4049401,102529] 0 2024-04-02T04:47:46.997 INFO:tasks.workunit.client.0.smithi070.stdout:4/318: link d1/f26 d1/d3/d9/d41/d65/f6a 0 2024-04-02T04:47:47.001 INFO:tasks.workunit.client.1.smithi179.stdout:6/380: creat d3/d12/d23/d39/d4e/d66/d7c/f87 x:0 0 0 2024-04-02T04:47:47.002 INFO:tasks.workunit.client.1.smithi179.stdout:1/365: dwrite d3/d5/f13 [0,4194304] 0 2024-04-02T04:47:47.002 INFO:tasks.workunit.client.1.smithi179.stdout:6/381: read - d3/d12/d44/f5e zero size 2024-04-02T04:47:47.003 INFO:tasks.workunit.client.1.smithi179.stdout:0/416: rename d1/d5b to d1/da/d48/d65/d7d 0 2024-04-02T04:47:47.006 INFO:tasks.workunit.client.0.smithi070.stdout:5/400: readlink d5/d66/l5b 0 2024-04-02T04:47:47.008 INFO:tasks.workunit.client.0.smithi070.stdout:9/300: dwrite d4/d10/d11/f3f [0,4194304] 0 2024-04-02T04:47:47.013 INFO:tasks.workunit.client.0.smithi070.stdout:5/401: dread d5/d9/d10/f11 [0,4194304] 0 2024-04-02T04:47:47.016 INFO:tasks.workunit.client.0.smithi070.stdout:4/319: dwrite d1/fe [0,4194304] 0 2024-04-02T04:47:47.019 INFO:tasks.workunit.client.1.smithi179.stdout:9/356: creat d0/d1/d10/f6f x:0 0 0 2024-04-02T04:47:47.027 INFO:tasks.workunit.client.0.smithi070.stdout:4/320: fdatasync d1/f32 0 2024-04-02T04:47:47.028 INFO:tasks.workunit.client.1.smithi179.stdout:5/300: truncate d8/d1b/d23/d32/f3d 596607 0 2024-04-02T04:47:47.030 INFO:tasks.workunit.client.0.smithi070.stdout:9/301: mknod d4/d13/d16/d19/d2f/d39/c64 0 2024-04-02T04:47:47.031 INFO:tasks.workunit.client.1.smithi179.stdout:1/366: creat d3/d5/de/d33/f7b x:0 0 0 2024-04-02T04:47:47.032 INFO:tasks.workunit.client.1.smithi179.stdout:1/367: chown d3/d5/d2f/d4f/d42/d55/l76 66 1 2024-04-02T04:47:47.035 INFO:tasks.workunit.client.1.smithi179.stdout:4/432: mkdir d1/d3/df/d8e 0 2024-04-02T04:47:47.036 INFO:tasks.workunit.client.1.smithi179.stdout:4/433: stat d1/d3/dc/f18 0 2024-04-02T04:47:47.037 INFO:tasks.workunit.client.1.smithi179.stdout:0/417: creat d1/da/d28/d34/f7e x:0 0 0 2024-04-02T04:47:47.038 INFO:tasks.workunit.client.1.smithi179.stdout:0/418: readlink d1/da/df/l59 0 2024-04-02T04:47:47.038 INFO:tasks.workunit.client.0.smithi070.stdout:4/321: read d1/fa [2236305,119850] 0 2024-04-02T04:47:47.038 INFO:tasks.workunit.client.0.smithi070.stdout:9/302: write d4/d10/d12/d27/f5c [724485,76890] 0 2024-04-02T04:47:47.040 INFO:tasks.workunit.client.0.smithi070.stdout:4/322: read d1/fa [1546644,72453] 0 2024-04-02T04:47:47.044 INFO:tasks.workunit.client.0.smithi070.stdout:3/354: write d10/d17/d4c/f62 [643312,117422] 0 2024-04-02T04:47:47.046 INFO:tasks.workunit.client.1.smithi179.stdout:8/337: link d3/d32/d21/d22/f62 d3/f76 0 2024-04-02T04:47:47.046 INFO:tasks.workunit.client.1.smithi179.stdout:3/364: write d8/f1b [2540156,73671] 0 2024-04-02T04:47:47.048 INFO:tasks.workunit.client.0.smithi070.stdout:5/402: write d5/d17/d6c/f41 [218886,115126] 0 2024-04-02T04:47:47.050 INFO:tasks.workunit.client.0.smithi070.stdout:3/355: creat d10/d32/d4b/f7d x:0 0 0 2024-04-02T04:47:47.054 INFO:tasks.workunit.client.1.smithi179.stdout:4/434: dread d1/d3/dc/d26/f29 [0,4194304] 0 2024-04-02T04:47:47.056 INFO:tasks.workunit.client.0.smithi070.stdout:5/403: symlink d5/d9/d23/d30/l85 0 2024-04-02T04:47:47.057 INFO:tasks.workunit.client.0.smithi070.stdout:5/404: truncate d5/d66/d75/f80 587815 0 2024-04-02T04:47:47.060 INFO:tasks.workunit.client.0.smithi070.stdout:4/323: link d1/d3/c4b d1/d3/d9/d41/d65/c6b 0 2024-04-02T04:47:47.061 INFO:tasks.workunit.client.0.smithi070.stdout:9/303: dread d4/f9 [4194304,4194304] 0 2024-04-02T04:47:47.061 INFO:tasks.workunit.client.1.smithi179.stdout:2/237: truncate dc/d29/d4a/f22 389907 0 2024-04-02T04:47:47.062 INFO:tasks.workunit.client.0.smithi070.stdout:3/356: mknod d10/d17/c7e 0 2024-04-02T04:47:47.063 INFO:tasks.workunit.client.0.smithi070.stdout:9/304: truncate d4/d10/d12/d27/f30 38541 0 2024-04-02T04:47:47.064 INFO:tasks.workunit.client.0.smithi070.stdout:9/305: write d4/d10/d11/f4e [1493118,82110] 0 2024-04-02T04:47:47.067 INFO:tasks.workunit.client.0.smithi070.stdout:4/324: fsync d1/f33 0 2024-04-02T04:47:47.069 INFO:tasks.workunit.client.0.smithi070.stdout:5/405: dread d5/d9/f5e [0,4194304] 0 2024-04-02T04:47:47.071 INFO:tasks.workunit.client.0.smithi070.stdout:9/306: dwrite d4/d13/d16/f2b [0,4194304] 0 2024-04-02T04:47:47.073 INFO:tasks.workunit.client.0.smithi070.stdout:9/307: write d4/d13/d16/f61 [785210,22654] 0 2024-04-02T04:47:47.074 INFO:tasks.workunit.client.0.smithi070.stdout:9/308: chown d4/d10/d12/d56/f5b 65 1 2024-04-02T04:47:47.094 INFO:tasks.workunit.client.0.smithi070.stdout:3/357: mknod d10/d32/d4b/c7f 0 2024-04-02T04:47:47.099 INFO:tasks.workunit.client.0.smithi070.stdout:7/370: sync 2024-04-02T04:47:47.099 INFO:tasks.workunit.client.0.smithi070.stdout:8/389: sync 2024-04-02T04:47:47.100 INFO:tasks.workunit.client.0.smithi070.stdout:8/390: fsync d2/d3/f61 0 2024-04-02T04:47:47.101 INFO:tasks.workunit.client.0.smithi070.stdout:7/371: write d7/d9/fa [8272035,64178] 0 2024-04-02T04:47:47.101 INFO:tasks.workunit.client.0.smithi070.stdout:8/391: chown d2/d8/d29/d22/f60 648750 1 2024-04-02T04:47:47.102 INFO:tasks.workunit.client.0.smithi070.stdout:8/392: write d2/d3/f61 [808862,101643] 0 2024-04-02T04:47:47.104 INFO:tasks.workunit.client.0.smithi070.stdout:8/393: write d2/d8/d23/d24/f4b [868014,2883] 0 2024-04-02T04:47:47.126 INFO:tasks.workunit.client.0.smithi070.stdout:2/424: sync 2024-04-02T04:47:47.126 INFO:tasks.workunit.client.0.smithi070.stdout:1/379: sync 2024-04-02T04:47:47.126 INFO:tasks.workunit.client.0.smithi070.stdout:0/415: sync 2024-04-02T04:47:47.126 INFO:tasks.workunit.client.0.smithi070.stdout:6/268: sync 2024-04-02T04:47:47.127 INFO:tasks.workunit.client.0.smithi070.stdout:6/269: dread - d0/d3/f3a zero size 2024-04-02T04:47:47.129 INFO:tasks.workunit.client.1.smithi179.stdout:3/365: write d8/f6f [356817,24486] 0 2024-04-02T04:47:47.129 INFO:tasks.workunit.client.0.smithi070.stdout:0/416: read d4/d5/d13/d2a/d29/f46 [2759289,129177] 0 2024-04-02T04:47:47.133 INFO:tasks.workunit.client.1.smithi179.stdout:9/357: rename d0/d19/d1c/l2e to d0/d1/d10/l70 0 2024-04-02T04:47:47.134 INFO:tasks.workunit.client.1.smithi179.stdout:6/382: write d3/d12/d23/f2b [2102658,32736] 0 2024-04-02T04:47:47.134 INFO:tasks.workunit.client.0.smithi070.stdout:0/417: read d4/d5/da/d2d/f66 [3869032,118837] 0 2024-04-02T04:47:47.135 INFO:tasks.workunit.client.0.smithi070.stdout:1/380: dread d0/d7/d1a/f24 [0,4194304] 0 2024-04-02T04:47:47.137 INFO:tasks.workunit.client.0.smithi070.stdout:2/425: dwrite d2/d10/d19/f77 [0,4194304] 0 2024-04-02T04:47:47.139 INFO:tasks.workunit.client.0.smithi070.stdout:0/418: write d4/d5/d13/d43/f53 [76776,46567] 0 2024-04-02T04:47:47.152 INFO:tasks.workunit.client.1.smithi179.stdout:7/412: link d1/dd/d16/d4d/c89 d1/dd/d16/d74/c94 0 2024-04-02T04:47:47.153 INFO:tasks.workunit.client.0.smithi070.stdout:7/372: read d7/dd/df/f4d [72248,3580] 0 2024-04-02T04:47:47.154 INFO:tasks.workunit.client.0.smithi070.stdout:7/373: fsync d7/d9/fa 0 2024-04-02T04:47:47.162 INFO:tasks.workunit.client.0.smithi070.stdout:8/394: dread d2/d3/d52/f55 [0,4194304] 0 2024-04-02T04:47:47.162 INFO:tasks.workunit.client.1.smithi179.stdout:3/366: fsync d8/d26/f34 0 2024-04-02T04:47:47.163 INFO:tasks.workunit.client.1.smithi179.stdout:3/367: write d8/d26/d36/f47 [1520397,122062] 0 2024-04-02T04:47:47.166 INFO:tasks.workunit.client.1.smithi179.stdout:9/358: mknod d0/d1/d11/d3a/c71 0 2024-04-02T04:47:47.167 INFO:tasks.workunit.client.1.smithi179.stdout:9/359: read d0/d1/d11/d29/f50 [3751248,109716] 0 2024-04-02T04:47:47.168 INFO:tasks.workunit.client.1.smithi179.stdout:9/360: fdatasync d0/d19/d1c/f6b 0 2024-04-02T04:47:47.174 INFO:tasks.workunit.client.1.smithi179.stdout:9/361: dread d0/d1/f36 [0,4194304] 0 2024-04-02T04:47:47.174 INFO:tasks.workunit.client.1.smithi179.stdout:5/301: link d8/c22 d8/d1b/d23/d42/c58 0 2024-04-02T04:47:47.175 INFO:tasks.workunit.client.1.smithi179.stdout:9/362: write d0/d1/f44 [1811644,7997] 0 2024-04-02T04:47:47.176 INFO:tasks.workunit.client.1.smithi179.stdout:9/363: readlink d0/d1/d10/d6d/l1b 0 2024-04-02T04:47:47.177 INFO:tasks.workunit.client.0.smithi070.stdout:1/381: mkdir d0/d9/d1d/d22/d2c/d4f/d77 0 2024-04-02T04:47:47.179 INFO:tasks.workunit.client.1.smithi179.stdout:7/413: rename d1/dd/d16/d4d/l52 to d1/d9/d27/d87/l95 0 2024-04-02T04:47:47.180 INFO:tasks.workunit.client.0.smithi070.stdout:2/426: fsync d2/d10/d21/d2b/f82 0 2024-04-02T04:47:47.183 INFO:tasks.workunit.client.0.smithi070.stdout:5/406: mknod d5/d9/d10/d6a/d7d/c86 0 2024-04-02T04:47:47.186 INFO:tasks.workunit.client.0.smithi070.stdout:0/419: dread d4/d5/da/d2d/f4c [0,4194304] 0 2024-04-02T04:47:47.188 INFO:tasks.workunit.client.0.smithi070.stdout:1/382: dread d0/d9/d1d/f4e [0,4194304] 0 2024-04-02T04:47:47.189 INFO:tasks.workunit.client.1.smithi179.stdout:3/368: unlink d8/d26/c29 0 2024-04-02T04:47:47.193 INFO:tasks.workunit.client.1.smithi179.stdout:3/369: dread d8/d57/d63/f6c [0,4194304] 0 2024-04-02T04:47:47.194 INFO:tasks.workunit.client.1.smithi179.stdout:3/370: chown d8/d26/f3f 451643 1 2024-04-02T04:47:47.194 INFO:tasks.workunit.client.0.smithi070.stdout:7/374: mkdir d7/d9/d19/d75 0 2024-04-02T04:47:47.196 INFO:tasks.workunit.client.1.smithi179.stdout:6/383: unlink d3/d12/d44/l84 0 2024-04-02T04:47:47.197 INFO:tasks.workunit.client.1.smithi179.stdout:3/371: dread d8/d26/f72 [0,4194304] 0 2024-04-02T04:47:47.211 INFO:tasks.workunit.client.1.smithi179.stdout:0/419: link d1/da/d48/d65/d7d/f7a d1/da/d67/f7f 0 2024-04-02T04:47:47.216 INFO:tasks.workunit.client.1.smithi179.stdout:5/302: dread d8/d1b/f56 [0,4194304] 0 2024-04-02T04:47:47.226 INFO:tasks.workunit.client.1.smithi179.stdout:6/384: unlink d3/d12/d44/f5e 0 2024-04-02T04:47:47.227 INFO:tasks.workunit.client.1.smithi179.stdout:6/385: chown d3/d12/d23/f55 41109 1 2024-04-02T04:47:47.230 INFO:tasks.workunit.client.1.smithi179.stdout:0/420: mkdir d1/da/d48/d65/d7d/d80 0 2024-04-02T04:47:47.232 INFO:tasks.workunit.client.0.smithi070.stdout:0/420: creat d4/d5/d13/d23/f74 x:0 0 0 2024-04-02T04:47:47.236 INFO:tasks.workunit.client.0.smithi070.stdout:1/383: creat d0/d54/d64/d6d/f78 x:0 0 0 2024-04-02T04:47:47.239 INFO:tasks.workunit.client.0.smithi070.stdout:7/375: mknod d7/dd/d66/d57/c76 0 2024-04-02T04:47:47.243 INFO:tasks.workunit.client.0.smithi070.stdout:8/395: creat d2/d8/d29/d22/d5a/f7e x:0 0 0 2024-04-02T04:47:47.244 INFO:tasks.workunit.client.1.smithi179.stdout:3/372: creat d8/d6d/f76 x:0 0 0 2024-04-02T04:47:47.244 INFO:tasks.workunit.client.1.smithi179.stdout:3/373: write d8/d26/f2b [2109714,3332] 0 2024-04-02T04:47:47.246 INFO:tasks.workunit.client.0.smithi070.stdout:8/396: dread d2/d8/d23/d24/d66/f6a [0,4194304] 0 2024-04-02T04:47:47.273 INFO:tasks.workunit.client.1.smithi179.stdout:7/414: sync 2024-04-02T04:47:47.326 INFO:tasks.workunit.client.1.smithi179.stdout:3/374: sync 2024-04-02T04:47:47.372 INFO:tasks.workunit.client.0.smithi070.stdout:0/421: rename d4/d5/d21/c54 to d4/d5/d13/d2a/d3a/c75 0 2024-04-02T04:47:47.373 INFO:tasks.workunit.client.0.smithi070.stdout:7/376: truncate d7/d9/f27 135286 0 2024-04-02T04:47:47.377 INFO:tasks.workunit.client.0.smithi070.stdout:4/325: dwrite d1/d3/d9/f22 [0,4194304] 0 2024-04-02T04:47:47.378 INFO:tasks.workunit.client.1.smithi179.stdout:7/415: creat d1/d9/d55/f96 x:0 0 0 2024-04-02T04:47:47.379 INFO:tasks.workunit.client.0.smithi070.stdout:3/358: write d10/d32/f55 [241675,126327] 0 2024-04-02T04:47:47.381 INFO:tasks.workunit.client.0.smithi070.stdout:8/397: read d2/d8/d23/f51 [2343071,16724] 0 2024-04-02T04:47:47.382 INFO:tasks.workunit.client.0.smithi070.stdout:8/398: dread - d2/d8/d29/d22/d5a/f7e zero size 2024-04-02T04:47:47.382 INFO:tasks.workunit.client.1.smithi179.stdout:3/375: mknod d8/d57/d63/c77 0 2024-04-02T04:47:47.383 INFO:tasks.workunit.client.0.smithi070.stdout:5/407: link d5/d9/d10/l13 d5/d66/d75/l87 0 2024-04-02T04:47:47.383 INFO:tasks.workunit.client.0.smithi070.stdout:8/399: dread - d2/d3/d35/d3a/f47 zero size 2024-04-02T04:47:47.384 INFO:tasks.workunit.client.0.smithi070.stdout:8/400: fdatasync d2/d8/d29/d22/d5a/f7e 0 2024-04-02T04:47:47.386 INFO:tasks.workunit.client.0.smithi070.stdout:8/401: write d2/d8/d29/d22/f46 [1417974,46586] 0 2024-04-02T04:47:47.386 INFO:tasks.workunit.client.0.smithi070.stdout:0/422: rename d4/d5/l38 to d4/d5/d13/d2a/d3a/l76 0 2024-04-02T04:47:47.387 INFO:tasks.workunit.client.0.smithi070.stdout:8/402: fsync d2/d8/d23/d24/f5b 0 2024-04-02T04:47:47.392 INFO:tasks.workunit.client.1.smithi179.stdout:7/416: fdatasync d1/f32 0 2024-04-02T04:47:47.393 INFO:tasks.workunit.client.1.smithi179.stdout:8/338: write d3/f4 [4751702,9204] 0 2024-04-02T04:47:47.393 INFO:tasks.workunit.client.0.smithi070.stdout:0/423: dwrite d4/d5/d13/f32 [0,4194304] 0 2024-04-02T04:47:47.394 INFO:tasks.workunit.client.1.smithi179.stdout:3/376: mknod d8/d57/d63/c78 0 2024-04-02T04:47:47.395 INFO:tasks.workunit.client.0.smithi070.stdout:0/424: read - d4/d5/d13/d2a/d3a/f67 zero size 2024-04-02T04:47:47.396 INFO:tasks.workunit.client.0.smithi070.stdout:5/408: mknod d5/d17/d2b/c88 0 2024-04-02T04:47:47.397 INFO:tasks.workunit.client.1.smithi179.stdout:0/421: getdents d1/db 0 2024-04-02T04:47:47.402 INFO:tasks.workunit.client.1.smithi179.stdout:8/339: creat d3/d32/d21/d50/d56/f77 x:0 0 0 2024-04-02T04:47:47.407 INFO:tasks.workunit.client.0.smithi070.stdout:4/326: symlink d1/l6c 0 2024-04-02T04:47:47.408 INFO:tasks.workunit.client.1.smithi179.stdout:0/422: truncate d1/da/df/f21 1030012 0 2024-04-02T04:47:47.408 INFO:tasks.workunit.client.1.smithi179.stdout:8/340: dwrite d3/d32/d21/d51/f55 [0,4194304] 0 2024-04-02T04:47:47.409 INFO:tasks.workunit.client.0.smithi070.stdout:5/409: stat d5/f22 0 2024-04-02T04:47:47.410 INFO:tasks.workunit.client.0.smithi070.stdout:5/410: read d5/d9/d10/f11 [3762418,68169] 0 2024-04-02T04:47:47.411 INFO:tasks.workunit.client.1.smithi179.stdout:7/417: symlink d1/dd/d16/l97 0 2024-04-02T04:47:47.411 INFO:tasks.workunit.client.1.smithi179.stdout:4/435: write d1/d3/dc/d27/f7f [1255942,111999] 0 2024-04-02T04:47:47.416 INFO:tasks.workunit.client.0.smithi070.stdout:9/309: unlink d4/d10/d12/c23 0 2024-04-02T04:47:47.418 INFO:tasks.workunit.client.1.smithi179.stdout:3/377: unlink d8/d57/c68 0 2024-04-02T04:47:47.420 INFO:tasks.workunit.client.1.smithi179.stdout:8/341: fsync d3/f76 0 2024-04-02T04:47:47.422 INFO:tasks.workunit.client.0.smithi070.stdout:9/310: symlink d4/d13/d16/d19/d53/l65 0 2024-04-02T04:47:47.424 INFO:tasks.workunit.client.0.smithi070.stdout:9/311: dread d4/d10/f2c [0,4194304] 0 2024-04-02T04:47:47.424 INFO:tasks.workunit.client.1.smithi179.stdout:0/423: mkdir d1/da/d28/d81 0 2024-04-02T04:47:47.425 INFO:tasks.workunit.client.1.smithi179.stdout:0/424: write d1/db/fc [2012015,130983] 0 2024-04-02T04:47:47.427 INFO:tasks.workunit.client.1.smithi179.stdout:8/342: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:47:47.427 INFO:tasks.workunit.client.1.smithi179.stdout:3/378: chown d8/d26/f2c 1 1 2024-04-02T04:47:47.428 INFO:tasks.workunit.client.0.smithi070.stdout:4/327: creat d1/d3/f6d x:0 0 0 2024-04-02T04:47:47.429 INFO:tasks.workunit.client.0.smithi070.stdout:9/312: mkdir d4/d10/d12/d55/d66 0 2024-04-02T04:47:47.430 INFO:tasks.workunit.client.0.smithi070.stdout:9/313: chown d4/d13/d16/d19/d53/f5a 58321 1 2024-04-02T04:47:47.431 INFO:tasks.workunit.client.0.smithi070.stdout:9/314: read d4/d13/f51 [7436,69953] 0 2024-04-02T04:47:47.434 INFO:tasks.workunit.client.0.smithi070.stdout:7/377: rename d7/d9/d19 to d7/d4e/d55/d6c/d77 0 2024-04-02T04:47:47.435 INFO:tasks.workunit.client.0.smithi070.stdout:5/411: dread f4 [0,4194304] 0 2024-04-02T04:47:47.436 INFO:tasks.workunit.client.0.smithi070.stdout:7/378: write d7/d2f/d46/d65/f67 [76944,78071] 0 2024-04-02T04:47:47.437 INFO:tasks.workunit.client.0.smithi070.stdout:7/379: write d7/d2f/d46/f53 [720040,130088] 0 2024-04-02T04:47:47.439 INFO:tasks.workunit.client.0.smithi070.stdout:9/315: rename d4/d10/d12/f5e to d4/d10/d12/f67 0 2024-04-02T04:47:47.440 INFO:tasks.workunit.client.0.smithi070.stdout:9/316: fsync d4/d13/d16/d19/d53/f5a 0 2024-04-02T04:47:47.441 INFO:tasks.workunit.client.0.smithi070.stdout:9/317: dread - d4/d34/f4a zero size 2024-04-02T04:47:47.445 INFO:tasks.workunit.client.1.smithi179.stdout:8/343: getdents d3/d32/d21/d22/d6b 0 2024-04-02T04:47:47.446 INFO:tasks.workunit.client.0.smithi070.stdout:9/318: dwrite d4/d13/f37 [0,4194304] 0 2024-04-02T04:47:47.447 INFO:tasks.workunit.client.1.smithi179.stdout:0/425: link d1/db/l31 d1/da/d48/l82 0 2024-04-02T04:47:47.450 INFO:tasks.workunit.client.1.smithi179.stdout:8/344: mknod d3/d1b/d2a/c78 0 2024-04-02T04:47:47.453 INFO:tasks.workunit.client.0.smithi070.stdout:9/319: dwrite d4/d13/d16/f35 [0,4194304] 0 2024-04-02T04:47:47.453 INFO:tasks.workunit.client.1.smithi179.stdout:0/426: mkdir d1/da/d48/d5c/d83 0 2024-04-02T04:47:47.458 INFO:tasks.workunit.client.1.smithi179.stdout:0/427: mkdir d1/da/df/d56/d84 0 2024-04-02T04:47:47.458 INFO:tasks.workunit.client.1.smithi179.stdout:4/436: creat d1/d3/df/f8f x:0 0 0 2024-04-02T04:47:47.458 INFO:tasks.workunit.client.1.smithi179.stdout:0/428: dread - d1/da/f4a zero size 2024-04-02T04:47:47.460 INFO:tasks.workunit.client.0.smithi070.stdout:5/412: getdents d5/d9/d10/d6a/d7d 0 2024-04-02T04:47:47.462 INFO:tasks.workunit.client.1.smithi179.stdout:0/429: chown d1/da/c60 119 1 2024-04-02T04:47:47.464 INFO:tasks.workunit.client.0.smithi070.stdout:6/270: dwrite d0/d20/f27 [0,4194304] 0 2024-04-02T04:47:47.464 INFO:tasks.workunit.client.1.smithi179.stdout:4/437: rename d1/d3/dc/d26/d8a/c7e to d1/d3/dc/d27/d62/c90 0 2024-04-02T04:47:47.466 INFO:tasks.workunit.client.0.smithi070.stdout:9/320: dwrite d4/d13/d16/f61 [0,4194304] 0 2024-04-02T04:47:47.467 INFO:tasks.workunit.client.1.smithi179.stdout:1/368: mknod d3/d5/d2f/c7c 0 2024-04-02T04:47:47.468 INFO:tasks.workunit.client.1.smithi179.stdout:0/430: mknod d1/da/df/d56/d84/c85 0 2024-04-02T04:47:47.468 INFO:tasks.workunit.client.1.smithi179.stdout:0/431: chown d1/da/f1b 2 1 2024-04-02T04:47:47.469 INFO:tasks.workunit.client.0.smithi070.stdout:2/427: write d2/d10/d21/d2b/f6e [719497,24410] 0 2024-04-02T04:47:47.469 INFO:tasks.workunit.client.1.smithi179.stdout:0/432: rename d1/db/d64 to d1/db/d64/d86 22 2024-04-02T04:47:47.470 INFO:tasks.workunit.client.1.smithi179.stdout:4/438: creat d1/d3/df/d10/d2f/d7b/f91 x:0 0 0 2024-04-02T04:47:47.470 INFO:tasks.workunit.client.0.smithi070.stdout:5/413: dread d5/d9/d10/f11 [0,4194304] 0 2024-04-02T04:47:47.473 INFO:tasks.workunit.client.0.smithi070.stdout:6/271: dread d0/d43/f4a [0,4194304] 0 2024-04-02T04:47:47.473 INFO:tasks.workunit.client.1.smithi179.stdout:4/439: dread d1/d3/df/d10/d2f/f4e [0,4194304] 0 2024-04-02T04:47:47.475 INFO:tasks.workunit.client.1.smithi179.stdout:0/433: unlink d1/c32 0 2024-04-02T04:47:47.476 INFO:tasks.workunit.client.1.smithi179.stdout:0/434: readlink d1/da/df/l59 0 2024-04-02T04:47:47.477 INFO:tasks.workunit.client.0.smithi070.stdout:9/321: dwrite d4/d10/f18 [8388608,4194304] 0 2024-04-02T04:47:47.481 INFO:tasks.workunit.client.1.smithi179.stdout:4/440: mknod d1/d3/dc/d26/d8a/c92 0 2024-04-02T04:47:47.482 INFO:tasks.workunit.client.1.smithi179.stdout:0/435: symlink d1/da/d52/l87 0 2024-04-02T04:47:47.483 INFO:tasks.workunit.client.0.smithi070.stdout:6/272: mknod d0/d20/c56 0 2024-04-02T04:47:47.487 INFO:tasks.workunit.client.1.smithi179.stdout:8/345: dread d3/d32/d21/f37 [0,4194304] 0 2024-04-02T04:47:47.488 INFO:tasks.workunit.client.0.smithi070.stdout:2/428: rename d2/d10/d19/l2e to d2/d12/l87 0 2024-04-02T04:47:47.488 INFO:tasks.workunit.client.1.smithi179.stdout:0/436: readlink d1/da/df/l5f 0 2024-04-02T04:47:47.489 INFO:tasks.workunit.client.0.smithi070.stdout:2/429: write d2/d10/d21/d2b/f82 [703698,46948] 0 2024-04-02T04:47:47.490 INFO:tasks.workunit.client.0.smithi070.stdout:2/430: rename d2 to d2/d10/d53/d88 22 2024-04-02T04:47:47.491 INFO:tasks.workunit.client.0.smithi070.stdout:9/322: mkdir d4/d10/d12/d27/d68 0 2024-04-02T04:47:47.491 INFO:tasks.workunit.client.0.smithi070.stdout:2/431: write d2/d10/d21/d3d/f47 [1214225,15896] 0 2024-04-02T04:47:47.494 INFO:tasks.workunit.client.1.smithi179.stdout:0/437: mknod d1/da/d28/d6d/c88 0 2024-04-02T04:47:47.495 INFO:tasks.workunit.client.1.smithi179.stdout:8/346: dwrite d3/d32/d21/d22/d33/f60 [0,4194304] 0 2024-04-02T04:47:47.500 INFO:tasks.workunit.client.0.smithi070.stdout:9/323: creat d4/d10/d12/d27/f69 x:0 0 0 2024-04-02T04:47:47.500 INFO:tasks.workunit.client.1.smithi179.stdout:4/441: dread d1/d3/dc/d27/d62/f5e [0,4194304] 0 2024-04-02T04:47:47.501 INFO:tasks.workunit.client.0.smithi070.stdout:6/273: dread d0/d20/f34 [0,4194304] 0 2024-04-02T04:47:47.502 INFO:tasks.workunit.client.1.smithi179.stdout:8/347: mknod d3/c79 0 2024-04-02T04:47:47.502 INFO:tasks.workunit.client.1.smithi179.stdout:7/418: sync 2024-04-02T04:47:47.502 INFO:tasks.workunit.client.0.smithi070.stdout:2/432: rmdir d2/d10/d21/d71 39 2024-04-02T04:47:47.503 INFO:tasks.workunit.client.1.smithi179.stdout:9/364: symlink d0/d1/d11/l72 0 2024-04-02T04:47:47.504 INFO:tasks.workunit.client.1.smithi179.stdout:8/348: chown d3/d32/d21/d22/d33/d3c/c43 317956 1 2024-04-02T04:47:47.505 INFO:tasks.workunit.client.0.smithi070.stdout:5/414: rmdir d5/d17/d27/d2c 0 2024-04-02T04:47:47.506 INFO:tasks.workunit.client.1.smithi179.stdout:5/303: truncate d8/d1b/d23/d32/f33 3559819 0 2024-04-02T04:47:47.506 INFO:tasks.workunit.client.0.smithi070.stdout:5/415: chown d5/d9/d23/d30/l85 218 1 2024-04-02T04:47:47.507 INFO:tasks.workunit.client.0.smithi070.stdout:5/416: dread - d5/d17/d6c/f6d zero size 2024-04-02T04:47:47.509 INFO:tasks.workunit.client.0.smithi070.stdout:9/324: unlink d4/d10/d11/f21 0 2024-04-02T04:47:47.509 INFO:tasks.workunit.client.0.smithi070.stdout:9/325: readlink d4/d10/d12/d27/l52 0 2024-04-02T04:47:47.510 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:47 smithi179 ceph-mon[36603]: pgmap v12: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 178 MiB/s wr, 387 op/s 2024-04-02T04:47:47.510 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:47 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:47.510 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:47 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:47.510 INFO:tasks.workunit.client.0.smithi070.stdout:9/326: chown d4/d10/d12 832287 1 2024-04-02T04:47:47.510 INFO:tasks.workunit.client.1.smithi179.stdout:1/369: dread d3/d5/d2f/d4f/f37 [0,4194304] 0 2024-04-02T04:47:47.511 INFO:tasks.workunit.client.1.smithi179.stdout:7/419: rename d1/dd/d16/c4c to d1/dd/d16/d74/c98 0 2024-04-02T04:47:47.511 INFO:tasks.workunit.client.1.smithi179.stdout:0/438: dread d1/f2 [0,4194304] 0 2024-04-02T04:47:47.512 INFO:tasks.workunit.client.1.smithi179.stdout:7/420: chown d1/dd/d16/l67 1 1 2024-04-02T04:47:47.512 INFO:tasks.workunit.client.1.smithi179.stdout:0/439: chown d1/f14 1538 1 2024-04-02T04:47:47.514 INFO:tasks.workunit.client.1.smithi179.stdout:2/238: mknod dc/d29/d4a/c4d 0 2024-04-02T04:47:47.515 INFO:tasks.workunit.client.1.smithi179.stdout:5/304: read d8/f18 [530132,84504] 0 2024-04-02T04:47:47.518 INFO:tasks.workunit.client.1.smithi179.stdout:1/370: creat d3/d5/d2f/d4f/d42/f7d x:0 0 0 2024-04-02T04:47:47.519 INFO:tasks.workunit.client.1.smithi179.stdout:6/386: dwrite d3/d12/f22 [0,4194304] 0 2024-04-02T04:47:47.519 INFO:tasks.workunit.client.1.smithi179.stdout:5/305: write d8/f2c [2687140,120062] 0 2024-04-02T04:47:47.521 INFO:tasks.workunit.client.0.smithi070.stdout:6/274: symlink d0/d3/d19/d31/d37/d50/l57 0 2024-04-02T04:47:47.523 INFO:tasks.workunit.client.0.smithi070.stdout:5/417: rmdir d5/d17/d2b 39 2024-04-02T04:47:47.524 INFO:tasks.workunit.client.0.smithi070.stdout:9/327: symlink d4/d13/d16/l6a 0 2024-04-02T04:47:47.525 INFO:tasks.workunit.client.0.smithi070.stdout:9/328: readlink d4/d13/d16/d19/d53/l65 0 2024-04-02T04:47:47.526 INFO:tasks.workunit.client.0.smithi070.stdout:2/433: creat d2/d10/d21/f89 x:0 0 0 2024-04-02T04:47:47.527 INFO:tasks.workunit.client.1.smithi179.stdout:0/440: dwrite d1/da/d48/d5c/f78 [0,4194304] 0 2024-04-02T04:47:47.528 INFO:tasks.workunit.client.0.smithi070.stdout:6/275: symlink d0/d3/d19/d31/d46/l58 0 2024-04-02T04:47:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:47 smithi070 ceph-mon[29212]: pgmap v12: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 178 MiB/s wr, 387 op/s 2024-04-02T04:47:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:47 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:47 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:47.531 INFO:tasks.workunit.client.0.smithi070.stdout:5/418: unlink d5/d14/f54 0 2024-04-02T04:47:47.532 INFO:tasks.workunit.client.1.smithi179.stdout:1/371: dwrite d3/d5/de/d33/f51 [0,4194304] 0 2024-04-02T04:47:47.533 INFO:tasks.workunit.client.0.smithi070.stdout:6/276: dread d0/d43/f4a [0,4194304] 0 2024-04-02T04:47:47.533 INFO:tasks.workunit.client.1.smithi179.stdout:1/372: readlink d3/d5/d2f/l63 0 2024-04-02T04:47:47.534 INFO:tasks.workunit.client.0.smithi070.stdout:8/403: truncate d2/f1f 1696319 0 2024-04-02T04:47:47.534 INFO:tasks.workunit.client.0.smithi070.stdout:6/277: chown d0/d20/c56 1961285 1 2024-04-02T04:47:47.540 INFO:tasks.workunit.client.0.smithi070.stdout:2/434: dread d2/f8 [0,4194304] 0 2024-04-02T04:47:47.540 INFO:tasks.workunit.client.1.smithi179.stdout:9/365: dread d0/d1/d11/f32 [0,4194304] 0 2024-04-02T04:47:47.540 INFO:tasks.workunit.client.1.smithi179.stdout:1/373: dwrite d3/d5/f25 [0,4194304] 0 2024-04-02T04:47:47.541 INFO:tasks.workunit.client.0.smithi070.stdout:9/329: dread d4/d10/d12/d56/f45 [0,4194304] 0 2024-04-02T04:47:47.541 INFO:tasks.workunit.client.1.smithi179.stdout:6/387: mknod d3/d42/c88 0 2024-04-02T04:47:47.547 INFO:tasks.workunit.client.1.smithi179.stdout:9/366: dwrite d0/d1/d11/d29/d40/f67 [0,4194304] 0 2024-04-02T04:47:47.556 INFO:tasks.workunit.client.0.smithi070.stdout:2/435: stat d2/d10/d5c/c84 0 2024-04-02T04:47:47.558 INFO:tasks.workunit.client.1.smithi179.stdout:2/239: dread dc/f12 [4194304,4194304] 0 2024-04-02T04:47:47.558 INFO:tasks.workunit.client.1.smithi179.stdout:2/240: chown dc/d29/d4a/d1e/d43/f47 44544218 1 2024-04-02T04:47:47.559 INFO:tasks.workunit.client.1.smithi179.stdout:5/306: creat d8/d1b/d23/d32/d4e/f59 x:0 0 0 2024-04-02T04:47:47.561 INFO:tasks.workunit.client.1.smithi179.stdout:0/441: fsync d1/f2 0 2024-04-02T04:47:47.562 INFO:tasks.workunit.client.1.smithi179.stdout:2/241: dread dc/d29/d4a/f24 [4194304,4194304] 0 2024-04-02T04:47:47.567 INFO:tasks.workunit.client.1.smithi179.stdout:8/349: getdents d3/d32/d21/d22 0 2024-04-02T04:47:47.568 INFO:tasks.workunit.client.1.smithi179.stdout:2/242: dwrite dc/d29/d4a/f15 [0,4194304] 0 2024-04-02T04:47:47.570 INFO:tasks.workunit.client.1.smithi179.stdout:8/350: dread - d3/d32/d21/d50/d56/f6a zero size 2024-04-02T04:47:47.573 INFO:tasks.workunit.client.1.smithi179.stdout:3/379: dwrite d8/f22 [0,4194304] 0 2024-04-02T04:47:47.573 INFO:tasks.workunit.client.0.smithi070.stdout:5/419: creat d5/d17/d2b/f89 x:0 0 0 2024-04-02T04:47:47.573 INFO:tasks.workunit.client.0.smithi070.stdout:5/420: fsync d5/d17/d50/f5f 0 2024-04-02T04:47:47.574 INFO:tasks.workunit.client.1.smithi179.stdout:2/243: dread dc/d29/d4a/f45 [0,4194304] 0 2024-04-02T04:47:47.576 INFO:tasks.workunit.client.1.smithi179.stdout:7/421: creat d1/dd/f99 x:0 0 0 2024-04-02T04:47:47.585 INFO:tasks.workunit.client.1.smithi179.stdout:1/374: creat d3/d5/d2f/d4f/d35/d49/f7e x:0 0 0 2024-04-02T04:47:47.589 INFO:tasks.workunit.client.0.smithi070.stdout:5/421: dread d5/f6 [0,4194304] 0 2024-04-02T04:47:47.590 INFO:tasks.workunit.client.0.smithi070.stdout:5/422: write d5/f2f [2582389,29037] 0 2024-04-02T04:47:47.591 INFO:tasks.workunit.client.1.smithi179.stdout:9/367: chown d0/d1/d11/c6a 339837 1 2024-04-02T04:47:47.591 INFO:tasks.workunit.client.1.smithi179.stdout:9/368: readlink d0/d19/d27/d48/l6e 0 2024-04-02T04:47:47.596 INFO:tasks.workunit.client.0.smithi070.stdout:2/436: creat d2/d12/d59/d69/f8a x:0 0 0 2024-04-02T04:47:47.601 INFO:tasks.workunit.client.0.smithi070.stdout:7/380: unlink d7/d9/f5b 0 2024-04-02T04:47:47.605 INFO:tasks.workunit.client.1.smithi179.stdout:3/380: mknod d8/d26/d36/c79 0 2024-04-02T04:47:47.606 INFO:tasks.workunit.client.1.smithi179.stdout:3/381: chown d8/l14 1827 1 2024-04-02T04:47:47.606 INFO:tasks.workunit.client.1.smithi179.stdout:3/382: dread - d8/d6d/f76 zero size 2024-04-02T04:47:47.606 INFO:tasks.workunit.client.1.smithi179.stdout:3/383: fdatasync d8/d26/f2c 0 2024-04-02T04:47:47.607 INFO:tasks.workunit.client.1.smithi179.stdout:3/384: write d8/d26/f46 [358079,60679] 0 2024-04-02T04:47:47.608 INFO:tasks.workunit.client.1.smithi179.stdout:3/385: dread - d8/d26/d36/f49 zero size 2024-04-02T04:47:47.610 INFO:tasks.workunit.client.1.smithi179.stdout:3/386: write d8/d26/d36/f4d [3365306,102503] 0 2024-04-02T04:47:47.611 INFO:tasks.workunit.client.1.smithi179.stdout:3/387: truncate d8/d26/f72 1154703 0 2024-04-02T04:47:47.612 INFO:tasks.workunit.client.1.smithi179.stdout:3/388: truncate d8/d26/d36/f59 1389655 0 2024-04-02T04:47:47.613 INFO:tasks.workunit.client.1.smithi179.stdout:3/389: write d8/d26/f3f [74433,82107] 0 2024-04-02T04:47:47.615 INFO:tasks.workunit.client.1.smithi179.stdout:1/375: symlink d3/l7f 0 2024-04-02T04:47:47.616 INFO:tasks.workunit.client.0.smithi070.stdout:2/437: dread d2/f1b [0,4194304] 0 2024-04-02T04:47:47.617 INFO:tasks.workunit.client.1.smithi179.stdout:6/388: symlink d3/d12/d23/l89 0 2024-04-02T04:47:47.620 INFO:tasks.workunit.client.1.smithi179.stdout:9/369: mkdir d0/d19/d1c/d73 0 2024-04-02T04:47:47.620 INFO:tasks.workunit.client.1.smithi179.stdout:6/389: read d3/d12/d23/d2f/f4a [2286134,54120] 0 2024-04-02T04:47:47.620 INFO:tasks.workunit.client.1.smithi179.stdout:4/442: dwrite d1/d3/df/f51 [0,4194304] 0 2024-04-02T04:47:47.620 INFO:tasks.workunit.client.0.smithi070.stdout:8/404: link d2/d3/d35/d3a/d4f/d59/f6e d2/f7f 0 2024-04-02T04:47:47.621 INFO:tasks.workunit.client.1.smithi179.stdout:9/370: chown d0/d1/d10/f2b 327200 1 2024-04-02T04:47:47.621 INFO:tasks.workunit.client.1.smithi179.stdout:4/443: dread - d1/d3/dc/d26/d8a/d5d/f89 zero size 2024-04-02T04:47:47.624 INFO:tasks.workunit.client.0.smithi070.stdout:2/438: unlink d2/d12/d3e/c80 0 2024-04-02T04:47:47.628 INFO:tasks.workunit.client.1.smithi179.stdout:8/351: mknod d3/d32/d21/d50/d65/c7a 0 2024-04-02T04:47:47.629 INFO:tasks.workunit.client.0.smithi070.stdout:8/405: fdatasync d2/d8/d23/f51 0 2024-04-02T04:47:47.633 INFO:tasks.workunit.client.0.smithi070.stdout:7/381: creat d7/d9/f78 x:0 0 0 2024-04-02T04:47:47.638 INFO:tasks.workunit.client.0.smithi070.stdout:8/406: rmdir d2/d8/d23/d24/d79 39 2024-04-02T04:47:47.641 INFO:tasks.workunit.client.0.smithi070.stdout:7/382: dread d7/d9/f27 [0,4194304] 0 2024-04-02T04:47:47.642 INFO:tasks.workunit.client.0.smithi070.stdout:6/278: truncate d0/f15 500405 0 2024-04-02T04:47:47.644 INFO:tasks.workunit.client.0.smithi070.stdout:9/330: chown d4/d10/d11/f58 6204956 1 2024-04-02T04:47:47.644 INFO:tasks.workunit.client.1.smithi179.stdout:3/390: mkdir d8/d7a 0 2024-04-02T04:47:47.645 INFO:tasks.workunit.client.0.smithi070.stdout:2/439: dread d2/d10/d21/d3d/f5d [0,4194304] 0 2024-04-02T04:47:47.646 INFO:tasks.workunit.client.1.smithi179.stdout:1/376: symlink d3/d5/d2f/d65/l80 0 2024-04-02T04:47:47.647 INFO:tasks.workunit.client.1.smithi179.stdout:1/377: chown d3/d5/d2f/d4f/d35/d49/l75 16 1 2024-04-02T04:47:47.652 INFO:tasks.workunit.client.1.smithi179.stdout:6/390: chown d3/l5a 19858 1 2024-04-02T04:47:47.653 INFO:tasks.workunit.client.0.smithi070.stdout:6/279: mknod d0/d3/d19/d31/d37/c59 0 2024-04-02T04:47:47.655 INFO:tasks.workunit.client.0.smithi070.stdout:7/383: truncate d7/d2f/d46/f51 1617057 0 2024-04-02T04:47:47.657 INFO:tasks.workunit.client.1.smithi179.stdout:4/444: rename d1/d3/df/d8e to d1/d93 0 2024-04-02T04:47:47.659 INFO:tasks.workunit.client.1.smithi179.stdout:4/445: dread d1/d15/f31 [0,4194304] 0 2024-04-02T04:47:47.660 INFO:tasks.workunit.client.0.smithi070.stdout:2/440: mknod d2/d10/d21/d2b/c8b 0 2024-04-02T04:47:47.661 INFO:tasks.workunit.client.0.smithi070.stdout:5/423: write d5/d66/f38 [1788726,108475] 0 2024-04-02T04:47:47.662 INFO:tasks.workunit.client.0.smithi070.stdout:5/424: write d5/d57/f5d [963116,32735] 0 2024-04-02T04:47:47.662 INFO:tasks.workunit.client.1.smithi179.stdout:8/352: mknod d3/d32/d21/d45/c7b 0 2024-04-02T04:47:47.664 INFO:tasks.workunit.client.0.smithi070.stdout:6/280: chown d0/c33 3896352 1 2024-04-02T04:47:47.665 INFO:tasks.workunit.client.0.smithi070.stdout:6/281: fdatasync d0/d3/d19/f47 0 2024-04-02T04:47:47.669 INFO:tasks.workunit.client.0.smithi070.stdout:8/407: write d2/d8/d23/f51 [2558845,73662] 0 2024-04-02T04:47:47.670 INFO:tasks.workunit.client.1.smithi179.stdout:0/442: dwrite d1/da/df/f20 [0,4194304] 0 2024-04-02T04:47:47.670 INFO:tasks.workunit.client.0.smithi070.stdout:2/441: stat d2/d24/f7e 0 2024-04-02T04:47:47.670 INFO:tasks.workunit.client.1.smithi179.stdout:0/443: dread - d1/da/f4a zero size 2024-04-02T04:47:47.671 INFO:tasks.workunit.client.0.smithi070.stdout:2/442: write d2/d10/d5c/f83 [1045923,1204] 0 2024-04-02T04:47:47.673 INFO:tasks.workunit.client.0.smithi070.stdout:5/425: write d5/f1d [4883950,50176] 0 2024-04-02T04:47:47.674 INFO:tasks.workunit.client.0.smithi070.stdout:5/426: chown d5/d17/d27/c39 341782748 1 2024-04-02T04:47:47.677 INFO:tasks.workunit.client.1.smithi179.stdout:6/391: unlink d3/d12/d23/d26/d57/c7a 0 2024-04-02T04:47:47.677 INFO:tasks.workunit.client.1.smithi179.stdout:7/422: write d1/dd/f17 [2210286,33899] 0 2024-04-02T04:47:47.678 INFO:tasks.workunit.client.1.smithi179.stdout:7/423: readlink d1/dd/d23/l65 0 2024-04-02T04:47:47.678 INFO:tasks.workunit.client.1.smithi179.stdout:6/392: chown d3/d12/d23/d26/d57/c60 4 1 2024-04-02T04:47:47.679 INFO:tasks.workunit.client.0.smithi070.stdout:6/282: creat d0/d3/d19/d31/d37/f5a x:0 0 0 2024-04-02T04:47:47.682 INFO:tasks.workunit.client.1.smithi179.stdout:9/371: link d0/d19/d1c/d64/f69 d0/d19/d1c/d73/f74 0 2024-04-02T04:47:47.685 INFO:tasks.workunit.client.0.smithi070.stdout:6/283: dwrite d0/d43/f51 [0,4194304] 0 2024-04-02T04:47:47.687 INFO:tasks.workunit.client.0.smithi070.stdout:9/331: rename d4/c1a to d4/c6b 0 2024-04-02T04:47:47.691 INFO:tasks.workunit.client.0.smithi070.stdout:7/384: rename d7/d2f to d7/dd/df/d79 0 2024-04-02T04:47:47.692 INFO:tasks.workunit.client.0.smithi070.stdout:7/385: write d7/d4e/d55/d6c/f74 [121696,86556] 0 2024-04-02T04:47:47.692 INFO:tasks.workunit.client.1.smithi179.stdout:5/307: getdents d8/d1b/d23/d32/d4e 0 2024-04-02T04:47:47.696 INFO:tasks.workunit.client.1.smithi179.stdout:4/446: creat d1/d3/dc/d26/d8a/f94 x:0 0 0 2024-04-02T04:47:47.697 INFO:tasks.workunit.client.1.smithi179.stdout:4/447: readlink d1/d3/df/d10/d2f/d60/l77 0 2024-04-02T04:47:47.698 INFO:tasks.workunit.client.1.smithi179.stdout:3/391: write d8/f1c [1196825,4195] 0 2024-04-02T04:47:47.698 INFO:tasks.workunit.client.0.smithi070.stdout:6/284: creat d0/d3/d19/d31/d37/f5b x:0 0 0 2024-04-02T04:47:47.699 INFO:tasks.workunit.client.1.smithi179.stdout:8/353: symlink d3/d32/d21/d45/d5a/l7c 0 2024-04-02T04:47:47.701 INFO:tasks.workunit.client.1.smithi179.stdout:8/354: write d3/d32/d21/f66 [733065,105012] 0 2024-04-02T04:47:47.701 INFO:tasks.workunit.client.1.smithi179.stdout:2/244: link cb dc/d29/d31/d39/c4e 0 2024-04-02T04:47:47.703 INFO:tasks.workunit.client.1.smithi179.stdout:2/245: write dc/fe [2661118,11914] 0 2024-04-02T04:47:47.704 INFO:tasks.workunit.client.1.smithi179.stdout:4/448: dwrite d1/d3/df/f48 [0,4194304] 0 2024-04-02T04:47:47.711 INFO:tasks.workunit.client.1.smithi179.stdout:6/393: creat d3/d12/d23/d26/d57/f8a x:0 0 0 2024-04-02T04:47:47.712 INFO:tasks.workunit.client.1.smithi179.stdout:6/394: dread - d3/d12/f62 zero size 2024-04-02T04:47:47.714 INFO:tasks.workunit.client.1.smithi179.stdout:7/424: rename d1/d9/f68 to d1/d9/d27/d87/d8b/d75/f9a 0 2024-04-02T04:47:47.714 INFO:tasks.workunit.client.1.smithi179.stdout:7/425: truncate d1/f83 136057 0 2024-04-02T04:47:47.715 INFO:tasks.workunit.client.1.smithi179.stdout:9/372: truncate d0/d1/d11/d29/f50 1015503 0 2024-04-02T04:47:47.715 INFO:tasks.workunit.client.0.smithi070.stdout:9/332: write d4/d10/d12/d27/f30 [149448,59509] 0 2024-04-02T04:47:47.717 INFO:tasks.workunit.client.1.smithi179.stdout:5/308: chown d8/l48 1326542 1 2024-04-02T04:47:47.717 INFO:tasks.workunit.client.0.smithi070.stdout:8/408: write d2/f7f [967929,2700] 0 2024-04-02T04:47:47.718 INFO:tasks.workunit.client.1.smithi179.stdout:5/309: truncate d8/d1b/f49 598950 0 2024-04-02T04:47:47.719 INFO:tasks.workunit.client.0.smithi070.stdout:2/443: write d2/f1c [2382852,115082] 0 2024-04-02T04:47:47.722 INFO:tasks.workunit.client.0.smithi070.stdout:5/427: truncate d5/d9/d23/f49 227886 0 2024-04-02T04:47:47.722 INFO:tasks.workunit.client.0.smithi070.stdout:9/333: dwrite d4/d10/d11/f4e [0,4194304] 0 2024-04-02T04:47:47.724 INFO:tasks.workunit.client.1.smithi179.stdout:0/444: write d1/da/f2d [825031,54798] 0 2024-04-02T04:47:47.726 INFO:tasks.workunit.client.0.smithi070.stdout:6/285: mkdir d0/d55/d5c 0 2024-04-02T04:47:47.728 INFO:tasks.workunit.client.0.smithi070.stdout:1/384: sync 2024-04-02T04:47:47.728 INFO:tasks.workunit.client.1.smithi179.stdout:4/449: rmdir d1/d15/d7c 39 2024-04-02T04:47:47.729 INFO:tasks.workunit.client.1.smithi179.stdout:9/373: dread d0/d1/d10/f2b [0,4194304] 0 2024-04-02T04:47:47.730 INFO:tasks.workunit.client.0.smithi070.stdout:1/385: truncate d0/d9/d1d/d22/d56/f69 1313365 0 2024-04-02T04:47:47.734 INFO:tasks.workunit.client.1.smithi179.stdout:1/378: creat d3/d5/d2f/f81 x:0 0 0 2024-04-02T04:47:47.736 INFO:tasks.workunit.client.1.smithi179.stdout:1/379: chown d3/d5/d2f/d4f/d42/d53/d5b/f61 332 1 2024-04-02T04:47:47.736 INFO:tasks.workunit.client.0.smithi070.stdout:7/386: write d7/dd/df/d79/f5e [359808,123757] 0 2024-04-02T04:47:47.737 INFO:tasks.workunit.client.0.smithi070.stdout:2/444: readlink d2/d12/l87 0 2024-04-02T04:47:47.739 INFO:tasks.workunit.client.0.smithi070.stdout:5/428: symlink d5/d17/d27/d2e/l8a 0 2024-04-02T04:47:47.739 INFO:tasks.workunit.client.1.smithi179.stdout:3/392: dwrite d8/d26/f33 [0,4194304] 0 2024-04-02T04:47:47.739 INFO:tasks.workunit.client.0.smithi070.stdout:5/429: fsync d5/f1d 0 2024-04-02T04:47:47.741 INFO:tasks.workunit.client.1.smithi179.stdout:6/395: rename d3/d12/d23/d39/d4e/d6c/c73 to d3/d12/d23/d39/d4e/d66/c8b 0 2024-04-02T04:47:47.745 INFO:tasks.workunit.client.0.smithi070.stdout:6/286: chown d0/d3/l18 123150 1 2024-04-02T04:47:47.747 INFO:tasks.workunit.client.0.smithi070.stdout:1/386: mkdir d0/d9/d1d/d22/d56/d79 0 2024-04-02T04:47:47.749 INFO:tasks.workunit.client.1.smithi179.stdout:4/450: mknod d1/d3/d19/c95 0 2024-04-02T04:47:47.751 INFO:tasks.workunit.client.1.smithi179.stdout:9/374: chown d0/d1/d10/d6d/f25 1923499 1 2024-04-02T04:47:47.752 INFO:tasks.workunit.client.0.smithi070.stdout:6/287: dwrite d0/d3/f3a [0,4194304] 0 2024-04-02T04:47:47.756 INFO:tasks.workunit.client.0.smithi070.stdout:7/387: rename d7/d4e/d55 to d7/dd/df/d7a 0 2024-04-02T04:47:47.757 INFO:tasks.workunit.client.0.smithi070.stdout:6/288: write d0/d3/d19/d31/d37/f5a [157100,40499] 0 2024-04-02T04:47:47.758 INFO:tasks.workunit.client.0.smithi070.stdout:7/388: read d7/dd/f2e [7415,63343] 0 2024-04-02T04:47:47.759 INFO:tasks.workunit.client.0.smithi070.stdout:6/289: write d0/d3/d19/d31/d37/f5a [1009771,62801] 0 2024-04-02T04:47:47.762 INFO:tasks.workunit.client.0.smithi070.stdout:5/430: rmdir d5/d9/d10/d51 39 2024-04-02T04:47:47.764 INFO:tasks.workunit.client.1.smithi179.stdout:1/380: mknod d3/d5/d2f/d5a/c82 0 2024-04-02T04:47:47.764 INFO:tasks.workunit.client.1.smithi179.stdout:5/310: dwrite f1 [0,4194304] 0 2024-04-02T04:47:47.764 INFO:tasks.workunit.client.0.smithi070.stdout:9/334: mknod d4/d13/d16/d19/c6c 0 2024-04-02T04:47:47.765 INFO:tasks.workunit.client.1.smithi179.stdout:1/381: chown d3/d5/d2f/d4f/l70 425922454 1 2024-04-02T04:47:47.767 INFO:tasks.workunit.client.1.smithi179.stdout:2/246: truncate dc/d29/d4a/d1e/f40 1125520 0 2024-04-02T04:47:47.768 INFO:tasks.workunit.client.0.smithi070.stdout:1/387: creat d0/d7/d28/f7a x:0 0 0 2024-04-02T04:47:47.769 INFO:tasks.workunit.client.1.smithi179.stdout:1/382: dread d3/f28 [4194304,4194304] 0 2024-04-02T04:47:47.771 INFO:tasks.workunit.client.1.smithi179.stdout:7/426: rename d1/dd/d23/c53 to d1/d9/d27/c9b 0 2024-04-02T04:47:47.772 INFO:tasks.workunit.client.1.smithi179.stdout:7/427: fdatasync d1/dd/f99 0 2024-04-02T04:47:47.778 INFO:tasks.workunit.client.1.smithi179.stdout:7/428: dwrite d1/dd/d16/f62 [0,4194304] 0 2024-04-02T04:47:47.780 INFO:tasks.workunit.client.1.smithi179.stdout:4/451: symlink d1/d15/d6e/l96 0 2024-04-02T04:47:47.786 INFO:tasks.workunit.client.1.smithi179.stdout:5/311: mkdir d8/d1b/d23/d42/d5a 0 2024-04-02T04:47:47.789 INFO:tasks.workunit.client.0.smithi070.stdout:7/389: unlink d7/dd/d66/d57/c76 0 2024-04-02T04:47:47.790 INFO:tasks.workunit.client.0.smithi070.stdout:5/431: fdatasync d5/fa 0 2024-04-02T04:47:47.792 INFO:tasks.workunit.client.1.smithi179.stdout:9/375: write d0/d1/f36 [424341,45722] 0 2024-04-02T04:47:47.793 INFO:tasks.workunit.client.1.smithi179.stdout:9/376: dread - d0/d1/d10/f6f zero size 2024-04-02T04:47:47.793 INFO:tasks.workunit.client.0.smithi070.stdout:9/335: rename d4/d10/d12/d56/f45 to d4/d10/d12/d55/d66/f6d 0 2024-04-02T04:47:47.794 INFO:tasks.workunit.client.1.smithi179.stdout:9/377: write d0/d1/d11/f51 [301144,21011] 0 2024-04-02T04:47:47.794 INFO:tasks.workunit.client.0.smithi070.stdout:9/336: fsync d4/d10/d12/d27/f5c 0 2024-04-02T04:47:47.794 INFO:tasks.workunit.client.1.smithi179.stdout:6/396: mknod d3/c8c 0 2024-04-02T04:47:47.795 INFO:tasks.workunit.client.0.smithi070.stdout:1/388: symlink d0/d7/d28/l7b 0 2024-04-02T04:47:47.796 INFO:tasks.workunit.client.0.smithi070.stdout:8/409: link d2/c12 d2/d3/d52/d62/c80 0 2024-04-02T04:47:47.797 INFO:tasks.workunit.client.1.smithi179.stdout:8/355: getdents d3/d32/d21/d49 0 2024-04-02T04:47:47.798 INFO:tasks.workunit.client.1.smithi179.stdout:3/393: truncate d8/d26/f46 1731921 0 2024-04-02T04:47:47.801 INFO:tasks.workunit.client.1.smithi179.stdout:3/394: read f5 [15756,107127] 0 2024-04-02T04:47:47.802 INFO:tasks.workunit.client.1.smithi179.stdout:3/395: fdatasync d8/d26/f69 0 2024-04-02T04:47:47.802 INFO:tasks.workunit.client.1.smithi179.stdout:2/247: creat dc/d29/d31/d39/f4f x:0 0 0 2024-04-02T04:47:47.802 INFO:tasks.workunit.client.1.smithi179.stdout:2/248: chown dc/c4c 431822 1 2024-04-02T04:47:47.803 INFO:tasks.workunit.client.0.smithi070.stdout:8/410: dwrite d2/d3/d35/d3a/f47 [0,4194304] 0 2024-04-02T04:47:47.803 INFO:tasks.workunit.client.0.smithi070.stdout:9/337: read d4/d10/d11/f2d [352872,61749] 0 2024-04-02T04:47:47.804 INFO:tasks.workunit.client.1.smithi179.stdout:2/249: write dc/d29/d31/f3a [246476,79819] 0 2024-04-02T04:47:47.804 INFO:tasks.workunit.client.1.smithi179.stdout:2/250: readlink dc/l13 0 2024-04-02T04:47:47.805 INFO:tasks.workunit.client.1.smithi179.stdout:2/251: stat dc/d29/d4a/f45 0 2024-04-02T04:47:47.805 INFO:tasks.workunit.client.0.smithi070.stdout:9/338: chown d4/d13 508187867 1 2024-04-02T04:47:47.805 INFO:tasks.workunit.client.0.smithi070.stdout:8/411: write d2/d8/f5e [634802,112717] 0 2024-04-02T04:47:47.807 INFO:tasks.workunit.client.0.smithi070.stdout:8/412: write d2/d8/d23/d24/d5f/f4c [1037920,106284] 0 2024-04-02T04:47:47.807 INFO:tasks.workunit.client.1.smithi179.stdout:1/383: link d3/d5/d2f/d65/f6b d3/d5/d2f/d4f/d42/d53/d5b/f83 0 2024-04-02T04:47:47.811 INFO:tasks.workunit.client.0.smithi070.stdout:5/432: truncate d5/f1d 264294 0 2024-04-02T04:47:47.814 INFO:tasks.workunit.client.0.smithi070.stdout:5/433: chown d5/d17/d2b/f6f 137208368 1 2024-04-02T04:47:47.815 INFO:tasks.workunit.client.1.smithi179.stdout:5/312: getdents d8/d1b/d23/d30 0 2024-04-02T04:47:47.816 INFO:tasks.workunit.client.0.smithi070.stdout:9/339: dwrite d4/d13/d16/f1d [0,4194304] 0 2024-04-02T04:47:47.818 INFO:tasks.workunit.client.0.smithi070.stdout:8/413: dread d2/d3/d35/d3a/f47 [0,4194304] 0 2024-04-02T04:47:47.818 INFO:tasks.workunit.client.1.smithi179.stdout:9/378: creat d0/d1/d10/d6d/d58/f75 x:0 0 0 2024-04-02T04:47:47.820 INFO:tasks.workunit.client.0.smithi070.stdout:8/414: write d2/d8/d29/d22/f7a [1891365,97552] 0 2024-04-02T04:47:47.820 INFO:tasks.workunit.client.1.smithi179.stdout:3/396: mknod d8/d57/d63/c7b 0 2024-04-02T04:47:47.820 INFO:tasks.workunit.client.1.smithi179.stdout:9/379: dread d0/d1/f36 [0,4194304] 0 2024-04-02T04:47:47.821 INFO:tasks.workunit.client.0.smithi070.stdout:8/415: chown d2/d8/d23/d24/f67 1946 1 2024-04-02T04:47:47.824 INFO:tasks.workunit.client.0.smithi070.stdout:1/389: getdents d0/d9/d1d/d22/d2c/d4d 0 2024-04-02T04:47:47.824 INFO:tasks.workunit.client.0.smithi070.stdout:1/390: chown d0/d7/f20 0 1 2024-04-02T04:47:47.826 INFO:tasks.workunit.client.0.smithi070.stdout:1/391: chown d0/d9/d1d/d22/d2c/d4f/f61 5375 1 2024-04-02T04:47:47.826 INFO:tasks.workunit.client.0.smithi070.stdout:1/392: chown d0/d9/d1d/d22/d2c/f76 314486954 1 2024-04-02T04:47:47.828 INFO:tasks.workunit.client.0.smithi070.stdout:1/393: read - d0/d9/d1d/d22/d2c/d4f/f50 zero size 2024-04-02T04:47:47.829 INFO:tasks.workunit.client.1.smithi179.stdout:2/252: read dc/d29/d4a/d1e/f40 [917983,7945] 0 2024-04-02T04:47:47.830 INFO:tasks.workunit.client.0.smithi070.stdout:8/416: write d2/d3/d35/f78 [2232106,83383] 0 2024-04-02T04:47:47.830 INFO:tasks.workunit.client.1.smithi179.stdout:2/253: read - dc/d29/f34 zero size 2024-04-02T04:47:47.832 INFO:tasks.workunit.client.1.smithi179.stdout:2/254: chown dc/d29/d4a/f15 32 1 2024-04-02T04:47:47.833 INFO:tasks.workunit.client.0.smithi070.stdout:1/394: mknod d0/d9/d1d/d22/d56/c7c 0 2024-04-02T04:47:47.835 INFO:tasks.workunit.client.1.smithi179.stdout:6/397: dwrite d3/d2d/f65 [8388608,4194304] 0 2024-04-02T04:47:47.836 INFO:tasks.workunit.client.0.smithi070.stdout:8/417: mknod d2/d8/d29/d22/d5a/c81 0 2024-04-02T04:47:47.837 INFO:tasks.workunit.client.0.smithi070.stdout:1/395: mknod d0/d7/d28/d5c/d6e/c7d 0 2024-04-02T04:47:47.842 INFO:tasks.workunit.client.1.smithi179.stdout:7/429: truncate d1/dd/d23/f25 675796 0 2024-04-02T04:47:47.845 INFO:tasks.workunit.client.1.smithi179.stdout:3/397: read f5 [2393099,34049] 0 2024-04-02T04:47:47.846 INFO:tasks.workunit.client.0.smithi070.stdout:9/340: write d4/d13/d16/d19/d2f/f3c [902813,72144] 0 2024-04-02T04:47:47.847 INFO:tasks.workunit.client.0.smithi070.stdout:1/396: mknod d0/d7/d1a/d66/c7e 0 2024-04-02T04:47:47.848 INFO:tasks.workunit.client.0.smithi070.stdout:9/341: chown d4/d13/d16/l1f 4 1 2024-04-02T04:47:47.848 INFO:tasks.workunit.client.1.smithi179.stdout:8/356: link d3/d32/d21/f37 d3/d32/d30/f7d 0 2024-04-02T04:47:47.849 INFO:tasks.workunit.client.1.smithi179.stdout:8/357: chown d3/d32/d21/d50 85 1 2024-04-02T04:47:47.851 INFO:tasks.workunit.client.1.smithi179.stdout:6/398: truncate d3/fb 8611007 0 2024-04-02T04:47:47.851 INFO:tasks.workunit.client.0.smithi070.stdout:9/342: symlink d4/d10/d12/d56/l6e 0 2024-04-02T04:47:47.852 INFO:tasks.workunit.client.1.smithi179.stdout:5/313: symlink d8/d1b/d23/d32/d46/l5b 0 2024-04-02T04:47:47.853 INFO:tasks.workunit.client.1.smithi179.stdout:5/314: dread f6 [0,4194304] 0 2024-04-02T04:47:47.854 INFO:tasks.workunit.client.0.smithi070.stdout:1/397: dwrite d0/d9/d1d/f16 [0,4194304] 0 2024-04-02T04:47:47.854 INFO:tasks.workunit.client.1.smithi179.stdout:5/315: fsync d8/d1b/d23/f47 0 2024-04-02T04:47:47.855 INFO:tasks.workunit.client.1.smithi179.stdout:9/380: symlink d0/l76 0 2024-04-02T04:47:47.863 INFO:tasks.workunit.client.1.smithi179.stdout:1/384: truncate d3/d5/d2f/d36/f67 38322 0 2024-04-02T04:47:47.867 INFO:tasks.workunit.client.1.smithi179.stdout:8/358: creat d3/d31/f7e x:0 0 0 2024-04-02T04:47:47.867 INFO:tasks.workunit.client.0.smithi070.stdout:9/343: link d4/d10/d12/d27/l52 d4/l6f 0 2024-04-02T04:47:47.868 INFO:tasks.workunit.client.0.smithi070.stdout:9/344: read d4/f8 [4370980,16617] 0 2024-04-02T04:47:47.870 INFO:tasks.workunit.client.1.smithi179.stdout:9/381: rename d0/d19/d1c/d73 to d0/d1/d11/d77 0 2024-04-02T04:47:47.871 INFO:tasks.workunit.client.1.smithi179.stdout:1/385: dwrite d3/d5/de/f24 [0,4194304] 0 2024-04-02T04:47:47.872 INFO:tasks.workunit.client.1.smithi179.stdout:9/382: chown d0/d19/d1c/c4f 57 1 2024-04-02T04:47:47.873 INFO:tasks.workunit.client.1.smithi179.stdout:2/255: creat dc/f50 x:0 0 0 2024-04-02T04:47:47.874 INFO:tasks.workunit.client.0.smithi070.stdout:9/345: symlink d4/l70 0 2024-04-02T04:47:47.877 INFO:tasks.workunit.client.1.smithi179.stdout:6/399: unlink d3/c14 0 2024-04-02T04:47:47.879 INFO:tasks.workunit.client.0.smithi070.stdout:9/346: write d4/d13/d16/d19/d53/f59 [636426,35718] 0 2024-04-02T04:47:47.880 INFO:tasks.workunit.client.0.smithi070.stdout:9/347: chown d4/d13/c49 72688 1 2024-04-02T04:47:47.881 INFO:tasks.workunit.client.1.smithi179.stdout:2/256: dwrite dc/f50 [0,4194304] 0 2024-04-02T04:47:47.882 INFO:tasks.workunit.client.0.smithi070.stdout:1/398: write d0/d7/d28/d5c/d6e/f6f [356033,68069] 0 2024-04-02T04:47:47.889 INFO:tasks.workunit.client.1.smithi179.stdout:9/383: creat d0/d1/d11/f78 x:0 0 0 2024-04-02T04:47:47.890 INFO:tasks.workunit.client.0.smithi070.stdout:1/399: creat d0/d54/f7f x:0 0 0 2024-04-02T04:47:47.894 INFO:tasks.workunit.client.1.smithi179.stdout:8/359: fsync d3/d32/d21/d22/d33/d3c/f44 0 2024-04-02T04:47:47.894 INFO:tasks.workunit.client.1.smithi179.stdout:5/316: write d8/d1b/f1f [205450,118559] 0 2024-04-02T04:47:47.896 INFO:tasks.workunit.client.1.smithi179.stdout:7/430: getdents d1/d9/d35 0 2024-04-02T04:47:47.898 INFO:tasks.workunit.client.1.smithi179.stdout:1/386: symlink d3/d5/l84 0 2024-04-02T04:47:47.901 INFO:tasks.workunit.client.1.smithi179.stdout:9/384: rename d0/f57 to d0/d19/d1c/d64/f79 0 2024-04-02T04:47:47.901 INFO:tasks.workunit.client.1.smithi179.stdout:3/398: link d8/c25 d8/d6d/c7c 0 2024-04-02T04:47:47.902 INFO:tasks.workunit.client.1.smithi179.stdout:3/399: fdatasync d8/d26/f62 0 2024-04-02T04:47:47.903 INFO:tasks.workunit.client.0.smithi070.stdout:1/400: dread d0/d9/d1d/f42 [0,4194304] 0 2024-04-02T04:47:47.905 INFO:tasks.workunit.client.1.smithi179.stdout:5/317: creat d8/d1b/d23/d30/f5c x:0 0 0 2024-04-02T04:47:47.906 INFO:tasks.workunit.client.1.smithi179.stdout:2/257: creat dc/d29/d4a/d4b/f51 x:0 0 0 2024-04-02T04:47:47.906 INFO:tasks.workunit.client.0.smithi070.stdout:9/348: dwrite d4/f9 [0,4194304] 0 2024-04-02T04:47:47.907 INFO:tasks.workunit.client.1.smithi179.stdout:2/258: write dc/d29/d4a/d4b/f51 [625342,55432] 0 2024-04-02T04:47:47.909 INFO:tasks.workunit.client.0.smithi070.stdout:1/401: creat d0/d7/d28/d5c/f80 x:0 0 0 2024-04-02T04:47:47.911 INFO:tasks.workunit.client.1.smithi179.stdout:1/387: mkdir d3/d85 0 2024-04-02T04:47:47.913 INFO:tasks.workunit.client.1.smithi179.stdout:9/385: rmdir d0/d1/d11/d29/d40 39 2024-04-02T04:47:47.913 INFO:tasks.workunit.client.0.smithi070.stdout:1/402: rmdir d0/d9/d1d/d22/d2c/d4d 39 2024-04-02T04:47:47.914 INFO:tasks.workunit.client.1.smithi179.stdout:9/386: chown d0/l24 320406 1 2024-04-02T04:47:47.915 INFO:tasks.workunit.client.1.smithi179.stdout:3/400: mknod d8/d6d/c7d 0 2024-04-02T04:47:47.915 INFO:tasks.workunit.client.1.smithi179.stdout:5/318: fdatasync d8/f13 0 2024-04-02T04:47:47.918 INFO:tasks.workunit.client.1.smithi179.stdout:6/400: truncate d3/d42/f54 80631 0 2024-04-02T04:47:47.921 INFO:tasks.workunit.client.0.smithi070.stdout:1/403: dwrite d0/d7/d28/d5c/d6e/f6f [0,4194304] 0 2024-04-02T04:47:47.923 INFO:tasks.workunit.client.1.smithi179.stdout:1/388: unlink d3/d5/d2f/d4f/d42/f7d 0 2024-04-02T04:47:47.927 INFO:tasks.workunit.client.1.smithi179.stdout:3/401: mkdir d8/d26/d52/d7e 0 2024-04-02T04:47:47.928 INFO:tasks.workunit.client.1.smithi179.stdout:3/402: rename d8/d6d to d8/d6d/d7f 22 2024-04-02T04:47:47.929 INFO:tasks.workunit.client.1.smithi179.stdout:7/431: link d1/d9/f11 d1/d9/d27/d87/d8b/f9c 0 2024-04-02T04:47:47.930 INFO:tasks.workunit.client.1.smithi179.stdout:3/403: read f5 [551686,7636] 0 2024-04-02T04:47:47.931 INFO:tasks.workunit.client.1.smithi179.stdout:5/319: getdents d8/d1b/d23/d36/d53 0 2024-04-02T04:47:47.933 INFO:tasks.workunit.client.1.smithi179.stdout:6/401: mknod d3/d2c/d85/c8d 0 2024-04-02T04:47:47.933 INFO:tasks.workunit.client.1.smithi179.stdout:2/259: creat dc/f52 x:0 0 0 2024-04-02T04:47:47.942 INFO:tasks.workunit.client.1.smithi179.stdout:1/389: dread d3/f2d [0,4194304] 0 2024-04-02T04:47:47.942 INFO:tasks.workunit.client.0.smithi070.stdout:0/425: sync 2024-04-02T04:47:47.943 INFO:tasks.workunit.client.0.smithi070.stdout:3/359: sync 2024-04-02T04:47:47.943 INFO:tasks.workunit.client.0.smithi070.stdout:4/328: sync 2024-04-02T04:47:47.943 INFO:tasks.workunit.client.1.smithi179.stdout:1/390: dread - d3/d5/d2f/f81 zero size 2024-04-02T04:47:47.943 INFO:tasks.workunit.client.1.smithi179.stdout:1/391: chown d3/c58 200587813 1 2024-04-02T04:47:47.944 INFO:tasks.workunit.client.1.smithi179.stdout:1/392: read d3/d5/de/f57 [4109234,2240] 0 2024-04-02T04:47:47.977 INFO:tasks.workunit.client.1.smithi179.stdout:6/402: mknod d3/d12/d44/c8e 0 2024-04-02T04:47:47.980 INFO:tasks.workunit.client.1.smithi179.stdout:7/432: rename d1/dd/f14 to d1/d9/d7e/f9d 0 2024-04-02T04:47:47.981 INFO:tasks.workunit.client.0.smithi070.stdout:4/329: creat d1/d3/d9/d41/f6e x:0 0 0 2024-04-02T04:47:47.982 INFO:tasks.workunit.client.0.smithi070.stdout:1/404: write d0/d9/d1d/d22/d2c/f76 [59175,35712] 0 2024-04-02T04:47:47.987 INFO:tasks.workunit.client.0.smithi070.stdout:4/330: creat d1/d3/d9/f6f x:0 0 0 2024-04-02T04:47:47.992 INFO:tasks.workunit.client.0.smithi070.stdout:3/360: creat d10/d17/d51/d73/f80 x:0 0 0 2024-04-02T04:47:47.992 INFO:tasks.workunit.client.1.smithi179.stdout:9/387: dwrite d0/d1/d11/d29/f50 [0,4194304] 0 2024-04-02T04:47:47.994 INFO:tasks.workunit.client.1.smithi179.stdout:6/403: rename d3/d12/f46 to d3/d12/d23/d39/d80/f8f 0 2024-04-02T04:47:47.994 INFO:tasks.workunit.client.1.smithi179.stdout:5/320: write f6 [1101343,71149] 0 2024-04-02T04:47:47.999 INFO:tasks.workunit.client.1.smithi179.stdout:7/433: dread d1/dd/d23/f25 [0,4194304] 0 2024-04-02T04:47:48.003 INFO:tasks.workunit.client.1.smithi179.stdout:7/434: truncate d1/d9/f6e 1738339 0 2024-04-02T04:47:48.003 INFO:tasks.workunit.client.1.smithi179.stdout:3/404: link d8/d26/d36/c66 d8/d7a/c80 0 2024-04-02T04:47:48.004 INFO:tasks.workunit.client.1.smithi179.stdout:7/435: stat d1/d9/c22 0 2024-04-02T04:47:48.004 INFO:tasks.workunit.client.1.smithi179.stdout:7/436: fsync d1/dd/f90 0 2024-04-02T04:47:48.005 INFO:tasks.workunit.client.1.smithi179.stdout:7/437: chown d1/dd/f90 105 1 2024-04-02T04:47:48.005 INFO:tasks.workunit.client.0.smithi070.stdout:0/426: getdents d4/d5 0 2024-04-02T04:47:48.005 INFO:tasks.workunit.client.1.smithi179.stdout:7/438: fsync d1/dd/f90 0 2024-04-02T04:47:48.008 INFO:tasks.workunit.client.1.smithi179.stdout:9/388: mkdir d0/d19/d27/d7a 0 2024-04-02T04:47:48.008 INFO:tasks.workunit.client.1.smithi179.stdout:1/393: write d3/f2d [3433970,93832] 0 2024-04-02T04:47:48.010 INFO:tasks.workunit.client.1.smithi179.stdout:6/404: symlink d3/d42/l90 0 2024-04-02T04:47:48.018 INFO:tasks.workunit.client.1.smithi179.stdout:3/405: mknod d8/d26/d52/c81 0 2024-04-02T04:47:48.019 INFO:tasks.workunit.client.1.smithi179.stdout:3/406: readlink d8/d26/d36/l42 0 2024-04-02T04:47:48.019 INFO:tasks.workunit.client.1.smithi179.stdout:7/439: creat d1/d9/d55/f9e x:0 0 0 2024-04-02T04:47:48.020 INFO:tasks.workunit.client.1.smithi179.stdout:9/389: stat d0/d1/d11/d29/d40/c4b 0 2024-04-02T04:47:48.024 INFO:tasks.workunit.client.1.smithi179.stdout:1/394: mkdir d3/d5/d2f/d4f/d42/d53/d72/d86 0 2024-04-02T04:47:48.024 INFO:tasks.workunit.client.1.smithi179.stdout:0/445: sync 2024-04-02T04:47:48.026 INFO:tasks.workunit.client.1.smithi179.stdout:6/405: mknod d3/c91 0 2024-04-02T04:47:48.028 INFO:tasks.workunit.client.1.smithi179.stdout:3/407: dread d8/f11 [0,4194304] 0 2024-04-02T04:47:48.029 INFO:tasks.workunit.client.1.smithi179.stdout:5/321: getdents d8/d1b/d23/d36 0 2024-04-02T04:47:48.029 INFO:tasks.workunit.client.1.smithi179.stdout:1/395: dread d3/d5/de/d33/f51 [0,4194304] 0 2024-04-02T04:47:48.029 INFO:tasks.workunit.client.1.smithi179.stdout:0/446: rmdir d1/da/d48/d65/d7d 39 2024-04-02T04:47:48.030 INFO:tasks.workunit.client.1.smithi179.stdout:1/396: chown d3/d5/d2f/f47 0 1 2024-04-02T04:47:48.031 INFO:tasks.workunit.client.1.smithi179.stdout:9/390: creat d0/d1/f7b x:0 0 0 2024-04-02T04:47:48.031 INFO:tasks.workunit.client.1.smithi179.stdout:1/397: readlink d3/d5/d2f/d4f/d42/d53/l5f 0 2024-04-02T04:47:48.033 INFO:tasks.workunit.client.1.smithi179.stdout:6/406: rename d3/d12/d23/d39/d4e/d6c/c40 to d3/d2d/d43/c92 0 2024-04-02T04:47:48.034 INFO:tasks.workunit.client.1.smithi179.stdout:7/440: getdents d1/d9/d27/d87/d8b 0 2024-04-02T04:47:48.035 INFO:tasks.workunit.client.1.smithi179.stdout:0/447: dwrite d1/da/d52/f73 [0,4194304] 0 2024-04-02T04:47:48.036 INFO:tasks.workunit.client.1.smithi179.stdout:7/441: chown d1/d9/d7e/f81 365 1 2024-04-02T04:47:48.037 INFO:tasks.workunit.client.1.smithi179.stdout:1/398: mknod d3/c87 0 2024-04-02T04:47:48.037 INFO:tasks.workunit.client.1.smithi179.stdout:7/442: write d1/dd/d23/f2f [4870172,98349] 0 2024-04-02T04:47:48.037 INFO:tasks.workunit.client.1.smithi179.stdout:1/399: stat d3/c4 0 2024-04-02T04:47:48.039 INFO:tasks.workunit.client.1.smithi179.stdout:7/443: dread - d1/d9/d55/f9e zero size 2024-04-02T04:47:48.041 INFO:tasks.workunit.client.1.smithi179.stdout:6/407: mkdir d3/d12/d23/d39/d80/d93 0 2024-04-02T04:47:48.042 INFO:tasks.workunit.client.1.smithi179.stdout:9/391: symlink d0/d19/l7c 0 2024-04-02T04:47:48.042 INFO:tasks.workunit.client.1.smithi179.stdout:7/444: dread d1/d9/d27/f69 [0,4194304] 0 2024-04-02T04:47:48.043 INFO:tasks.workunit.client.1.smithi179.stdout:0/448: rmdir d1/da/df/d56/d84 39 2024-04-02T04:47:48.092 INFO:tasks.workunit.client.0.smithi070.stdout:1/405: creat d0/d7/f81 x:0 0 0 2024-04-02T04:47:48.094 INFO:tasks.workunit.client.0.smithi070.stdout:4/331: dwrite d1/f3f [0,4194304] 0 2024-04-02T04:47:48.098 INFO:tasks.workunit.client.0.smithi070.stdout:1/406: dread - d0/d9/d1d/f5e zero size 2024-04-02T04:47:48.098 INFO:tasks.workunit.client.0.smithi070.stdout:3/361: dwrite d10/d17/f74 [0,4194304] 0 2024-04-02T04:47:48.098 INFO:tasks.workunit.client.0.smithi070.stdout:4/332: stat d1/d3/c5d 0 2024-04-02T04:47:48.099 INFO:tasks.workunit.client.0.smithi070.stdout:0/427: rename d4/d5/d13/d2a/d3a/f67 to d4/d3c/f77 0 2024-04-02T04:47:48.101 INFO:tasks.workunit.client.0.smithi070.stdout:1/407: mknod d0/d4c/c82 0 2024-04-02T04:47:48.104 INFO:tasks.workunit.client.0.smithi070.stdout:1/408: write d0/d7/d28/d5c/d6e/f6f [3875574,9325] 0 2024-04-02T04:47:48.105 INFO:tasks.workunit.client.0.smithi070.stdout:3/362: mknod d10/d17/d51/d73/c81 0 2024-04-02T04:47:48.109 INFO:tasks.workunit.client.0.smithi070.stdout:1/409: fsync d0/d54/f63 0 2024-04-02T04:47:48.111 INFO:tasks.workunit.client.1.smithi179.stdout:5/322: rename d8/f27 to d8/d1b/d23/d42/f5d 0 2024-04-02T04:47:48.111 INFO:tasks.workunit.client.1.smithi179.stdout:3/408: link d8/l14 d8/d26/d52/d7e/l82 0 2024-04-02T04:47:48.112 INFO:tasks.workunit.client.0.smithi070.stdout:4/333: dwrite d1/f46 [0,4194304] 0 2024-04-02T04:47:48.112 INFO:tasks.workunit.client.1.smithi179.stdout:7/445: truncate d1/d9/d55/f64 5203387 0 2024-04-02T04:47:48.113 INFO:tasks.workunit.client.0.smithi070.stdout:4/334: chown d1/d3/d9/f11 573734 1 2024-04-02T04:47:48.114 INFO:tasks.workunit.client.0.smithi070.stdout:0/428: link d4/d5/d13/d2a/d29/f59 d4/d5/d13/d43/f78 0 2024-04-02T04:47:48.116 INFO:tasks.workunit.client.1.smithi179.stdout:0/449: dwrite d1/f1e [0,4194304] 0 2024-04-02T04:47:48.118 INFO:tasks.workunit.client.0.smithi070.stdout:1/410: mkdir d0/d7/d28/d83 0 2024-04-02T04:47:48.119 INFO:tasks.workunit.client.1.smithi179.stdout:3/409: fsync f5 0 2024-04-02T04:47:48.123 INFO:tasks.workunit.client.1.smithi179.stdout:5/323: creat d8/d1b/d23/d32/d46/f5e x:0 0 0 2024-04-02T04:47:48.124 INFO:tasks.workunit.client.1.smithi179.stdout:7/446: dwrite d1/d9/d27/f49 [0,4194304] 0 2024-04-02T04:47:48.124 INFO:tasks.workunit.client.0.smithi070.stdout:0/429: unlink d4/f1b 0 2024-04-02T04:47:48.126 INFO:tasks.workunit.client.1.smithi179.stdout:0/450: mknod d1/db/c89 0 2024-04-02T04:47:48.127 INFO:tasks.workunit.client.0.smithi070.stdout:4/335: symlink d1/d3/l70 0 2024-04-02T04:47:48.128 INFO:tasks.workunit.client.1.smithi179.stdout:3/410: mkdir d8/d26/d83 0 2024-04-02T04:47:48.129 INFO:tasks.workunit.client.0.smithi070.stdout:1/411: rmdir d0/d9/d1d/d22/d56 39 2024-04-02T04:47:48.131 INFO:tasks.workunit.client.0.smithi070.stdout:1/412: truncate d0/d9/d1d/d22/d2c/f76 644119 0 2024-04-02T04:47:48.133 INFO:tasks.workunit.client.0.smithi070.stdout:1/413: write d0/d7/d28/f74 [1628302,68852] 0 2024-04-02T04:47:48.133 INFO:tasks.workunit.client.1.smithi179.stdout:5/324: unlink d8/d1b/d23/d32/f38 0 2024-04-02T04:47:48.137 INFO:tasks.workunit.client.1.smithi179.stdout:0/451: unlink d1/da/d28/d37/f5a 0 2024-04-02T04:47:48.138 INFO:tasks.workunit.client.1.smithi179.stdout:0/452: read d1/f2 [4345122,18279] 0 2024-04-02T04:47:48.139 INFO:tasks.workunit.client.0.smithi070.stdout:0/430: truncate d4/f9 4822609 0 2024-04-02T04:47:48.140 INFO:tasks.workunit.client.1.smithi179.stdout:3/411: creat d8/d57/d63/f84 x:0 0 0 2024-04-02T04:47:48.141 INFO:tasks.workunit.client.1.smithi179.stdout:3/412: chown f5 115190 1 2024-04-02T04:47:48.145 INFO:tasks.workunit.client.0.smithi070.stdout:1/414: creat d0/d7/d1a/d66/f84 x:0 0 0 2024-04-02T04:47:48.146 INFO:tasks.workunit.client.0.smithi070.stdout:1/415: stat d0/d9/d1d/d22/d2c/c41 0 2024-04-02T04:47:48.147 INFO:tasks.workunit.client.1.smithi179.stdout:1/400: dwrite d3/d5/de/f1a [0,4194304] 0 2024-04-02T04:47:48.148 INFO:tasks.workunit.client.0.smithi070.stdout:3/363: getdents d10 0 2024-04-02T04:47:48.150 INFO:tasks.workunit.client.1.smithi179.stdout:9/392: dwrite d0/d1/d10/d6d/f2d [0,4194304] 0 2024-04-02T04:47:48.150 INFO:tasks.workunit.client.0.smithi070.stdout:0/431: fdatasync d4/d5/d13/d43/f78 0 2024-04-02T04:47:48.152 INFO:tasks.workunit.client.1.smithi179.stdout:9/393: read d0/d1/d11/f51 [32286,84866] 0 2024-04-02T04:47:48.152 INFO:tasks.workunit.client.1.smithi179.stdout:3/413: mkdir d8/d26/d52/d85 0 2024-04-02T04:47:48.154 INFO:tasks.workunit.client.0.smithi070.stdout:4/336: creat d1/d3/d9/d5e/f71 x:0 0 0 2024-04-02T04:47:48.155 INFO:tasks.workunit.client.0.smithi070.stdout:4/337: dread - d1/d3/d9/f6f zero size 2024-04-02T04:47:48.157 INFO:tasks.workunit.client.1.smithi179.stdout:0/453: dwrite d1/f2a [0,4194304] 0 2024-04-02T04:47:48.157 INFO:tasks.workunit.client.0.smithi070.stdout:1/416: mknod d0/d9/d1d/d22/d2c/d4f/c85 0 2024-04-02T04:47:48.161 INFO:tasks.workunit.client.0.smithi070.stdout:3/364: symlink d10/d32/l82 0 2024-04-02T04:47:48.162 INFO:tasks.workunit.client.1.smithi179.stdout:1/401: symlink d3/d5/d2f/d4f/d42/l88 0 2024-04-02T04:47:48.168 INFO:tasks.workunit.client.0.smithi070.stdout:1/417: rmdir d0/d7/d28 39 2024-04-02T04:47:48.168 INFO:tasks.workunit.client.1.smithi179.stdout:1/402: dwrite d3/f28 [4194304,4194304] 0 2024-04-02T04:47:48.169 INFO:tasks.workunit.client.0.smithi070.stdout:4/338: dwrite d1/f28 [0,4194304] 0 2024-04-02T04:47:48.171 INFO:tasks.workunit.client.0.smithi070.stdout:4/339: write d1/f32 [1858502,79222] 0 2024-04-02T04:47:48.172 INFO:tasks.workunit.client.0.smithi070.stdout:3/365: dread d10/d17/f78 [0,4194304] 0 2024-04-02T04:47:48.173 INFO:tasks.workunit.client.1.smithi179.stdout:3/414: write d8/d26/d36/f5c [3370028,30696] 0 2024-04-02T04:47:48.174 INFO:tasks.workunit.client.0.smithi070.stdout:0/432: rename d4/d5/d13/d2a/d3a/c75 to d4/d5/d13/d2a/d29/c79 0 2024-04-02T04:47:48.175 INFO:tasks.workunit.client.0.smithi070.stdout:0/433: chown d4/l39 6309 1 2024-04-02T04:47:48.178 INFO:tasks.workunit.client.0.smithi070.stdout:1/418: unlink d0/d7/d1a/f2e 0 2024-04-02T04:47:48.178 INFO:tasks.workunit.client.0.smithi070.stdout:1/419: fsync d0/d7/d1a/f1b 0 2024-04-02T04:47:48.180 INFO:tasks.workunit.client.1.smithi179.stdout:3/415: dwrite d8/d26/f2b [0,4194304] 0 2024-04-02T04:47:48.180 INFO:tasks.workunit.client.1.smithi179.stdout:0/454: creat d1/da/df/f8a x:0 0 0 2024-04-02T04:47:48.182 INFO:tasks.workunit.client.0.smithi070.stdout:3/366: symlink d10/d32/d4b/l83 0 2024-04-02T04:47:48.190 INFO:tasks.workunit.client.0.smithi070.stdout:3/367: dwrite d10/d32/f56 [0,4194304] 0 2024-04-02T04:47:48.198 INFO:tasks.workunit.client.0.smithi070.stdout:1/420: fdatasync d0/d7/d28/f7a 0 2024-04-02T04:47:48.200 INFO:tasks.workunit.client.0.smithi070.stdout:3/368: dwrite d10/d32/d3f/f4d [0,4194304] 0 2024-04-02T04:47:48.205 INFO:tasks.workunit.client.0.smithi070.stdout:3/369: dread d10/d32/d3f/f6a [0,4194304] 0 2024-04-02T04:47:48.205 INFO:tasks.workunit.client.0.smithi070.stdout:1/421: mknod d0/d54/c86 0 2024-04-02T04:47:48.206 INFO:tasks.workunit.client.0.smithi070.stdout:4/340: rename d1/d3/c61 to d1/c72 0 2024-04-02T04:47:48.208 INFO:tasks.workunit.client.0.smithi070.stdout:0/434: write d4/d5/d13/d43/f53 [2176436,39953] 0 2024-04-02T04:47:48.210 INFO:tasks.workunit.client.0.smithi070.stdout:3/370: readlink d10/d17/l2f 0 2024-04-02T04:47:48.213 INFO:tasks.workunit.client.0.smithi070.stdout:0/435: rename d4/d5/da/c4b to d4/d5/d13/d2a/d3a/c7a 0 2024-04-02T04:47:48.213 INFO:tasks.workunit.client.0.smithi070.stdout:2/445: sync 2024-04-02T04:47:48.213 INFO:tasks.workunit.client.0.smithi070.stdout:6/290: sync 2024-04-02T04:47:48.214 INFO:tasks.workunit.client.0.smithi070.stdout:0/436: truncate d4/d3c/f6a 399885 0 2024-04-02T04:47:48.216 INFO:tasks.workunit.client.0.smithi070.stdout:3/371: creat d10/d17/d4c/f84 x:0 0 0 2024-04-02T04:47:48.217 INFO:tasks.workunit.client.0.smithi070.stdout:6/291: creat d0/d3/d19/d31/d37/f5d x:0 0 0 2024-04-02T04:47:48.218 INFO:tasks.workunit.client.1.smithi179.stdout:3/416: chown f5 182405 1 2024-04-02T04:47:48.218 INFO:tasks.workunit.client.0.smithi070.stdout:1/422: rmdir d0/d54 39 2024-04-02T04:47:48.219 INFO:tasks.workunit.client.1.smithi179.stdout:5/325: getdents d8/d1b/d23/d30 0 2024-04-02T04:47:48.221 INFO:tasks.workunit.client.1.smithi179.stdout:1/403: creat d3/d85/f89 x:0 0 0 2024-04-02T04:47:48.223 INFO:tasks.workunit.client.0.smithi070.stdout:6/292: rmdir d0/d3/d19/d31 39 2024-04-02T04:47:48.224 INFO:tasks.workunit.client.0.smithi070.stdout:6/293: chown d0/d3/l29 4165891 1 2024-04-02T04:47:48.225 INFO:tasks.workunit.client.1.smithi179.stdout:5/326: mkdir d8/d1b/d23/d30/d5f 0 2024-04-02T04:47:48.225 INFO:tasks.workunit.client.0.smithi070.stdout:4/341: dwrite d1/f2d [0,4194304] 0 2024-04-02T04:47:48.227 INFO:tasks.workunit.client.1.smithi179.stdout:9/394: getdents d0/d1/d11/d29 0 2024-04-02T04:47:48.228 INFO:tasks.workunit.client.1.smithi179.stdout:9/395: readlink d0/d1/d11/l54 0 2024-04-02T04:47:48.229 INFO:tasks.workunit.client.0.smithi070.stdout:3/372: symlink d10/d17/l85 0 2024-04-02T04:47:48.231 INFO:tasks.workunit.client.0.smithi070.stdout:1/423: creat d0/d54/f87 x:0 0 0 2024-04-02T04:47:48.232 INFO:tasks.workunit.client.0.smithi070.stdout:1/424: write d0/d9/d1d/f5e [281899,35777] 0 2024-04-02T04:47:48.233 INFO:tasks.workunit.client.1.smithi179.stdout:3/417: symlink d8/d26/d52/d7e/l86 0 2024-04-02T04:47:48.235 INFO:tasks.workunit.client.1.smithi179.stdout:5/327: creat d8/d1b/d23/f60 x:0 0 0 2024-04-02T04:47:48.237 INFO:tasks.workunit.client.1.smithi179.stdout:1/404: write d3/d5/de/f57 [2342344,53935] 0 2024-04-02T04:47:48.239 INFO:tasks.workunit.client.0.smithi070.stdout:2/446: getdents d2/d10 0 2024-04-02T04:47:48.241 INFO:tasks.workunit.client.0.smithi070.stdout:2/447: readlink d2/d12/d3e/d68/l7c 0 2024-04-02T04:47:48.242 INFO:tasks.workunit.client.0.smithi070.stdout:0/437: dwrite d4/d5/d13/f19 [8388608,4194304] 0 2024-04-02T04:47:48.244 INFO:tasks.workunit.client.0.smithi070.stdout:1/425: creat d0/d7/d1a/d66/f88 x:0 0 0 2024-04-02T04:47:48.247 INFO:tasks.workunit.client.1.smithi179.stdout:5/328: write d8/d1b/f24 [495407,82529] 0 2024-04-02T04:47:48.249 INFO:tasks.workunit.client.0.smithi070.stdout:4/342: link d1/fa d1/d3/d9/d41/f73 0 2024-04-02T04:47:48.251 INFO:tasks.workunit.client.1.smithi179.stdout:5/329: dread f6 [0,4194304] 0 2024-04-02T04:47:48.251 INFO:tasks.workunit.client.0.smithi070.stdout:3/373: link d10/d32/d3f/f38 d10/f86 0 2024-04-02T04:47:48.253 INFO:tasks.workunit.client.1.smithi179.stdout:1/405: dwrite d3/d5/de/f32 [0,4194304] 0 2024-04-02T04:47:48.253 INFO:tasks.workunit.client.0.smithi070.stdout:2/448: creat d2/d10/d5c/f8c x:0 0 0 2024-04-02T04:47:48.254 INFO:tasks.workunit.client.0.smithi070.stdout:2/449: chown d2/d10 1 1 2024-04-02T04:47:48.255 INFO:tasks.workunit.client.1.smithi179.stdout:9/396: creat d0/d1/f7d x:0 0 0 2024-04-02T04:47:48.255 INFO:tasks.workunit.client.1.smithi179.stdout:9/397: stat d0/c5a 0 2024-04-02T04:47:48.257 INFO:tasks.workunit.client.0.smithi070.stdout:1/426: unlink d0/d9/d1d/f40 0 2024-04-02T04:47:48.258 INFO:tasks.workunit.client.1.smithi179.stdout:0/455: link d1/da/d48/d65/d7d/f7a d1/da/d48/d5c/d83/f8b 0 2024-04-02T04:47:48.260 INFO:tasks.workunit.client.0.smithi070.stdout:6/294: truncate d0/d3/d19/f3f 1031444 0 2024-04-02T04:47:48.262 INFO:tasks.workunit.client.1.smithi179.stdout:3/418: link d8/d26/d36/c43 d8/d26/d52/c87 0 2024-04-02T04:47:48.262 INFO:tasks.workunit.client.1.smithi179.stdout:3/419: stat d8/d57/d63/c78 0 2024-04-02T04:47:48.263 INFO:tasks.workunit.client.0.smithi070.stdout:4/343: fsync d1/d3/d9/f13 0 2024-04-02T04:47:48.264 INFO:tasks.workunit.client.0.smithi070.stdout:4/344: dread - d1/d3/d9/d41/f66 zero size 2024-04-02T04:47:48.265 INFO:tasks.workunit.client.1.smithi179.stdout:5/330: creat d8/d1b/d23/d32/d4e/f61 x:0 0 0 2024-04-02T04:47:48.267 INFO:tasks.workunit.client.0.smithi070.stdout:0/438: creat d4/d5/d21/d6b/f7b x:0 0 0 2024-04-02T04:47:48.269 INFO:tasks.workunit.client.1.smithi179.stdout:3/420: dwrite d8/d26/d36/f4a [0,4194304] 0 2024-04-02T04:47:48.271 INFO:tasks.workunit.client.1.smithi179.stdout:0/456: symlink d1/da/d28/d6d/l8c 0 2024-04-02T04:47:48.272 INFO:tasks.workunit.client.0.smithi070.stdout:4/345: chown d1/l4 1117 1 2024-04-02T04:47:48.274 INFO:tasks.workunit.client.1.smithi179.stdout:5/331: unlink d8/c1e 0 2024-04-02T04:47:48.275 INFO:tasks.workunit.client.0.smithi070.stdout:6/295: symlink d0/d3/d19/d39/l5e 0 2024-04-02T04:47:48.275 INFO:tasks.workunit.client.0.smithi070.stdout:0/439: dwrite d4/d5/d13/d2a/f6c [0,4194304] 0 2024-04-02T04:47:48.279 INFO:tasks.workunit.client.1.smithi179.stdout:9/398: mkdir d0/d19/d27/d7a/d7e 0 2024-04-02T04:47:48.279 INFO:tasks.workunit.client.1.smithi179.stdout:3/421: mkdir d8/d6d/d88 0 2024-04-02T04:47:48.281 INFO:tasks.workunit.client.0.smithi070.stdout:1/427: dread d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:48.282 INFO:tasks.workunit.client.0.smithi070.stdout:1/428: stat d0/d9/d1d/d22/d2c/d4f 0 2024-04-02T04:47:48.283 INFO:tasks.workunit.client.0.smithi070.stdout:3/374: getdents d10 0 2024-04-02T04:47:48.284 INFO:tasks.workunit.client.1.smithi179.stdout:5/332: mkdir d8/d1b/d23/d36/d53/d62 0 2024-04-02T04:47:48.284 INFO:tasks.workunit.client.0.smithi070.stdout:6/296: mkdir d0/d20/d5f 0 2024-04-02T04:47:48.285 INFO:tasks.workunit.client.0.smithi070.stdout:6/297: write d0/f2d [656414,92821] 0 2024-04-02T04:47:48.286 INFO:tasks.workunit.client.1.smithi179.stdout:1/406: link d3/d5/de/d33/c48 d3/d5/d2f/d5a/c8a 0 2024-04-02T04:47:48.286 INFO:tasks.workunit.client.1.smithi179.stdout:5/333: dread - d8/d1b/d23/d30/f34 zero size 2024-04-02T04:47:48.287 INFO:tasks.workunit.client.0.smithi070.stdout:0/440: creat d4/d5/d58/d60/f7c x:0 0 0 2024-04-02T04:47:48.288 INFO:tasks.workunit.client.1.smithi179.stdout:9/399: dwrite d0/d1/f7b [0,4194304] 0 2024-04-02T04:47:48.288 INFO:tasks.workunit.client.0.smithi070.stdout:0/441: readlink d4/d3c/l5c 0 2024-04-02T04:47:48.289 INFO:tasks.workunit.client.0.smithi070.stdout:0/442: chown d4/d5/d13/d17 293430 1 2024-04-02T04:47:48.290 INFO:tasks.workunit.client.0.smithi070.stdout:1/429: fdatasync d0/d9/d1d/f4e 0 2024-04-02T04:47:48.292 INFO:tasks.workunit.client.1.smithi179.stdout:3/422: mkdir d8/d26/d83/d89 0 2024-04-02T04:47:48.293 INFO:tasks.workunit.client.1.smithi179.stdout:3/423: fdatasync d8/f22 0 2024-04-02T04:47:48.293 INFO:tasks.workunit.client.0.smithi070.stdout:3/375: mknod d10/d17/d51/c87 0 2024-04-02T04:47:48.297 INFO:tasks.workunit.client.0.smithi070.stdout:0/443: dwrite d4/d5/d13/d23/f74 [0,4194304] 0 2024-04-02T04:47:48.297 INFO:tasks.workunit.client.1.smithi179.stdout:5/334: symlink d8/d1b/d23/d42/l63 0 2024-04-02T04:47:48.304 INFO:tasks.workunit.client.1.smithi179.stdout:3/424: unlink d8/d26/d36/f4a 0 2024-04-02T04:47:48.308 INFO:tasks.workunit.client.1.smithi179.stdout:3/425: dread d8/d57/f5a [0,4194304] 0 2024-04-02T04:47:48.310 INFO:tasks.workunit.client.0.smithi070.stdout:4/346: rename d1/d3/d9/d41/d65/c68 to d1/d3/c74 0 2024-04-02T04:47:48.313 INFO:tasks.workunit.client.1.smithi179.stdout:3/426: dwrite d8/d26/f56 [0,4194304] 0 2024-04-02T04:47:48.313 INFO:tasks.workunit.client.1.smithi179.stdout:5/335: unlink d8/f2c 0 2024-04-02T04:47:48.313 INFO:tasks.workunit.client.0.smithi070.stdout:0/444: dread d4/d5/f5e [4194304,4194304] 0 2024-04-02T04:47:48.314 INFO:tasks.workunit.client.0.smithi070.stdout:0/445: stat d4/d5/d13/d2a/d29/f46 0 2024-04-02T04:47:48.317 INFO:tasks.workunit.client.0.smithi070.stdout:6/298: unlink d0/d3/d19/f2f 0 2024-04-02T04:47:48.324 INFO:tasks.workunit.client.1.smithi179.stdout:4/452: sync 2024-04-02T04:47:48.324 INFO:tasks.workunit.client.1.smithi179.stdout:8/360: sync 2024-04-02T04:47:48.326 INFO:tasks.workunit.client.0.smithi070.stdout:0/446: dwrite d4/d5/d58/d60/f7c [0,4194304] 0 2024-04-02T04:47:48.328 INFO:tasks.workunit.client.0.smithi070.stdout:3/376: dread fd [0,4194304] 0 2024-04-02T04:47:48.328 INFO:tasks.workunit.client.0.smithi070.stdout:6/299: dwrite d0/f26 [0,4194304] 0 2024-04-02T04:47:48.329 INFO:tasks.workunit.client.0.smithi070.stdout:6/300: fdatasync d0/f1c 0 2024-04-02T04:47:48.330 INFO:tasks.workunit.client.1.smithi179.stdout:1/407: truncate d3/d5/f27 4224937 0 2024-04-02T04:47:48.331 INFO:tasks.workunit.client.1.smithi179.stdout:1/408: fsync d3/d5/d2f/d5a/f5d 0 2024-04-02T04:47:48.332 INFO:tasks.workunit.client.0.smithi070.stdout:1/430: mkdir d0/d9/d1d/d22/d56/d79/d89 0 2024-04-02T04:47:48.337 INFO:tasks.workunit.client.1.smithi179.stdout:5/336: rename d8/d1b/d23/d30/l3a to d8/d1b/d23/d32/d4b/l64 0 2024-04-02T04:47:48.341 INFO:tasks.workunit.client.0.smithi070.stdout:0/447: write d4/d3c/f77 [279345,84455] 0 2024-04-02T04:47:48.342 INFO:tasks.workunit.client.0.smithi070.stdout:0/448: read - d4/d5/d13/d2a/d29/f71 zero size 2024-04-02T04:47:48.343 INFO:tasks.workunit.client.0.smithi070.stdout:3/377: chown d10/d17/c5f 3 1 2024-04-02T04:47:48.343 INFO:tasks.workunit.client.1.smithi179.stdout:9/400: getdents d0/d1/d11/d77 0 2024-04-02T04:47:48.344 INFO:tasks.workunit.client.1.smithi179.stdout:9/401: chown d0/d1/d10/c38 11 1 2024-04-02T04:47:48.344 INFO:tasks.workunit.client.1.smithi179.stdout:1/409: creat d3/d5/d2f/d4f/d35/d49/f8b x:0 0 0 2024-04-02T04:47:48.345 INFO:tasks.workunit.client.1.smithi179.stdout:5/337: dwrite d8/d1b/f37 [0,4194304] 0 2024-04-02T04:47:48.345 INFO:tasks.workunit.client.0.smithi070.stdout:1/431: fdatasync d0/d9/f14 0 2024-04-02T04:47:48.348 INFO:tasks.workunit.client.1.smithi179.stdout:4/453: link d1/d3/df/f57 d1/d3/dc/d27/f97 0 2024-04-02T04:47:48.349 INFO:tasks.workunit.client.0.smithi070.stdout:6/301: creat d0/d55/d5c/f60 x:0 0 0 2024-04-02T04:47:48.352 INFO:tasks.workunit.client.1.smithi179.stdout:9/402: dwrite d0/d1/d10/f18 [0,4194304] 0 2024-04-02T04:47:48.353 INFO:tasks.workunit.client.0.smithi070.stdout:0/449: rename d4/d5/d21/f64 to d4/d5/da/d2d/f7d 0 2024-04-02T04:47:48.354 INFO:tasks.workunit.client.1.smithi179.stdout:1/410: rmdir d3/d5 39 2024-04-02T04:47:48.355 INFO:tasks.workunit.client.0.smithi070.stdout:0/450: fsync d4/d5/d13/f32 0 2024-04-02T04:47:48.356 INFO:tasks.workunit.client.0.smithi070.stdout:0/451: chown d4 1745147368 1 2024-04-02T04:47:48.356 INFO:tasks.workunit.client.0.smithi070.stdout:3/378: dwrite d10/d32/d3f/f65 [0,4194304] 0 2024-04-02T04:47:48.357 INFO:tasks.workunit.client.0.smithi070.stdout:3/379: stat d10/c70 0 2024-04-02T04:47:48.358 INFO:tasks.workunit.client.1.smithi179.stdout:5/338: dread d8/d1b/f1f [0,4194304] 0 2024-04-02T04:47:48.359 INFO:tasks.workunit.client.0.smithi070.stdout:3/380: fdatasync d10/d32/d4b/f7d 0 2024-04-02T04:47:48.360 INFO:tasks.workunit.client.0.smithi070.stdout:3/381: chown d10/d17/c7e 5 1 2024-04-02T04:47:48.362 INFO:tasks.workunit.client.0.smithi070.stdout:0/452: dread d4/d5/d13/d23/f5b [0,4194304] 0 2024-04-02T04:47:48.365 INFO:tasks.workunit.client.0.smithi070.stdout:4/347: getdents d1/d3 0 2024-04-02T04:47:48.365 INFO:tasks.workunit.client.0.smithi070.stdout:0/453: write d4/d5/d13/d2a/d29/f71 [82395,6395] 0 2024-04-02T04:47:48.366 INFO:tasks.workunit.client.0.smithi070.stdout:0/454: stat d4/d5/d13/d17 0 2024-04-02T04:47:48.369 INFO:tasks.workunit.client.0.smithi070.stdout:0/455: write d4/d5/d13/f32 [3726372,41253] 0 2024-04-02T04:47:48.370 INFO:tasks.workunit.client.1.smithi179.stdout:8/361: rename d3/f16 to d3/d31/f7f 0 2024-04-02T04:47:48.372 INFO:tasks.workunit.client.1.smithi179.stdout:3/427: getdents d8/d57 0 2024-04-02T04:47:48.372 INFO:tasks.workunit.client.1.smithi179.stdout:3/428: chown d8/d7a 6 1 2024-04-02T04:47:48.373 INFO:tasks.workunit.client.0.smithi070.stdout:4/348: dwrite d1/d3/f62 [0,4194304] 0 2024-04-02T04:47:48.373 INFO:tasks.workunit.client.0.smithi070.stdout:0/456: truncate d4/d5/d13/d2a/d29/f71 816802 0 2024-04-02T04:47:48.374 INFO:tasks.workunit.client.1.smithi179.stdout:4/454: fdatasync d1/d3/dc/d27/d62/f54 0 2024-04-02T04:47:48.374 INFO:tasks.workunit.client.0.smithi070.stdout:5/434: sync 2024-04-02T04:47:48.374 INFO:tasks.workunit.client.0.smithi070.stdout:9/349: sync 2024-04-02T04:47:48.375 INFO:tasks.workunit.client.0.smithi070.stdout:8/418: sync 2024-04-02T04:47:48.375 INFO:tasks.workunit.client.0.smithi070.stdout:7/390: sync 2024-04-02T04:47:48.375 INFO:tasks.workunit.client.1.smithi179.stdout:4/455: dread - d1/d3/df/d10/d2f/d60/d71/f85 zero size 2024-04-02T04:47:48.376 INFO:tasks.workunit.client.0.smithi070.stdout:9/350: stat d4/d13/l46 0 2024-04-02T04:47:48.379 INFO:tasks.workunit.client.1.smithi179.stdout:9/403: unlink d0/d1/d10/d6d/l1f 0 2024-04-02T04:47:48.382 INFO:tasks.workunit.client.0.smithi070.stdout:8/419: dread d2/d8/d23/d24/f67 [0,4194304] 0 2024-04-02T04:47:48.386 INFO:tasks.workunit.client.0.smithi070.stdout:4/349: unlink d1/d3/d9/d5e/f71 0 2024-04-02T04:47:48.388 INFO:tasks.workunit.client.0.smithi070.stdout:0/457: mknod d4/d5/d13/d2a/d29/c7e 0 2024-04-02T04:47:48.390 INFO:tasks.workunit.client.0.smithi070.stdout:5/435: mkdir d5/d9/d23/d8b 0 2024-04-02T04:47:48.391 INFO:tasks.workunit.client.1.smithi179.stdout:2/260: sync 2024-04-02T04:47:48.391 INFO:tasks.workunit.client.1.smithi179.stdout:6/408: sync 2024-04-02T04:47:48.391 INFO:tasks.workunit.client.0.smithi070.stdout:5/436: write d5/d66/f73 [79414,95456] 0 2024-04-02T04:47:48.393 INFO:tasks.workunit.client.0.smithi070.stdout:5/437: read d5/f2f [1818960,101186] 0 2024-04-02T04:47:48.397 INFO:tasks.workunit.client.1.smithi179.stdout:9/404: rmdir d0/d1/d11/d77 39 2024-04-02T04:47:48.398 INFO:tasks.workunit.client.1.smithi179.stdout:5/339: dwrite d8/d1b/d23/f2b [0,4194304] 0 2024-04-02T04:47:48.400 INFO:tasks.workunit.client.1.smithi179.stdout:1/411: mknod d3/d5/d2f/d36/c8c 0 2024-04-02T04:47:48.401 INFO:tasks.workunit.client.0.smithi070.stdout:7/391: mkdir d7/dd/df/d79/d46/d65/d7b 0 2024-04-02T04:47:48.402 INFO:tasks.workunit.client.0.smithi070.stdout:7/392: read - d7/dd/df/d79/d46/d65/f71 zero size 2024-04-02T04:47:48.403 INFO:tasks.workunit.client.0.smithi070.stdout:3/382: symlink d10/d17/l88 0 2024-04-02T04:47:48.403 INFO:tasks.workunit.client.1.smithi179.stdout:4/456: dwrite d1/d15/d7c/f84 [0,4194304] 0 2024-04-02T04:47:48.404 INFO:tasks.workunit.client.0.smithi070.stdout:7/393: stat d7/dd/df/d7a/d6c/d77/d75 0 2024-04-02T04:47:48.404 INFO:tasks.workunit.client.1.smithi179.stdout:5/340: dread d8/d1b/f49 [0,4194304] 0 2024-04-02T04:47:48.407 INFO:tasks.workunit.client.1.smithi179.stdout:1/412: write d3/d5/d2f/d4f/d35/d49/f8b [932456,52484] 0 2024-04-02T04:47:48.409 INFO:tasks.workunit.client.0.smithi070.stdout:0/458: creat d4/d5/d13/d23/f7f x:0 0 0 2024-04-02T04:47:48.409 INFO:tasks.workunit.client.1.smithi179.stdout:3/429: dwrite d8/d57/f5a [0,4194304] 0 2024-04-02T04:47:48.410 INFO:tasks.workunit.client.0.smithi070.stdout:0/459: chown d4/d5/d13/f32 357 1 2024-04-02T04:47:48.412 INFO:tasks.workunit.client.1.smithi179.stdout:6/409: unlink d3/d12/d23/l2a 0 2024-04-02T04:47:48.415 INFO:tasks.workunit.client.0.smithi070.stdout:4/350: dread d1/d3/d9/f1d [0,4194304] 0 2024-04-02T04:47:48.416 INFO:tasks.workunit.client.1.smithi179.stdout:2/261: write dc/d29/d4a/f45 [1494720,24534] 0 2024-04-02T04:47:48.418 INFO:tasks.workunit.client.1.smithi179.stdout:9/405: dwrite d0/d1/d11/d29/d40/f5d [0,4194304] 0 2024-04-02T04:47:48.419 INFO:tasks.workunit.client.0.smithi070.stdout:0/460: dwrite d4/d5/d13/f2b [0,4194304] 0 2024-04-02T04:47:48.421 INFO:tasks.workunit.client.1.smithi179.stdout:1/413: dread d3/d5/de/f24 [0,4194304] 0 2024-04-02T04:47:48.421 INFO:tasks.workunit.client.1.smithi179.stdout:2/262: write dc/d29/f34 [286630,120190] 0 2024-04-02T04:47:48.423 INFO:tasks.workunit.client.0.smithi070.stdout:0/461: write d4/d5/d13/f2b [1420070,54634] 0 2024-04-02T04:47:48.424 INFO:tasks.workunit.client.1.smithi179.stdout:3/430: dwrite d8/f1b [4194304,4194304] 0 2024-04-02T04:47:48.424 INFO:tasks.workunit.client.1.smithi179.stdout:8/362: truncate d3/f4 2098516 0 2024-04-02T04:47:48.426 INFO:tasks.workunit.client.0.smithi070.stdout:5/438: mknod d5/d14/c8c 0 2024-04-02T04:47:48.426 INFO:tasks.workunit.client.0.smithi070.stdout:0/462: fsync d4/d5/d13/d2a/f68 0 2024-04-02T04:47:48.429 INFO:tasks.workunit.client.0.smithi070.stdout:6/302: rename d0/d3/d19/d31/d37/d50 to d0/d3/d19/d31/d61 0 2024-04-02T04:47:48.430 INFO:tasks.workunit.client.1.smithi179.stdout:9/406: dwrite d0/d1/d10/f2b [0,4194304] 0 2024-04-02T04:47:48.434 INFO:tasks.workunit.client.1.smithi179.stdout:4/457: mkdir d1/d15/d7c/d98 0 2024-04-02T04:47:48.437 INFO:tasks.workunit.client.0.smithi070.stdout:8/420: mknod d2/d8/c82 0 2024-04-02T04:47:48.438 INFO:tasks.workunit.client.1.smithi179.stdout:3/431: dwrite d8/d57/d63/f84 [0,4194304] 0 2024-04-02T04:47:48.448 INFO:tasks.workunit.client.0.smithi070.stdout:4/351: creat d1/d3/d9/f75 x:0 0 0 2024-04-02T04:47:48.452 INFO:tasks.workunit.client.1.smithi179.stdout:8/363: rename d3/d32/d21/d49/c6f to d3/d32/d21/d51/c80 0 2024-04-02T04:47:48.454 INFO:tasks.workunit.client.0.smithi070.stdout:5/439: read d5/d17/d27/f45 [4163833,30370] 0 2024-04-02T04:47:48.454 INFO:tasks.workunit.client.0.smithi070.stdout:0/463: mkdir d4/d5/d13/d2a/d3a/d80 0 2024-04-02T04:47:48.456 INFO:tasks.workunit.client.0.smithi070.stdout:7/394: fsync d7/d9/f1b 0 2024-04-02T04:47:48.458 INFO:tasks.workunit.client.0.smithi070.stdout:7/395: truncate d7/dd/df/d79/d46/d65/f67 1321084 0 2024-04-02T04:47:48.459 INFO:tasks.workunit.client.0.smithi070.stdout:7/396: dread d7/d36/f62 [0,4194304] 0 2024-04-02T04:47:48.459 INFO:tasks.workunit.client.1.smithi179.stdout:5/341: link d8/d1b/d23/d32/d46/f5e d8/d1b/d23/d36/d53/f65 0 2024-04-02T04:47:48.460 INFO:tasks.workunit.client.0.smithi070.stdout:8/421: rmdir d2/d3 39 2024-04-02T04:47:48.461 INFO:tasks.workunit.client.0.smithi070.stdout:8/422: fsync d2/d8/d23/d24/f5b 0 2024-04-02T04:47:48.462 INFO:tasks.workunit.client.0.smithi070.stdout:4/352: mknod d1/d3/d9/c76 0 2024-04-02T04:47:48.465 INFO:tasks.workunit.client.0.smithi070.stdout:3/383: dwrite d10/d32/d3f/f6a [0,4194304] 0 2024-04-02T04:47:48.467 INFO:tasks.workunit.client.0.smithi070.stdout:5/440: rename d5/d17/d2b/f55 to d5/d17/d2b/f8d 0 2024-04-02T04:47:48.468 INFO:tasks.workunit.client.0.smithi070.stdout:0/464: creat d4/d5/d58/d60/f81 x:0 0 0 2024-04-02T04:47:48.470 INFO:tasks.workunit.client.1.smithi179.stdout:8/364: symlink d3/d32/d21/d49/l81 0 2024-04-02T04:47:48.470 INFO:tasks.workunit.client.1.smithi179.stdout:8/365: readlink l2 0 2024-04-02T04:47:48.472 INFO:tasks.workunit.client.1.smithi179.stdout:9/407: mknod d0/d19/c7f 0 2024-04-02T04:47:48.477 INFO:tasks.workunit.client.1.smithi179.stdout:5/342: dread d8/d1b/f1f [0,4194304] 0 2024-04-02T04:47:48.477 INFO:tasks.workunit.client.1.smithi179.stdout:1/414: creat d3/d5/f8d x:0 0 0 2024-04-02T04:47:48.479 INFO:tasks.workunit.client.1.smithi179.stdout:3/432: rename d8/d26/f46 to d8/d7a/f8a 0 2024-04-02T04:47:48.480 INFO:tasks.workunit.client.0.smithi070.stdout:4/353: unlink d1/d3/d9/c57 0 2024-04-02T04:47:48.480 INFO:tasks.workunit.client.0.smithi070.stdout:4/354: chown d1/d3 22 1 2024-04-02T04:47:48.481 INFO:tasks.workunit.client.1.smithi179.stdout:8/366: creat d3/f82 x:0 0 0 2024-04-02T04:47:48.484 INFO:tasks.workunit.client.0.smithi070.stdout:9/351: link d4/d10/f2c d4/d10/f71 0 2024-04-02T04:47:48.487 INFO:tasks.workunit.client.1.smithi179.stdout:2/263: getdents dc/d29/d4a 0 2024-04-02T04:47:48.487 INFO:tasks.workunit.client.1.smithi179.stdout:2/264: chown dc/d29/l42 54 1 2024-04-02T04:47:48.489 INFO:tasks.workunit.client.1.smithi179.stdout:1/415: symlink d3/d5/d2f/d65/l8e 0 2024-04-02T04:47:48.492 INFO:tasks.workunit.client.0.smithi070.stdout:0/465: dread d4/d5/d13/d2a/d29/f5d [0,4194304] 0 2024-04-02T04:47:48.492 INFO:tasks.workunit.client.0.smithi070.stdout:6/303: rmdir d0/d3/d19/d31/d46/d52 0 2024-04-02T04:47:48.493 INFO:tasks.workunit.client.1.smithi179.stdout:4/458: write d1/d3/df/d10/d2f/d36/f58 [2101039,126262] 0 2024-04-02T04:47:48.493 INFO:tasks.workunit.client.1.smithi179.stdout:7/447: sync 2024-04-02T04:47:48.494 INFO:tasks.workunit.client.0.smithi070.stdout:0/466: dread d4/d5/da/d2d/f4c [0,4194304] 0 2024-04-02T04:47:48.495 INFO:tasks.workunit.client.1.smithi179.stdout:4/459: fdatasync d1/d3/df/d10/d2f/d60/d71/f85 0 2024-04-02T04:47:48.495 INFO:tasks.workunit.client.0.smithi070.stdout:7/397: symlink d7/l7c 0 2024-04-02T04:47:48.495 INFO:tasks.workunit.client.1.smithi179.stdout:4/460: stat d1/d3/dc/c23 0 2024-04-02T04:47:48.496 INFO:tasks.workunit.client.1.smithi179.stdout:8/367: mkdir d3/d1b/d2a/d83 0 2024-04-02T04:47:48.497 INFO:tasks.workunit.client.0.smithi070.stdout:7/398: write d7/dd/df/d79/d46/d65/f67 [1940255,122874] 0 2024-04-02T04:47:48.499 INFO:tasks.workunit.client.0.smithi070.stdout:4/355: readlink d1/l2f 0 2024-04-02T04:47:48.501 INFO:tasks.workunit.client.1.smithi179.stdout:2/265: creat dc/d29/d4a/f53 x:0 0 0 2024-04-02T04:47:48.502 INFO:tasks.workunit.client.0.smithi070.stdout:9/352: symlink d4/d10/d12/d56/l72 0 2024-04-02T04:47:48.504 INFO:tasks.workunit.client.1.smithi179.stdout:1/416: mknod d3/d5/d2f/d4f/d42/d55/c8f 0 2024-04-02T04:47:48.505 INFO:tasks.workunit.client.1.smithi179.stdout:9/408: dwrite d0/d1/f36 [0,4194304] 0 2024-04-02T04:47:48.505 INFO:tasks.workunit.client.1.smithi179.stdout:5/343: creat d8/d1b/d23/d32/d4e/d51/f66 x:0 0 0 2024-04-02T04:47:48.506 INFO:tasks.workunit.client.0.smithi070.stdout:5/441: symlink d5/d17/d2b/d7b/l8e 0 2024-04-02T04:47:48.507 INFO:tasks.workunit.client.1.smithi179.stdout:5/344: write d8/ff [660642,55893] 0 2024-04-02T04:47:48.507 INFO:tasks.workunit.client.0.smithi070.stdout:3/384: dwrite d10/d32/d3f/f38 [0,4194304] 0 2024-04-02T04:47:48.508 INFO:tasks.workunit.client.1.smithi179.stdout:7/448: mkdir d1/dd/d16/d74/d9f 0 2024-04-02T04:47:48.510 INFO:tasks.workunit.client.1.smithi179.stdout:1/417: dwrite d3/d5/de/f57 [0,4194304] 0 2024-04-02T04:47:48.511 INFO:tasks.workunit.client.0.smithi070.stdout:9/353: dwrite d4/f36 [0,4194304] 0 2024-04-02T04:47:48.514 INFO:tasks.workunit.client.1.smithi179.stdout:1/418: write d3/d5/d2f/d4f/d35/d49/f7e [542932,74909] 0 2024-04-02T04:47:48.514 INFO:tasks.workunit.client.1.smithi179.stdout:4/461: mknod d1/d3/dc/d27/d62/c99 0 2024-04-02T04:47:48.514 INFO:tasks.workunit.client.1.smithi179.stdout:8/368: mkdir d3/d32/d21/d22/d33/d84 0 2024-04-02T04:47:48.514 INFO:tasks.workunit.client.1.smithi179.stdout:4/462: write d1/fb [3692963,55967] 0 2024-04-02T04:47:48.520 INFO:tasks.workunit.client.0.smithi070.stdout:7/399: truncate d7/dd/f2e 3340400 0 2024-04-02T04:47:48.521 INFO:tasks.workunit.client.0.smithi070.stdout:0/467: dwrite d4/d5/da/f6d [0,4194304] 0 2024-04-02T04:47:48.522 INFO:tasks.workunit.client.1.smithi179.stdout:8/369: dwrite d3/d32/d21/d51/f55 [0,4194304] 0 2024-04-02T04:47:48.522 INFO:tasks.workunit.client.0.smithi070.stdout:4/356: dread - d1/f3e zero size 2024-04-02T04:47:48.524 INFO:tasks.workunit.client.0.smithi070.stdout:0/468: chown d4/d5/d13/d2a/d29/f59 5 1 2024-04-02T04:47:48.524 INFO:tasks.workunit.client.0.smithi070.stdout:4/357: stat d1/d3/d9/d41/f66 0 2024-04-02T04:47:48.525 INFO:tasks.workunit.client.1.smithi179.stdout:1/419: dread d3/d5/d2f/d4f/d35/d49/f7e [0,4194304] 0 2024-04-02T04:47:48.525 INFO:tasks.workunit.client.0.smithi070.stdout:0/469: fsync d4/d3c/f6a 0 2024-04-02T04:47:48.526 INFO:tasks.workunit.client.1.smithi179.stdout:9/409: mkdir d0/d1/d11/d29/d80 0 2024-04-02T04:47:48.528 INFO:tasks.workunit.client.1.smithi179.stdout:9/410: write d0/d1/f65 [747853,115466] 0 2024-04-02T04:47:48.529 INFO:tasks.workunit.client.1.smithi179.stdout:5/345: rename d8/l11 to d8/d1b/d23/d36/d53/d62/l67 0 2024-04-02T04:47:48.540 INFO:tasks.workunit.client.1.smithi179.stdout:3/433: getdents d8/d6d 0 2024-04-02T04:47:48.541 INFO:tasks.workunit.client.1.smithi179.stdout:7/449: truncate d1/d9/d55/f64 3284153 0 2024-04-02T04:47:48.542 INFO:tasks.workunit.client.1.smithi179.stdout:4/463: mknod d1/d15/d7c/d98/c9a 0 2024-04-02T04:47:48.543 INFO:tasks.workunit.client.1.smithi179.stdout:1/420: creat d3/d5/d2f/d4f/d42/d53/d72/d86/f90 x:0 0 0 2024-04-02T04:47:48.543 INFO:tasks.workunit.client.1.smithi179.stdout:4/464: stat d1/d3/dc/f18 0 2024-04-02T04:47:48.545 INFO:tasks.workunit.client.1.smithi179.stdout:3/434: dread d8/f48 [0,4194304] 0 2024-04-02T04:47:48.546 INFO:tasks.workunit.client.1.smithi179.stdout:5/346: truncate d8/d1b/d23/d32/f33 2431622 0 2024-04-02T04:47:48.547 INFO:tasks.workunit.client.0.smithi070.stdout:5/442: symlink d5/d9/d23/d30/l8f 0 2024-04-02T04:47:48.547 INFO:tasks.workunit.client.0.smithi070.stdout:4/358: truncate d1/d3/f6d 723535 0 2024-04-02T04:47:48.547 INFO:tasks.workunit.client.1.smithi179.stdout:3/435: read d8/d26/d36/f38 [239566,119537] 0 2024-04-02T04:47:48.547 INFO:tasks.workunit.client.1.smithi179.stdout:3/436: stat d8/c75 0 2024-04-02T04:47:48.548 INFO:tasks.workunit.client.0.smithi070.stdout:3/385: mknod d10/d17/d4c/c89 0 2024-04-02T04:47:48.548 INFO:tasks.workunit.client.0.smithi070.stdout:6/304: creat d0/d20/d5f/f62 x:0 0 0 2024-04-02T04:47:48.548 INFO:tasks.workunit.client.0.smithi070.stdout:8/423: link d2/d3/c7d d2/d8/d29/d72/c83 0 2024-04-02T04:47:48.549 INFO:tasks.workunit.client.0.smithi070.stdout:4/359: write d1/d3/d9/f13 [5016149,99174] 0 2024-04-02T04:47:48.550 INFO:tasks.workunit.client.1.smithi179.stdout:1/421: symlink d3/d5/d2f/d4f/l91 0 2024-04-02T04:47:48.550 INFO:tasks.workunit.client.0.smithi070.stdout:8/424: chown d2/d8/d23/d24/f4b 117 1 2024-04-02T04:47:48.551 INFO:tasks.workunit.client.1.smithi179.stdout:4/465: mknod d1/d15/d7c/d98/c9b 0 2024-04-02T04:47:48.551 INFO:tasks.workunit.client.1.smithi179.stdout:8/370: fdatasync d3/d32/d21/d51/f55 0 2024-04-02T04:47:48.552 INFO:tasks.workunit.client.0.smithi070.stdout:3/386: write d10/d17/d51/d73/f80 [189433,47876] 0 2024-04-02T04:47:48.552 INFO:tasks.workunit.client.1.smithi179.stdout:4/466: chown d1/d3/dc 189 1 2024-04-02T04:47:48.553 INFO:tasks.workunit.client.1.smithi179.stdout:4/467: truncate d1/d3/dc/d27/d62/f5e 5170957 0 2024-04-02T04:47:48.556 INFO:tasks.workunit.client.1.smithi179.stdout:5/347: creat d8/d1b/d23/d32/d4e/d50/f68 x:0 0 0 2024-04-02T04:47:48.556 INFO:tasks.workunit.client.1.smithi179.stdout:9/411: getdents d0/d1/d11/d3a 0 2024-04-02T04:47:48.557 INFO:tasks.workunit.client.1.smithi179.stdout:5/348: fsync d8/d1b/d23/d32/d4e/f61 0 2024-04-02T04:47:48.557 INFO:tasks.workunit.client.1.smithi179.stdout:9/412: stat d0/d1/d11/l54 0 2024-04-02T04:47:48.558 INFO:tasks.workunit.client.1.smithi179.stdout:2/266: readlink dc/d29/d4a/l28 0 2024-04-02T04:47:48.558 INFO:tasks.workunit.client.1.smithi179.stdout:9/413: write d0/d1/f7d [369852,77469] 0 2024-04-02T04:47:48.559 INFO:tasks.workunit.client.1.smithi179.stdout:2/267: fdatasync dc/f50 0 2024-04-02T04:47:48.561 INFO:tasks.workunit.client.1.smithi179.stdout:8/371: readlink d3/d31/l29 0 2024-04-02T04:47:48.562 INFO:tasks.workunit.client.1.smithi179.stdout:4/468: mkdir d1/d15/d6e/d9c 0 2024-04-02T04:47:48.562 INFO:tasks.workunit.client.1.smithi179.stdout:5/349: dread d8/d1b/d23/d36/f41 [0,4194304] 0 2024-04-02T04:47:48.563 INFO:tasks.workunit.client.1.smithi179.stdout:7/450: getdents d1/d9/d35 0 2024-04-02T04:47:48.564 INFO:tasks.workunit.client.1.smithi179.stdout:5/350: dread - d8/d1b/d23/f2f zero size 2024-04-02T04:47:48.564 INFO:tasks.workunit.client.1.smithi179.stdout:4/469: dread - d1/d3/df/d10/d2f/d7b/f86 zero size 2024-04-02T04:47:48.567 INFO:tasks.workunit.client.1.smithi179.stdout:9/414: creat d0/d1/d11/d3a/f81 x:0 0 0 2024-04-02T04:47:48.567 INFO:tasks.workunit.client.1.smithi179.stdout:0/457: sync 2024-04-02T04:47:48.569 INFO:tasks.workunit.client.1.smithi179.stdout:2/268: chown dc/d29/d4a/f15 197543286 1 2024-04-02T04:47:48.569 INFO:tasks.workunit.client.1.smithi179.stdout:0/458: fsync d1/da/f2d 0 2024-04-02T04:47:48.569 INFO:tasks.workunit.client.1.smithi179.stdout:8/372: unlink d3/d32/d21/d22/c26 0 2024-04-02T04:47:48.570 INFO:tasks.workunit.client.1.smithi179.stdout:4/470: dwrite d1/d15/d7c/f84 [0,4194304] 0 2024-04-02T04:47:48.572 INFO:tasks.workunit.client.1.smithi179.stdout:5/351: creat d8/d1b/d23/f69 x:0 0 0 2024-04-02T04:47:48.575 INFO:tasks.workunit.client.1.smithi179.stdout:9/415: mkdir d0/d19/d27/d7a/d82 0 2024-04-02T04:47:48.577 INFO:tasks.workunit.client.1.smithi179.stdout:2/269: read - dc/d29/d4a/f27 zero size 2024-04-02T04:47:48.579 INFO:tasks.workunit.client.1.smithi179.stdout:3/437: getdents d8/d26 0 2024-04-02T04:47:48.580 INFO:tasks.workunit.client.1.smithi179.stdout:3/438: write d8/d26/d36/f4d [2124925,75654] 0 2024-04-02T04:47:48.580 INFO:tasks.workunit.client.1.smithi179.stdout:7/451: mknod d1/d9/ca0 0 2024-04-02T04:47:48.582 INFO:tasks.workunit.client.1.smithi179.stdout:3/439: dread d8/f48 [0,4194304] 0 2024-04-02T04:47:48.583 INFO:tasks.workunit.client.1.smithi179.stdout:0/459: symlink d1/da/d48/d5c/l8d 0 2024-04-02T04:47:48.584 INFO:tasks.workunit.client.1.smithi179.stdout:4/471: rmdir d1 39 2024-04-02T04:47:48.587 INFO:tasks.workunit.client.1.smithi179.stdout:5/352: rename d8/d1b/d23/d32/l52 to d8/d1b/d23/d36/d53/l6a 0 2024-04-02T04:47:48.587 INFO:tasks.workunit.client.1.smithi179.stdout:7/452: dwrite d1/f83 [0,4194304] 0 2024-04-02T04:47:48.588 INFO:tasks.workunit.client.1.smithi179.stdout:5/353: fdatasync d8/d1b/f24 0 2024-04-02T04:47:48.590 INFO:tasks.workunit.client.1.smithi179.stdout:7/453: fsync d1/d9/d27/d87/d8b/f6f 0 2024-04-02T04:47:48.592 INFO:tasks.workunit.client.1.smithi179.stdout:8/373: unlink d3/d32/d21/d22/l4f 0 2024-04-02T04:47:48.593 INFO:tasks.workunit.client.1.smithi179.stdout:3/440: rmdir d8/d57 39 2024-04-02T04:47:48.596 INFO:tasks.workunit.client.1.smithi179.stdout:8/374: truncate d3/f61 2672 0 2024-04-02T04:47:48.597 INFO:tasks.workunit.client.1.smithi179.stdout:1/422: dwrite d3/d5/f1d [0,4194304] 0 2024-04-02T04:47:48.597 INFO:tasks.workunit.client.1.smithi179.stdout:9/416: chown d0/d1/d11/d29/d40/c55 11454 1 2024-04-02T04:47:48.598 INFO:tasks.workunit.client.1.smithi179.stdout:2/270: dread dc/fe [0,4194304] 0 2024-04-02T04:47:48.600 INFO:tasks.workunit.client.1.smithi179.stdout:1/423: write d3/d5/f25 [2254545,34780] 0 2024-04-02T04:47:48.600 INFO:tasks.workunit.client.1.smithi179.stdout:7/454: symlink d1/d84/d8d/la1 0 2024-04-02T04:47:48.602 INFO:tasks.workunit.client.1.smithi179.stdout:7/455: read d1/f5d [3760108,6595] 0 2024-04-02T04:47:48.602 INFO:tasks.workunit.client.1.smithi179.stdout:4/472: mkdir d1/d9d 0 2024-04-02T04:47:48.603 INFO:tasks.workunit.client.1.smithi179.stdout:1/424: fsync d3/d5/de/d33/f7b 0 2024-04-02T04:47:48.603 INFO:tasks.workunit.client.1.smithi179.stdout:8/375: mknod d3/d31/c85 0 2024-04-02T04:47:48.605 INFO:tasks.workunit.client.1.smithi179.stdout:3/441: dread d8/d26/f33 [0,4194304] 0 2024-04-02T04:47:48.606 INFO:tasks.workunit.client.1.smithi179.stdout:1/425: readlink d3/d5/d2f/d4f/l91 0 2024-04-02T04:47:48.607 INFO:tasks.workunit.client.1.smithi179.stdout:2/271: fdatasync f6 0 2024-04-02T04:47:48.609 INFO:tasks.workunit.client.1.smithi179.stdout:3/442: write d8/d26/d36/f59 [1697965,74461] 0 2024-04-02T04:47:48.609 INFO:tasks.workunit.client.1.smithi179.stdout:5/354: link d8/d1b/f24 d8/d1b/d23/d42/d5a/f6b 0 2024-04-02T04:47:48.610 INFO:tasks.workunit.client.1.smithi179.stdout:1/426: read d3/d5/d2f/f47 [1741705,30943] 0 2024-04-02T04:47:48.611 INFO:tasks.workunit.client.1.smithi179.stdout:3/443: dread - d8/d26/d36/f49 zero size 2024-04-02T04:47:48.612 INFO:tasks.workunit.client.0.smithi070.stdout:5/443: read - d5/d9/d10/d51/f59 zero size 2024-04-02T04:47:48.612 INFO:tasks.workunit.client.1.smithi179.stdout:7/456: dwrite d1/dd/d16/f48 [0,4194304] 0 2024-04-02T04:47:48.613 INFO:tasks.workunit.client.0.smithi070.stdout:6/305: rmdir d0/d55 39 2024-04-02T04:47:48.613 INFO:tasks.workunit.client.0.smithi070.stdout:7/400: creat d7/dd/df/d79/d46/d65/d7b/f7d x:0 0 0 2024-04-02T04:47:48.613 INFO:tasks.workunit.client.0.smithi070.stdout:4/360: write d1/d3/d9/f13 [9483097,2656] 0 2024-04-02T04:47:48.616 INFO:tasks.workunit.client.0.smithi070.stdout:5/444: dread - d5/d17/d6c/f6d zero size 2024-04-02T04:47:48.617 INFO:tasks.workunit.client.0.smithi070.stdout:7/401: write d7/dd/d66/f59 [1852575,31385] 0 2024-04-02T04:47:48.619 INFO:tasks.workunit.client.0.smithi070.stdout:3/387: rmdir d10/d32 39 2024-04-02T04:47:48.621 INFO:tasks.workunit.client.0.smithi070.stdout:0/470: dwrite d4/d5/d13/d2a/d29/f65 [0,4194304] 0 2024-04-02T04:47:48.622 INFO:tasks.workunit.client.0.smithi070.stdout:7/402: chown d7/dd/df/d7a/d6c/d77/d75 73794081 1 2024-04-02T04:47:48.628 INFO:tasks.workunit.client.1.smithi179.stdout:4/473: rename d1/d15/l32 to d1/d3/df/d10/d2f/d7b/l9e 0 2024-04-02T04:47:48.629 INFO:tasks.workunit.client.0.smithi070.stdout:8/425: dread d2/d8/d29/d22/f7a [0,4194304] 0 2024-04-02T04:47:48.629 INFO:tasks.workunit.client.1.smithi179.stdout:7/457: dread d1/fc [0,4194304] 0 2024-04-02T04:47:48.630 INFO:tasks.workunit.client.0.smithi070.stdout:8/426: write d2/d8/d23/d24/f4b [1463705,32072] 0 2024-04-02T04:47:48.632 INFO:tasks.workunit.client.1.smithi179.stdout:9/417: dread d0/d1/d11/d29/f34 [0,4194304] 0 2024-04-02T04:47:48.633 INFO:tasks.workunit.client.0.smithi070.stdout:4/361: creat d1/d3/d9/d5e/f77 x:0 0 0 2024-04-02T04:47:48.637 INFO:tasks.workunit.client.1.smithi179.stdout:0/460: creat d1/db/f8e x:0 0 0 2024-04-02T04:47:48.639 INFO:tasks.workunit.client.1.smithi179.stdout:2/272: dwrite dc/f44 [0,4194304] 0 2024-04-02T04:47:48.641 INFO:tasks.workunit.client.0.smithi070.stdout:0/471: creat d4/d5/d13/d2a/d29/d52/f82 x:0 0 0 2024-04-02T04:47:48.643 INFO:tasks.workunit.client.1.smithi179.stdout:8/376: symlink d3/d6c/l86 0 2024-04-02T04:47:48.643 INFO:tasks.workunit.client.0.smithi070.stdout:0/472: chown d4/d5/d13/d2a/d29/l57 179606 1 2024-04-02T04:47:48.644 INFO:tasks.workunit.client.0.smithi070.stdout:8/427: mknod d2/d3/d35/c84 0 2024-04-02T04:47:48.644 INFO:tasks.workunit.client.0.smithi070.stdout:0/473: chown d4/d5/d13/d23/l4e 1 1 2024-04-02T04:47:48.644 INFO:tasks.workunit.client.1.smithi179.stdout:4/474: mkdir d1/d93/d9f 0 2024-04-02T04:47:48.645 INFO:tasks.workunit.client.0.smithi070.stdout:7/403: dread d7/dd/df/d79/d46/d65/f69 [8388608,4194304] 0 2024-04-02T04:47:48.646 INFO:tasks.workunit.client.0.smithi070.stdout:4/362: fdatasync d1/d3/d9/f11 0 2024-04-02T04:47:48.648 INFO:tasks.workunit.client.0.smithi070.stdout:4/363: truncate d1/d3/f4d 585002 0 2024-04-02T04:47:48.648 INFO:tasks.workunit.client.0.smithi070.stdout:7/404: fsync d7/dd/df/d34/f4a 0 2024-04-02T04:47:48.651 INFO:tasks.workunit.client.0.smithi070.stdout:3/388: mkdir d10/d32/d8a 0 2024-04-02T04:47:48.654 INFO:tasks.workunit.client.0.smithi070.stdout:8/428: mknod d2/d3/d35/d3a/d4f/c85 0 2024-04-02T04:47:48.656 INFO:tasks.workunit.client.0.smithi070.stdout:8/429: write d2/d8/d23/d24/d5f/f4c [488580,5773] 0 2024-04-02T04:47:48.660 INFO:tasks.workunit.client.0.smithi070.stdout:5/445: dwrite d5/d9/d23/f42 [4194304,4194304] 0 2024-04-02T04:47:48.665 INFO:tasks.workunit.client.0.smithi070.stdout:4/364: unlink d1/d3/f4d 0 2024-04-02T04:47:48.667 INFO:tasks.workunit.client.0.smithi070.stdout:6/306: link d0/d3/l8 d0/d3/d19/d39/l63 0 2024-04-02T04:47:48.668 INFO:tasks.workunit.client.1.smithi179.stdout:4/475: mkdir d1/d93/da0 0 2024-04-02T04:47:48.669 INFO:tasks.workunit.client.0.smithi070.stdout:7/405: mknod d7/dd/df/d7a/c7e 0 2024-04-02T04:47:48.672 INFO:tasks.workunit.client.1.smithi179.stdout:7/458: link d1/d9/f12 d1/d9/d27/d87/d8b/fa2 0 2024-04-02T04:47:48.673 INFO:tasks.workunit.client.0.smithi070.stdout:3/389: truncate f8 7044400 0 2024-04-02T04:47:48.675 INFO:tasks.workunit.client.1.smithi179.stdout:9/418: creat d0/d1/d11/d77/f83 x:0 0 0 2024-04-02T04:47:48.677 INFO:tasks.workunit.client.0.smithi070.stdout:5/446: dread d5/d9/f5e [0,4194304] 0 2024-04-02T04:47:48.679 INFO:tasks.workunit.client.0.smithi070.stdout:8/430: dread d2/d8/d23/d24/f67 [0,4194304] 0 2024-04-02T04:47:48.680 INFO:tasks.workunit.client.1.smithi179.stdout:2/273: symlink dc/l54 0 2024-04-02T04:47:48.680 INFO:tasks.workunit.client.0.smithi070.stdout:0/474: dwrite d4/d5/da/d2d/f4c [0,4194304] 0 2024-04-02T04:47:48.685 INFO:tasks.workunit.client.0.smithi070.stdout:6/307: chown d0/d55/d5c 112 1 2024-04-02T04:47:48.686 INFO:tasks.workunit.client.1.smithi179.stdout:3/444: dwrite d8/d7a/f8a [0,4194304] 0 2024-04-02T04:47:48.686 INFO:tasks.workunit.client.0.smithi070.stdout:6/308: stat d0/d20/d5f 0 2024-04-02T04:47:48.686 INFO:tasks.workunit.client.1.smithi179.stdout:0/461: dwrite d1/db/d64/f6a [0,4194304] 0 2024-04-02T04:47:48.687 INFO:tasks.workunit.client.1.smithi179.stdout:4/476: dread d1/d15/f56 [0,4194304] 0 2024-04-02T04:47:48.687 INFO:tasks.workunit.client.1.smithi179.stdout:7/459: readlink d1/dd/l2b 0 2024-04-02T04:47:48.687 INFO:tasks.workunit.client.1.smithi179.stdout:7/460: fdatasync d1/fc 0 2024-04-02T04:47:48.688 INFO:tasks.workunit.client.1.smithi179.stdout:7/461: dread - d1/d9/d55/f96 zero size 2024-04-02T04:47:48.689 INFO:tasks.workunit.client.0.smithi070.stdout:7/406: fsync d7/dd/df/d79/d46/d65/f69 0 2024-04-02T04:47:48.690 INFO:tasks.workunit.client.0.smithi070.stdout:7/407: write d7/dd/df/d79/d46/d65/f67 [64480,65570] 0 2024-04-02T04:47:48.690 INFO:tasks.workunit.client.1.smithi179.stdout:9/419: mkdir d0/d19/d27/d7a/d84 0 2024-04-02T04:47:48.693 INFO:tasks.workunit.client.0.smithi070.stdout:3/390: mknod d10/d17/d51/c8b 0 2024-04-02T04:47:48.694 INFO:tasks.workunit.client.0.smithi070.stdout:3/391: fsync d10/d17/d4c/f84 0 2024-04-02T04:47:48.694 INFO:tasks.workunit.client.1.smithi179.stdout:1/427: link d3/d5/d2f/d36/f67 d3/d5/d2f/d65/f92 0 2024-04-02T04:47:48.695 INFO:tasks.workunit.client.1.smithi179.stdout:1/428: fsync d3/d5/de/f57 0 2024-04-02T04:47:48.696 INFO:tasks.workunit.client.1.smithi179.stdout:1/429: stat d3/d5/d2f/d4f/d35/f6e 0 2024-04-02T04:47:48.700 INFO:tasks.workunit.client.0.smithi070.stdout:5/447: symlink d5/d14/l90 0 2024-04-02T04:47:48.703 INFO:tasks.workunit.client.0.smithi070.stdout:0/475: mkdir d4/d5/d58/d60/d83 0 2024-04-02T04:47:48.704 INFO:tasks.workunit.client.0.smithi070.stdout:0/476: dread - d4/d5/d13/d23/f7f zero size 2024-04-02T04:47:48.711 INFO:tasks.workunit.client.0.smithi070.stdout:7/408: symlink d7/dd/df/d7a/d6c/d77/l7f 0 2024-04-02T04:47:48.715 INFO:tasks.workunit.client.0.smithi070.stdout:0/477: creat d4/d5/f84 x:0 0 0 2024-04-02T04:47:48.717 INFO:tasks.workunit.client.0.smithi070.stdout:8/431: rename d2/l40 to d2/d8/d23/d24/d79/l86 0 2024-04-02T04:47:48.719 INFO:tasks.workunit.client.0.smithi070.stdout:4/365: mkdir d1/d3/d78 0 2024-04-02T04:47:48.721 INFO:tasks.workunit.client.0.smithi070.stdout:2/450: sync 2024-04-02T04:47:48.724 INFO:tasks.workunit.client.0.smithi070.stdout:8/432: rmdir d2/d8/d29/d22 39 2024-04-02T04:47:48.728 INFO:tasks.workunit.client.0.smithi070.stdout:7/409: mkdir d7/dd/df/d7a/d6c/d77/d75/d80 0 2024-04-02T04:47:48.728 INFO:tasks.workunit.client.0.smithi070.stdout:8/433: dread d2/d8/f5e [0,4194304] 0 2024-04-02T04:47:48.729 INFO:tasks.workunit.client.0.smithi070.stdout:7/410: chown d7/dd/df/d79/d46/d65/f67 59806025 1 2024-04-02T04:47:48.730 INFO:tasks.workunit.client.0.smithi070.stdout:4/366: dread d1/d3/d9/f25 [0,4194304] 0 2024-04-02T04:47:48.731 INFO:tasks.workunit.client.0.smithi070.stdout:6/309: write d0/d3/d19/f3f [1424712,50554] 0 2024-04-02T04:47:48.734 INFO:tasks.workunit.client.0.smithi070.stdout:3/392: truncate d10/d32/d3f/f38 955384 0 2024-04-02T04:47:48.735 INFO:tasks.workunit.client.0.smithi070.stdout:8/434: creat d2/d3/d35/d3a/d4f/f87 x:0 0 0 2024-04-02T04:47:48.735 INFO:tasks.workunit.client.0.smithi070.stdout:3/393: dread - d10/d17/d51/f76 zero size 2024-04-02T04:47:48.736 INFO:tasks.workunit.client.0.smithi070.stdout:7/411: dread d7/dd/df/d7a/f5a [0,4194304] 0 2024-04-02T04:47:48.737 INFO:tasks.workunit.client.0.smithi070.stdout:2/451: rename d2/d12/f13 to d2/d24/f8d 0 2024-04-02T04:47:48.739 INFO:tasks.workunit.client.0.smithi070.stdout:0/478: dread d4/d5/d13/f19 [8388608,4194304] 0 2024-04-02T04:47:48.740 INFO:tasks.workunit.client.0.smithi070.stdout:6/310: mknod d0/d3/d19/d31/d37/c64 0 2024-04-02T04:47:48.743 INFO:tasks.workunit.client.0.smithi070.stdout:7/412: write d7/dd/df/d79/d46/d65/f71 [42963,6007] 0 2024-04-02T04:47:48.745 INFO:tasks.workunit.client.0.smithi070.stdout:5/448: dwrite d5/d17/d27/f7c [0,4194304] 0 2024-04-02T04:47:48.746 INFO:tasks.workunit.client.0.smithi070.stdout:2/452: creat d2/d12/d59/f8e x:0 0 0 2024-04-02T04:47:48.746 INFO:tasks.workunit.client.0.smithi070.stdout:6/311: write d0/d20/f45 [276935,116108] 0 2024-04-02T04:47:48.748 INFO:tasks.workunit.client.0.smithi070.stdout:8/435: dwrite d2/f7f [0,4194304] 0 2024-04-02T04:47:48.752 INFO:tasks.workunit.client.0.smithi070.stdout:8/436: write d2/d8/d23/d24/f34 [2522234,30502] 0 2024-04-02T04:47:48.752 INFO:tasks.workunit.client.0.smithi070.stdout:4/367: rename d1/d3/f67 to d1/d3/d9/d41/f79 0 2024-04-02T04:47:48.755 INFO:tasks.workunit.client.0.smithi070.stdout:2/453: creat d2/d24/f8f x:0 0 0 2024-04-02T04:47:48.755 INFO:tasks.workunit.client.0.smithi070.stdout:2/454: readlink d2/l86 0 2024-04-02T04:47:48.760 INFO:tasks.workunit.client.0.smithi070.stdout:0/479: creat d4/d5/d58/f85 x:0 0 0 2024-04-02T04:47:48.761 INFO:tasks.workunit.client.0.smithi070.stdout:6/312: creat d0/d3/d19/d39/f65 x:0 0 0 2024-04-02T04:47:48.761 INFO:tasks.workunit.client.0.smithi070.stdout:7/413: write d7/d9/fa [406855,129822] 0 2024-04-02T04:47:48.763 INFO:tasks.workunit.client.0.smithi070.stdout:5/449: creat d5/d17/d27/d2e/f91 x:0 0 0 2024-04-02T04:47:48.763 INFO:tasks.workunit.client.1.smithi179.stdout:6/410: sync 2024-04-02T04:47:48.764 INFO:tasks.workunit.client.0.smithi070.stdout:4/368: mkdir d1/d3/d9/d41/d65/d7a 0 2024-04-02T04:47:48.764 INFO:tasks.workunit.client.0.smithi070.stdout:3/394: dread f0 [0,4194304] 0 2024-04-02T04:47:48.764 INFO:tasks.workunit.client.0.smithi070.stdout:6/313: stat d0/d3/d19/d31/d46 0 2024-04-02T04:47:48.765 INFO:tasks.workunit.client.0.smithi070.stdout:7/414: write d7/dd/f38 [2614115,43569] 0 2024-04-02T04:47:48.770 INFO:tasks.workunit.client.0.smithi070.stdout:7/415: read d7/d9/fa [5181376,71831] 0 2024-04-02T04:47:48.771 INFO:tasks.workunit.client.0.smithi070.stdout:7/416: stat d7/l58 0 2024-04-02T04:47:48.772 INFO:tasks.workunit.client.1.smithi179.stdout:2/274: symlink dc/d29/d31/d39/l55 0 2024-04-02T04:47:48.772 INFO:tasks.workunit.client.1.smithi179.stdout:8/377: write d3/d32/d21/f37 [134912,6418] 0 2024-04-02T04:47:48.773 INFO:tasks.workunit.client.0.smithi070.stdout:8/437: rename d2/d8/c82 to d2/d3/d35/d3a/d4f/d59/c88 0 2024-04-02T04:47:48.775 INFO:tasks.workunit.client.1.smithi179.stdout:3/445: chown d8/c25 27 1 2024-04-02T04:47:48.775 INFO:tasks.workunit.client.1.smithi179.stdout:6/411: write d3/d42/f7e [639032,109148] 0 2024-04-02T04:47:48.775 INFO:tasks.workunit.client.0.smithi070.stdout:3/395: dwrite d10/f4f [0,4194304] 0 2024-04-02T04:47:48.776 INFO:tasks.workunit.client.1.smithi179.stdout:1/430: rename d3/d5/d2f/d4f/d42/d53/d5b/f69 to d3/d5/d2f/d36/f93 0 2024-04-02T04:47:48.777 INFO:tasks.workunit.client.1.smithi179.stdout:4/477: write d1/f3c [1030863,37203] 0 2024-04-02T04:47:48.777 INFO:tasks.workunit.client.1.smithi179.stdout:8/378: write d3/d32/d30/f4e [170067,59745] 0 2024-04-02T04:47:48.783 INFO:tasks.workunit.client.1.smithi179.stdout:2/275: truncate dc/d29/d4a/f27 156738 0 2024-04-02T04:47:48.785 INFO:tasks.workunit.client.0.smithi070.stdout:4/369: dread d1/d3/d9/f11 [0,4194304] 0 2024-04-02T04:47:48.785 INFO:tasks.workunit.client.0.smithi070.stdout:7/417: mknod d7/dd/df/d7a/d6c/d77/d75/c81 0 2024-04-02T04:47:48.787 INFO:tasks.workunit.client.0.smithi070.stdout:8/438: dwrite d2/f48 [0,4194304] 0 2024-04-02T04:47:48.787 INFO:tasks.workunit.client.1.smithi179.stdout:7/462: dwrite d1/d9/fb [0,4194304] 0 2024-04-02T04:47:48.790 INFO:tasks.workunit.client.0.smithi070.stdout:3/396: chown d10/d17/l2d 415915994 1 2024-04-02T04:47:48.790 INFO:tasks.workunit.client.1.smithi179.stdout:3/446: dread d8/d57/f5a [0,4194304] 0 2024-04-02T04:47:48.790 INFO:tasks.workunit.client.1.smithi179.stdout:6/412: dread d3/d2d/d43/f4b [0,4194304] 0 2024-04-02T04:47:48.791 INFO:tasks.workunit.client.1.smithi179.stdout:4/478: truncate d1/d3/dc/d27/d62/f4d 93862 0 2024-04-02T04:47:48.793 INFO:tasks.workunit.client.1.smithi179.stdout:6/413: chown d3/d12/l78 0 1 2024-04-02T04:47:48.794 INFO:tasks.workunit.client.1.smithi179.stdout:4/479: chown d1/d15/f31 14981001 1 2024-04-02T04:47:48.795 INFO:tasks.workunit.client.0.smithi070.stdout:9/354: sync 2024-04-02T04:47:48.795 INFO:tasks.workunit.client.0.smithi070.stdout:1/432: sync 2024-04-02T04:47:48.796 INFO:tasks.workunit.client.0.smithi070.stdout:3/397: dwrite d10/d17/d51/f64 [0,4194304] 0 2024-04-02T04:47:48.797 INFO:tasks.workunit.client.0.smithi070.stdout:1/433: write d0/d7/d28/f7a [487920,102497] 0 2024-04-02T04:47:48.798 INFO:tasks.workunit.client.0.smithi070.stdout:9/355: write d4/f36 [4915619,66531] 0 2024-04-02T04:47:48.799 INFO:tasks.workunit.client.1.smithi179.stdout:9/420: dread d0/d1/d10/f18 [4194304,4194304] 0 2024-04-02T04:47:48.803 INFO:tasks.workunit.client.0.smithi070.stdout:5/450: rmdir d5/d14/d7e 0 2024-04-02T04:47:48.804 INFO:tasks.workunit.client.0.smithi070.stdout:5/451: truncate d5/d66/d75/f72 262003 0 2024-04-02T04:47:48.806 INFO:tasks.workunit.client.1.smithi179.stdout:7/463: truncate d1/dd/d23/f57 566229 0 2024-04-02T04:47:48.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/452: stat d5/d9/d10/d6a/f81 0 2024-04-02T04:47:48.806 INFO:tasks.workunit.client.0.smithi070.stdout:4/370: mknod d1/d3/d9/d41/c7b 0 2024-04-02T04:47:48.807 INFO:tasks.workunit.client.0.smithi070.stdout:4/371: chown d1/d3/d9/d41/l5c 418 1 2024-04-02T04:47:48.813 INFO:tasks.workunit.client.0.smithi070.stdout:7/418: rename d7/dd/d66 to d7/d4e/d82 0 2024-04-02T04:47:48.815 INFO:tasks.workunit.client.1.smithi179.stdout:8/379: dread d3/d32/d21/d22/d33/d3c/f40 [0,4194304] 0 2024-04-02T04:47:48.816 INFO:tasks.workunit.client.0.smithi070.stdout:6/314: link d0/d3/d19/f1f d0/d3/d19/d31/d61/f66 0 2024-04-02T04:47:48.816 INFO:tasks.workunit.client.0.smithi070.stdout:2/455: write d2/d24/f8d [2050072,50580] 0 2024-04-02T04:47:48.820 INFO:tasks.workunit.client.1.smithi179.stdout:3/447: creat d8/d26/d52/d85/f8b x:0 0 0 2024-04-02T04:47:48.820 INFO:tasks.workunit.client.1.smithi179.stdout:6/414: dread d3/fa [0,4194304] 0 2024-04-02T04:47:48.822 INFO:tasks.workunit.client.1.smithi179.stdout:3/448: dread d8/f11 [0,4194304] 0 2024-04-02T04:47:48.822 INFO:tasks.workunit.client.1.smithi179.stdout:1/431: write d3/d5/f27 [2904738,60686] 0 2024-04-02T04:47:48.825 INFO:tasks.workunit.client.0.smithi070.stdout:0/480: getdents d4/d3c 0 2024-04-02T04:47:48.826 INFO:tasks.workunit.client.0.smithi070.stdout:0/481: rename d4/d5/d58/d60/d83 to d4/d5/d58/d60/d83/d86 22 2024-04-02T04:47:48.827 INFO:tasks.workunit.client.1.smithi179.stdout:9/421: link d0/d1/d10/f18 d0/d19/d1c/d64/f85 0 2024-04-02T04:47:48.828 INFO:tasks.workunit.client.1.smithi179.stdout:4/480: rename d1/d3/dc/d27/d62/f4d to d1/d3/df/d10/fa1 0 2024-04-02T04:47:48.829 INFO:tasks.workunit.client.0.smithi070.stdout:9/356: chown d4/d10/f25 18 1 2024-04-02T04:47:48.830 INFO:tasks.workunit.client.1.smithi179.stdout:8/380: dwrite d3/d1b/f23 [0,4194304] 0 2024-04-02T04:47:48.832 INFO:tasks.workunit.client.1.smithi179.stdout:7/464: creat d1/dd/d16/d74/d9f/fa3 x:0 0 0 2024-04-02T04:47:48.832 INFO:tasks.workunit.client.0.smithi070.stdout:8/439: symlink d2/d8/d29/l89 0 2024-04-02T04:47:48.835 INFO:tasks.workunit.client.1.smithi179.stdout:6/415: mknod d3/d12/d23/d39/d4e/d66/d7c/c94 0 2024-04-02T04:47:48.835 INFO:tasks.workunit.client.0.smithi070.stdout:7/419: rmdir d7/dd/df/d79/d46/d65 39 2024-04-02T04:47:48.838 INFO:tasks.workunit.client.1.smithi179.stdout:3/449: symlink d8/d26/d36/l8c 0 2024-04-02T04:47:48.840 INFO:tasks.workunit.client.1.smithi179.stdout:1/432: creat d3/d5/d2f/d36/f94 x:0 0 0 2024-04-02T04:47:48.840 INFO:tasks.workunit.client.0.smithi070.stdout:3/398: mknod d10/d17/c8c 0 2024-04-02T04:47:48.841 INFO:tasks.workunit.client.1.smithi179.stdout:6/416: dwrite d3/d12/d23/d39/f71 [0,4194304] 0 2024-04-02T04:47:48.843 INFO:tasks.workunit.client.1.smithi179.stdout:1/433: chown d3/d5/de/d33/l45 0 1 2024-04-02T04:47:48.845 INFO:tasks.workunit.client.0.smithi070.stdout:9/357: mkdir d4/d10/d12/d73 0 2024-04-02T04:47:48.845 INFO:tasks.workunit.client.1.smithi179.stdout:2/276: dwrite dc/d29/d4a/f27 [0,4194304] 0 2024-04-02T04:47:48.846 INFO:tasks.workunit.client.1.smithi179.stdout:2/277: dread - dc/d29/d4a/f3f zero size 2024-04-02T04:47:48.848 INFO:tasks.workunit.client.0.smithi070.stdout:9/358: chown d4/d10/d12/f67 69468 1 2024-04-02T04:47:48.848 INFO:tasks.workunit.client.1.smithi179.stdout:7/465: symlink d1/dd/d76/la4 0 2024-04-02T04:47:48.849 INFO:tasks.workunit.client.0.smithi070.stdout:1/434: dwrite d0/d9/fc [0,4194304] 0 2024-04-02T04:47:48.850 INFO:tasks.workunit.client.1.smithi179.stdout:5/355: sync 2024-04-02T04:47:48.852 INFO:tasks.workunit.client.1.smithi179.stdout:6/417: read d3/d12/d23/d26/f4f [1020995,47830] 0 2024-04-02T04:47:48.852 INFO:tasks.workunit.client.0.smithi070.stdout:9/359: write d4/d13/d16/d19/d53/f5a [298328,74544] 0 2024-04-02T04:47:48.853 INFO:tasks.workunit.client.0.smithi070.stdout:6/315: symlink d0/d3/d19/l67 0 2024-04-02T04:47:48.854 INFO:tasks.workunit.client.0.smithi070.stdout:6/316: dread - d0/d20/d5f/f62 zero size 2024-04-02T04:47:48.855 INFO:tasks.workunit.client.0.smithi070.stdout:6/317: chown d0/d3 7485242 1 2024-04-02T04:47:48.858 INFO:tasks.workunit.client.0.smithi070.stdout:7/420: dread d7/dd/df/d79/d46/d65/f69 [8388608,4194304] 0 2024-04-02T04:47:48.859 INFO:tasks.workunit.client.0.smithi070.stdout:7/421: readlink d7/dd/df/d34/l33 0 2024-04-02T04:47:48.860 INFO:tasks.workunit.client.1.smithi179.stdout:9/422: mkdir d0/d1/d11/d29/d86 0 2024-04-02T04:47:48.861 INFO:tasks.workunit.client.0.smithi070.stdout:0/482: fsync d4/d5/da/d2d/f7d 0 2024-04-02T04:47:48.863 INFO:tasks.workunit.client.0.smithi070.stdout:0/483: readlink d4/d3c/l5c 0 2024-04-02T04:47:48.863 INFO:tasks.workunit.client.0.smithi070.stdout:9/360: dwrite d4/d10/d12/d27/f5c [0,4194304] 0 2024-04-02T04:47:48.864 INFO:tasks.workunit.client.0.smithi070.stdout:0/484: write d4/d3c/f6a [1409484,57295] 0 2024-04-02T04:47:48.864 INFO:tasks.workunit.client.0.smithi070.stdout:9/361: chown d4/f36 4 1 2024-04-02T04:47:48.867 INFO:tasks.workunit.client.0.smithi070.stdout:0/485: write d4/d5/d13/d2a/d29/d52/f82 [324821,71790] 0 2024-04-02T04:47:48.869 INFO:tasks.workunit.client.1.smithi179.stdout:4/481: rename d1/d3/df/f59 to d1/d3/df/d10/d2f/d7b/fa2 0 2024-04-02T04:47:48.871 INFO:tasks.workunit.client.1.smithi179.stdout:4/482: chown d1/d3/df/d10/d2f/d60/f6c 102269 1 2024-04-02T04:47:48.872 INFO:tasks.workunit.client.0.smithi070.stdout:8/440: rename d2/c26 to d2/d8/d23/d24/d79/c8a 0 2024-04-02T04:47:48.872 INFO:tasks.workunit.client.0.smithi070.stdout:5/453: write d5/d9/d23/f49 [1078874,2083] 0 2024-04-02T04:47:48.875 INFO:tasks.workunit.client.0.smithi070.stdout:8/441: truncate d2/d8/d23/d24/d79/f4e 3387010 0 2024-04-02T04:47:48.875 INFO:tasks.workunit.client.0.smithi070.stdout:3/399: mkdir d10/d32/d8a/d8d 0 2024-04-02T04:47:48.875 INFO:tasks.workunit.client.1.smithi179.stdout:1/434: dread d3/f2d [0,4194304] 0 2024-04-02T04:47:48.877 INFO:tasks.workunit.client.0.smithi070.stdout:7/422: creat d7/dd/df/f83 x:0 0 0 2024-04-02T04:47:48.878 INFO:tasks.workunit.client.1.smithi179.stdout:2/278: stat dc/d29/d4a/d1e/c1f 0 2024-04-02T04:47:48.878 INFO:tasks.workunit.client.0.smithi070.stdout:8/442: write d2/d3/d52/f3c [1560974,63798] 0 2024-04-02T04:47:48.880 INFO:tasks.workunit.client.1.smithi179.stdout:5/356: creat d8/f6c x:0 0 0 2024-04-02T04:47:48.880 INFO:tasks.workunit.client.0.smithi070.stdout:5/454: dwrite d5/d9/d23/d30/f79 [0,4194304] 0 2024-04-02T04:47:48.883 INFO:tasks.workunit.client.0.smithi070.stdout:1/435: rename d0/d9/d1d/le to d0/d7/d28/l8a 0 2024-04-02T04:47:48.884 INFO:tasks.workunit.client.0.smithi070.stdout:3/400: unlink d10/d17/d51/c77 0 2024-04-02T04:47:48.888 INFO:tasks.workunit.client.1.smithi179.stdout:3/450: rename d8/d26/f5f to d8/d26/d36/f8d 0 2024-04-02T04:47:48.889 INFO:tasks.workunit.client.0.smithi070.stdout:0/486: mknod d4/d5/c87 0 2024-04-02T04:47:48.889 INFO:tasks.workunit.client.0.smithi070.stdout:0/487: stat f0 0 2024-04-02T04:47:48.890 INFO:tasks.workunit.client.0.smithi070.stdout:0/488: dread - d4/d5/f84 zero size 2024-04-02T04:47:48.892 INFO:tasks.workunit.client.0.smithi070.stdout:8/443: creat d2/d8/d29/db/f8b x:0 0 0 2024-04-02T04:47:48.893 INFO:tasks.workunit.client.1.smithi179.stdout:7/466: mknod d1/ca5 0 2024-04-02T04:47:48.895 INFO:tasks.workunit.client.0.smithi070.stdout:6/318: rename d0/f1c to d0/d3/d19/d31/d46/f68 0 2024-04-02T04:47:48.896 INFO:tasks.workunit.client.1.smithi179.stdout:6/418: fdatasync f2 0 2024-04-02T04:47:48.899 INFO:tasks.workunit.client.0.smithi070.stdout:9/362: link d4/d13/l46 d4/d13/d16/d19/d2f/d4d/l74 0 2024-04-02T04:47:48.899 INFO:tasks.workunit.client.1.smithi179.stdout:7/467: dwrite d1/f7b [0,4194304] 0 2024-04-02T04:47:48.903 INFO:tasks.workunit.client.0.smithi070.stdout:6/319: dwrite d0/d3/d19/d39/f4b [0,4194304] 0 2024-04-02T04:47:48.904 INFO:tasks.workunit.client.0.smithi070.stdout:6/320: chown d0/d20/d5f 5 1 2024-04-02T04:47:48.904 INFO:tasks.workunit.client.0.smithi070.stdout:6/321: stat d0/c33 0 2024-04-02T04:47:48.906 INFO:tasks.workunit.client.0.smithi070.stdout:6/322: write d0/d20/d5f/f62 [755042,32774] 0 2024-04-02T04:47:48.907 INFO:tasks.workunit.client.1.smithi179.stdout:4/483: link d1/d15/d6e/f80 d1/d3/d19/fa3 0 2024-04-02T04:47:48.908 INFO:tasks.workunit.client.1.smithi179.stdout:2/279: dread dc/f1a [0,4194304] 0 2024-04-02T04:47:48.908 INFO:tasks.workunit.client.1.smithi179.stdout:2/280: chown dc 234 1 2024-04-02T04:47:48.909 INFO:tasks.workunit.client.0.smithi070.stdout:6/323: write d0/d3/d19/d39/f4b [161418,47927] 0 2024-04-02T04:47:48.910 INFO:tasks.workunit.client.0.smithi070.stdout:6/324: chown d0/d3/d19/d31 238801 1 2024-04-02T04:47:48.911 INFO:tasks.workunit.client.0.smithi070.stdout:8/444: chown d2/d8/d29/d22/f46 21 1 2024-04-02T04:47:48.912 INFO:tasks.workunit.client.1.smithi179.stdout:8/381: truncate d3/d32/d21/d22/d33/f60 458173 0 2024-04-02T04:47:48.913 INFO:tasks.workunit.client.0.smithi070.stdout:5/455: getdents d5/d9/d10/d6a/d7d 0 2024-04-02T04:47:48.920 INFO:tasks.workunit.client.0.smithi070.stdout:6/325: dwrite d0/d3/f3a [0,4194304] 0 2024-04-02T04:47:48.920 INFO:tasks.workunit.client.0.smithi070.stdout:1/436: dread d0/d4c/f51 [0,4194304] 0 2024-04-02T04:47:48.927 INFO:tasks.workunit.client.1.smithi179.stdout:7/468: rename d1/l3 to d1/dd/d16/d4d/la6 0 2024-04-02T04:47:48.929 INFO:tasks.workunit.client.0.smithi070.stdout:0/489: mknod d4/d5/d58/c88 0 2024-04-02T04:47:48.933 INFO:tasks.workunit.client.0.smithi070.stdout:7/423: dwrite d7/d36/f62 [0,4194304] 0 2024-04-02T04:47:48.940 INFO:tasks.workunit.client.1.smithi179.stdout:9/423: truncate d0/d1/f36 3400810 0 2024-04-02T04:47:48.941 INFO:tasks.workunit.client.1.smithi179.stdout:3/451: write d8/f11 [1753255,66461] 0 2024-04-02T04:47:48.942 INFO:tasks.workunit.client.0.smithi070.stdout:7/424: dwrite d7/dd/df/d7a/d6c/f74 [0,4194304] 0 2024-04-02T04:47:48.944 INFO:tasks.workunit.client.0.smithi070.stdout:7/425: chown d7/dd/df/d7a 27982 1 2024-04-02T04:47:48.944 INFO:tasks.workunit.client.0.smithi070.stdout:5/456: write d5/d17/d2b/f8d [823776,11383] 0 2024-04-02T04:47:48.946 INFO:tasks.workunit.client.0.smithi070.stdout:5/457: stat d5/d9/d23/f6e 0 2024-04-02T04:47:48.948 INFO:tasks.workunit.client.0.smithi070.stdout:6/326: mknod d0/d55/d5c/c69 0 2024-04-02T04:47:48.951 INFO:tasks.workunit.client.1.smithi179.stdout:8/382: rename d3/d31/c85 to d3/d32/d21/d49/c87 0 2024-04-02T04:47:48.955 INFO:tasks.workunit.client.0.smithi070.stdout:1/437: symlink d0/d7/d1a/l8b 0 2024-04-02T04:47:48.959 INFO:tasks.workunit.client.1.smithi179.stdout:5/357: getdents d8/d1b 0 2024-04-02T04:47:48.961 INFO:tasks.workunit.client.0.smithi070.stdout:6/327: dread d0/d43/f4a [0,4194304] 0 2024-04-02T04:47:48.962 INFO:tasks.workunit.client.1.smithi179.stdout:9/424: dread d0/d19/d27/f68 [0,4194304] 0 2024-04-02T04:47:48.963 INFO:tasks.workunit.client.0.smithi070.stdout:0/490: read d4/d5/d13/d2a/d29/f71 [340956,58541] 0 2024-04-02T04:47:48.963 INFO:tasks.workunit.client.1.smithi179.stdout:9/425: chown d0/l3 40195944 1 2024-04-02T04:47:48.965 INFO:tasks.workunit.client.1.smithi179.stdout:3/452: fsync d8/d57/f5a 0 2024-04-02T04:47:48.970 INFO:tasks.workunit.client.1.smithi179.stdout:8/383: symlink d3/d32/d21/d45/d5a/l88 0 2024-04-02T04:47:48.971 INFO:tasks.workunit.client.1.smithi179.stdout:6/419: truncate d3/fb 5579890 0 2024-04-02T04:47:48.972 INFO:tasks.workunit.client.1.smithi179.stdout:3/453: dwrite d8/d26/d36/f59 [0,4194304] 0 2024-04-02T04:47:48.973 INFO:tasks.workunit.client.1.smithi179.stdout:6/420: readlink d3/d12/d44/l67 0 2024-04-02T04:47:48.974 INFO:tasks.workunit.client.0.smithi070.stdout:7/426: symlink d7/dd/df/d7a/l84 0 2024-04-02T04:47:48.978 INFO:tasks.workunit.client.1.smithi179.stdout:2/281: creat dc/d29/d4a/f56 x:0 0 0 2024-04-02T04:47:48.979 INFO:tasks.workunit.client.0.smithi070.stdout:3/401: getdents d10/d17/d51/d73 0 2024-04-02T04:47:48.980 INFO:tasks.workunit.client.1.smithi179.stdout:5/358: stat d8/d1b/d23/d32/f3d 0 2024-04-02T04:47:48.982 INFO:tasks.workunit.client.1.smithi179.stdout:0/462: sync 2024-04-02T04:47:48.984 INFO:tasks.workunit.client.0.smithi070.stdout:9/363: rmdir d4/d10/d12/d73 0 2024-04-02T04:47:48.984 INFO:tasks.workunit.client.0.smithi070.stdout:9/364: chown d4/d10/d12/d56 0 1 2024-04-02T04:47:48.985 INFO:tasks.workunit.client.1.smithi179.stdout:9/426: mkdir d0/d19/d27/d7a/d87 0 2024-04-02T04:47:48.986 INFO:tasks.workunit.client.0.smithi070.stdout:9/365: write d4/d10/d12/d27/f5c [3961055,80380] 0 2024-04-02T04:47:48.988 INFO:tasks.workunit.client.1.smithi179.stdout:8/384: creat d3/d6c/f89 x:0 0 0 2024-04-02T04:47:48.992 INFO:tasks.workunit.client.0.smithi070.stdout:7/427: creat d7/dd/df/d7a/f85 x:0 0 0 2024-04-02T04:47:48.995 INFO:tasks.workunit.client.1.smithi179.stdout:4/484: getdents d1/d3/dc 0 2024-04-02T04:47:48.996 INFO:tasks.workunit.client.0.smithi070.stdout:1/438: mkdir d0/d9/d1d/d22/d2c/d4f/d77/d8c 0 2024-04-02T04:47:48.998 INFO:tasks.workunit.client.1.smithi179.stdout:7/469: write d1/d9/d27/d87/d8b/f9c [1587391,122260] 0 2024-04-02T04:47:48.999 INFO:tasks.workunit.client.1.smithi179.stdout:7/470: chown d1/dd/d23/f2f 10601733 1 2024-04-02T04:47:49.000 INFO:tasks.workunit.client.1.smithi179.stdout:7/471: write d1/d9/d55/f9e [769093,38601] 0 2024-04-02T04:47:49.000 INFO:tasks.workunit.client.1.smithi179.stdout:2/282: mkdir dc/d29/d31/d57 0 2024-04-02T04:47:49.001 INFO:tasks.workunit.client.0.smithi070.stdout:5/458: write d5/f2f [4587737,16374] 0 2024-04-02T04:47:49.003 INFO:tasks.workunit.client.0.smithi070.stdout:9/366: truncate d4/d10/d11/f2d 1651858 0 2024-04-02T04:47:49.005 INFO:tasks.workunit.client.0.smithi070.stdout:0/491: creat d4/d5/da/d2d/f89 x:0 0 0 2024-04-02T04:47:49.006 INFO:tasks.workunit.client.0.smithi070.stdout:0/492: dread - d4/d5/d13/d2a/d29/d52/f69 zero size 2024-04-02T04:47:49.007 INFO:tasks.workunit.client.1.smithi179.stdout:9/427: unlink d0/c5a 0 2024-04-02T04:47:49.009 INFO:tasks.workunit.client.0.smithi070.stdout:9/367: dread d4/d10/d12/d55/d66/f6d [0,4194304] 0 2024-04-02T04:47:49.009 INFO:tasks.workunit.client.1.smithi179.stdout:6/421: dread d3/f18 [0,4194304] 0 2024-04-02T04:47:49.010 INFO:tasks.workunit.client.1.smithi179.stdout:8/385: unlink d3/d32/l2c 0 2024-04-02T04:47:49.010 INFO:tasks.workunit.client.1.smithi179.stdout:7/472: dwrite d1/d9/d55/f9e [0,4194304] 0 2024-04-02T04:47:49.011 INFO:tasks.workunit.client.0.smithi070.stdout:8/445: rename d2/d8/d23/d24/d79/l86 to d2/d8/d29/l8c 0 2024-04-02T04:47:49.013 INFO:tasks.workunit.client.0.smithi070.stdout:7/428: mknod d7/dd/df/d34/c86 0 2024-04-02T04:47:49.014 INFO:tasks.workunit.client.0.smithi070.stdout:7/429: stat d7/dd/df/d79/d46 0 2024-04-02T04:47:49.019 INFO:tasks.workunit.client.0.smithi070.stdout:5/459: creat d5/d17/d2b/d7b/f92 x:0 0 0 2024-04-02T04:47:49.019 INFO:tasks.workunit.client.1.smithi179.stdout:4/485: symlink d1/d3/df/d10/d2f/d60/la4 0 2024-04-02T04:47:49.020 INFO:tasks.workunit.client.1.smithi179.stdout:4/486: fsync d1/d3/dc/fd 0 2024-04-02T04:47:49.021 INFO:tasks.workunit.client.0.smithi070.stdout:8/446: dwrite d2/d3/d35/d3a/d4f/d59/f6e [0,4194304] 0 2024-04-02T04:47:49.022 INFO:tasks.workunit.client.1.smithi179.stdout:2/283: write dc/f12 [8623668,123846] 0 2024-04-02T04:47:49.023 INFO:tasks.workunit.client.1.smithi179.stdout:2/284: write dc/f46 [306290,29928] 0 2024-04-02T04:47:49.024 INFO:tasks.workunit.client.0.smithi070.stdout:9/368: symlink d4/d10/d11/l75 0 2024-04-02T04:47:49.027 INFO:tasks.workunit.client.0.smithi070.stdout:3/402: rename d10/d17/c7e to d10/d32/d8a/c8e 0 2024-04-02T04:47:49.028 INFO:tasks.workunit.client.0.smithi070.stdout:1/439: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/l8d 0 2024-04-02T04:47:49.036 INFO:tasks.workunit.client.1.smithi179.stdout:9/428: dread d0/f4c [0,4194304] 0 2024-04-02T04:47:49.037 INFO:tasks.workunit.client.1.smithi179.stdout:9/429: read d0/d1/f7d [346029,28225] 0 2024-04-02T04:47:49.039 INFO:tasks.workunit.client.0.smithi070.stdout:6/328: getdents d0/d3/d19 0 2024-04-02T04:47:49.042 INFO:tasks.workunit.client.0.smithi070.stdout:9/369: rename d4/l70 to d4/d13/d16/d19/d53/l76 0 2024-04-02T04:47:49.044 INFO:tasks.workunit.client.0.smithi070.stdout:3/403: mkdir d10/d17/d51/d73/d8f 0 2024-04-02T04:47:49.045 INFO:tasks.workunit.client.1.smithi179.stdout:0/463: truncate d1/da/d48/d5c/d83/f8b 511664 0 2024-04-02T04:47:49.045 INFO:tasks.workunit.client.0.smithi070.stdout:9/370: read d4/d13/d16/f35 [2160188,15677] 0 2024-04-02T04:47:49.046 INFO:tasks.workunit.client.0.smithi070.stdout:1/440: symlink d0/d9/l8e 0 2024-04-02T04:47:49.049 INFO:tasks.workunit.client.1.smithi179.stdout:7/473: symlink d1/d9/d35/la7 0 2024-04-02T04:47:49.050 INFO:tasks.workunit.client.0.smithi070.stdout:3/404: dwrite d10/d17/d51/f76 [0,4194304] 0 2024-04-02T04:47:49.052 INFO:tasks.workunit.client.0.smithi070.stdout:0/493: creat d4/d5/d13/f8a x:0 0 0 2024-04-02T04:47:49.052 INFO:tasks.workunit.client.0.smithi070.stdout:5/460: write d5/fa [738439,1181] 0 2024-04-02T04:47:49.053 INFO:tasks.workunit.client.1.smithi179.stdout:8/386: mknod d3/d32/d21/d22/d33/c8a 0 2024-04-02T04:47:49.053 INFO:tasks.workunit.client.0.smithi070.stdout:6/329: mknod d0/d3/d19/d31/d46/c6a 0 2024-04-02T04:47:49.054 INFO:tasks.workunit.client.0.smithi070.stdout:6/330: fdatasync d0/d20/f27 0 2024-04-02T04:47:49.054 INFO:tasks.workunit.client.1.smithi179.stdout:8/387: read d3/d32/d21/d50/f5f [36787,89493] 0 2024-04-02T04:47:49.061 INFO:tasks.workunit.client.0.smithi070.stdout:1/441: mknod d0/d4c/d5a/c8f 0 2024-04-02T04:47:49.063 INFO:tasks.workunit.client.0.smithi070.stdout:1/442: write d0/d9/d1d/d22/d2c/f4a [512304,9076] 0 2024-04-02T04:47:49.065 INFO:tasks.workunit.client.1.smithi179.stdout:1/435: sync 2024-04-02T04:47:49.072 INFO:tasks.workunit.client.0.smithi070.stdout:3/405: symlink d10/d32/l90 0 2024-04-02T04:47:49.074 INFO:tasks.workunit.client.0.smithi070.stdout:9/371: dread d4/d13/d16/d19/d53/f59 [0,4194304] 0 2024-04-02T04:47:49.078 INFO:tasks.workunit.client.0.smithi070.stdout:0/494: rename d4/d5/c87 to d4/d5/d13/d2a/d29/c8b 0 2024-04-02T04:47:49.079 INFO:tasks.workunit.client.1.smithi179.stdout:7/474: rmdir d1/d9/d27/d87/d8b 39 2024-04-02T04:47:49.079 INFO:tasks.workunit.client.1.smithi179.stdout:1/436: dread d3/f28 [4194304,4194304] 0 2024-04-02T04:47:49.080 INFO:tasks.workunit.client.0.smithi070.stdout:6/331: mkdir d0/d3/d19/d31/d46/d6b 0 2024-04-02T04:47:49.082 INFO:tasks.workunit.client.0.smithi070.stdout:1/443: dread d0/d7/f8 [4194304,4194304] 0 2024-04-02T04:47:49.083 INFO:tasks.workunit.client.1.smithi179.stdout:3/454: rename d8/d7a/c80 to d8/c8e 0 2024-04-02T04:47:49.083 INFO:tasks.workunit.client.0.smithi070.stdout:1/444: fsync d0/d7/d1a/d66/f84 0 2024-04-02T04:47:49.083 INFO:tasks.workunit.client.1.smithi179.stdout:3/455: write d8/d26/f62 [693740,66232] 0 2024-04-02T04:47:49.086 INFO:tasks.workunit.client.1.smithi179.stdout:6/422: truncate d3/f18 797020 0 2024-04-02T04:47:49.087 INFO:tasks.workunit.client.1.smithi179.stdout:6/423: write d3/d12/d23/d26/d57/f8a [792278,92269] 0 2024-04-02T04:47:49.088 INFO:tasks.workunit.client.1.smithi179.stdout:8/388: creat d3/d1b/d2a/f8b x:0 0 0 2024-04-02T04:47:49.088 INFO:tasks.workunit.client.0.smithi070.stdout:5/461: dwrite d5/d9/d23/f48 [0,4194304] 0 2024-04-02T04:47:49.091 INFO:tasks.workunit.client.1.smithi179.stdout:8/389: read d3/d32/d21/d49/f72 [345920,90523] 0 2024-04-02T04:47:49.092 INFO:tasks.workunit.client.1.smithi179.stdout:8/390: chown d3/c3d 24431 1 2024-04-02T04:47:49.092 INFO:tasks.workunit.client.1.smithi179.stdout:5/359: getdents d8/d1b/d23/d36/d53/d62 0 2024-04-02T04:47:49.093 INFO:tasks.workunit.client.0.smithi070.stdout:7/430: link d7/dd/df/d79/c3d d7/d4e/d82/c87 0 2024-04-02T04:47:49.094 INFO:tasks.workunit.client.1.smithi179.stdout:5/360: rename d8/d1b/d23/d36/d53/d62 to d8/d1b/d23/d36/d53/d62/d6d 22 2024-04-02T04:47:49.095 INFO:tasks.workunit.client.1.smithi179.stdout:5/361: chown d8/d1b/d23/f47 213 1 2024-04-02T04:47:49.096 INFO:tasks.workunit.client.0.smithi070.stdout:0/495: fdatasync d4/d5/f5e 0 2024-04-02T04:47:49.098 INFO:tasks.workunit.client.1.smithi179.stdout:5/362: write d8/d1b/d23/f69 [780638,82864] 0 2024-04-02T04:47:49.098 INFO:tasks.workunit.client.0.smithi070.stdout:8/447: getdents d2/d3/d35/d3a/d4f 0 2024-04-02T04:47:49.101 INFO:tasks.workunit.client.1.smithi179.stdout:7/475: mkdir d1/dd/d16/d74/d9f/da8 0 2024-04-02T04:47:49.101 INFO:tasks.workunit.client.0.smithi070.stdout:1/445: mknod d0/d9/d1d/d22/d56/d79/c90 0 2024-04-02T04:47:49.101 INFO:tasks.workunit.client.0.smithi070.stdout:6/332: dread d0/d43/f51 [0,4194304] 0 2024-04-02T04:47:49.102 INFO:tasks.workunit.client.0.smithi070.stdout:6/333: chown d0/d43/f4a 0 1 2024-04-02T04:47:49.106 INFO:tasks.workunit.client.0.smithi070.stdout:3/406: mkdir d10/d32/d8a/d8d/d91 0 2024-04-02T04:47:49.110 INFO:tasks.workunit.client.1.smithi179.stdout:3/456: dwrite d8/d26/d36/f38 [0,4194304] 0 2024-04-02T04:47:49.110 INFO:tasks.workunit.client.1.smithi179.stdout:4/487: rmdir d1/d93/da0 0 2024-04-02T04:47:49.116 INFO:tasks.workunit.client.1.smithi179.stdout:9/430: dwrite d0/d1/d11/d29/f37 [4194304,4194304] 0 2024-04-02T04:47:49.116 INFO:tasks.workunit.client.0.smithi070.stdout:0/496: rmdir d4/d5/d13 39 2024-04-02T04:47:49.117 INFO:tasks.workunit.client.1.smithi179.stdout:9/431: write d0/d1/d11/f32 [1271795,77784] 0 2024-04-02T04:47:49.118 INFO:tasks.workunit.client.1.smithi179.stdout:9/432: chown d0/d1/d11/f78 101101 1 2024-04-02T04:47:49.121 INFO:tasks.workunit.client.0.smithi070.stdout:8/448: symlink d2/d3/d35/d3a/d4f/d59/l8d 0 2024-04-02T04:47:49.125 INFO:tasks.workunit.client.0.smithi070.stdout:3/407: write d10/d32/d3f/f43 [3786715,84290] 0 2024-04-02T04:47:49.131 INFO:tasks.workunit.client.0.smithi070.stdout:8/449: fdatasync d2/d3/fd 0 2024-04-02T04:47:49.132 INFO:tasks.workunit.client.1.smithi179.stdout:4/488: mknod d1/d3/dc/d27/ca5 0 2024-04-02T04:47:49.133 INFO:tasks.workunit.client.0.smithi070.stdout:3/408: dread d10/d32/f42 [0,4194304] 0 2024-04-02T04:47:49.135 INFO:tasks.workunit.client.0.smithi070.stdout:6/334: link d0/d3/d19/d31/f4e d0/d3/d19/d39/f6c 0 2024-04-02T04:47:49.135 INFO:tasks.workunit.client.0.smithi070.stdout:8/450: dread d2/d8/f44 [0,4194304] 0 2024-04-02T04:47:49.136 INFO:tasks.workunit.client.1.smithi179.stdout:3/457: dwrite d8/d26/f33 [4194304,4194304] 0 2024-04-02T04:47:49.136 INFO:tasks.workunit.client.1.smithi179.stdout:2/285: getdents dc/d29/d31 0 2024-04-02T04:47:49.137 INFO:tasks.workunit.client.0.smithi070.stdout:6/335: write d0/d3/d19/d31/d37/f5b [90959,88154] 0 2024-04-02T04:47:49.137 INFO:tasks.workunit.client.0.smithi070.stdout:8/451: truncate d2/d8/f6d 2716 0 2024-04-02T04:47:49.137 INFO:tasks.workunit.client.1.smithi179.stdout:7/476: write d1/f32 [1721005,18962] 0 2024-04-02T04:47:49.138 INFO:tasks.workunit.client.1.smithi179.stdout:7/477: readlink d1/dd/d23/l29 0 2024-04-02T04:47:49.140 INFO:tasks.workunit.client.0.smithi070.stdout:9/372: getdents d4/d13/d16/d19 0 2024-04-02T04:47:49.141 INFO:tasks.workunit.client.1.smithi179.stdout:0/464: getdents d1/da/d52 0 2024-04-02T04:47:49.145 INFO:tasks.workunit.client.1.smithi179.stdout:1/437: creat d3/f95 x:0 0 0 2024-04-02T04:47:49.145 INFO:tasks.workunit.client.1.smithi179.stdout:9/433: dread d0/d1/d11/d29/f37 [0,4194304] 0 2024-04-02T04:47:49.146 INFO:tasks.workunit.client.1.smithi179.stdout:5/363: unlink d8/d1b/d23/d42/c4f 0 2024-04-02T04:47:49.146 INFO:tasks.workunit.client.1.smithi179.stdout:9/434: write d0/d19/d1c/f6b [873727,18388] 0 2024-04-02T04:47:49.148 INFO:tasks.workunit.client.1.smithi179.stdout:9/435: chown d0/d1/d11/d29/d40/f67 80210485 1 2024-04-02T04:47:49.148 INFO:tasks.workunit.client.0.smithi070.stdout:7/431: rename d7/l2c to d7/dd/df/d79/l88 0 2024-04-02T04:47:49.149 INFO:tasks.workunit.client.1.smithi179.stdout:4/489: rmdir d1/d3 39 2024-04-02T04:47:49.150 INFO:tasks.workunit.client.0.smithi070.stdout:8/452: fsync d2/d8/f5e 0 2024-04-02T04:47:49.152 INFO:tasks.workunit.client.0.smithi070.stdout:9/373: truncate d4/fc 2396402 0 2024-04-02T04:47:49.156 INFO:tasks.workunit.client.0.smithi070.stdout:1/446: getdents d0/d4c/d5a 0 2024-04-02T04:47:49.159 INFO:tasks.workunit.client.0.smithi070.stdout:3/409: rename d10/d32/d3f/f6a to d10/d32/d8a/d8d/f92 0 2024-04-02T04:47:49.160 INFO:tasks.workunit.client.0.smithi070.stdout:7/432: truncate d7/d9/f27 155819 0 2024-04-02T04:47:49.160 INFO:tasks.workunit.client.0.smithi070.stdout:3/410: write d10/d32/d4b/f61 [4434769,125688] 0 2024-04-02T04:47:49.164 INFO:tasks.workunit.client.0.smithi070.stdout:8/453: mkdir d2/d3/d35/d8e 0 2024-04-02T04:47:49.166 INFO:tasks.workunit.client.0.smithi070.stdout:8/454: write d2/f7f [5048347,65917] 0 2024-04-02T04:47:49.168 INFO:tasks.workunit.client.0.smithi070.stdout:3/411: dwrite d10/d17/d51/d73/f80 [0,4194304] 0 2024-04-02T04:47:49.169 INFO:tasks.workunit.client.0.smithi070.stdout:3/412: dread - d10/d32/f63 zero size 2024-04-02T04:47:49.170 INFO:tasks.workunit.client.0.smithi070.stdout:3/413: fdatasync d10/d32/d3f/f4d 0 2024-04-02T04:47:49.172 INFO:tasks.workunit.client.0.smithi070.stdout:0/497: rename d4/d5/d13/f2b to d4/d5/d21/f8c 0 2024-04-02T04:47:49.173 INFO:tasks.workunit.client.0.smithi070.stdout:7/433: creat d7/d4e/f89 x:0 0 0 2024-04-02T04:47:49.180 INFO:tasks.workunit.client.0.smithi070.stdout:9/374: rename d4/f36 to d4/d13/d16/d19/d2f/d39/d4f/f77 0 2024-04-02T04:47:49.181 INFO:tasks.workunit.client.0.smithi070.stdout:0/498: mkdir d4/d5/d13/d2a/d8d 0 2024-04-02T04:47:49.184 INFO:tasks.workunit.client.0.smithi070.stdout:8/455: rename d2/d8/d23/d24/f34 to d2/d3/d35/d3a/d4f/d59/f8f 0 2024-04-02T04:47:49.184 INFO:tasks.workunit.client.0.smithi070.stdout:8/456: readlink d2/d8/l2e 0 2024-04-02T04:47:49.184 INFO:tasks.workunit.client.0.smithi070.stdout:9/375: creat d4/d13/d16/d19/d2f/d39/d4f/f78 x:0 0 0 2024-04-02T04:47:49.187 INFO:tasks.workunit.client.0.smithi070.stdout:0/499: creat d4/d5/d13/d2a/d3a/f8e x:0 0 0 2024-04-02T04:47:49.188 INFO:tasks.workunit.client.0.smithi070.stdout:8/457: read d2/d8/f44 [1219784,9179] 0 2024-04-02T04:47:49.189 INFO:tasks.workunit.client.0.smithi070.stdout:1/447: truncate d0/d9/f3e 2318494 0 2024-04-02T04:47:49.190 INFO:tasks.workunit.client.0.smithi070.stdout:9/376: mkdir d4/d10/d12/d56/d79 0 2024-04-02T04:47:49.190 INFO:tasks.workunit.client.0.smithi070.stdout:1/448: write d0/d9/d1d/f35 [607081,93001] 0 2024-04-02T04:47:49.192 INFO:tasks.workunit.client.0.smithi070.stdout:1/449: chown d0/d4c/f70 1 1 2024-04-02T04:47:49.193 INFO:tasks.workunit.client.0.smithi070.stdout:4/372: sync 2024-04-02T04:47:49.193 INFO:tasks.workunit.client.0.smithi070.stdout:3/414: rename d10/d17/l57 to d10/d32/d8a/l93 0 2024-04-02T04:47:49.195 INFO:tasks.workunit.client.0.smithi070.stdout:9/377: write d4/d10/f18 [6375415,56943] 0 2024-04-02T04:47:49.196 INFO:tasks.workunit.client.0.smithi070.stdout:3/415: chown d10/f86 3 1 2024-04-02T04:47:49.198 INFO:tasks.workunit.client.0.smithi070.stdout:4/373: mknod d1/d3/d9/d41/d65/d7a/c7c 0 2024-04-02T04:47:49.201 INFO:tasks.workunit.client.0.smithi070.stdout:8/458: dread d2/d3/d35/d3a/d4f/d59/f8f [0,4194304] 0 2024-04-02T04:47:49.210 INFO:tasks.workunit.client.1.smithi179.stdout:2/286: write dc/f1a [4710692,55364] 0 2024-04-02T04:47:49.211 INFO:tasks.workunit.client.1.smithi179.stdout:3/458: write d8/d57/d63/f6c [314954,93682] 0 2024-04-02T04:47:49.211 INFO:tasks.workunit.client.1.smithi179.stdout:0/465: write d1/da/df/f11 [1596348,11205] 0 2024-04-02T04:47:49.213 INFO:tasks.workunit.client.1.smithi179.stdout:5/364: write d8/f1a [304658,27885] 0 2024-04-02T04:47:49.215 INFO:tasks.workunit.client.1.smithi179.stdout:9/436: dwrite d0/d1/d11/d3a/f81 [0,4194304] 0 2024-04-02T04:47:49.219 INFO:tasks.workunit.client.1.smithi179.stdout:4/490: dwrite d1/d3/df/f51 [0,4194304] 0 2024-04-02T04:47:49.220 INFO:tasks.workunit.client.1.smithi179.stdout:4/491: readlink d1/d3/dc/d27/d62/l73 0 2024-04-02T04:47:49.221 INFO:tasks.workunit.client.1.smithi179.stdout:4/492: chown d1/d3/dc/d27/f7f 254231 1 2024-04-02T04:47:49.222 INFO:tasks.workunit.client.1.smithi179.stdout:4/493: chown d1/fb 52 1 2024-04-02T04:47:49.239 INFO:tasks.workunit.client.1.smithi179.stdout:7/478: mknod d1/ca9 0 2024-04-02T04:47:49.240 INFO:tasks.workunit.client.0.smithi070.stdout:4/374: rename d1/d3/d9/d41/f66 to d1/d3/d9/d41/d65/d7a/f7d 0 2024-04-02T04:47:49.245 INFO:tasks.workunit.client.0.smithi070.stdout:0/500: write d4/f9 [2715213,34785] 0 2024-04-02T04:47:49.246 INFO:tasks.workunit.client.0.smithi070.stdout:0/501: dread - d4/d5/d13/d43/f6f zero size 2024-04-02T04:47:49.246 INFO:tasks.workunit.client.1.smithi179.stdout:1/438: mkdir d3/d5/d96 0 2024-04-02T04:47:49.247 INFO:tasks.workunit.client.0.smithi070.stdout:4/375: symlink d1/d3/d9/d41/d65/l7e 0 2024-04-02T04:47:49.249 INFO:tasks.workunit.client.1.smithi179.stdout:2/287: unlink dc/l54 0 2024-04-02T04:47:49.250 INFO:tasks.workunit.client.1.smithi179.stdout:3/459: chown d8/c8e 7 1 2024-04-02T04:47:49.251 INFO:tasks.workunit.client.0.smithi070.stdout:9/378: truncate d4/d10/d12/d27/f5c 3146112 0 2024-04-02T04:47:49.251 INFO:tasks.workunit.client.1.smithi179.stdout:3/460: write d8/d26/d36/f38 [2431774,100409] 0 2024-04-02T04:47:49.252 INFO:tasks.workunit.client.1.smithi179.stdout:3/461: write d8/d57/d63/f6b [481232,59784] 0 2024-04-02T04:47:49.253 INFO:tasks.workunit.client.1.smithi179.stdout:3/462: fdatasync d8/d26/f2c 0 2024-04-02T04:47:49.253 INFO:tasks.workunit.client.0.smithi070.stdout:9/379: dread - d4/d13/d16/d19/f54 zero size 2024-04-02T04:47:49.254 INFO:tasks.workunit.client.1.smithi179.stdout:3/463: chown d8/d26/f62 1094 1 2024-04-02T04:47:49.254 INFO:tasks.workunit.client.1.smithi179.stdout:3/464: chown d8/d57/f5a 1239 1 2024-04-02T04:47:49.255 INFO:tasks.workunit.client.0.smithi070.stdout:3/416: dwrite d10/d32/d3f/f38 [0,4194304] 0 2024-04-02T04:47:49.256 INFO:tasks.workunit.client.1.smithi179.stdout:3/465: write d8/f1b [2070948,104047] 0 2024-04-02T04:47:49.257 INFO:tasks.workunit.client.0.smithi070.stdout:3/417: chown d10/d17/d51/d73/f80 211890045 1 2024-04-02T04:47:49.262 INFO:tasks.workunit.client.1.smithi179.stdout:5/365: truncate d8/d1b/d23/d32/f3d 801455 0 2024-04-02T04:47:49.262 INFO:tasks.workunit.client.0.smithi070.stdout:8/459: dwrite d2/d3/f7 [0,4194304] 0 2024-04-02T04:47:49.266 INFO:tasks.workunit.client.0.smithi070.stdout:0/502: write d4/d5/d13/d43/f49 [4399593,60487] 0 2024-04-02T04:47:49.267 INFO:tasks.workunit.client.1.smithi179.stdout:6/424: sync 2024-04-02T04:47:49.269 INFO:tasks.workunit.client.1.smithi179.stdout:7/479: mknod d1/d9/d7e/caa 0 2024-04-02T04:47:49.271 INFO:tasks.workunit.client.1.smithi179.stdout:7/480: read - d1/dd/f99 zero size 2024-04-02T04:47:49.272 INFO:tasks.workunit.client.0.smithi070.stdout:9/380: unlink d4/d13/d16/d19/f2e 0 2024-04-02T04:47:49.272 INFO:tasks.workunit.client.0.smithi070.stdout:4/376: write d1/fa [101458,56064] 0 2024-04-02T04:47:49.272 INFO:tasks.workunit.client.0.smithi070.stdout:9/381: stat d4/d10/d12/c38 0 2024-04-02T04:47:49.275 INFO:tasks.workunit.client.1.smithi179.stdout:6/425: dwrite d3/d12/d23/d26/d57/f8a [0,4194304] 0 2024-04-02T04:47:49.276 INFO:tasks.workunit.client.1.smithi179.stdout:2/288: read dc/d29/d4a/d1e/f36 [197563,116988] 0 2024-04-02T04:47:49.277 INFO:tasks.workunit.client.0.smithi070.stdout:4/377: dwrite d1/f32 [0,4194304] 0 2024-04-02T04:47:49.281 INFO:tasks.workunit.client.1.smithi179.stdout:3/466: mkdir d8/d57/d8f 0 2024-04-02T04:47:49.283 INFO:tasks.workunit.client.0.smithi070.stdout:3/418: creat d10/d32/d8a/d8d/d91/f94 x:0 0 0 2024-04-02T04:47:49.284 INFO:tasks.workunit.client.0.smithi070.stdout:3/419: readlink d10/l18 0 2024-04-02T04:47:49.287 INFO:tasks.workunit.client.1.smithi179.stdout:5/366: mkdir d8/d1b/d23/d32/d6e 0 2024-04-02T04:47:49.291 INFO:tasks.workunit.client.0.smithi070.stdout:0/503: creat d4/d5/d13/d2a/d8d/f8f x:0 0 0 2024-04-02T04:47:49.292 INFO:tasks.workunit.client.0.smithi070.stdout:8/460: dwrite d2/d8/d23/d24/f67 [0,4194304] 0 2024-04-02T04:47:49.292 INFO:tasks.workunit.client.1.smithi179.stdout:1/439: creat d3/d5/d2f/d4f/d42/d53/d77/f97 x:0 0 0 2024-04-02T04:47:49.294 INFO:tasks.workunit.client.0.smithi070.stdout:4/378: symlink d1/d3/d9/l7f 0 2024-04-02T04:47:49.297 INFO:tasks.workunit.client.1.smithi179.stdout:7/481: rmdir d1/d84 39 2024-04-02T04:47:49.297 INFO:tasks.workunit.client.0.smithi070.stdout:3/420: mknod d10/d17/d4c/c95 0 2024-04-02T04:47:49.299 INFO:tasks.workunit.client.0.smithi070.stdout:3/421: stat d10/d32/d4b/c7f 0 2024-04-02T04:47:49.302 INFO:tasks.workunit.client.0.smithi070.stdout:8/461: creat d2/d3/d35/d3a/d4f/d59/f90 x:0 0 0 2024-04-02T04:47:49.302 INFO:tasks.workunit.client.0.smithi070.stdout:0/504: dwrite d4/d5/d13/d43/f6f [0,4194304] 0 2024-04-02T04:47:49.305 INFO:tasks.workunit.client.0.smithi070.stdout:4/379: dwrite d1/fe [0,4194304] 0 2024-04-02T04:47:49.305 INFO:tasks.workunit.client.1.smithi179.stdout:4/494: write d1/d3/df/d10/d2f/d7b/fa2 [982455,53528] 0 2024-04-02T04:47:49.307 INFO:tasks.workunit.client.1.smithi179.stdout:0/466: dwrite d1/da/d28/d37/f49 [0,4194304] 0 2024-04-02T04:47:49.310 INFO:tasks.workunit.client.0.smithi070.stdout:3/422: dwrite d10/d17/d4c/f62 [0,4194304] 0 2024-04-02T04:47:49.313 INFO:tasks.workunit.client.0.smithi070.stdout:8/462: symlink d2/d3/d35/d3a/l91 0 2024-04-02T04:47:49.314 INFO:tasks.workunit.client.1.smithi179.stdout:2/289: dwrite dc/d29/d4a/d1e/d43/f47 [0,4194304] 0 2024-04-02T04:47:49.314 INFO:tasks.workunit.client.0.smithi070.stdout:9/382: getdents d4/d13 0 2024-04-02T04:47:49.319 INFO:tasks.workunit.client.1.smithi179.stdout:2/290: write dc/d29/d4a/f56 [804244,27707] 0 2024-04-02T04:47:49.319 INFO:tasks.workunit.client.1.smithi179.stdout:7/482: mkdir d1/dd/d23/dab 0 2024-04-02T04:47:49.320 INFO:tasks.workunit.client.1.smithi179.stdout:7/483: stat d1/d9/d27/f49 0 2024-04-02T04:47:49.328 INFO:tasks.workunit.client.0.smithi070.stdout:4/380: fdatasync d1/d3/d9/d41/d65/f6a 0 2024-04-02T04:47:49.334 INFO:tasks.workunit.client.1.smithi179.stdout:4/495: mkdir d1/d3/df/d10/d2f/d60/da6 0 2024-04-02T04:47:49.336 INFO:tasks.workunit.client.1.smithi179.stdout:9/437: getdents d0/d1/d10 0 2024-04-02T04:47:49.339 INFO:tasks.workunit.client.0.smithi070.stdout:8/463: creat d2/d8/d23/f92 x:0 0 0 2024-04-02T04:47:49.341 INFO:tasks.workunit.client.1.smithi179.stdout:6/426: link d3/d42/l90 d3/d12/d23/d39/d4e/d66/l95 0 2024-04-02T04:47:49.343 INFO:tasks.workunit.client.0.smithi070.stdout:4/381: creat d1/d3/f80 x:0 0 0 2024-04-02T04:47:49.344 INFO:tasks.workunit.client.1.smithi179.stdout:0/467: getdents d1/da/d52/d75 0 2024-04-02T04:47:49.344 INFO:tasks.workunit.client.1.smithi179.stdout:2/291: dwrite dc/d29/d4a/f15 [0,4194304] 0 2024-04-02T04:47:49.345 INFO:tasks.workunit.client.1.smithi179.stdout:0/468: write d1/db/f8e [283231,99425] 0 2024-04-02T04:47:49.346 INFO:tasks.workunit.client.1.smithi179.stdout:6/427: mknod d3/d12/d23/d39/d4e/c96 0 2024-04-02T04:47:49.347 INFO:tasks.workunit.client.1.smithi179.stdout:9/438: symlink d0/d1/d11/d29/d80/l88 0 2024-04-02T04:47:49.348 INFO:tasks.workunit.client.1.smithi179.stdout:2/292: mknod dc/d29/d31/d39/c58 0 2024-04-02T04:47:49.349 INFO:tasks.workunit.client.1.smithi179.stdout:4/496: rmdir d1/d9d 0 2024-04-02T04:47:49.350 INFO:tasks.workunit.client.0.smithi070.stdout:4/382: dwrite d1/d3/d9/d41/f6e [0,4194304] 0 2024-04-02T04:47:49.352 INFO:tasks.workunit.client.1.smithi179.stdout:9/439: stat d0/d1/c21 0 2024-04-02T04:47:49.354 INFO:tasks.workunit.client.1.smithi179.stdout:0/469: mknod d1/da/d48/d7b/c8f 0 2024-04-02T04:47:49.357 INFO:tasks.workunit.client.0.smithi070.stdout:4/383: creat d1/d3/d9/f81 x:0 0 0 2024-04-02T04:47:49.357 INFO:tasks.workunit.client.0.smithi070.stdout:3/423: write ff [4018258,72613] 0 2024-04-02T04:47:49.358 INFO:tasks.workunit.client.0.smithi070.stdout:3/424: stat d10/d32/d4b/f7d 0 2024-04-02T04:47:49.358 INFO:tasks.workunit.client.0.smithi070.stdout:4/384: write d1/f46 [1672944,119562] 0 2024-04-02T04:47:49.359 INFO:tasks.workunit.client.1.smithi179.stdout:4/497: rmdir d1/d15/d6e 39 2024-04-02T04:47:49.359 INFO:tasks.workunit.client.1.smithi179.stdout:5/367: dwrite d8/d1b/d23/d36/d53/f65 [0,4194304] 0 2024-04-02T04:47:49.362 INFO:tasks.workunit.client.1.smithi179.stdout:6/428: fdatasync d3/d12/d23/d26/f37 0 2024-04-02T04:47:49.363 INFO:tasks.workunit.client.0.smithi070.stdout:3/425: creat d10/d32/d3f/f96 x:0 0 0 2024-04-02T04:47:49.364 INFO:tasks.workunit.client.1.smithi179.stdout:3/467: truncate d8/d26/d36/f38 868347 0 2024-04-02T04:47:49.366 INFO:tasks.workunit.client.0.smithi070.stdout:4/385: rmdir d1/d3/d9/d41/d65/d7a 39 2024-04-02T04:47:49.366 INFO:tasks.workunit.client.1.smithi179.stdout:6/429: fdatasync d3/d12/d23/d39/d4e/d66/d7c/f87 0 2024-04-02T04:47:49.368 INFO:tasks.workunit.client.1.smithi179.stdout:0/470: rmdir d1/da/d28/d37 39 2024-04-02T04:47:49.369 INFO:tasks.workunit.client.0.smithi070.stdout:3/426: mknod d10/d17/d51/c97 0 2024-04-02T04:47:49.372 INFO:tasks.workunit.client.0.smithi070.stdout:9/383: dwrite d4/d10/d12/d27/f5c [0,4194304] 0 2024-04-02T04:47:49.375 INFO:tasks.workunit.client.1.smithi179.stdout:4/498: creat d1/d3/df/d10/d2f/fa7 x:0 0 0 2024-04-02T04:47:49.376 INFO:tasks.workunit.client.1.smithi179.stdout:7/484: dwrite d1/dd/d23/f24 [0,4194304] 0 2024-04-02T04:47:49.377 INFO:tasks.workunit.client.1.smithi179.stdout:7/485: fsync d1/d9/d55/f9e 0 2024-04-02T04:47:49.378 INFO:tasks.workunit.client.1.smithi179.stdout:9/440: symlink d0/d19/l89 0 2024-04-02T04:47:49.380 INFO:tasks.workunit.client.0.smithi070.stdout:8/464: dwrite d2/d8/f5e [0,4194304] 0 2024-04-02T04:47:49.380 INFO:tasks.workunit.client.1.smithi179.stdout:5/368: creat d8/d1b/d23/d32/d4e/d50/f6f x:0 0 0 2024-04-02T04:47:49.381 INFO:tasks.workunit.client.0.smithi070.stdout:9/384: rmdir d4/d10/d12/d55/d66 39 2024-04-02T04:47:49.382 INFO:tasks.workunit.client.1.smithi179.stdout:5/369: write d8/d1b/d23/d32/d4e/d50/f6f [165181,79593] 0 2024-04-02T04:47:49.384 INFO:tasks.workunit.client.1.smithi179.stdout:6/430: fdatasync d3/d12/d23/f55 0 2024-04-02T04:47:49.387 INFO:tasks.workunit.client.0.smithi070.stdout:3/427: rename d10/d17/l28 to d10/d32/d8a/d8d/l98 0 2024-04-02T04:47:49.388 INFO:tasks.workunit.client.1.smithi179.stdout:2/293: write dc/d29/d4a/d1e/f36 [183532,85320] 0 2024-04-02T04:47:49.389 INFO:tasks.workunit.client.1.smithi179.stdout:2/294: chown f6 2153 1 2024-04-02T04:47:49.389 INFO:tasks.workunit.client.1.smithi179.stdout:2/295: readlink dc/d29/l30 0 2024-04-02T04:47:49.390 INFO:tasks.workunit.client.0.smithi070.stdout:9/385: dwrite d4/d13/d16/f1d [0,4194304] 0 2024-04-02T04:47:49.395 INFO:tasks.workunit.client.0.smithi070.stdout:8/465: write d2/d8/d29/db/f21 [3969538,119590] 0 2024-04-02T04:47:49.396 INFO:tasks.workunit.client.0.smithi070.stdout:8/466: chown d2/d8/d29 21576147 1 2024-04-02T04:47:49.398 INFO:tasks.workunit.client.0.smithi070.stdout:8/467: write d2/d8/d23/d24/f5b [6244089,32221] 0 2024-04-02T04:47:49.402 INFO:tasks.workunit.client.0.smithi070.stdout:3/428: symlink d10/d32/d8a/d8d/d91/l99 0 2024-04-02T04:47:49.404 INFO:tasks.workunit.client.1.smithi179.stdout:7/486: mknod d1/dd/d16/d74/cac 0 2024-04-02T04:47:49.404 INFO:tasks.workunit.client.1.smithi179.stdout:7/487: chown d1/d9/d27/f5f 19071 1 2024-04-02T04:47:49.406 INFO:tasks.workunit.client.1.smithi179.stdout:5/370: rmdir d8/d1b/d23/d32 39 2024-04-02T04:47:49.408 INFO:tasks.workunit.client.0.smithi070.stdout:4/386: link d1/l6c d1/l82 0 2024-04-02T04:47:49.413 INFO:tasks.workunit.client.0.smithi070.stdout:8/468: rename d2/d8/d23/d24/f67 to d2/d8/d23/d24/d5f/f93 0 2024-04-02T04:47:49.414 INFO:tasks.workunit.client.1.smithi179.stdout:8/391: sync 2024-04-02T04:47:49.416 INFO:tasks.workunit.client.1.smithi179.stdout:6/431: rename d3/d12/f22 to d3/d12/f97 0 2024-04-02T04:47:49.418 INFO:tasks.workunit.client.0.smithi070.stdout:3/429: unlink c6 0 2024-04-02T04:47:49.419 INFO:tasks.workunit.client.1.smithi179.stdout:8/392: dwrite d3/d32/d30/f3b [0,4194304] 0 2024-04-02T04:47:49.421 INFO:tasks.workunit.client.0.smithi070.stdout:4/387: symlink d1/d3/d9/d5e/l83 0 2024-04-02T04:47:49.424 INFO:tasks.workunit.client.1.smithi179.stdout:0/471: fsync d1/f36 0 2024-04-02T04:47:49.425 INFO:tasks.workunit.client.1.smithi179.stdout:0/472: dread - d1/da/f4a zero size 2024-04-02T04:47:49.426 INFO:tasks.workunit.client.0.smithi070.stdout:9/386: dwrite d4/d13/d16/d19/d2f/d39/f44 [0,4194304] 0 2024-04-02T04:47:49.430 INFO:tasks.workunit.client.0.smithi070.stdout:3/430: rename d10/d32/d3f/f96 to d10/d32/d8a/d8d/f9a 0 2024-04-02T04:47:49.432 INFO:tasks.workunit.client.1.smithi179.stdout:7/488: mknod d1/d9/d7e/cad 0 2024-04-02T04:47:49.435 INFO:tasks.workunit.client.1.smithi179.stdout:3/468: truncate d8/d57/d63/f6c 1108900 0 2024-04-02T04:47:49.437 INFO:tasks.workunit.client.1.smithi179.stdout:7/489: dread d1/d9/d27/f4b [0,4194304] 0 2024-04-02T04:47:49.437 INFO:tasks.workunit.client.1.smithi179.stdout:7/490: read - d1/d9/d55/f96 zero size 2024-04-02T04:47:49.438 INFO:tasks.workunit.client.1.smithi179.stdout:7/491: write d1/dd/d23/f82 [2535033,91482] 0 2024-04-02T04:47:49.439 INFO:tasks.workunit.client.0.smithi070.stdout:3/431: dread d10/f29 [0,4194304] 0 2024-04-02T04:47:49.440 INFO:tasks.workunit.client.0.smithi070.stdout:9/387: creat d4/d34/f7a x:0 0 0 2024-04-02T04:47:49.440 INFO:tasks.workunit.client.1.smithi179.stdout:4/499: write d1/d3/dc/d27/d62/f76 [837031,17042] 0 2024-04-02T04:47:49.444 INFO:tasks.workunit.client.0.smithi070.stdout:4/388: write d1/d3/d9/d41/d65/d7a/f7d [1019792,103124] 0 2024-04-02T04:47:49.446 INFO:tasks.workunit.client.1.smithi179.stdout:4/500: dwrite d1/d3/df/d10/d2f/d36/f58 [0,4194304] 0 2024-04-02T04:47:49.447 INFO:tasks.workunit.client.0.smithi070.stdout:3/432: symlink d10/d17/d4c/l9b 0 2024-04-02T04:47:49.447 INFO:tasks.workunit.client.1.smithi179.stdout:4/501: fsync d1/fb 0 2024-04-02T04:47:49.447 INFO:tasks.workunit.client.0.smithi070.stdout:3/433: chown d10/d17/f74 66966125 1 2024-04-02T04:47:49.448 INFO:tasks.workunit.client.1.smithi179.stdout:8/393: symlink d3/d31/l8c 0 2024-04-02T04:47:49.449 INFO:tasks.workunit.client.0.smithi070.stdout:9/388: rename d4/d13/d16 to d4/d10/d11/d7b 0 2024-04-02T04:47:49.452 INFO:tasks.workunit.client.1.smithi179.stdout:5/371: dwrite d8/d1b/d23/d36/f41 [0,4194304] 0 2024-04-02T04:47:49.454 INFO:tasks.workunit.client.1.smithi179.stdout:4/502: read d1/d3/dc/d26/d8a/f82 [606642,76941] 0 2024-04-02T04:47:49.455 INFO:tasks.workunit.client.1.smithi179.stdout:4/503: stat d1/d3/dc/l83 0 2024-04-02T04:47:49.457 INFO:tasks.workunit.client.1.smithi179.stdout:3/469: creat d8/d26/d36/f90 x:0 0 0 2024-04-02T04:47:49.457 INFO:tasks.workunit.client.1.smithi179.stdout:8/394: dwrite d3/d1b/f23 [0,4194304] 0 2024-04-02T04:47:49.459 INFO:tasks.workunit.client.0.smithi070.stdout:3/434: creat d10/d17/d51/d73/d8f/f9c x:0 0 0 2024-04-02T04:47:49.467 INFO:tasks.workunit.client.0.smithi070.stdout:4/389: rename d1/c1b to d1/c84 0 2024-04-02T04:47:49.474 INFO:tasks.workunit.client.0.smithi070.stdout:9/389: truncate d4/d10/d12/d27/f5c 1085157 0 2024-04-02T04:47:49.477 INFO:tasks.workunit.client.0.smithi070.stdout:3/435: getdents d10/d32/d3f 0 2024-04-02T04:47:49.478 INFO:tasks.workunit.client.1.smithi179.stdout:8/395: rmdir d3/d32/d30 39 2024-04-02T04:47:49.480 INFO:tasks.workunit.client.1.smithi179.stdout:7/492: unlink d1/d9/d27/c9b 0 2024-04-02T04:47:49.482 INFO:tasks.workunit.client.0.smithi070.stdout:3/436: creat d10/d66/f9d x:0 0 0 2024-04-02T04:47:49.483 INFO:tasks.workunit.client.0.smithi070.stdout:9/390: dwrite d4/d10/d11/d7b/f35 [0,4194304] 0 2024-04-02T04:47:49.484 INFO:tasks.workunit.client.1.smithi179.stdout:5/372: creat d8/d1b/d23/d32/d4b/f70 x:0 0 0 2024-04-02T04:47:49.484 INFO:tasks.workunit.client.1.smithi179.stdout:5/373: readlink d8/d1b/d23/l3f 0 2024-04-02T04:47:49.485 INFO:tasks.workunit.client.0.smithi070.stdout:9/391: write d4/d10/d11/d7b/f35 [655464,126647] 0 2024-04-02T04:47:49.486 INFO:tasks.workunit.client.0.smithi070.stdout:3/437: rmdir d10/d17 39 2024-04-02T04:47:49.487 INFO:tasks.workunit.client.0.smithi070.stdout:3/438: dread - d10/d32/d3f/f6d zero size 2024-04-02T04:47:49.488 INFO:tasks.workunit.client.0.smithi070.stdout:3/439: readlink d10/d32/d4b/l6e 0 2024-04-02T04:47:49.489 INFO:tasks.workunit.client.1.smithi179.stdout:8/396: creat d3/d32/d21/d51/d70/f8d x:0 0 0 2024-04-02T04:47:49.489 INFO:tasks.workunit.client.0.smithi070.stdout:4/390: truncate d1/f32 1316252 0 2024-04-02T04:47:49.492 INFO:tasks.workunit.client.1.smithi179.stdout:7/493: unlink d1/dd/d16/f2c 0 2024-04-02T04:47:49.493 INFO:tasks.workunit.client.0.smithi070.stdout:9/392: unlink d4/f9 0 2024-04-02T04:47:49.496 INFO:tasks.workunit.client.0.smithi070.stdout:3/440: unlink d10/l18 0 2024-04-02T04:47:49.497 INFO:tasks.workunit.client.1.smithi179.stdout:4/504: write d1/d15/d6e/f80 [284161,29619] 0 2024-04-02T04:47:49.498 INFO:tasks.workunit.client.1.smithi179.stdout:4/505: chown d1/d3/dc/d27/c4b 14 1 2024-04-02T04:47:49.499 INFO:tasks.workunit.client.1.smithi179.stdout:8/397: rmdir d3/d31 39 2024-04-02T04:47:49.503 INFO:tasks.workunit.client.1.smithi179.stdout:4/506: read d1/d3/d19/f2c [2786548,79679] 0 2024-04-02T04:47:49.504 INFO:tasks.workunit.client.1.smithi179.stdout:5/374: rename d8/f10 to d8/d1b/d23/d30/f71 0 2024-04-02T04:47:49.510 INFO:tasks.workunit.client.0.smithi070.stdout:3/441: dread ff [0,4194304] 0 2024-04-02T04:47:49.510 INFO:tasks.workunit.client.0.smithi070.stdout:3/442: readlink d10/d32/l37 0 2024-04-02T04:47:49.512 INFO:tasks.workunit.client.1.smithi179.stdout:4/507: symlink d1/d3/dc/d27/la8 0 2024-04-02T04:47:49.513 INFO:tasks.workunit.client.0.smithi070.stdout:9/393: creat d4/d10/d12/d55/d66/f7c x:0 0 0 2024-04-02T04:47:49.514 INFO:tasks.workunit.client.0.smithi070.stdout:4/391: dread d1/f2d [0,4194304] 0 2024-04-02T04:47:49.514 INFO:tasks.workunit.client.0.smithi070.stdout:9/394: write d4/d10/d11/d7b/f35 [4667327,78675] 0 2024-04-02T04:47:49.515 INFO:tasks.workunit.client.1.smithi179.stdout:7/494: rename d1/dd/d16/d74/cac to d1/d9/d35/d6b/cae 0 2024-04-02T04:47:49.516 INFO:tasks.workunit.client.0.smithi070.stdout:9/395: readlink d4/d10/d11/d7b/l1f 0 2024-04-02T04:47:49.516 INFO:tasks.workunit.client.0.smithi070.stdout:4/392: write d1/d3/d9/d41/f6e [2622033,37660] 0 2024-04-02T04:47:49.516 INFO:tasks.workunit.client.1.smithi179.stdout:5/375: stat d8/d1b/d23/d32/f3d 0 2024-04-02T04:47:49.518 INFO:tasks.workunit.client.0.smithi070.stdout:3/443: rmdir d10/d32/d8a 39 2024-04-02T04:47:49.519 INFO:tasks.workunit.client.0.smithi070.stdout:3/444: write d10/d32/f56 [3651957,21664] 0 2024-04-02T04:47:49.521 INFO:tasks.workunit.client.1.smithi179.stdout:8/398: write d3/d32/d21/d22/d33/f34 [102730,111625] 0 2024-04-02T04:47:49.522 INFO:tasks.workunit.client.1.smithi179.stdout:1/440: sync 2024-04-02T04:47:49.523 INFO:tasks.workunit.client.1.smithi179.stdout:7/495: write d1/d84/d8d/f8f [919734,30627] 0 2024-04-02T04:47:49.524 INFO:tasks.workunit.client.0.smithi070.stdout:9/396: mknod d4/d10/d11/d7b/d19/d2f/d39/d4f/c7d 0 2024-04-02T04:47:49.529 INFO:tasks.workunit.client.1.smithi179.stdout:8/399: dread d3/d32/d21/d50/f5f [0,4194304] 0 2024-04-02T04:47:49.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:49 smithi070 ceph-mon[29212]: pgmap v13: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 176 MiB/s wr, 262 op/s 2024-04-02T04:47:49.530 INFO:tasks.workunit.client.0.smithi070.stdout:3/445: unlink d10/f86 0 2024-04-02T04:47:49.531 INFO:tasks.workunit.client.1.smithi179.stdout:5/376: truncate d8/f13 2882960 0 2024-04-02T04:47:49.532 INFO:tasks.workunit.client.1.smithi179.stdout:5/377: truncate d8/d1b/d23/f2f 51442 0 2024-04-02T04:47:49.539 INFO:tasks.workunit.client.1.smithi179.stdout:1/441: truncate d3/f2d 1958803 0 2024-04-02T04:47:49.539 INFO:tasks.workunit.client.1.smithi179.stdout:5/378: dwrite d8/d1b/d23/f60 [0,4194304] 0 2024-04-02T04:47:49.542 INFO:tasks.workunit.client.0.smithi070.stdout:4/393: write d1/f32 [1165609,82196] 0 2024-04-02T04:47:49.542 INFO:tasks.workunit.client.0.smithi070.stdout:4/394: dread - d1/d3/d9/d5e/f77 zero size 2024-04-02T04:47:49.544 INFO:tasks.workunit.client.1.smithi179.stdout:5/379: stat d8/d1b/d23/d32/d6e 0 2024-04-02T04:47:49.544 INFO:tasks.workunit.client.1.smithi179.stdout:1/442: dread - d3/d5/d2f/d4f/d35/f6e zero size 2024-04-02T04:47:49.545 INFO:tasks.workunit.client.0.smithi070.stdout:9/397: link d4/d10/d11/f4e d4/d10/d12/d55/f7e 0 2024-04-02T04:47:49.547 INFO:tasks.workunit.client.1.smithi179.stdout:8/400: creat d3/d32/d21/d22/d33/d84/f8e x:0 0 0 2024-04-02T04:47:49.548 INFO:tasks.workunit.client.1.smithi179.stdout:5/380: creat d8/d1b/d23/d36/d53/f72 x:0 0 0 2024-04-02T04:47:49.549 INFO:tasks.workunit.client.1.smithi179.stdout:5/381: readlink d8/d1b/d23/d32/l35 0 2024-04-02T04:47:49.550 INFO:tasks.workunit.client.0.smithi070.stdout:4/395: mkdir d1/d3/d85 0 2024-04-02T04:47:49.553 INFO:tasks.workunit.client.1.smithi179.stdout:7/496: fsync d1/d9/d27/d87/d8b/d75/f9a 0 2024-04-02T04:47:49.556 INFO:tasks.workunit.client.0.smithi070.stdout:3/446: dread d10/d32/f55 [0,4194304] 0 2024-04-02T04:47:49.556 INFO:tasks.workunit.client.1.smithi179.stdout:1/443: unlink d3/d5/de/d33/c48 0 2024-04-02T04:47:49.557 INFO:tasks.workunit.client.0.smithi070.stdout:2/456: sync 2024-04-02T04:47:49.557 INFO:tasks.workunit.client.1.smithi179.stdout:5/382: dwrite d8/d1b/d23/d36/d53/f65 [0,4194304] 0 2024-04-02T04:47:49.559 INFO:tasks.workunit.client.0.smithi070.stdout:9/398: dread d4/d10/d12/d55/f7e [0,4194304] 0 2024-04-02T04:47:49.562 INFO:tasks.workunit.client.1.smithi179.stdout:8/401: dwrite d3/d32/d30/f7d [4194304,4194304] 0 2024-04-02T04:47:49.563 INFO:tasks.workunit.client.0.smithi070.stdout:2/457: mkdir d2/d12/d3e/d68/d90 0 2024-04-02T04:47:49.563 INFO:tasks.workunit.client.0.smithi070.stdout:2/458: fsync d2/d12/f52 0 2024-04-02T04:47:49.564 INFO:tasks.workunit.client.1.smithi179.stdout:8/402: write d3/f61 [250498,13668] 0 2024-04-02T04:47:49.564 INFO:tasks.workunit.client.1.smithi179.stdout:1/444: symlink d3/d5/d2f/d65/l98 0 2024-04-02T04:47:49.567 INFO:tasks.workunit.client.1.smithi179.stdout:5/383: readlink d8/d1b/d23/d36/d53/l6a 0 2024-04-02T04:47:49.568 INFO:tasks.workunit.client.0.smithi070.stdout:3/447: mkdir d10/d17/d51/d73/d9e 0 2024-04-02T04:47:49.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:49 smithi179 ceph-mon[36603]: pgmap v13: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 176 MiB/s wr, 262 op/s 2024-04-02T04:47:49.572 INFO:tasks.workunit.client.0.smithi070.stdout:2/459: dwrite d2/d10/d21/f33 [0,4194304] 0 2024-04-02T04:47:49.573 INFO:tasks.workunit.client.1.smithi179.stdout:7/497: write d1/dd/d23/f25 [1448356,129054] 0 2024-04-02T04:47:49.578 INFO:tasks.workunit.client.0.smithi070.stdout:3/448: truncate f1 907040 0 2024-04-02T04:47:49.584 INFO:tasks.workunit.client.0.smithi070.stdout:9/399: mknod d4/d10/d11/d7b/d19/d2f/d39/d57/c7f 0 2024-04-02T04:47:49.585 INFO:tasks.workunit.client.1.smithi179.stdout:5/384: mkdir d8/d1b/d23/d30/d73 0 2024-04-02T04:47:49.587 INFO:tasks.workunit.client.0.smithi070.stdout:9/400: truncate d4/d13/f51 344949 0 2024-04-02T04:47:49.588 INFO:tasks.workunit.client.1.smithi179.stdout:7/498: truncate d1/d9/d27/f69 1835636 0 2024-04-02T04:47:49.590 INFO:tasks.workunit.client.1.smithi179.stdout:8/403: mkdir d3/d31/d8f 0 2024-04-02T04:47:49.591 INFO:tasks.workunit.client.1.smithi179.stdout:5/385: symlink d8/d1b/d23/d36/d53/l74 0 2024-04-02T04:47:49.595 INFO:tasks.workunit.client.1.smithi179.stdout:8/404: symlink d3/d32/d21/d22/d33/d3c/l90 0 2024-04-02T04:47:49.595 INFO:tasks.workunit.client.1.smithi179.stdout:1/445: rmdir d3/d5/d2f 39 2024-04-02T04:47:49.596 INFO:tasks.workunit.client.1.smithi179.stdout:1/446: write d3/d85/f89 [984642,88295] 0 2024-04-02T04:47:49.598 INFO:tasks.workunit.client.0.smithi070.stdout:9/401: dwrite d4/d10/d11/d7b/d19/d53/f59 [0,4194304] 0 2024-04-02T04:47:49.598 INFO:tasks.workunit.client.1.smithi179.stdout:7/499: mkdir d1/d9/d27/d87/daf 0 2024-04-02T04:47:49.599 INFO:tasks.workunit.client.0.smithi070.stdout:9/402: fdatasync d4/d13/f37 0 2024-04-02T04:47:49.603 INFO:tasks.workunit.client.0.smithi070.stdout:3/449: truncate d10/d32/f55 1454110 0 2024-04-02T04:47:49.603 INFO:tasks.workunit.client.0.smithi070.stdout:3/450: stat d10/d66/c75 0 2024-04-02T04:47:49.609 INFO:tasks.workunit.client.1.smithi179.stdout:8/405: rename d3/d32/d21/d22/d33/d3c to d3/d32/d21/d22/d6b/d91 0 2024-04-02T04:47:49.611 INFO:tasks.workunit.client.1.smithi179.stdout:7/500: mkdir d1/dd/d23/dab/db0 0 2024-04-02T04:47:49.612 INFO:tasks.workunit.client.1.smithi179.stdout:7/501: fdatasync d1/dd/f90 0 2024-04-02T04:47:49.613 INFO:tasks.workunit.client.1.smithi179.stdout:5/386: dwrite d8/f3e [0,4194304] 0 2024-04-02T04:47:49.613 INFO:tasks.workunit.client.1.smithi179.stdout:1/447: rename d3/l9 to d3/d5/d2f/d4f/l99 0 2024-04-02T04:47:49.617 INFO:tasks.workunit.client.1.smithi179.stdout:5/387: rmdir d8/d1b/d23/d42 39 2024-04-02T04:47:49.617 INFO:tasks.workunit.client.1.smithi179.stdout:1/448: symlink d3/d5/d2f/d4f/d35/d49/l9a 0 2024-04-02T04:47:49.621 INFO:tasks.workunit.client.0.smithi070.stdout:9/403: rename d4/d10/c28 to d4/d10/d11/d7b/d19/c80 0 2024-04-02T04:47:49.622 INFO:tasks.workunit.client.0.smithi070.stdout:9/404: stat d4/d10/d11/d7b/f61 0 2024-04-02T04:47:49.623 INFO:tasks.workunit.client.1.smithi179.stdout:8/406: fdatasync d3/d32/d21/f25 0 2024-04-02T04:47:49.623 INFO:tasks.workunit.client.1.smithi179.stdout:8/407: fsync d3/f67 0 2024-04-02T04:47:49.625 INFO:tasks.workunit.client.1.smithi179.stdout:5/388: rmdir d8/d1b/d23/d36/d53/d62 39 2024-04-02T04:47:49.628 INFO:tasks.workunit.client.0.smithi070.stdout:9/405: symlink d4/d10/d11/d7b/l81 0 2024-04-02T04:47:49.630 INFO:tasks.workunit.client.0.smithi070.stdout:9/406: chown d4/d10/d11/d7b/d19/d2f/c41 1032680755 1 2024-04-02T04:47:49.632 INFO:tasks.workunit.client.0.smithi070.stdout:9/407: write d4/d10/d12/f62 [517321,26151] 0 2024-04-02T04:47:49.632 INFO:tasks.workunit.client.0.smithi070.stdout:3/451: dwrite d10/d32/d3f/f38 [0,4194304] 0 2024-04-02T04:47:49.636 INFO:tasks.workunit.client.0.smithi070.stdout:9/408: mkdir d4/d10/d11/d7b/d19/d82 0 2024-04-02T04:47:49.637 INFO:tasks.workunit.client.1.smithi179.stdout:9/441: sync 2024-04-02T04:47:49.637 INFO:tasks.workunit.client.1.smithi179.stdout:2/296: sync 2024-04-02T04:47:49.639 INFO:tasks.workunit.client.0.smithi070.stdout:3/452: dwrite d10/d17/f74 [0,4194304] 0 2024-04-02T04:47:49.640 INFO:tasks.workunit.client.0.smithi070.stdout:3/453: chown d10/l11 7 1 2024-04-02T04:47:49.641 INFO:tasks.workunit.client.0.smithi070.stdout:3/454: write d10/d32/f52 [1541642,107600] 0 2024-04-02T04:47:49.649 INFO:tasks.workunit.client.1.smithi179.stdout:2/297: write dc/d29/f34 [783137,22862] 0 2024-04-02T04:47:49.652 INFO:tasks.workunit.client.1.smithi179.stdout:2/298: chown dc/d29/d31/d39/l55 1572129 1 2024-04-02T04:47:49.652 INFO:tasks.workunit.client.1.smithi179.stdout:8/408: dwrite d3/d32/d30/f4e [0,4194304] 0 2024-04-02T04:47:49.654 INFO:tasks.workunit.client.1.smithi179.stdout:2/299: chown dc/d29/d31/d39/c4e 321554609 1 2024-04-02T04:47:49.654 INFO:tasks.workunit.client.1.smithi179.stdout:8/409: stat d3/d32/d21/d50/d56/f77 0 2024-04-02T04:47:49.654 INFO:tasks.workunit.client.1.smithi179.stdout:1/449: dwrite d3/f28 [0,4194304] 0 2024-04-02T04:47:49.656 INFO:tasks.workunit.client.1.smithi179.stdout:8/410: fsync d3/d32/d30/f3b 0 2024-04-02T04:47:49.656 INFO:tasks.workunit.client.1.smithi179.stdout:5/389: dread d8/d1b/f1f [0,4194304] 0 2024-04-02T04:47:49.662 INFO:tasks.workunit.client.1.smithi179.stdout:1/450: dwrite d3/d5/d2f/f81 [0,4194304] 0 2024-04-02T04:47:49.663 INFO:tasks.workunit.client.1.smithi179.stdout:1/451: chown d3/d5/f25 0 1 2024-04-02T04:47:49.664 INFO:tasks.workunit.client.1.smithi179.stdout:1/452: chown d3/lc 63 1 2024-04-02T04:47:49.664 INFO:tasks.workunit.client.1.smithi179.stdout:1/453: chown d3/d85 7490 1 2024-04-02T04:47:49.671 INFO:tasks.workunit.client.1.smithi179.stdout:8/411: dread d3/d32/d30/f7d [0,4194304] 0 2024-04-02T04:47:49.700 INFO:tasks.workunit.client.0.smithi070.stdout:5/462: sync 2024-04-02T04:47:49.708 INFO:tasks.workunit.client.1.smithi179.stdout:2/300: write dc/d29/d4a/f27 [294561,98299] 0 2024-04-02T04:47:49.708 INFO:tasks.workunit.client.0.smithi070.stdout:3/455: dread d10/f2c [0,4194304] 0 2024-04-02T04:47:49.709 INFO:tasks.workunit.client.1.smithi179.stdout:2/301: write dc/f44 [820771,84114] 0 2024-04-02T04:47:49.712 INFO:tasks.workunit.client.0.smithi070.stdout:5/463: creat d5/d17/f93 x:0 0 0 2024-04-02T04:47:49.720 INFO:tasks.workunit.client.0.smithi070.stdout:5/464: mknod d5/d9/d10/d6a/d7d/c94 0 2024-04-02T04:47:49.721 INFO:tasks.workunit.client.1.smithi179.stdout:9/442: dread d0/d1/d10/d6d/f2d [0,4194304] 0 2024-04-02T04:47:49.722 INFO:tasks.workunit.client.0.smithi070.stdout:3/456: readlink d10/d32/d8a/l93 0 2024-04-02T04:47:49.725 INFO:tasks.workunit.client.1.smithi179.stdout:9/443: dread d0/d1/d11/d29/f37 [0,4194304] 0 2024-04-02T04:47:49.725 INFO:tasks.workunit.client.0.smithi070.stdout:3/457: creat d10/d32/d8a/d8d/d91/f9f x:0 0 0 2024-04-02T04:47:49.726 INFO:tasks.workunit.client.0.smithi070.stdout:5/465: dread d5/d17/d27/f4b [0,4194304] 0 2024-04-02T04:47:49.726 INFO:tasks.workunit.client.1.smithi179.stdout:2/302: symlink dc/d29/d4a/d1e/l59 0 2024-04-02T04:47:49.726 INFO:tasks.workunit.client.0.smithi070.stdout:5/466: stat d5/d17/d6c/f41 0 2024-04-02T04:47:49.727 INFO:tasks.workunit.client.0.smithi070.stdout:5/467: fdatasync d5/d66/d75/f80 0 2024-04-02T04:47:49.730 INFO:tasks.workunit.client.0.smithi070.stdout:3/458: unlink d10/d32/d4b/c7f 0 2024-04-02T04:47:49.731 INFO:tasks.workunit.client.1.smithi179.stdout:5/390: truncate d8/d1b/d23/f47 5802178 0 2024-04-02T04:47:49.732 INFO:tasks.workunit.client.1.smithi179.stdout:5/391: write d8/f6c [679379,119794] 0 2024-04-02T04:47:49.732 INFO:tasks.workunit.client.1.smithi179.stdout:8/412: dwrite d3/d32/d21/d22/f62 [8388608,4194304] 0 2024-04-02T04:47:49.733 INFO:tasks.workunit.client.1.smithi179.stdout:0/473: sync 2024-04-02T04:47:49.734 INFO:tasks.workunit.client.1.smithi179.stdout:6/432: sync 2024-04-02T04:47:49.734 INFO:tasks.workunit.client.1.smithi179.stdout:3/470: sync 2024-04-02T04:47:49.734 INFO:tasks.workunit.client.0.smithi070.stdout:5/468: dwrite d5/d66/d75/f80 [0,4194304] 0 2024-04-02T04:47:49.735 INFO:tasks.workunit.client.1.smithi179.stdout:3/471: fsync d8/d26/d36/f4d 0 2024-04-02T04:47:49.737 INFO:tasks.workunit.client.0.smithi070.stdout:3/459: mknod d10/d32/d3f/ca0 0 2024-04-02T04:47:49.738 INFO:tasks.workunit.client.1.smithi179.stdout:6/433: dread d3/f7f [0,4194304] 0 2024-04-02T04:47:49.742 INFO:tasks.workunit.client.0.smithi070.stdout:3/460: mkdir d10/d32/d8a/d8d/da1 0 2024-04-02T04:47:49.746 INFO:tasks.workunit.client.0.smithi070.stdout:5/469: truncate d5/d17/d50/f5f 3442072 0 2024-04-02T04:47:49.747 INFO:tasks.workunit.client.1.smithi179.stdout:1/454: mknod d3/d5/d2f/d5a/c9b 0 2024-04-02T04:47:49.750 INFO:tasks.workunit.client.1.smithi179.stdout:2/303: dwrite dc/d29/d4a/d4b/f51 [0,4194304] 0 2024-04-02T04:47:49.751 INFO:tasks.workunit.client.0.smithi070.stdout:5/470: read d5/fa [114729,102808] 0 2024-04-02T04:47:49.753 INFO:tasks.workunit.client.1.smithi179.stdout:3/472: mknod d8/d26/d52/d7e/c91 0 2024-04-02T04:47:49.753 INFO:tasks.workunit.client.1.smithi179.stdout:3/473: readlink d8/l12 0 2024-04-02T04:47:49.754 INFO:tasks.workunit.client.0.smithi070.stdout:5/471: unlink d5/d9/d23/d30/l85 0 2024-04-02T04:47:49.759 INFO:tasks.workunit.client.0.smithi070.stdout:5/472: rename d5/d17/l20 to d5/d17/d2b/d7b/l95 0 2024-04-02T04:47:49.761 INFO:tasks.workunit.client.1.smithi179.stdout:5/392: creat d8/d1b/d23/d36/d53/d62/f75 x:0 0 0 2024-04-02T04:47:49.761 INFO:tasks.workunit.client.1.smithi179.stdout:5/393: chown d8/d1b/f37 93 1 2024-04-02T04:47:49.768 INFO:tasks.workunit.client.1.smithi179.stdout:5/394: read d8/f1d [17261,13504] 0 2024-04-02T04:47:49.769 INFO:tasks.workunit.client.1.smithi179.stdout:3/474: creat d8/d57/d8f/f92 x:0 0 0 2024-04-02T04:47:49.770 INFO:tasks.workunit.client.1.smithi179.stdout:0/474: write d1/da/d28/f41 [6045930,33423] 0 2024-04-02T04:47:49.773 INFO:tasks.workunit.client.1.smithi179.stdout:6/434: write d3/d12/f25 [334386,93400] 0 2024-04-02T04:47:49.774 INFO:tasks.workunit.client.1.smithi179.stdout:3/475: write d8/f48 [348266,116460] 0 2024-04-02T04:47:49.774 INFO:tasks.workunit.client.1.smithi179.stdout:9/444: truncate d0/d1/f44 1008343 0 2024-04-02T04:47:49.777 INFO:tasks.workunit.client.1.smithi179.stdout:9/445: chown d0/d19/d1c/f47 180659444 1 2024-04-02T04:47:49.778 INFO:tasks.workunit.client.1.smithi179.stdout:0/475: dwrite d1/db/d64/f6a [0,4194304] 0 2024-04-02T04:47:49.778 INFO:tasks.workunit.client.1.smithi179.stdout:9/446: fsync d0/d1/d11/d3a/f81 0 2024-04-02T04:47:49.780 INFO:tasks.workunit.client.1.smithi179.stdout:8/413: truncate d3/d32/d21/d22/f2d 822149 0 2024-04-02T04:47:49.781 INFO:tasks.workunit.client.1.smithi179.stdout:5/395: creat d8/d1b/d23/d42/d5a/f76 x:0 0 0 2024-04-02T04:47:49.784 INFO:tasks.workunit.client.1.smithi179.stdout:1/455: dwrite d3/d5/d2f/d4f/d42/d53/d5b/f61 [0,4194304] 0 2024-04-02T04:47:49.788 INFO:tasks.workunit.client.0.smithi070.stdout:5/473: write d5/d17/d50/f58 [956025,127732] 0 2024-04-02T04:47:49.790 INFO:tasks.workunit.client.1.smithi179.stdout:0/476: mkdir d1/da/d28/d34/d39/d90 0 2024-04-02T04:47:49.793 INFO:tasks.workunit.client.1.smithi179.stdout:8/414: symlink d3/d32/d21/d22/d33/d84/l92 0 2024-04-02T04:47:49.799 INFO:tasks.workunit.client.0.smithi070.stdout:5/474: getdents d5/d17 0 2024-04-02T04:47:49.801 INFO:tasks.workunit.client.1.smithi179.stdout:0/477: mknod d1/da/d48/d7b/c91 0 2024-04-02T04:47:49.801 INFO:tasks.workunit.client.1.smithi179.stdout:6/435: link d3/d2d/d38/c41 d3/d12/d23/d39/d80/d93/c98 0 2024-04-02T04:47:49.803 INFO:tasks.workunit.client.1.smithi179.stdout:8/415: unlink d3/d32/d21/d22/d33/c39 0 2024-04-02T04:47:49.804 INFO:tasks.workunit.client.1.smithi179.stdout:5/396: creat d8/d1b/d23/d30/d5f/f77 x:0 0 0 2024-04-02T04:47:49.805 INFO:tasks.workunit.client.1.smithi179.stdout:3/476: write d8/d26/d36/f38 [1567370,112501] 0 2024-04-02T04:47:49.805 INFO:tasks.workunit.client.0.smithi070.stdout:5/475: mknod d5/d9/d10/c96 0 2024-04-02T04:47:49.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/476: write d5/f2f [4576611,39398] 0 2024-04-02T04:47:49.806 INFO:tasks.workunit.client.1.smithi179.stdout:1/456: symlink d3/d5/d96/l9c 0 2024-04-02T04:47:49.808 INFO:tasks.workunit.client.1.smithi179.stdout:0/478: chown d1/da/d28/d37/f72 5 1 2024-04-02T04:47:49.813 INFO:tasks.workunit.client.1.smithi179.stdout:3/477: write d8/d26/f72 [2029400,105411] 0 2024-04-02T04:47:49.816 INFO:tasks.workunit.client.0.smithi070.stdout:5/477: creat d5/f97 x:0 0 0 2024-04-02T04:47:49.817 INFO:tasks.workunit.client.0.smithi070.stdout:5/478: fdatasync d5/d17/f93 0 2024-04-02T04:47:49.817 INFO:tasks.workunit.client.1.smithi179.stdout:0/479: stat d1/db/c25 0 2024-04-02T04:47:49.819 INFO:tasks.workunit.client.1.smithi179.stdout:5/397: dread d8/d1b/d23/f47 [4194304,4194304] 0 2024-04-02T04:47:49.820 INFO:tasks.workunit.client.0.smithi070.stdout:5/479: unlink f4 0 2024-04-02T04:47:49.823 INFO:tasks.workunit.client.1.smithi179.stdout:3/478: symlink d8/d26/d52/d85/l93 0 2024-04-02T04:47:49.824 INFO:tasks.workunit.client.1.smithi179.stdout:0/480: rename d1/da/d48/d65/l68 to d1/da/df/l92 0 2024-04-02T04:47:49.826 INFO:tasks.workunit.client.1.smithi179.stdout:6/436: dwrite d3/d12/d23/d2f/f4a [0,4194304] 0 2024-04-02T04:47:49.834 INFO:tasks.workunit.client.1.smithi179.stdout:4/508: sync 2024-04-02T04:47:49.834 INFO:tasks.workunit.client.1.smithi179.stdout:7/502: sync 2024-04-02T04:47:49.835 INFO:tasks.workunit.client.0.smithi070.stdout:6/336: sync 2024-04-02T04:47:49.835 INFO:tasks.workunit.client.1.smithi179.stdout:7/503: chown d1/d9/d27/f49 129872242 1 2024-04-02T04:47:49.837 INFO:tasks.workunit.client.0.smithi070.stdout:5/480: creat d5/d17/d27/f98 x:0 0 0 2024-04-02T04:47:49.839 INFO:tasks.workunit.client.0.smithi070.stdout:6/337: rename d0/d3/d19/d31/d37/f5b to d0/d3/d19/d31/d46/d6b/f6d 0 2024-04-02T04:47:49.841 INFO:tasks.workunit.client.1.smithi179.stdout:3/479: unlink d8/d57/d63/c77 0 2024-04-02T04:47:49.842 INFO:tasks.workunit.client.0.smithi070.stdout:5/481: symlink d5/d17/d2b/d7b/l99 0 2024-04-02T04:47:49.842 INFO:tasks.workunit.client.1.smithi179.stdout:6/437: fdatasync f2 0 2024-04-02T04:47:49.844 INFO:tasks.workunit.client.0.smithi070.stdout:6/338: rename d0/d3/d19/d39/f4b to d0/d20/d5f/f6e 0 2024-04-02T04:47:49.846 INFO:tasks.workunit.client.1.smithi179.stdout:4/509: rename d1/d3/df/d10/d3a/l4f to d1/d3/d19/la9 0 2024-04-02T04:47:49.848 INFO:tasks.workunit.client.0.smithi070.stdout:5/482: dwrite d5/d17/d2b/f8d [0,4194304] 0 2024-04-02T04:47:49.848 INFO:tasks.workunit.client.1.smithi179.stdout:7/504: mkdir d1/dd/d16/d4d/db1 0 2024-04-02T04:47:49.849 INFO:tasks.workunit.client.0.smithi070.stdout:5/483: write d5/d17/d27/d2e/f91 [109180,62707] 0 2024-04-02T04:47:49.850 INFO:tasks.workunit.client.1.smithi179.stdout:8/416: link d3/d32/d21/d49/c87 d3/d31/c93 0 2024-04-02T04:47:49.854 INFO:tasks.workunit.client.0.smithi070.stdout:5/484: creat d5/d17/d2b/f9a x:0 0 0 2024-04-02T04:47:49.855 INFO:tasks.workunit.client.1.smithi179.stdout:1/457: link d3/c87 d3/c9d 0 2024-04-02T04:47:49.857 INFO:tasks.workunit.client.0.smithi070.stdout:6/339: dread d0/f15 [0,4194304] 0 2024-04-02T04:47:49.858 INFO:tasks.workunit.client.0.smithi070.stdout:6/340: chown d0/d3/d19/d31/d37/f5d 15324 1 2024-04-02T04:47:49.858 INFO:tasks.workunit.client.1.smithi179.stdout:3/480: mkdir d8/d6d/d94 0 2024-04-02T04:47:49.863 INFO:tasks.workunit.client.0.smithi070.stdout:6/341: dread d0/d43/f4a [0,4194304] 0 2024-04-02T04:47:49.863 INFO:tasks.workunit.client.1.smithi179.stdout:4/510: symlink d1/d3/dc/d7a/laa 0 2024-04-02T04:47:49.863 INFO:tasks.workunit.client.0.smithi070.stdout:6/342: stat d0/d20/l32 0 2024-04-02T04:47:49.864 INFO:tasks.workunit.client.0.smithi070.stdout:5/485: dwrite d5/f6 [4194304,4194304] 0 2024-04-02T04:47:49.864 INFO:tasks.workunit.client.1.smithi179.stdout:8/417: truncate d3/f27 231159 0 2024-04-02T04:47:49.865 INFO:tasks.workunit.client.1.smithi179.stdout:8/418: stat d3/d32/d21/d51/d70 0 2024-04-02T04:47:49.865 INFO:tasks.workunit.client.0.smithi070.stdout:5/486: stat d5/d17/d27/c39 0 2024-04-02T04:47:49.866 INFO:tasks.workunit.client.1.smithi179.stdout:5/398: creat d8/d1b/d23/d36/d53/f78 x:0 0 0 2024-04-02T04:47:49.868 INFO:tasks.workunit.client.1.smithi179.stdout:5/399: write d8/d1b/d23/d32/d46/f5e [3708976,61576] 0 2024-04-02T04:47:49.869 INFO:tasks.workunit.client.1.smithi179.stdout:1/458: symlink d3/d5/de/d33/l9e 0 2024-04-02T04:47:49.870 INFO:tasks.workunit.client.1.smithi179.stdout:0/481: creat d1/da/f93 x:0 0 0 2024-04-02T04:47:49.872 INFO:tasks.workunit.client.1.smithi179.stdout:3/481: write d8/d26/d36/f8d [470758,46534] 0 2024-04-02T04:47:49.876 INFO:tasks.workunit.client.0.smithi070.stdout:6/343: rmdir d0/d3/d19/d39 39 2024-04-02T04:47:49.879 INFO:tasks.workunit.client.1.smithi179.stdout:8/419: rename d3/d32/d21/d22/d33 to d3/d32/d21/d50/d56/d94 0 2024-04-02T04:47:49.882 INFO:tasks.workunit.client.1.smithi179.stdout:5/400: truncate f6 782696 0 2024-04-02T04:47:49.883 INFO:tasks.workunit.client.0.smithi070.stdout:6/344: chown d0/d3/d19/f47 1 1 2024-04-02T04:47:49.884 INFO:tasks.workunit.client.1.smithi179.stdout:5/401: dread d8/d1b/d23/d32/d4e/d50/f6f [0,4194304] 0 2024-04-02T04:47:49.885 INFO:tasks.workunit.client.1.smithi179.stdout:5/402: fdatasync d8/d1b/f56 0 2024-04-02T04:47:49.886 INFO:tasks.workunit.client.1.smithi179.stdout:3/482: unlink f5 0 2024-04-02T04:47:49.888 INFO:tasks.workunit.client.0.smithi070.stdout:6/345: creat d0/d55/f6f x:0 0 0 2024-04-02T04:47:49.893 INFO:tasks.workunit.client.1.smithi179.stdout:6/438: truncate d3/d12/d23/d2f/f4a 923112 0 2024-04-02T04:47:49.894 INFO:tasks.workunit.client.0.smithi070.stdout:6/346: creat d0/d3/d19/d31/d46/d6b/f70 x:0 0 0 2024-04-02T04:47:49.895 INFO:tasks.workunit.client.1.smithi179.stdout:4/511: write d1/d15/f31 [543878,64501] 0 2024-04-02T04:47:49.896 INFO:tasks.workunit.client.1.smithi179.stdout:0/482: unlink d1/da/df/l92 0 2024-04-02T04:47:49.896 INFO:tasks.workunit.client.1.smithi179.stdout:0/483: readlink d1/da/l44 0 2024-04-02T04:47:49.898 INFO:tasks.workunit.client.0.smithi070.stdout:5/487: dwrite d5/d17/d50/f5a [0,4194304] 0 2024-04-02T04:47:49.900 INFO:tasks.workunit.client.1.smithi179.stdout:1/459: write d3/d5/d2f/d4f/d42/d53/d5b/f83 [520681,46460] 0 2024-04-02T04:47:49.904 INFO:tasks.workunit.client.1.smithi179.stdout:5/403: creat d8/d1b/d23/d30/d5f/f79 x:0 0 0 2024-04-02T04:47:49.909 INFO:tasks.workunit.client.1.smithi179.stdout:7/505: link d1/d9/l26 d1/dd/lb2 0 2024-04-02T04:47:49.909 INFO:tasks.workunit.client.1.smithi179.stdout:7/506: chown d1/fc 605 1 2024-04-02T04:47:49.911 INFO:tasks.workunit.client.0.smithi070.stdout:6/347: rename d0/c4c to d0/d3/d19/c71 0 2024-04-02T04:47:49.919 INFO:tasks.workunit.client.1.smithi179.stdout:0/484: mknod d1/da/d28/d34/c94 0 2024-04-02T04:47:49.923 INFO:tasks.workunit.client.0.smithi070.stdout:5/488: dwrite d5/f1d [4194304,4194304] 0 2024-04-02T04:47:49.923 INFO:tasks.workunit.client.0.smithi070.stdout:6/348: dwrite d0/d3/d19/d31/d46/f68 [4194304,4194304] 0 2024-04-02T04:47:49.923 INFO:tasks.workunit.client.1.smithi179.stdout:5/404: rename d8/d1b/d23/d32/d46/f5e to d8/d1b/d23/d32/f7a 0 2024-04-02T04:47:49.926 INFO:tasks.workunit.client.1.smithi179.stdout:7/507: dread d1/fc [4194304,4194304] 0 2024-04-02T04:47:49.927 INFO:tasks.workunit.client.1.smithi179.stdout:2/304: sync 2024-04-02T04:47:49.927 INFO:tasks.workunit.client.1.smithi179.stdout:9/447: sync 2024-04-02T04:47:49.928 INFO:tasks.workunit.client.1.smithi179.stdout:7/508: fsync d1/d9/d7e/f81 0 2024-04-02T04:47:49.929 INFO:tasks.workunit.client.1.smithi179.stdout:4/512: dread d1/d3/df/d10/f11 [0,4194304] 0 2024-04-02T04:47:49.929 INFO:tasks.workunit.client.1.smithi179.stdout:3/483: dwrite d8/d26/f34 [0,4194304] 0 2024-04-02T04:47:49.932 INFO:tasks.workunit.client.1.smithi179.stdout:8/420: truncate d3/d32/d21/d22/f62 5901973 0 2024-04-02T04:47:49.932 INFO:tasks.workunit.client.1.smithi179.stdout:6/439: unlink d3/d2d/d38/d64/c70 0 2024-04-02T04:47:49.933 INFO:tasks.workunit.client.0.smithi070.stdout:5/489: dwrite d5/d17/f53 [0,4194304] 0 2024-04-02T04:47:49.933 INFO:tasks.workunit.client.0.smithi070.stdout:5/490: stat d5/d14/c5c 0 2024-04-02T04:47:49.936 INFO:tasks.workunit.client.0.smithi070.stdout:6/349: rmdir d0/d3/d19/d31 39 2024-04-02T04:47:49.940 INFO:tasks.workunit.client.1.smithi179.stdout:3/484: dwrite d8/d26/f4e [0,4194304] 0 2024-04-02T04:47:49.942 INFO:tasks.workunit.client.1.smithi179.stdout:3/485: readlink d8/l9 0 2024-04-02T04:47:49.944 INFO:tasks.workunit.client.1.smithi179.stdout:0/485: creat d1/da/d28/d37/f95 x:0 0 0 2024-04-02T04:47:49.945 INFO:tasks.workunit.client.0.smithi070.stdout:5/491: chown d5/d17/d27/d2e/l4e 3422 1 2024-04-02T04:47:49.948 INFO:tasks.workunit.client.0.smithi070.stdout:5/492: symlink d5/d57/l9b 0 2024-04-02T04:47:49.953 INFO:tasks.workunit.client.0.smithi070.stdout:6/350: rename d0/f2d to d0/d3/d19/d31/d46/d6b/f72 0 2024-04-02T04:47:49.954 INFO:tasks.workunit.client.0.smithi070.stdout:6/351: chown d0/d3/d19/d31/d46/f68 6619612 1 2024-04-02T04:47:49.955 INFO:tasks.workunit.client.0.smithi070.stdout:5/493: unlink d5/d66/f73 0 2024-04-02T04:47:49.958 INFO:tasks.workunit.client.1.smithi179.stdout:9/448: fdatasync d0/d19/d1c/d64/f85 0 2024-04-02T04:47:49.961 INFO:tasks.workunit.client.0.smithi070.stdout:6/352: symlink d0/d20/d5f/l73 0 2024-04-02T04:47:49.961 INFO:tasks.workunit.client.1.smithi179.stdout:4/513: truncate d1/d3/f28 6116193 0 2024-04-02T04:47:49.965 INFO:tasks.workunit.client.1.smithi179.stdout:6/440: rename d3/f1c to d3/d12/d23/d26/d57/f99 0 2024-04-02T04:47:49.966 INFO:tasks.workunit.client.0.smithi070.stdout:5/494: rename d5/d17/d50/l70 to d5/d9/d10/d6a/l9c 0 2024-04-02T04:47:49.967 INFO:tasks.workunit.client.1.smithi179.stdout:1/460: creat d3/d5/d2f/d4f/d42/d53/f9f x:0 0 0 2024-04-02T04:47:49.973 INFO:tasks.workunit.client.1.smithi179.stdout:7/509: symlink d1/dd/d16/d4d/db1/lb3 0 2024-04-02T04:47:49.973 INFO:tasks.workunit.client.1.smithi179.stdout:8/421: getdents d3/d1b/d2a/d83 0 2024-04-02T04:47:49.974 INFO:tasks.workunit.client.1.smithi179.stdout:8/422: truncate d3/d32/d21/d50/d56/d94/d84/f8e 157160 0 2024-04-02T04:47:49.975 INFO:tasks.workunit.client.1.smithi179.stdout:3/486: creat d8/d6d/d94/f95 x:0 0 0 2024-04-02T04:47:49.976 INFO:tasks.workunit.client.1.smithi179.stdout:8/423: truncate d3/d32/d21/d51/d70/f8d 893444 0 2024-04-02T04:47:49.977 INFO:tasks.workunit.client.1.smithi179.stdout:5/405: creat d8/d1b/d23/d36/d53/f7b x:0 0 0 2024-04-02T04:47:49.977 INFO:tasks.workunit.client.1.smithi179.stdout:2/305: creat dc/f5a x:0 0 0 2024-04-02T04:47:49.978 INFO:tasks.workunit.client.1.smithi179.stdout:5/406: rename d8 to d8/d1b/d23/d36/d7c 22 2024-04-02T04:47:49.978 INFO:tasks.workunit.client.1.smithi179.stdout:2/306: readlink dc/d29/l42 0 2024-04-02T04:47:49.978 INFO:tasks.workunit.client.1.smithi179.stdout:7/510: chown d1/d9/d27/f69 1072159 1 2024-04-02T04:47:49.979 INFO:tasks.workunit.client.1.smithi179.stdout:4/514: unlink d1/d3/dc/d27/d62/c90 0 2024-04-02T04:47:49.979 INFO:tasks.workunit.client.1.smithi179.stdout:0/486: link d1/da/d48/c4e d1/db/d64/c96 0 2024-04-02T04:47:49.980 INFO:tasks.workunit.client.1.smithi179.stdout:1/461: mkdir d3/d5/d2f/d4f/d6a/da0 0 2024-04-02T04:47:49.983 INFO:tasks.workunit.client.1.smithi179.stdout:5/407: creat d8/d1b/d23/d32/d46/f7d x:0 0 0 2024-04-02T04:47:49.984 INFO:tasks.workunit.client.1.smithi179.stdout:7/511: unlink d1/d9/d27/d87/d8b/f9c 0 2024-04-02T04:47:49.984 INFO:tasks.workunit.client.1.smithi179.stdout:5/408: chown d8/d1b/d23/d32/d4b 107 1 2024-04-02T04:47:49.985 INFO:tasks.workunit.client.1.smithi179.stdout:4/515: dwrite d1/d3/fa [0,4194304] 0 2024-04-02T04:47:49.986 INFO:tasks.workunit.client.1.smithi179.stdout:7/512: write d1/dd/d16/d74/d9f/fa3 [331448,24942] 0 2024-04-02T04:47:49.987 INFO:tasks.workunit.client.1.smithi179.stdout:1/462: creat d3/d5/d2f/d4f/d42/d55/fa1 x:0 0 0 2024-04-02T04:47:49.988 INFO:tasks.workunit.client.1.smithi179.stdout:6/441: getdents d3/d12/d23/d39/d4e/d66 0 2024-04-02T04:47:49.989 INFO:tasks.workunit.client.1.smithi179.stdout:0/487: mkdir d1/da/d48/d5c/d83/d97 0 2024-04-02T04:47:49.990 INFO:tasks.workunit.client.1.smithi179.stdout:5/409: mkdir d8/d7e 0 2024-04-02T04:47:49.990 INFO:tasks.workunit.client.1.smithi179.stdout:8/424: link d3/d32/d21/l3f d3/d1b/l95 0 2024-04-02T04:47:49.991 INFO:tasks.workunit.client.1.smithi179.stdout:6/442: write d3/d12/d23/d2f/f3a [5291882,100202] 0 2024-04-02T04:47:49.991 INFO:tasks.workunit.client.1.smithi179.stdout:4/516: symlink d1/d3/df/d10/d2f/lab 0 2024-04-02T04:47:49.992 INFO:tasks.workunit.client.1.smithi179.stdout:7/513: creat d1/d9/d35/fb4 x:0 0 0 2024-04-02T04:47:49.993 INFO:tasks.workunit.client.1.smithi179.stdout:2/307: link dc/d29/c38 dc/c5b 0 2024-04-02T04:47:49.995 INFO:tasks.workunit.client.1.smithi179.stdout:0/488: rename d1/da/df/f11 to d1/da/d28/d5e/f98 0 2024-04-02T04:47:49.995 INFO:tasks.workunit.client.1.smithi179.stdout:7/514: symlink d1/dd/d76/lb5 0 2024-04-02T04:47:49.997 INFO:tasks.workunit.client.1.smithi179.stdout:0/489: unlink d1/da/d28/d34/l38 0 2024-04-02T04:47:49.998 INFO:tasks.workunit.client.1.smithi179.stdout:0/490: write d1/da/d28/d37/f95 [265286,123838] 0 2024-04-02T04:47:50.001 INFO:tasks.workunit.client.1.smithi179.stdout:0/491: rename d1/da/d67/f7f to d1/db/d64/f99 0 2024-04-02T04:47:50.001 INFO:tasks.workunit.client.1.smithi179.stdout:6/443: chown d3/d2d/f65 488 1 2024-04-02T04:47:50.003 INFO:tasks.workunit.client.1.smithi179.stdout:0/492: chown d1/da/d28/d6d/l6e 4643 1 2024-04-02T04:47:50.004 INFO:tasks.workunit.client.1.smithi179.stdout:2/308: dread dc/f46 [0,4194304] 0 2024-04-02T04:47:50.005 INFO:tasks.workunit.client.1.smithi179.stdout:7/515: creat d1/dd/d16/d4d/fb6 x:0 0 0 2024-04-02T04:47:50.005 INFO:tasks.workunit.client.1.smithi179.stdout:2/309: chown dc 0 1 2024-04-02T04:47:50.005 INFO:tasks.workunit.client.1.smithi179.stdout:4/517: creat d1/d15/d6e/d9c/fac x:0 0 0 2024-04-02T04:47:50.011 INFO:tasks.workunit.client.1.smithi179.stdout:0/493: mkdir d1/da/d52/d75/d9a 0 2024-04-02T04:47:50.011 INFO:tasks.workunit.client.1.smithi179.stdout:7/516: truncate d1/d9/d7e/f8a 218981 0 2024-04-02T04:47:50.011 INFO:tasks.workunit.client.0.smithi070.stdout:7/434: sync 2024-04-02T04:47:50.012 INFO:tasks.workunit.client.1.smithi179.stdout:7/517: dread - d1/dd/f99 zero size 2024-04-02T04:47:50.012 INFO:tasks.workunit.client.1.smithi179.stdout:1/463: dread d3/d5/f60 [0,4194304] 0 2024-04-02T04:47:50.012 INFO:tasks.workunit.client.1.smithi179.stdout:4/518: symlink d1/d3/dc/d26/d8a/d5d/lad 0 2024-04-02T04:47:50.014 INFO:tasks.workunit.client.1.smithi179.stdout:2/310: rename dc/c1b to dc/d29/d31/c5c 0 2024-04-02T04:47:50.014 INFO:tasks.workunit.client.1.smithi179.stdout:7/518: rename d1 to d1/dd/d23/db7 22 2024-04-02T04:47:50.015 INFO:tasks.workunit.client.1.smithi179.stdout:0/494: creat d1/da/d28/d34/d39/d77/f9b x:0 0 0 2024-04-02T04:47:50.016 INFO:tasks.workunit.client.0.smithi070.stdout:7/435: rename d7/dd/df/d7a/d6c/d77/c44 to d7/dd/df/d7a/d6c/d77/d54/c8a 0 2024-04-02T04:47:50.016 INFO:tasks.workunit.client.1.smithi179.stdout:1/464: mkdir d3/d85/da2 0 2024-04-02T04:47:50.018 INFO:tasks.workunit.client.1.smithi179.stdout:1/465: chown d3/d5/d2f/d4f/d42/d53/d77 218858 1 2024-04-02T04:47:50.019 INFO:tasks.workunit.client.1.smithi179.stdout:7/519: creat d1/d9/d7e/fb8 x:0 0 0 2024-04-02T04:47:50.019 INFO:tasks.workunit.client.0.smithi070.stdout:7/436: fsync d7/d9/f21 0 2024-04-02T04:47:50.019 INFO:tasks.workunit.client.1.smithi179.stdout:4/519: dwrite d1/d3/df/d10/fa1 [0,4194304] 0 2024-04-02T04:47:50.020 INFO:tasks.workunit.client.1.smithi179.stdout:7/520: dread - d1/dd/f90 zero size 2024-04-02T04:47:50.024 INFO:tasks.workunit.client.0.smithi070.stdout:7/437: link d7/dd/df/d34/c86 d7/dd/df/d34/d68/c8b 0 2024-04-02T04:47:50.025 INFO:tasks.workunit.client.0.smithi070.stdout:7/438: write d7/d36/f72 [427465,111916] 0 2024-04-02T04:47:50.026 INFO:tasks.workunit.client.1.smithi179.stdout:7/521: rename d1/d84 to d1/d9/d27/d87/d8b/d75/db9 0 2024-04-02T04:47:50.030 INFO:tasks.workunit.client.0.smithi070.stdout:7/439: creat d7/dd/df/d79/d46/d65/f8c x:0 0 0 2024-04-02T04:47:50.031 INFO:tasks.workunit.client.1.smithi179.stdout:7/522: truncate d1/d9/d27/d87/d8b/d75/f9a 625014 0 2024-04-02T04:47:50.032 INFO:tasks.workunit.client.1.smithi179.stdout:4/520: getdents d1/d3/df/d10/d2f/d36 0 2024-04-02T04:47:50.032 INFO:tasks.workunit.client.1.smithi179.stdout:7/523: chown d1/dd/d23/l28 33462318 1 2024-04-02T04:47:50.033 INFO:tasks.workunit.client.0.smithi070.stdout:7/440: creat d7/dd/f8d x:0 0 0 2024-04-02T04:47:50.034 INFO:tasks.workunit.client.1.smithi179.stdout:7/524: creat d1/dd/d23/dab/fba x:0 0 0 2024-04-02T04:47:50.034 INFO:tasks.workunit.client.1.smithi179.stdout:4/521: mkdir d1/d3/dc/d27/dae 0 2024-04-02T04:47:50.035 INFO:tasks.workunit.client.1.smithi179.stdout:7/525: mknod d1/d9/d35/cbb 0 2024-04-02T04:47:50.037 INFO:tasks.workunit.client.1.smithi179.stdout:4/522: link d1/d15/f56 d1/d3/df/d10/d3a/faf 0 2024-04-02T04:47:50.038 INFO:tasks.workunit.client.1.smithi179.stdout:4/523: chown d1/d3/l5c 281556 1 2024-04-02T04:47:50.039 INFO:tasks.workunit.client.1.smithi179.stdout:4/524: write d1/d3/dc/d26/d8a/d5d/f89 [914526,117372] 0 2024-04-02T04:47:50.039 INFO:tasks.workunit.client.1.smithi179.stdout:7/526: creat d1/dd/d16/fbc x:0 0 0 2024-04-02T04:47:50.040 INFO:tasks.workunit.client.1.smithi179.stdout:4/525: mknod d1/d93/cb0 0 2024-04-02T04:47:50.045 INFO:tasks.workunit.client.1.smithi179.stdout:7/527: dwrite d1/dd/d23/dab/fba [0,4194304] 0 2024-04-02T04:47:50.048 INFO:tasks.workunit.client.1.smithi179.stdout:7/528: creat d1/d9/d27/d87/d8b/d75/fbd x:0 0 0 2024-04-02T04:47:50.050 INFO:tasks.workunit.client.1.smithi179.stdout:7/529: creat d1/d9/d27/d87/fbe x:0 0 0 2024-04-02T04:47:50.053 INFO:tasks.workunit.client.1.smithi179.stdout:4/526: dread d1/d3/df/d10/d2f/d60/f6c [0,4194304] 0 2024-04-02T04:47:50.054 INFO:tasks.workunit.client.1.smithi179.stdout:4/527: mknod d1/d3/dc/d7a/cb1 0 2024-04-02T04:47:50.066 INFO:tasks.workunit.client.1.smithi179.stdout:4/528: dread d1/d3/dc/fd [0,4194304] 0 2024-04-02T04:47:50.067 INFO:tasks.workunit.client.1.smithi179.stdout:4/529: write d1/d15/f79 [785146,55727] 0 2024-04-02T04:47:50.140 INFO:tasks.workunit.client.0.smithi070.stdout:6/353: truncate d0/f26 3161662 0 2024-04-02T04:47:50.142 INFO:tasks.workunit.client.1.smithi179.stdout:9/449: write d0/d1/d11/f51 [134931,104970] 0 2024-04-02T04:47:50.146 INFO:tasks.workunit.client.0.smithi070.stdout:5/495: dwrite d5/d17/d27/f28 [0,4194304] 0 2024-04-02T04:47:50.147 INFO:tasks.workunit.client.0.smithi070.stdout:6/354: creat d0/d3/d19/d31/f74 x:0 0 0 2024-04-02T04:47:50.149 INFO:tasks.workunit.client.0.smithi070.stdout:5/496: readlink d5/d66/d75/l87 0 2024-04-02T04:47:50.150 INFO:tasks.workunit.client.0.smithi070.stdout:5/497: stat d5/d9/d23/d33 0 2024-04-02T04:47:50.153 INFO:tasks.workunit.client.0.smithi070.stdout:5/498: symlink d5/d17/d50/l9d 0 2024-04-02T04:47:50.157 INFO:tasks.workunit.client.0.smithi070.stdout:5/499: symlink d5/d17/d27/d2e/l9e 0 2024-04-02T04:47:50.158 INFO:tasks.workunit.client.0.smithi070.stdout:6/355: dwrite d0/d3/d19/d31/d46/d6b/f72 [0,4194304] 0 2024-04-02T04:47:50.160 INFO:tasks.workunit.client.1.smithi179.stdout:3/487: truncate d8/d26/d36/f59 1812390 0 2024-04-02T04:47:50.160 INFO:tasks.workunit.client.1.smithi179.stdout:3/488: dread - d8/d57/d8f/f92 zero size 2024-04-02T04:47:50.162 INFO:tasks.workunit.client.1.smithi179.stdout:9/450: dread d0/d19/d1c/f41 [0,4194304] 0 2024-04-02T04:47:50.162 INFO:tasks.workunit.client.0.smithi070.stdout:6/356: symlink d0/d3/d19/d31/l75 0 2024-04-02T04:47:50.164 INFO:tasks.workunit.client.0.smithi070.stdout:5/500: dwrite d5/ff [0,4194304] 0 2024-04-02T04:47:50.170 INFO:tasks.workunit.client.0.smithi070.stdout:5/501: dread d5/d66/d75/f80 [0,4194304] 0 2024-04-02T04:47:50.171 INFO:tasks.workunit.client.0.smithi070.stdout:5/502: read d5/d9/d23/f48 [576449,112396] 0 2024-04-02T04:47:50.175 INFO:tasks.workunit.client.0.smithi070.stdout:6/357: symlink d0/d3/d19/l76 0 2024-04-02T04:47:50.179 INFO:tasks.workunit.client.1.smithi179.stdout:3/489: getdents d8/d26/d36 0 2024-04-02T04:47:50.179 INFO:tasks.workunit.client.1.smithi179.stdout:3/490: fsync d8/d26/f2c 0 2024-04-02T04:47:50.180 INFO:tasks.workunit.client.0.smithi070.stdout:5/503: creat d5/d17/d27/d2e/f9f x:0 0 0 2024-04-02T04:47:50.181 INFO:tasks.workunit.client.0.smithi070.stdout:6/358: chown d0/d3/d19/f1f 3 1 2024-04-02T04:47:50.181 INFO:tasks.workunit.client.1.smithi179.stdout:5/410: truncate d8/f1d 373942 0 2024-04-02T04:47:50.182 INFO:tasks.workunit.client.1.smithi179.stdout:6/444: write d3/ff [944505,84846] 0 2024-04-02T04:47:50.184 INFO:tasks.workunit.client.0.smithi070.stdout:5/504: mknod d5/d17/d6c/ca0 0 2024-04-02T04:47:50.184 INFO:tasks.workunit.client.1.smithi179.stdout:5/411: write d8/d1b/d23/d30/d5f/f79 [411396,61582] 0 2024-04-02T04:47:50.186 INFO:tasks.workunit.client.1.smithi179.stdout:3/491: dwrite d8/d26/f62 [0,4194304] 0 2024-04-02T04:47:50.187 INFO:tasks.workunit.client.1.smithi179.stdout:3/492: fdatasync d8/d26/f69 0 2024-04-02T04:47:50.187 INFO:tasks.workunit.client.0.smithi070.stdout:5/505: mknod d5/d66/d75/ca1 0 2024-04-02T04:47:50.188 INFO:tasks.workunit.client.1.smithi179.stdout:3/493: readlink d8/d26/l67 0 2024-04-02T04:47:50.189 INFO:tasks.workunit.client.1.smithi179.stdout:6/445: mkdir d3/d9a 0 2024-04-02T04:47:50.192 INFO:tasks.workunit.client.1.smithi179.stdout:3/494: symlink d8/d26/d83/l96 0 2024-04-02T04:47:50.193 INFO:tasks.workunit.client.1.smithi179.stdout:5/412: link d8/d1b/d23/d36/f41 d8/d1b/d23/d32/f7f 0 2024-04-02T04:47:50.194 INFO:tasks.workunit.client.1.smithi179.stdout:5/413: chown d8/d1b/d23/d42/d5a/f76 0 1 2024-04-02T04:47:50.195 INFO:tasks.workunit.client.1.smithi179.stdout:7/530: sync 2024-04-02T04:47:50.199 INFO:tasks.workunit.client.1.smithi179.stdout:7/531: rename d1/dd/d16/d74 to d1/d9/d55/dbf 0 2024-04-02T04:47:50.202 INFO:tasks.workunit.client.1.smithi179.stdout:2/311: write dc/d29/d4a/f48 [1271134,34624] 0 2024-04-02T04:47:50.203 INFO:tasks.workunit.client.1.smithi179.stdout:0/495: truncate d1/da/d52/f5d 3705534 0 2024-04-02T04:47:50.204 INFO:tasks.workunit.client.1.smithi179.stdout:1/466: truncate d3/d5/f6d 112965 0 2024-04-02T04:47:50.204 INFO:tasks.workunit.client.0.smithi070.stdout:1/450: sync 2024-04-02T04:47:50.205 INFO:tasks.workunit.client.1.smithi179.stdout:6/446: dread d3/d12/d23/d26/d57/f99 [0,4194304] 0 2024-04-02T04:47:50.206 INFO:tasks.workunit.client.0.smithi070.stdout:1/451: write d0/d4c/d5a/f73 [176251,93059] 0 2024-04-02T04:47:50.207 INFO:tasks.workunit.client.1.smithi179.stdout:7/532: dwrite d1/dd/d16/f33 [0,4194304] 0 2024-04-02T04:47:50.210 INFO:tasks.workunit.client.1.smithi179.stdout:7/533: read d1/f15 [400262,80119] 0 2024-04-02T04:47:50.212 INFO:tasks.workunit.client.1.smithi179.stdout:6/447: rmdir d3/d2d/d43 39 2024-04-02T04:47:50.213 INFO:tasks.workunit.client.0.smithi070.stdout:1/452: mknod d0/d9/d1d/d22/c91 0 2024-04-02T04:47:50.216 INFO:tasks.workunit.client.1.smithi179.stdout:2/312: creat dc/f5d x:0 0 0 2024-04-02T04:47:50.219 INFO:tasks.workunit.client.1.smithi179.stdout:0/496: dread d1/da/d28/d37/f3f [0,4194304] 0 2024-04-02T04:47:50.220 INFO:tasks.workunit.client.0.smithi070.stdout:1/453: dwrite d0/d9/d1d/d22/d2c/f4a [0,4194304] 0 2024-04-02T04:47:50.223 INFO:tasks.workunit.client.1.smithi179.stdout:0/497: readlink d1/da/df/l55 0 2024-04-02T04:47:50.224 INFO:tasks.workunit.client.0.smithi070.stdout:1/454: dread d0/d9/d1d/d22/d2c/f76 [0,4194304] 0 2024-04-02T04:47:50.225 INFO:tasks.workunit.client.1.smithi179.stdout:1/467: dwrite d3/d5/f60 [0,4194304] 0 2024-04-02T04:47:50.226 INFO:tasks.workunit.client.1.smithi179.stdout:7/534: dwrite d1/d9/d7e/fb8 [0,4194304] 0 2024-04-02T04:47:50.231 INFO:tasks.workunit.client.0.smithi070.stdout:6/359: rmdir d0/d3/d19/d31 39 2024-04-02T04:47:50.235 INFO:tasks.workunit.client.1.smithi179.stdout:0/498: stat d1/db/d64/f99 0 2024-04-02T04:47:50.236 INFO:tasks.workunit.client.1.smithi179.stdout:1/468: symlink d3/d5/de/la3 0 2024-04-02T04:47:50.237 INFO:tasks.workunit.client.1.smithi179.stdout:4/530: dwrite d1/d3/df/d10/f2d [0,4194304] 0 2024-04-02T04:47:50.239 INFO:tasks.workunit.client.0.smithi070.stdout:6/360: write d0/d3/d19/d31/d46/f68 [3635764,84246] 0 2024-04-02T04:47:50.239 INFO:tasks.workunit.client.0.smithi070.stdout:1/455: link d0/d7/d1a/f24 d0/d9/d1d/d22/d2c/d4f/f92 0 2024-04-02T04:47:50.239 INFO:tasks.workunit.client.1.smithi179.stdout:4/531: write d1/fb [3027075,129949] 0 2024-04-02T04:47:50.245 INFO:tasks.workunit.client.0.smithi070.stdout:6/361: unlink d0/d20/d5f/l73 0 2024-04-02T04:47:50.246 INFO:tasks.workunit.client.1.smithi179.stdout:8/425: dwrite d3/d32/d21/d22/d6b/d91/f40 [4194304,4194304] 0 2024-04-02T04:47:50.247 INFO:tasks.workunit.client.0.smithi070.stdout:6/362: write d0/d3/d19/d31/d46/f68 [1806238,130585] 0 2024-04-02T04:47:50.250 INFO:tasks.workunit.client.1.smithi179.stdout:9/451: dwrite d0/d1/f36 [0,4194304] 0 2024-04-02T04:47:50.250 INFO:tasks.workunit.client.1.smithi179.stdout:7/535: creat d1/dd/d16/d4d/db1/fc0 x:0 0 0 2024-04-02T04:47:50.251 INFO:tasks.workunit.client.1.smithi179.stdout:0/499: mkdir d1/da/d28/d34/d9c 0 2024-04-02T04:47:50.251 INFO:tasks.workunit.client.1.smithi179.stdout:7/536: fsync d1/dd/f17 0 2024-04-02T04:47:50.252 INFO:tasks.workunit.client.0.smithi070.stdout:5/506: rename d5/d66 to d5/d9/d23/d30/da2 0 2024-04-02T04:47:50.252 INFO:tasks.workunit.client.1.smithi179.stdout:7/537: chown d1/dd/d16/d4d 1 1 2024-04-02T04:47:50.253 INFO:tasks.workunit.client.0.smithi070.stdout:5/507: read d5/d9/d23/d30/f79 [715817,65305] 0 2024-04-02T04:47:50.259 INFO:tasks.workunit.client.1.smithi179.stdout:1/469: rmdir d3/d5/d2f/d4f/d35/d49 39 2024-04-02T04:47:50.260 INFO:tasks.workunit.client.0.smithi070.stdout:6/363: truncate d0/f15 1289278 0 2024-04-02T04:47:50.261 INFO:tasks.workunit.client.1.smithi179.stdout:3/495: truncate d8/d7a/f8a 4118283 0 2024-04-02T04:47:50.262 INFO:tasks.workunit.client.1.smithi179.stdout:3/496: write d8/d57/d63/f84 [2513411,26200] 0 2024-04-02T04:47:50.263 INFO:tasks.workunit.client.1.smithi179.stdout:8/426: mkdir d3/d6c/d96 0 2024-04-02T04:47:50.268 INFO:tasks.workunit.client.0.smithi070.stdout:5/508: creat d5/d9/d10/d51/fa3 x:0 0 0 2024-04-02T04:47:50.270 INFO:tasks.workunit.client.1.smithi179.stdout:5/414: dwrite d8/f13 [0,4194304] 0 2024-04-02T04:47:50.270 INFO:tasks.workunit.client.0.smithi070.stdout:1/456: link d0/d7/d1a/l49 d0/d4c/d5a/l93 0 2024-04-02T04:47:50.271 INFO:tasks.workunit.client.0.smithi070.stdout:1/457: write d0/d9/d1d/f35 [1504361,31973] 0 2024-04-02T04:47:50.273 INFO:tasks.workunit.client.0.smithi070.stdout:6/364: mkdir d0/d20/d77 0 2024-04-02T04:47:50.275 INFO:tasks.workunit.client.1.smithi179.stdout:7/538: mknod d1/dd/d23/dab/cc1 0 2024-04-02T04:47:50.278 INFO:tasks.workunit.client.1.smithi179.stdout:1/470: symlink d3/d5/d2f/d4f/d6a/la4 0 2024-04-02T04:47:50.280 INFO:tasks.workunit.client.0.smithi070.stdout:1/458: mkdir d0/d9/d94 0 2024-04-02T04:47:50.281 INFO:tasks.workunit.client.1.smithi179.stdout:3/497: creat d8/d26/d52/d85/f97 x:0 0 0 2024-04-02T04:47:50.282 INFO:tasks.workunit.client.0.smithi070.stdout:4/396: sync 2024-04-02T04:47:50.282 INFO:tasks.workunit.client.0.smithi070.stdout:0/505: sync 2024-04-02T04:47:50.282 INFO:tasks.workunit.client.0.smithi070.stdout:2/460: sync 2024-04-02T04:47:50.282 INFO:tasks.workunit.client.0.smithi070.stdout:8/469: sync 2024-04-02T04:47:50.282 INFO:tasks.workunit.client.0.smithi070.stdout:9/409: sync 2024-04-02T04:47:50.282 INFO:tasks.workunit.client.0.smithi070.stdout:7/441: sync 2024-04-02T04:47:50.283 INFO:tasks.workunit.client.0.smithi070.stdout:3/461: sync 2024-04-02T04:47:50.283 INFO:tasks.workunit.client.1.smithi179.stdout:6/448: truncate d3/d12/d44/f52 49199 0 2024-04-02T04:47:50.283 INFO:tasks.workunit.client.1.smithi179.stdout:6/449: readlink d3/d42/l7b 0 2024-04-02T04:47:50.284 INFO:tasks.workunit.client.1.smithi179.stdout:3/498: read d8/f11 [1809349,27586] 0 2024-04-02T04:47:50.284 INFO:tasks.workunit.client.1.smithi179.stdout:8/427: rename d3/d1b/d2a/c78 to d3/d32/d21/d49/c97 0 2024-04-02T04:47:50.285 INFO:tasks.workunit.client.0.smithi070.stdout:2/461: write d2/d10/d21/f89 [586269,25567] 0 2024-04-02T04:47:50.285 INFO:tasks.workunit.client.1.smithi179.stdout:2/313: write dc/d29/d4a/f24 [2607374,113472] 0 2024-04-02T04:47:50.286 INFO:tasks.workunit.client.0.smithi070.stdout:2/462: read - d2/d10/d21/d2b/f7d zero size 2024-04-02T04:47:50.287 INFO:tasks.workunit.client.1.smithi179.stdout:9/452: mknod d0/d1/d11/c8a 0 2024-04-02T04:47:50.287 INFO:tasks.workunit.client.0.smithi070.stdout:2/463: dread - d2/d10/f78 zero size 2024-04-02T04:47:50.288 INFO:tasks.workunit.client.1.smithi179.stdout:9/453: chown d0/d1/d11/d3a/c4a 207520 1 2024-04-02T04:47:50.289 INFO:tasks.workunit.client.0.smithi070.stdout:1/459: mkdir d0/d7/d1a/d95 0 2024-04-02T04:47:50.291 INFO:tasks.workunit.client.1.smithi179.stdout:2/314: dwrite dc/d29/d31/f3a [0,4194304] 0 2024-04-02T04:47:50.292 INFO:tasks.workunit.client.0.smithi070.stdout:6/365: dread d0/d20/f23 [0,4194304] 0 2024-04-02T04:47:50.295 INFO:tasks.workunit.client.1.smithi179.stdout:7/539: read d1/dd/d23/f57 [346112,61864] 0 2024-04-02T04:47:50.297 INFO:tasks.workunit.client.0.smithi070.stdout:0/506: chown d4/d5/d13/d2a/d3a/c3e 4 1 2024-04-02T04:47:50.298 INFO:tasks.workunit.client.1.smithi179.stdout:7/540: stat d1/d9/c22 0 2024-04-02T04:47:50.298 INFO:tasks.workunit.client.1.smithi179.stdout:9/454: dwrite d0/d1/f7b [0,4194304] 0 2024-04-02T04:47:50.299 INFO:tasks.workunit.client.1.smithi179.stdout:9/455: write d0/d1/d11/f51 [156153,58078] 0 2024-04-02T04:47:50.300 INFO:tasks.workunit.client.1.smithi179.stdout:2/315: dread dc/d29/d4a/f15 [0,4194304] 0 2024-04-02T04:47:50.301 INFO:tasks.workunit.client.1.smithi179.stdout:6/450: write d3/d12/d23/d26/d57/f99 [1514339,20369] 0 2024-04-02T04:47:50.304 INFO:tasks.workunit.client.1.smithi179.stdout:4/532: write d1/d3/d19/f75 [657213,59106] 0 2024-04-02T04:47:50.304 INFO:tasks.workunit.client.0.smithi070.stdout:0/507: dwrite d4/d5/d13/d2a/f68 [0,4194304] 0 2024-04-02T04:47:50.306 INFO:tasks.workunit.client.0.smithi070.stdout:2/464: symlink d2/d24/l91 0 2024-04-02T04:47:50.308 INFO:tasks.workunit.client.1.smithi179.stdout:5/415: rename d8/f13 to d8/d1b/d23/d32/d4e/d50/f80 0 2024-04-02T04:47:50.311 INFO:tasks.workunit.client.1.smithi179.stdout:8/428: mknod d3/d31/c98 0 2024-04-02T04:47:50.311 INFO:tasks.workunit.client.0.smithi070.stdout:1/460: creat d0/d4c/f96 x:0 0 0 2024-04-02T04:47:50.312 INFO:tasks.workunit.client.1.smithi179.stdout:5/416: chown d8/d1b/d23/d32/d4b 17126 1 2024-04-02T04:47:50.312 INFO:tasks.workunit.client.1.smithi179.stdout:9/456: dwrite d0/d19/d1c/d45/f66 [0,4194304] 0 2024-04-02T04:47:50.316 INFO:tasks.workunit.client.0.smithi070.stdout:0/508: dwrite d4/d5/d13/f32 [0,4194304] 0 2024-04-02T04:47:50.318 INFO:tasks.workunit.client.1.smithi179.stdout:6/451: dwrite d3/d12/d6a/f6e [0,4194304] 0 2024-04-02T04:47:50.318 INFO:tasks.workunit.client.0.smithi070.stdout:3/462: mknod d10/d17/d51/d73/d9e/ca2 0 2024-04-02T04:47:50.319 INFO:tasks.workunit.client.0.smithi070.stdout:1/461: dread d0/d9/fc [0,4194304] 0 2024-04-02T04:47:50.321 INFO:tasks.workunit.client.0.smithi070.stdout:1/462: stat d0/d9/d1d/f42 0 2024-04-02T04:47:50.327 INFO:tasks.workunit.client.0.smithi070.stdout:5/509: getdents d5/d17/d50 0 2024-04-02T04:47:50.328 INFO:tasks.workunit.client.0.smithi070.stdout:0/509: dread d4/d3c/f6a [0,4194304] 0 2024-04-02T04:47:50.336 INFO:tasks.workunit.client.0.smithi070.stdout:2/465: creat d2/d10/d5c/d7a/f92 x:0 0 0 2024-04-02T04:47:50.343 INFO:tasks.workunit.client.0.smithi070.stdout:7/442: write d7/dd/df/d7a/f5a [1707598,113529] 0 2024-04-02T04:47:50.344 INFO:tasks.workunit.client.1.smithi179.stdout:1/471: write d3/fa [3874203,50952] 0 2024-04-02T04:47:50.347 INFO:tasks.workunit.client.0.smithi070.stdout:3/463: creat d10/d17/d4c/fa3 x:0 0 0 2024-04-02T04:47:50.348 INFO:tasks.workunit.client.0.smithi070.stdout:3/464: rename d10/d32/d8a to d10/d32/d8a/d8d/d91/da4 22 2024-04-02T04:47:50.348 INFO:tasks.workunit.client.1.smithi179.stdout:4/533: unlink d1/d3/d19/c95 0 2024-04-02T04:47:50.350 INFO:tasks.workunit.client.1.smithi179.stdout:7/541: rename d1/d9/f43 to d1/dd/d23/fc2 0 2024-04-02T04:47:50.352 INFO:tasks.workunit.client.0.smithi070.stdout:5/510: symlink d5/d9/d23/d30/da2/d75/la4 0 2024-04-02T04:47:50.352 INFO:tasks.workunit.client.0.smithi070.stdout:5/511: chown d5/d17/d2b/d7b 25067 1 2024-04-02T04:47:50.353 INFO:tasks.workunit.client.1.smithi179.stdout:5/417: fdatasync f2 0 2024-04-02T04:47:50.353 INFO:tasks.workunit.client.0.smithi070.stdout:5/512: read d5/d17/d27/f28 [2226430,23235] 0 2024-04-02T04:47:50.355 INFO:tasks.workunit.client.0.smithi070.stdout:5/513: write d5/d9/d10/d51/fa3 [311826,57663] 0 2024-04-02T04:47:50.356 INFO:tasks.workunit.client.0.smithi070.stdout:5/514: chown d5/d17/d2b/d7b 24658941 1 2024-04-02T04:47:50.356 INFO:tasks.workunit.client.0.smithi070.stdout:2/466: rmdir d2/d10/d53 39 2024-04-02T04:47:50.359 INFO:tasks.workunit.client.0.smithi070.stdout:9/410: getdents d4 0 2024-04-02T04:47:50.361 INFO:tasks.workunit.client.1.smithi179.stdout:4/534: dread d1/d3/f7 [0,4194304] 0 2024-04-02T04:47:50.362 INFO:tasks.workunit.client.0.smithi070.stdout:5/515: dwrite d5/d17/d2b/d7b/f92 [0,4194304] 0 2024-04-02T04:47:50.363 INFO:tasks.workunit.client.0.smithi070.stdout:6/366: creat d0/f78 x:0 0 0 2024-04-02T04:47:50.366 INFO:tasks.workunit.client.1.smithi179.stdout:6/452: mkdir d3/d12/d23/d26/d9b 0 2024-04-02T04:47:50.368 INFO:tasks.workunit.client.1.smithi179.stdout:6/453: write d3/d12/d23/d39/d4e/d6c/f50 [5217117,57829] 0 2024-04-02T04:47:50.369 INFO:tasks.workunit.client.0.smithi070.stdout:3/465: mkdir d10/d17/d51/da5 0 2024-04-02T04:47:50.370 INFO:tasks.workunit.client.0.smithi070.stdout:3/466: fdatasync d10/d32/f71 0 2024-04-02T04:47:50.372 INFO:tasks.workunit.client.1.smithi179.stdout:2/316: creat dc/d29/d31/d57/f5e x:0 0 0 2024-04-02T04:47:50.374 INFO:tasks.workunit.client.1.smithi179.stdout:3/499: creat d8/f98 x:0 0 0 2024-04-02T04:47:50.374 INFO:tasks.workunit.client.0.smithi070.stdout:0/510: creat d4/d5/d13/d17/d72/f90 x:0 0 0 2024-04-02T04:47:50.376 INFO:tasks.workunit.client.0.smithi070.stdout:0/511: write d4/d5/da/d2d/f7d [5235226,97982] 0 2024-04-02T04:47:50.377 INFO:tasks.workunit.client.0.smithi070.stdout:0/512: fsync d4/d5/d13/d2a/f20 0 2024-04-02T04:47:50.378 INFO:tasks.workunit.client.1.smithi179.stdout:2/317: dwrite dc/d29/d4a/f56 [0,4194304] 0 2024-04-02T04:47:50.379 INFO:tasks.workunit.client.1.smithi179.stdout:2/318: chown dc/d29/d4a/d1e/l59 26713 1 2024-04-02T04:47:50.381 INFO:tasks.workunit.client.0.smithi070.stdout:6/367: mkdir d0/d55/d79 0 2024-04-02T04:47:50.382 INFO:tasks.workunit.client.1.smithi179.stdout:7/542: truncate d1/dd/d23/f57 1115479 0 2024-04-02T04:47:50.382 INFO:tasks.workunit.client.0.smithi070.stdout:6/368: fdatasync d0/d3/d19/d31/d37/f5a 0 2024-04-02T04:47:50.383 INFO:tasks.workunit.client.1.smithi179.stdout:9/457: write d0/d1/d10/f18 [10477142,48706] 0 2024-04-02T04:47:50.385 INFO:tasks.workunit.client.0.smithi070.stdout:7/443: mkdir d7/d8e 0 2024-04-02T04:47:50.387 INFO:tasks.workunit.client.0.smithi070.stdout:7/444: write d7/dd/df/d79/d46/d65/f69 [9416114,88131] 0 2024-04-02T04:47:50.389 INFO:tasks.workunit.client.0.smithi070.stdout:5/516: dwrite d5/d9/d23/f74 [0,4194304] 0 2024-04-02T04:47:50.393 INFO:tasks.workunit.client.0.smithi070.stdout:1/463: link d0/d7/d28/c3a d0/d9/d1d/d22/d56/d79/d89/c97 0 2024-04-02T04:47:50.394 INFO:tasks.workunit.client.1.smithi179.stdout:4/535: rename d1/d3/dc/d27/d62/l73 to d1/d93/lb2 0 2024-04-02T04:47:50.397 INFO:tasks.workunit.client.0.smithi070.stdout:5/517: dwrite d5/ff [8388608,4194304] 0 2024-04-02T04:47:50.398 INFO:tasks.workunit.client.0.smithi070.stdout:5/518: read - d5/d14/f68 zero size 2024-04-02T04:47:50.403 INFO:tasks.workunit.client.1.smithi179.stdout:0/500: getdents d1/da/df/d56/d84 0 2024-04-02T04:47:50.409 INFO:tasks.workunit.client.1.smithi179.stdout:3/500: rmdir d8/d26 39 2024-04-02T04:47:50.412 INFO:tasks.workunit.client.1.smithi179.stdout:5/418: dwrite d8/f18 [0,4194304] 0 2024-04-02T04:47:50.413 INFO:tasks.workunit.client.1.smithi179.stdout:1/472: chown d3/d5/de/d33/c52 51 1 2024-04-02T04:47:50.417 INFO:tasks.workunit.client.1.smithi179.stdout:5/419: dread - d8/d1b/d23/d42/d5a/f76 zero size 2024-04-02T04:47:50.417 INFO:tasks.workunit.client.1.smithi179.stdout:9/458: chown d0/d1/d10/d6d/l39 9 1 2024-04-02T04:47:50.417 INFO:tasks.workunit.client.1.smithi179.stdout:8/429: dwrite d3/d32/d21/d50/d56/d94/f60 [0,4194304] 0 2024-04-02T04:47:50.418 INFO:tasks.workunit.client.0.smithi070.stdout:2/467: rmdir d2/d12/d85 0 2024-04-02T04:47:50.419 INFO:tasks.workunit.client.1.smithi179.stdout:6/454: write d3/d12/d23/d26/f37 [3568466,9428] 0 2024-04-02T04:47:50.421 INFO:tasks.workunit.client.1.smithi179.stdout:4/536: truncate d1/d3/dc/d26/d8a/f55 5057374 0 2024-04-02T04:47:50.421 INFO:tasks.workunit.client.0.smithi070.stdout:3/467: truncate d10/d32/d3f/f43 2824230 0 2024-04-02T04:47:50.422 INFO:tasks.workunit.client.0.smithi070.stdout:6/369: mknod d0/d55/d79/c7a 0 2024-04-02T04:47:50.426 INFO:tasks.workunit.client.0.smithi070.stdout:1/464: write d0/f1c [4118720,121479] 0 2024-04-02T04:47:50.427 INFO:tasks.workunit.client.1.smithi179.stdout:6/455: dwrite d3/d42/f7e [0,4194304] 0 2024-04-02T04:47:50.427 INFO:tasks.workunit.client.0.smithi070.stdout:5/519: creat d5/d9/d23/d8b/fa5 x:0 0 0 2024-04-02T04:47:50.433 INFO:tasks.workunit.client.0.smithi070.stdout:3/468: mknod d10/d17/d51/d73/ca6 0 2024-04-02T04:47:50.434 INFO:tasks.workunit.client.0.smithi070.stdout:7/445: link d7/dd/df/d79/c3d d7/dd/df/d79/d46/d65/d7b/c8f 0 2024-04-02T04:47:50.434 INFO:tasks.workunit.client.0.smithi070.stdout:6/370: dread d0/d3/d19/d31/d37/f5a [0,4194304] 0 2024-04-02T04:47:50.437 INFO:tasks.workunit.client.0.smithi070.stdout:5/520: creat d5/d17/d50/fa6 x:0 0 0 2024-04-02T04:47:50.439 INFO:tasks.workunit.client.0.smithi070.stdout:5/521: chown d5/d9/d23/f48 20976544 1 2024-04-02T04:47:50.442 INFO:tasks.workunit.client.0.smithi070.stdout:0/513: link d4/d5/d13/d2a/d29/c11 d4/d5/c91 0 2024-04-02T04:47:50.445 INFO:tasks.workunit.client.0.smithi070.stdout:3/469: unlink d10/d17/d51/d73/f80 0 2024-04-02T04:47:50.446 INFO:tasks.workunit.client.1.smithi179.stdout:8/430: fdatasync d3/d32/d21/d49/f72 0 2024-04-02T04:47:50.447 INFO:tasks.workunit.client.0.smithi070.stdout:7/446: unlink d7/dd/df/d7a/c7e 0 2024-04-02T04:47:50.449 INFO:tasks.workunit.client.1.smithi179.stdout:2/319: truncate dc/d29/d4a/f48 1172232 0 2024-04-02T04:47:50.452 INFO:tasks.workunit.client.1.smithi179.stdout:6/456: mknod d3/d42/c9c 0 2024-04-02T04:47:50.452 INFO:tasks.workunit.client.1.smithi179.stdout:8/431: dwrite d3/d32/d21/d51/f55 [0,4194304] 0 2024-04-02T04:47:50.453 INFO:tasks.workunit.client.1.smithi179.stdout:3/501: stat d8/d26/d52/d7e/l82 0 2024-04-02T04:47:50.454 INFO:tasks.workunit.client.0.smithi070.stdout:5/522: chown d5/d17/d2b/d7b/l95 0 1 2024-04-02T04:47:50.455 INFO:tasks.workunit.client.1.smithi179.stdout:7/543: creat d1/dd/fc3 x:0 0 0 2024-04-02T04:47:50.456 INFO:tasks.workunit.client.0.smithi070.stdout:5/523: chown d5/d9/d23/d33/c71 16033 1 2024-04-02T04:47:50.456 INFO:tasks.workunit.client.0.smithi070.stdout:5/524: read d5/f1d [4609505,82942] 0 2024-04-02T04:47:50.457 INFO:tasks.workunit.client.0.smithi070.stdout:5/525: stat d5/f97 0 2024-04-02T04:47:50.459 INFO:tasks.workunit.client.0.smithi070.stdout:5/526: write d5/d17/d50/f58 [1260486,41551] 0 2024-04-02T04:47:50.460 INFO:tasks.workunit.client.0.smithi070.stdout:5/527: dread - d5/d14/f68 zero size 2024-04-02T04:47:50.461 INFO:tasks.workunit.client.0.smithi070.stdout:6/371: write d0/d3/d19/d31/f4e [1430082,51120] 0 2024-04-02T04:47:50.462 INFO:tasks.workunit.client.1.smithi179.stdout:7/544: dwrite d1/d9/d7e/f81 [0,4194304] 0 2024-04-02T04:47:50.463 INFO:tasks.workunit.client.0.smithi070.stdout:3/470: mknod d10/d32/d3f/ca7 0 2024-04-02T04:47:50.465 INFO:tasks.workunit.client.1.smithi179.stdout:5/420: creat d8/d1b/d23/d30/d73/f81 x:0 0 0 2024-04-02T04:47:50.468 INFO:tasks.workunit.client.1.smithi179.stdout:4/537: rename d1/d3/l35 to d1/d3/df/d10/d2f/d36/lb3 0 2024-04-02T04:47:50.468 INFO:tasks.workunit.client.0.smithi070.stdout:2/468: getdents d2/d12/d3e/d68 0 2024-04-02T04:47:50.502 INFO:tasks.workunit.client.1.smithi179.stdout:6/457: mknod d3/d12/d23/d39/d4e/d66/d7c/c9d 0 2024-04-02T04:47:50.503 INFO:tasks.workunit.client.0.smithi070.stdout:0/514: link d4/d5/d13/d43/f78 d4/d5/d58/d60/f92 0 2024-04-02T04:47:50.505 INFO:tasks.workunit.client.1.smithi179.stdout:1/473: creat d3/d5/d2f/d4f/d35/fa5 x:0 0 0 2024-04-02T04:47:50.507 INFO:tasks.workunit.client.1.smithi179.stdout:1/474: read d3/d5/f13 [3798756,109756] 0 2024-04-02T04:47:50.509 INFO:tasks.workunit.client.1.smithi179.stdout:9/459: creat d0/d1/d10/d6d/f8b x:0 0 0 2024-04-02T04:47:50.510 INFO:tasks.workunit.client.0.smithi070.stdout:6/372: mkdir d0/d3/d19/d31/d61/d7b 0 2024-04-02T04:47:50.512 INFO:tasks.workunit.client.1.smithi179.stdout:7/545: fdatasync d1/d9/d55/f61 0 2024-04-02T04:47:50.516 INFO:tasks.workunit.client.1.smithi179.stdout:8/432: dwrite d3/d32/d21/f25 [0,4194304] 0 2024-04-02T04:47:50.517 INFO:tasks.workunit.client.0.smithi070.stdout:1/465: rename d0/d7/d28/l8a to d0/l98 0 2024-04-02T04:47:50.519 INFO:tasks.workunit.client.1.smithi179.stdout:5/421: mknod d8/d1b/d23/d32/d46/c82 0 2024-04-02T04:47:50.520 INFO:tasks.workunit.client.0.smithi070.stdout:2/469: mknod d2/d12/c93 0 2024-04-02T04:47:50.521 INFO:tasks.workunit.client.1.smithi179.stdout:9/460: dwrite d0/d19/d1c/d45/f66 [0,4194304] 0 2024-04-02T04:47:50.523 INFO:tasks.workunit.client.1.smithi179.stdout:6/458: dread d3/d12/d23/d39/d4e/d6c/f50 [0,4194304] 0 2024-04-02T04:47:50.524 INFO:tasks.workunit.client.1.smithi179.stdout:9/461: stat d0/d1/f7 0 2024-04-02T04:47:50.524 INFO:tasks.workunit.client.1.smithi179.stdout:3/502: rename d8/d6d/c7c to d8/d57/c99 0 2024-04-02T04:47:50.525 INFO:tasks.workunit.client.0.smithi070.stdout:7/447: write d7/dd/df/d79/d46/f51 [2011132,13130] 0 2024-04-02T04:47:50.525 INFO:tasks.workunit.client.1.smithi179.stdout:9/462: stat d0/d1/d11/d29/f37 0 2024-04-02T04:47:50.526 INFO:tasks.workunit.client.1.smithi179.stdout:7/546: dwrite d1/dd/d23/f2f [0,4194304] 0 2024-04-02T04:47:50.526 INFO:tasks.workunit.client.1.smithi179.stdout:9/463: chown d0/d1/c21 1911 1 2024-04-02T04:47:50.526 INFO:tasks.workunit.client.0.smithi070.stdout:0/515: truncate d4/d5/d13/f19 4695353 0 2024-04-02T04:47:50.527 INFO:tasks.workunit.client.1.smithi179.stdout:7/547: fsync d1/dd/d23/dab/fba 0 2024-04-02T04:47:50.528 INFO:tasks.workunit.client.1.smithi179.stdout:4/538: rmdir d1/d3/dc/d7a 39 2024-04-02T04:47:50.528 INFO:tasks.workunit.client.0.smithi070.stdout:5/528: dread d5/d9/d10/f19 [0,4194304] 0 2024-04-02T04:47:50.529 INFO:tasks.workunit.client.0.smithi070.stdout:1/466: creat d0/d9/d1d/d22/d2c/d4f/f99 x:0 0 0 2024-04-02T04:47:50.533 INFO:tasks.workunit.client.0.smithi070.stdout:2/470: mknod d2/d12/d3e/d68/c94 0 2024-04-02T04:47:50.533 INFO:tasks.workunit.client.1.smithi179.stdout:3/503: dwrite d8/d6d/f76 [0,4194304] 0 2024-04-02T04:47:50.536 INFO:tasks.workunit.client.0.smithi070.stdout:7/448: rmdir d7/d4e/d82/d57 39 2024-04-02T04:47:50.537 INFO:tasks.workunit.client.0.smithi070.stdout:6/373: dread d0/d20/f27 [0,4194304] 0 2024-04-02T04:47:50.541 INFO:tasks.workunit.client.1.smithi179.stdout:2/320: truncate dc/d29/d4a/f24 1959156 0 2024-04-02T04:47:50.542 INFO:tasks.workunit.client.1.smithi179.stdout:4/539: dwrite d1/d3/df/d10/d2f/d7b/f86 [0,4194304] 0 2024-04-02T04:47:50.543 INFO:tasks.workunit.client.0.smithi070.stdout:3/471: creat d10/d32/fa8 x:0 0 0 2024-04-02T04:47:50.545 INFO:tasks.workunit.client.0.smithi070.stdout:0/516: write d4/d5/d13/d23/f36 [901506,118491] 0 2024-04-02T04:47:50.547 INFO:tasks.workunit.client.0.smithi070.stdout:7/449: dwrite d7/dd/df/d34/d68/f70 [0,4194304] 0 2024-04-02T04:47:50.550 INFO:tasks.workunit.client.1.smithi179.stdout:8/433: mkdir d3/d32/d21/d50/d56/d94/d99 0 2024-04-02T04:47:50.550 INFO:tasks.workunit.client.1.smithi179.stdout:5/422: symlink d8/d1b/l83 0 2024-04-02T04:47:50.554 INFO:tasks.workunit.client.0.smithi070.stdout:6/374: write d0/d3/d19/d39/f6c [5389548,103900] 0 2024-04-02T04:47:50.555 INFO:tasks.workunit.client.0.smithi070.stdout:2/471: fsync d2/f32 0 2024-04-02T04:47:50.557 INFO:tasks.workunit.client.1.smithi179.stdout:6/459: creat d3/d12/d23/d26/d57/f9e x:0 0 0 2024-04-02T04:47:50.557 INFO:tasks.workunit.client.1.smithi179.stdout:9/464: creat d0/d19/d27/d48/f8c x:0 0 0 2024-04-02T04:47:50.558 INFO:tasks.workunit.client.0.smithi070.stdout:7/450: symlink d7/d4e/l90 0 2024-04-02T04:47:50.559 INFO:tasks.workunit.client.1.smithi179.stdout:6/460: write d3/d12/d23/d26/f37 [4626594,130677] 0 2024-04-02T04:47:50.559 INFO:tasks.workunit.client.0.smithi070.stdout:0/517: dwrite d4/d5/d13/f8a [0,4194304] 0 2024-04-02T04:47:50.559 INFO:tasks.workunit.client.0.smithi070.stdout:2/472: chown d2/d10/d5c/f81 178594062 1 2024-04-02T04:47:50.560 INFO:tasks.workunit.client.1.smithi179.stdout:0/501: sync 2024-04-02T04:47:50.566 INFO:tasks.workunit.client.0.smithi070.stdout:5/529: dread d5/d17/d27/f45 [0,4194304] 0 2024-04-02T04:47:50.568 INFO:tasks.workunit.client.1.smithi179.stdout:1/475: getdents d3/d85/da2 0 2024-04-02T04:47:50.568 INFO:tasks.workunit.client.1.smithi179.stdout:2/321: creat dc/d29/d4a/d1e/d43/f5f x:0 0 0 2024-04-02T04:47:50.569 INFO:tasks.workunit.client.0.smithi070.stdout:6/375: dwrite d0/d20/f45 [0,4194304] 0 2024-04-02T04:47:50.569 INFO:tasks.workunit.client.1.smithi179.stdout:1/476: write d3/d5/d2f/d4f/f4c [371143,116280] 0 2024-04-02T04:47:50.569 INFO:tasks.workunit.client.1.smithi179.stdout:2/322: chown dc/d29/d4a/d1e/d43/f47 35749960 1 2024-04-02T04:47:50.574 INFO:tasks.workunit.client.1.smithi179.stdout:8/434: creat d3/d1b/f9a x:0 0 0 2024-04-02T04:47:50.576 INFO:tasks.workunit.client.1.smithi179.stdout:8/435: write d3/d32/d21/d22/d6b/d91/f40 [4070239,45854] 0 2024-04-02T04:47:50.576 INFO:tasks.workunit.client.0.smithi070.stdout:3/472: getdents d10/d17 0 2024-04-02T04:47:50.577 INFO:tasks.workunit.client.1.smithi179.stdout:8/436: write d3/d32/d30/f4e [969245,11706] 0 2024-04-02T04:47:50.578 INFO:tasks.workunit.client.1.smithi179.stdout:8/437: fsync d3/f82 0 2024-04-02T04:47:50.579 INFO:tasks.workunit.client.0.smithi070.stdout:2/473: rename d2/d10/d19/c35 to d2/d24/c95 0 2024-04-02T04:47:50.579 INFO:tasks.workunit.client.0.smithi070.stdout:5/530: dwrite d5/d9/d10/d6a/f81 [0,4194304] 0 2024-04-02T04:47:50.587 INFO:tasks.workunit.client.1.smithi179.stdout:6/461: rmdir d3/d12/d23/d39/d4e 39 2024-04-02T04:47:50.588 INFO:tasks.workunit.client.1.smithi179.stdout:6/462: chown d3/d2c/l53 33 1 2024-04-02T04:47:50.593 INFO:tasks.workunit.client.0.smithi070.stdout:5/531: mkdir d5/d17/da7 0 2024-04-02T04:47:50.596 INFO:tasks.workunit.client.0.smithi070.stdout:1/467: dwrite d0/d9/d1d/f13 [0,4194304] 0 2024-04-02T04:47:50.597 INFO:tasks.workunit.client.1.smithi179.stdout:4/540: creat d1/d93/d9f/fb4 x:0 0 0 2024-04-02T04:47:50.597 INFO:tasks.workunit.client.0.smithi070.stdout:2/474: dread d2/d12/f52 [0,4194304] 0 2024-04-02T04:47:50.600 INFO:tasks.workunit.client.1.smithi179.stdout:2/323: unlink dc/d29/d31/d57/f5e 0 2024-04-02T04:47:50.602 INFO:tasks.workunit.client.0.smithi070.stdout:1/468: rename d0/d9/d1d/f35 to d0/d7/d1a/f9a 0 2024-04-02T04:47:50.604 INFO:tasks.workunit.client.1.smithi179.stdout:4/541: dwrite d1/d3/df/d10/d2f/d7b/fa2 [0,4194304] 0 2024-04-02T04:47:50.605 INFO:tasks.workunit.client.0.smithi070.stdout:5/532: mknod d5/d17/d27/ca8 0 2024-04-02T04:47:50.605 INFO:tasks.workunit.client.1.smithi179.stdout:4/542: fsync d1/d3/df/d10/fa1 0 2024-04-02T04:47:50.606 INFO:tasks.workunit.client.0.smithi070.stdout:5/533: readlink d5/d17/d2b/l4c 0 2024-04-02T04:47:50.607 INFO:tasks.workunit.client.1.smithi179.stdout:7/548: creat d1/fc4 x:0 0 0 2024-04-02T04:47:50.607 INFO:tasks.workunit.client.0.smithi070.stdout:3/473: getdents d10/d17/d51/d73/d9e 0 2024-04-02T04:47:50.614 INFO:tasks.workunit.client.1.smithi179.stdout:4/543: dwrite d1/d3/df/d10/d2f/fa7 [0,4194304] 0 2024-04-02T04:47:50.614 INFO:tasks.workunit.client.0.smithi070.stdout:2/475: rename d2/d12/d59/d69/f8a to d2/d24/f96 0 2024-04-02T04:47:50.615 INFO:tasks.workunit.client.1.smithi179.stdout:3/504: dwrite d8/f11 [0,4194304] 0 2024-04-02T04:47:50.616 INFO:tasks.workunit.client.0.smithi070.stdout:2/476: chown d2/d10/d5c/f81 146 1 2024-04-02T04:47:50.616 INFO:tasks.workunit.client.1.smithi179.stdout:9/465: mknod d0/d1/d11/d29/d86/c8d 0 2024-04-02T04:47:50.619 INFO:tasks.workunit.client.1.smithi179.stdout:6/463: unlink d3/d2c/l53 0 2024-04-02T04:47:50.620 INFO:tasks.workunit.client.0.smithi070.stdout:0/518: write d4/d5/d13/d43/f78 [674782,38540] 0 2024-04-02T04:47:50.622 INFO:tasks.workunit.client.1.smithi179.stdout:0/502: mkdir d1/da/d28/d81/d9d 0 2024-04-02T04:47:50.623 INFO:tasks.workunit.client.0.smithi070.stdout:7/451: write d7/dd/df/d79/f31 [1563191,96402] 0 2024-04-02T04:47:50.627 INFO:tasks.workunit.client.1.smithi179.stdout:6/464: dread f2 [0,4194304] 0 2024-04-02T04:47:50.627 INFO:tasks.workunit.client.0.smithi070.stdout:3/474: rename f8 to d10/d66/fa9 0 2024-04-02T04:47:50.634 INFO:tasks.workunit.client.0.smithi070.stdout:3/475: dwrite d10/d17/f74 [0,4194304] 0 2024-04-02T04:47:50.639 INFO:tasks.workunit.client.0.smithi070.stdout:7/452: dread d7/d36/f62 [0,4194304] 0 2024-04-02T04:47:50.640 INFO:tasks.workunit.client.1.smithi179.stdout:2/324: symlink dc/d29/d4a/d4b/l60 0 2024-04-02T04:47:50.640 INFO:tasks.workunit.client.1.smithi179.stdout:2/325: chown dc/f46 1611 1 2024-04-02T04:47:50.646 INFO:tasks.workunit.client.0.smithi070.stdout:7/453: dwrite d7/d9/f78 [0,4194304] 0 2024-04-02T04:47:50.646 INFO:tasks.workunit.client.1.smithi179.stdout:7/549: creat d1/d9/d27/d87/d8b/d75/db9/d8d/fc5 x:0 0 0 2024-04-02T04:47:50.647 INFO:tasks.workunit.client.1.smithi179.stdout:9/466: dread d0/d19/d1c/d64/f79 [0,4194304] 0 2024-04-02T04:47:50.651 INFO:tasks.workunit.client.1.smithi179.stdout:8/438: fdatasync d3/d32/d21/d22/f2d 0 2024-04-02T04:47:50.656 INFO:tasks.workunit.client.1.smithi179.stdout:4/544: symlink d1/d3/df/d10/lb5 0 2024-04-02T04:47:50.658 INFO:tasks.workunit.client.0.smithi070.stdout:1/469: truncate d0/d7/d1a/f9a 1270550 0 2024-04-02T04:47:50.659 INFO:tasks.workunit.client.1.smithi179.stdout:3/505: mkdir d8/d26/d52/d85/d9a 0 2024-04-02T04:47:50.660 INFO:tasks.workunit.client.1.smithi179.stdout:3/506: chown d8/d26/d36/f8d 163374567 1 2024-04-02T04:47:50.660 INFO:tasks.workunit.client.1.smithi179.stdout:3/507: stat d8/d26/d36/c3d 0 2024-04-02T04:47:50.661 INFO:tasks.workunit.client.1.smithi179.stdout:1/477: truncate d3/d5/f21 4783222 0 2024-04-02T04:47:50.662 INFO:tasks.workunit.client.0.smithi070.stdout:1/470: dread d0/d7/d28/f7a [0,4194304] 0 2024-04-02T04:47:50.663 INFO:tasks.workunit.client.1.smithi179.stdout:5/423: mknod d8/d1b/d23/d32/c84 0 2024-04-02T04:47:50.664 INFO:tasks.workunit.client.0.smithi070.stdout:1/471: chown d0/d4c/f51 3770 1 2024-04-02T04:47:50.668 INFO:tasks.workunit.client.0.smithi070.stdout:5/534: dwrite d5/d9/d23/d30/da2/f3d [0,4194304] 0 2024-04-02T04:47:50.669 INFO:tasks.workunit.client.1.smithi179.stdout:6/465: dread d3/f18 [0,4194304] 0 2024-04-02T04:47:50.669 INFO:tasks.workunit.client.0.smithi070.stdout:3/476: truncate d10/d32/f71 2120428 0 2024-04-02T04:47:50.670 INFO:tasks.workunit.client.1.smithi179.stdout:2/326: creat dc/d29/d4a/d4b/f61 x:0 0 0 2024-04-02T04:47:50.671 INFO:tasks.workunit.client.0.smithi070.stdout:0/519: link d4/d5/d13/c5a d4/d3c/c93 0 2024-04-02T04:47:50.671 INFO:tasks.workunit.client.0.smithi070.stdout:3/477: fsync d10/d17/d51/d73/d8f/f9c 0 2024-04-02T04:47:50.672 INFO:tasks.workunit.client.0.smithi070.stdout:0/520: fdatasync d4/d5/da/d2d/f89 0 2024-04-02T04:47:50.673 INFO:tasks.workunit.client.0.smithi070.stdout:3/478: write d10/d32/d4b/f7d [299807,46719] 0 2024-04-02T04:47:50.674 INFO:tasks.workunit.client.0.smithi070.stdout:5/535: dread d5/f2f [4194304,4194304] 0 2024-04-02T04:47:50.678 INFO:tasks.workunit.client.0.smithi070.stdout:2/477: rmdir d2/d12/d59/d69 0 2024-04-02T04:47:50.680 INFO:tasks.workunit.client.1.smithi179.stdout:8/439: creat d3/d32/d21/d50/d65/f9b x:0 0 0 2024-04-02T04:47:50.684 INFO:tasks.workunit.client.0.smithi070.stdout:1/472: creat d0/d4c/d5a/f9b x:0 0 0 2024-04-02T04:47:50.685 INFO:tasks.workunit.client.0.smithi070.stdout:1/473: fdatasync d0/d7/d1a/f1b 0 2024-04-02T04:47:50.687 INFO:tasks.workunit.client.1.smithi179.stdout:7/550: dread d1/f5d [0,4194304] 0 2024-04-02T04:47:50.687 INFO:tasks.workunit.client.1.smithi179.stdout:3/508: rename d8/d57/d8f/f92 to d8/d57/f9b 0 2024-04-02T04:47:50.691 INFO:tasks.workunit.client.1.smithi179.stdout:5/424: symlink d8/d1b/d23/d32/d46/l85 0 2024-04-02T04:47:50.692 INFO:tasks.workunit.client.1.smithi179.stdout:5/425: chown d8/d1b/d23/d32/d4e/d50 3790786 1 2024-04-02T04:47:50.692 INFO:tasks.workunit.client.0.smithi070.stdout:3/479: mknod d10/d66/caa 0 2024-04-02T04:47:50.693 INFO:tasks.workunit.client.0.smithi070.stdout:3/480: read - d10/d32/f63 zero size 2024-04-02T04:47:50.695 INFO:tasks.workunit.client.0.smithi070.stdout:0/521: unlink d4/d5/d58/f85 0 2024-04-02T04:47:50.696 INFO:tasks.workunit.client.1.smithi179.stdout:9/467: dwrite d0/d19/d27/f68 [0,4194304] 0 2024-04-02T04:47:50.696 INFO:tasks.workunit.client.1.smithi179.stdout:9/468: chown d0/d19/d27 2537 1 2024-04-02T04:47:50.698 INFO:tasks.workunit.client.0.smithi070.stdout:7/454: rename d7/dd/df/c56 to d7/c91 0 2024-04-02T04:47:50.700 INFO:tasks.workunit.client.1.smithi179.stdout:0/503: write d1/db/d64/f79 [457649,121852] 0 2024-04-02T04:47:50.701 INFO:tasks.workunit.client.0.smithi070.stdout:2/478: symlink d2/d10/d21/d2b/l97 0 2024-04-02T04:47:50.703 INFO:tasks.workunit.client.1.smithi179.stdout:2/327: dwrite dc/f46 [0,4194304] 0 2024-04-02T04:47:50.704 INFO:tasks.workunit.client.1.smithi179.stdout:4/545: readlink d1/d3/dc/d27/la8 0 2024-04-02T04:47:50.706 INFO:tasks.workunit.client.1.smithi179.stdout:2/328: write dc/d29/d4a/d1e/d43/f47 [676236,29994] 0 2024-04-02T04:47:50.707 INFO:tasks.workunit.client.1.smithi179.stdout:7/551: mknod d1/d9/d35/d6b/cc6 0 2024-04-02T04:47:50.708 INFO:tasks.workunit.client.1.smithi179.stdout:0/504: dwrite d1/f2a [0,4194304] 0 2024-04-02T04:47:50.709 INFO:tasks.workunit.client.1.smithi179.stdout:7/552: dread - d1/d9/d27/d87/d8b/d75/fbd zero size 2024-04-02T04:47:50.710 INFO:tasks.workunit.client.0.smithi070.stdout:3/481: rename d10/d32/f56 to d10/d17/d51/d73/d8f/fab 0 2024-04-02T04:47:50.712 INFO:tasks.workunit.client.0.smithi070.stdout:5/536: mkdir d5/d9/d10/da9 0 2024-04-02T04:47:50.712 INFO:tasks.workunit.client.1.smithi179.stdout:6/466: creat d3/d9a/f9f x:0 0 0 2024-04-02T04:47:50.713 INFO:tasks.workunit.client.1.smithi179.stdout:2/329: truncate dc/d29/d4a/d1e/d43/f5f 884170 0 2024-04-02T04:47:50.714 INFO:tasks.workunit.client.1.smithi179.stdout:5/426: mknod d8/d1b/d23/d32/d4e/d50/c86 0 2024-04-02T04:47:50.715 INFO:tasks.workunit.client.1.smithi179.stdout:2/330: stat dc/d29/d4a/d4b/f61 0 2024-04-02T04:47:50.716 INFO:tasks.workunit.client.1.smithi179.stdout:9/469: creat d0/d19/d27/d7a/f8e x:0 0 0 2024-04-02T04:47:50.717 INFO:tasks.workunit.client.0.smithi070.stdout:1/474: mknod d0/d7/c9c 0 2024-04-02T04:47:50.719 INFO:tasks.workunit.client.1.smithi179.stdout:0/505: dwrite d1/db/d64/f70 [0,4194304] 0 2024-04-02T04:47:50.719 INFO:tasks.workunit.client.0.smithi070.stdout:0/522: creat d4/d5/d58/d60/d83/f94 x:0 0 0 2024-04-02T04:47:50.720 INFO:tasks.workunit.client.0.smithi070.stdout:0/523: chown d4/d3c/l5c 104 1 2024-04-02T04:47:50.720 INFO:tasks.workunit.client.1.smithi179.stdout:0/506: truncate d1/da/d28/d34/f7e 452512 0 2024-04-02T04:47:50.722 INFO:tasks.workunit.client.1.smithi179.stdout:8/440: truncate d3/f59 101642 0 2024-04-02T04:47:50.722 INFO:tasks.workunit.client.0.smithi070.stdout:3/482: mknod d10/d66/cac 0 2024-04-02T04:47:50.723 INFO:tasks.workunit.client.0.smithi070.stdout:5/537: creat d5/d9/d10/d51/faa x:0 0 0 2024-04-02T04:47:50.725 INFO:tasks.workunit.client.0.smithi070.stdout:1/475: rmdir d0/d9/d1d/d22/d56/d79/d89 39 2024-04-02T04:47:50.725 INFO:tasks.workunit.client.0.smithi070.stdout:2/479: dread d2/d10/d21/f34 [0,4194304] 0 2024-04-02T04:47:50.726 INFO:tasks.workunit.client.1.smithi179.stdout:3/509: mkdir d8/d26/d83/d89/d9c 0 2024-04-02T04:47:50.727 INFO:tasks.workunit.client.0.smithi070.stdout:0/524: rmdir d4/d5/d13/d43 39 2024-04-02T04:47:50.727 INFO:tasks.workunit.client.0.smithi070.stdout:0/525: chown d4/d5/f84 48 1 2024-04-02T04:47:50.728 INFO:tasks.workunit.client.1.smithi179.stdout:0/507: fsync d1/f2a 0 2024-04-02T04:47:50.728 INFO:tasks.workunit.client.1.smithi179.stdout:3/510: write d8/d26/f4e [4387467,52596] 0 2024-04-02T04:47:50.729 INFO:tasks.workunit.client.1.smithi179.stdout:1/478: rename d3/d5/d2f/d5a/f5d to d3/d5/d2f/d4f/d42/d53/d72/d86/fa6 0 2024-04-02T04:47:50.730 INFO:tasks.workunit.client.0.smithi070.stdout:5/538: dread d5/d17/d2b/d7b/f92 [0,4194304] 0 2024-04-02T04:47:50.730 INFO:tasks.workunit.client.0.smithi070.stdout:3/483: rmdir d10/d17/d4c 39 2024-04-02T04:47:50.731 INFO:tasks.workunit.client.1.smithi179.stdout:8/441: dwrite d3/d32/f6e [0,4194304] 0 2024-04-02T04:47:50.733 INFO:tasks.workunit.client.0.smithi070.stdout:5/539: dread - d5/d17/d2b/f6f zero size 2024-04-02T04:47:50.733 INFO:tasks.workunit.client.0.smithi070.stdout:2/480: dwrite d2/d10/d5c/f83 [0,4194304] 0 2024-04-02T04:47:50.733 INFO:tasks.workunit.client.1.smithi179.stdout:5/427: symlink d8/d1b/d23/d36/l87 0 2024-04-02T04:47:50.735 INFO:tasks.workunit.client.1.smithi179.stdout:0/508: dread d1/db/d64/f70 [0,4194304] 0 2024-04-02T04:47:50.738 INFO:tasks.workunit.client.0.smithi070.stdout:2/481: truncate d2/d10/d5c/f8c 12396 0 2024-04-02T04:47:50.738 INFO:tasks.workunit.client.0.smithi070.stdout:1/476: truncate d0/d9/f14 4128350 0 2024-04-02T04:47:50.738 INFO:tasks.workunit.client.1.smithi179.stdout:4/546: creat d1/d3/df/d10/d2f/d60/da6/fb6 x:0 0 0 2024-04-02T04:47:50.739 INFO:tasks.workunit.client.0.smithi070.stdout:0/526: dwrite d4/d5/d13/d2a/f20 [0,4194304] 0 2024-04-02T04:47:50.745 INFO:tasks.workunit.client.1.smithi179.stdout:4/547: dread d1/d3/d19/fa3 [0,4194304] 0 2024-04-02T04:47:50.747 INFO:tasks.workunit.client.0.smithi070.stdout:1/477: symlink d0/d4c/d5a/l9d 0 2024-04-02T04:47:50.748 INFO:tasks.workunit.client.0.smithi070.stdout:1/478: fsync d0/d7/f6a 0 2024-04-02T04:47:50.750 INFO:tasks.workunit.client.0.smithi070.stdout:5/540: dread d5/d57/f5d [0,4194304] 0 2024-04-02T04:47:50.751 INFO:tasks.workunit.client.0.smithi070.stdout:0/527: rename d4/d5/d13/d2a/d29/c10 to d4/d3c/c95 0 2024-04-02T04:47:50.752 INFO:tasks.workunit.client.0.smithi070.stdout:0/528: fdatasync d4/d5/f84 0 2024-04-02T04:47:50.754 INFO:tasks.workunit.client.1.smithi179.stdout:6/467: rename d3/d2d/c45 to d3/d2d/d38/d64/ca0 0 2024-04-02T04:47:50.754 INFO:tasks.workunit.client.1.smithi179.stdout:4/548: dwrite d1/d3/df/d10/d2f/d7b/f86 [0,4194304] 0 2024-04-02T04:47:50.755 INFO:tasks.workunit.client.0.smithi070.stdout:0/529: write d4/d5/d13/d2a/f68 [2328186,82181] 0 2024-04-02T04:47:50.758 INFO:tasks.workunit.client.0.smithi070.stdout:0/530: stat d4/d5/d13/d2a/d29/d52/f69 0 2024-04-02T04:47:50.758 INFO:tasks.workunit.client.0.smithi070.stdout:5/541: dwrite d5/d9/d23/f49 [0,4194304] 0 2024-04-02T04:47:50.759 INFO:tasks.workunit.client.1.smithi179.stdout:1/479: creat d3/d5/d96/fa7 x:0 0 0 2024-04-02T04:47:50.760 INFO:tasks.workunit.client.1.smithi179.stdout:1/480: fsync d3/d5/de/d33/f7b 0 2024-04-02T04:47:50.771 INFO:tasks.workunit.client.0.smithi070.stdout:0/531: mknod d4/d3c/c96 0 2024-04-02T04:47:50.778 INFO:tasks.workunit.client.1.smithi179.stdout:2/331: rename dc/f1a to dc/d29/d31/d57/f62 0 2024-04-02T04:47:50.780 INFO:tasks.workunit.client.1.smithi179.stdout:6/468: mknod d3/d2d/d38/d64/ca1 0 2024-04-02T04:47:50.785 INFO:tasks.workunit.client.1.smithi179.stdout:2/332: dwrite dc/d29/d31/f3a [0,4194304] 0 2024-04-02T04:47:50.785 INFO:tasks.workunit.client.0.smithi070.stdout:2/482: dread d2/f32 [0,4194304] 0 2024-04-02T04:47:50.789 INFO:tasks.workunit.client.0.smithi070.stdout:3/484: dread d10/d32/d3f/f4d [0,4194304] 0 2024-04-02T04:47:50.790 INFO:tasks.workunit.client.0.smithi070.stdout:3/485: chown d10/d32/d8a/d8d/da1 2143727 1 2024-04-02T04:47:50.790 INFO:tasks.workunit.client.1.smithi179.stdout:4/549: dwrite d1/d3/dc/d26/f6d [4194304,4194304] 0 2024-04-02T04:47:50.791 INFO:tasks.workunit.client.1.smithi179.stdout:4/550: chown d1/d3/dc/d27 366 1 2024-04-02T04:47:50.792 INFO:tasks.workunit.client.1.smithi179.stdout:0/509: truncate d1/db/f2f 1926031 0 2024-04-02T04:47:50.793 INFO:tasks.workunit.client.1.smithi179.stdout:0/510: chown d1/da/d48/d5c/d83/d97 1 1 2024-04-02T04:47:50.794 INFO:tasks.workunit.client.1.smithi179.stdout:4/551: read d1/d3/df/d10/d2f/d60/d71/f74 [3861377,75749] 0 2024-04-02T04:47:50.795 INFO:tasks.workunit.client.1.smithi179.stdout:0/511: stat d1/da/d48/d5c/d83/f8b 0 2024-04-02T04:47:50.795 INFO:tasks.workunit.client.1.smithi179.stdout:3/511: rmdir d8/d6d/d88 0 2024-04-02T04:47:50.804 INFO:tasks.workunit.client.1.smithi179.stdout:6/469: creat d3/d12/d23/d26/d57/fa2 x:0 0 0 2024-04-02T04:47:50.804 INFO:tasks.workunit.client.0.smithi070.stdout:1/479: write d0/d9/d1d/f42 [4680749,93682] 0 2024-04-02T04:47:50.804 INFO:tasks.workunit.client.1.smithi179.stdout:0/512: dread d1/da/d28/d34/f35 [0,4194304] 0 2024-04-02T04:47:50.807 INFO:tasks.workunit.client.1.smithi179.stdout:1/481: mkdir d3/da8 0 2024-04-02T04:47:50.807 INFO:tasks.workunit.client.1.smithi179.stdout:0/513: dread d1/da/d28/d34/f7e [0,4194304] 0 2024-04-02T04:47:50.808 INFO:tasks.workunit.client.0.smithi070.stdout:0/532: rename d4/d5/d13/d2a/d29/c7e to d4/d5/d13/d23/c97 0 2024-04-02T04:47:50.809 INFO:tasks.workunit.client.1.smithi179.stdout:7/553: getdents d1/d9/d27/d87/d8b 0 2024-04-02T04:47:50.809 INFO:tasks.workunit.client.1.smithi179.stdout:7/554: write d1/fc [3368002,5383] 0 2024-04-02T04:47:50.810 INFO:tasks.workunit.client.0.smithi070.stdout:0/533: read d4/d5/da/d2d/f4c [3180841,34845] 0 2024-04-02T04:47:50.812 INFO:tasks.workunit.client.1.smithi179.stdout:2/333: creat dc/d29/d4a/d1e/f63 x:0 0 0 2024-04-02T04:47:50.813 INFO:tasks.workunit.client.1.smithi179.stdout:5/428: creat d8/d1b/f88 x:0 0 0 2024-04-02T04:47:50.820 INFO:tasks.workunit.client.1.smithi179.stdout:3/512: symlink d8/d26/d83/d89/l9d 0 2024-04-02T04:47:50.820 INFO:tasks.workunit.client.0.smithi070.stdout:0/534: mkdir d4/d5/d13/d17/d98 0 2024-04-02T04:47:50.821 INFO:tasks.workunit.client.1.smithi179.stdout:7/555: dread d1/d9/f12 [0,4194304] 0 2024-04-02T04:47:50.823 INFO:tasks.workunit.client.1.smithi179.stdout:7/556: write d1/dd/d16/d4d/fb6 [257316,109022] 0 2024-04-02T04:47:50.823 INFO:tasks.workunit.client.0.smithi070.stdout:3/486: fsync d10/d32/d3f/f43 0 2024-04-02T04:47:50.824 INFO:tasks.workunit.client.1.smithi179.stdout:3/513: dread d8/d57/d63/f6b [0,4194304] 0 2024-04-02T04:47:50.825 INFO:tasks.workunit.client.0.smithi070.stdout:1/480: symlink d0/d9/d94/l9e 0 2024-04-02T04:47:50.825 INFO:tasks.workunit.client.0.smithi070.stdout:1/481: stat d0/d7/c15 0 2024-04-02T04:47:50.826 INFO:tasks.workunit.client.1.smithi179.stdout:1/482: rmdir d3/d5/d2f 39 2024-04-02T04:47:50.827 INFO:tasks.workunit.client.0.smithi070.stdout:5/542: getdents d5/d9/d23 0 2024-04-02T04:47:50.827 INFO:tasks.workunit.client.0.smithi070.stdout:0/535: dwrite d4/d5/d58/d60/f7c [0,4194304] 0 2024-04-02T04:47:50.828 INFO:tasks.workunit.client.0.smithi070.stdout:5/543: truncate d5/f97 363001 0 2024-04-02T04:47:50.831 INFO:tasks.workunit.client.1.smithi179.stdout:7/557: dwrite d1/d9/d55/dbf/f7a [0,4194304] 0 2024-04-02T04:47:50.832 INFO:tasks.workunit.client.1.smithi179.stdout:7/558: chown d1/d9/d27 1 1 2024-04-02T04:47:50.834 INFO:tasks.workunit.client.1.smithi179.stdout:7/559: dread - d1/d9/d27/d87/d8b/d75/db9/f92 zero size 2024-04-02T04:47:50.836 INFO:tasks.workunit.client.1.smithi179.stdout:2/334: fdatasync dc/d29/d4a/f45 0 2024-04-02T04:47:50.842 INFO:tasks.workunit.client.0.smithi070.stdout:2/483: write d2/f8 [1504046,4070] 0 2024-04-02T04:47:50.843 INFO:tasks.workunit.client.0.smithi070.stdout:1/482: read d0/d9/f3e [1680123,123682] 0 2024-04-02T04:47:50.845 INFO:tasks.workunit.client.0.smithi070.stdout:5/544: mknod d5/d9/d10/d6a/d7d/cab 0 2024-04-02T04:47:50.850 INFO:tasks.workunit.client.1.smithi179.stdout:9/470: rename d0/d1/d11/c6a to d0/d19/c8f 0 2024-04-02T04:47:50.852 INFO:tasks.workunit.client.0.smithi070.stdout:3/487: dwrite d10/d17/d4c/fa3 [0,4194304] 0 2024-04-02T04:47:50.852 INFO:tasks.workunit.client.0.smithi070.stdout:4/397: sync 2024-04-02T04:47:50.852 INFO:tasks.workunit.client.0.smithi070.stdout:2/484: unlink d2/d24/f8d 0 2024-04-02T04:47:50.856 INFO:tasks.workunit.client.0.smithi070.stdout:3/488: dread d10/f29 [0,4194304] 0 2024-04-02T04:47:50.857 INFO:tasks.workunit.client.0.smithi070.stdout:1/483: mkdir d0/d4c/d5a/d9f 0 2024-04-02T04:47:50.860 INFO:tasks.workunit.client.1.smithi179.stdout:3/514: mknod d8/d26/d52/d85/c9e 0 2024-04-02T04:47:50.864 INFO:tasks.workunit.client.0.smithi070.stdout:1/484: dwrite d0/d54/d64/d6d/f78 [0,4194304] 0 2024-04-02T04:47:50.865 INFO:tasks.workunit.client.0.smithi070.stdout:5/545: dread d5/d17/d2b/d7b/f92 [0,4194304] 0 2024-04-02T04:47:50.865 INFO:tasks.workunit.client.1.smithi179.stdout:3/515: dread d8/d26/f3f [0,4194304] 0 2024-04-02T04:47:50.865 INFO:tasks.workunit.client.1.smithi179.stdout:0/514: creat d1/da/d48/d65/d7d/d80/f9e x:0 0 0 2024-04-02T04:47:50.866 INFO:tasks.workunit.client.1.smithi179.stdout:3/516: write d8/d26/f69 [1034307,63131] 0 2024-04-02T04:47:50.867 INFO:tasks.workunit.client.0.smithi070.stdout:5/546: write d5/d17/d27/d2e/f9f [841260,71914] 0 2024-04-02T04:47:50.868 INFO:tasks.workunit.client.0.smithi070.stdout:5/547: readlink d5/d17/d2b/d7b/l8e 0 2024-04-02T04:47:50.869 INFO:tasks.workunit.client.1.smithi179.stdout:4/552: write d1/d3/f28 [4676931,15833] 0 2024-04-02T04:47:50.873 INFO:tasks.workunit.client.0.smithi070.stdout:3/489: unlink d10/d32/d3f/l7c 0 2024-04-02T04:47:50.874 INFO:tasks.workunit.client.1.smithi179.stdout:3/517: dwrite d8/d26/f4e [0,4194304] 0 2024-04-02T04:47:50.876 INFO:tasks.workunit.client.0.smithi070.stdout:0/536: dwrite d4/d5/d13/d2a/d29/f46 [0,4194304] 0 2024-04-02T04:47:50.876 INFO:tasks.workunit.client.0.smithi070.stdout:3/490: write d10/d32/d3f/f38 [693670,70150] 0 2024-04-02T04:47:50.878 INFO:tasks.workunit.client.0.smithi070.stdout:5/548: dwrite d5/d9/d23/d30/da2/f38 [0,4194304] 0 2024-04-02T04:47:50.890 INFO:tasks.workunit.client.0.smithi070.stdout:3/491: dwrite d10/d66/f9d [0,4194304] 0 2024-04-02T04:47:50.890 INFO:tasks.workunit.client.1.smithi179.stdout:8/442: rename d3/d32/d30/f3b to d3/d32/d21/d50/d56/f9c 0 2024-04-02T04:47:50.891 INFO:tasks.workunit.client.0.smithi070.stdout:3/492: write d10/d32/d8a/d8d/d91/f94 [890408,95766] 0 2024-04-02T04:47:50.892 INFO:tasks.workunit.client.0.smithi070.stdout:2/485: symlink d2/d12/d3e/d68/d90/l98 0 2024-04-02T04:47:50.900 INFO:tasks.workunit.client.0.smithi070.stdout:5/549: rename d5/d17/d27/d2e/l8a to d5/d9/d23/d8b/lac 0 2024-04-02T04:47:50.902 INFO:tasks.workunit.client.0.smithi070.stdout:4/398: dwrite d1/d3/d9/d41/d65/f6a [0,4194304] 0 2024-04-02T04:47:50.905 INFO:tasks.workunit.client.1.smithi179.stdout:0/515: mkdir d1/da/d67/d9f 0 2024-04-02T04:47:50.905 INFO:tasks.workunit.client.1.smithi179.stdout:5/429: dwrite d8/d1b/f24 [0,4194304] 0 2024-04-02T04:47:50.908 INFO:tasks.workunit.client.0.smithi070.stdout:2/486: mknod d2/d12/d59/c99 0 2024-04-02T04:47:50.909 INFO:tasks.workunit.client.1.smithi179.stdout:7/560: mkdir d1/d9/d27/d87/dc7 0 2024-04-02T04:47:50.909 INFO:tasks.workunit.client.0.smithi070.stdout:0/537: dwrite d4/d5/d13/d43/f78 [0,4194304] 0 2024-04-02T04:47:50.909 INFO:tasks.workunit.client.0.smithi070.stdout:3/493: write d10/d17/d51/f64 [5069708,73292] 0 2024-04-02T04:47:50.912 INFO:tasks.workunit.client.1.smithi179.stdout:2/335: mknod dc/c64 0 2024-04-02T04:47:50.917 INFO:tasks.workunit.client.0.smithi070.stdout:1/485: dwrite d0/d7/f20 [0,4194304] 0 2024-04-02T04:47:50.921 INFO:tasks.workunit.client.0.smithi070.stdout:1/486: chown d0/d9/d1d/c36 0 1 2024-04-02T04:47:50.923 INFO:tasks.workunit.client.1.smithi179.stdout:9/471: fdatasync d0/d1/f44 0 2024-04-02T04:47:50.924 INFO:tasks.workunit.client.0.smithi070.stdout:4/399: dwrite d1/d3/d9/f1d [0,4194304] 0 2024-04-02T04:47:50.925 INFO:tasks.workunit.client.0.smithi070.stdout:1/487: chown d0/d9/d1d/d22/d56/d79/d89/c97 9277849 1 2024-04-02T04:47:50.926 INFO:tasks.workunit.client.0.smithi070.stdout:3/494: dwrite d10/d17/d51/f64 [0,4194304] 0 2024-04-02T04:47:50.926 INFO:tasks.workunit.client.0.smithi070.stdout:2/487: rename d2/d12/d59/f70 to d2/d12/d3e/d68/f9a 0 2024-04-02T04:47:50.930 INFO:tasks.workunit.client.0.smithi070.stdout:5/550: link d5/d14/c8c d5/d17/d2b/d7b/cad 0 2024-04-02T04:47:50.930 INFO:tasks.workunit.client.0.smithi070.stdout:5/551: readlink d5/d17/l36 0 2024-04-02T04:47:50.931 INFO:tasks.workunit.client.0.smithi070.stdout:4/400: chown d1/d3/f30 0 1 2024-04-02T04:47:50.935 INFO:tasks.workunit.client.1.smithi179.stdout:1/483: dwrite d3/f2d [0,4194304] 0 2024-04-02T04:47:50.936 INFO:tasks.workunit.client.1.smithi179.stdout:2/336: creat dc/d29/d31/d39/f65 x:0 0 0 2024-04-02T04:47:50.938 INFO:tasks.workunit.client.0.smithi070.stdout:2/488: creat d2/d12/d3e/d68/f9b x:0 0 0 2024-04-02T04:47:50.940 INFO:tasks.workunit.client.0.smithi070.stdout:4/401: dwrite d1/f3e [0,4194304] 0 2024-04-02T04:47:50.944 INFO:tasks.workunit.client.0.smithi070.stdout:0/538: mknod d4/d5/d13/c99 0 2024-04-02T04:47:50.953 INFO:tasks.workunit.client.1.smithi179.stdout:8/443: write d3/d32/d21/d50/f5f [1092756,42352] 0 2024-04-02T04:47:50.957 INFO:tasks.workunit.client.1.smithi179.stdout:2/337: mkdir dc/d29/d31/d66 0 2024-04-02T04:47:50.958 INFO:tasks.workunit.client.0.smithi070.stdout:9/411: sync 2024-04-02T04:47:50.958 INFO:tasks.workunit.client.0.smithi070.stdout:8/470: sync 2024-04-02T04:47:50.959 INFO:tasks.workunit.client.0.smithi070.stdout:9/412: chown d4/d10/d11/d7b/d19/d2f/d39/d4f/c7d 1712496184 1 2024-04-02T04:47:50.961 INFO:tasks.workunit.client.1.smithi179.stdout:0/516: getdents d1/db 0 2024-04-02T04:47:50.962 INFO:tasks.workunit.client.0.smithi070.stdout:0/539: rmdir d4/d5/d58/d60/d83 39 2024-04-02T04:47:50.964 INFO:tasks.workunit.client.1.smithi179.stdout:5/430: mkdir d8/d1b/d23/d36/d53/d89 0 2024-04-02T04:47:50.964 INFO:tasks.workunit.client.0.smithi070.stdout:8/471: dread d2/d8/d29/d22/f7a [0,4194304] 0 2024-04-02T04:47:50.965 INFO:tasks.workunit.client.1.smithi179.stdout:5/431: write d8/d1b/d23/d32/d4e/d50/f68 [639335,11651] 0 2024-04-02T04:47:50.965 INFO:tasks.workunit.client.0.smithi070.stdout:3/495: creat d10/fad x:0 0 0 2024-04-02T04:47:50.966 INFO:tasks.workunit.client.1.smithi179.stdout:8/444: rmdir d3/d32/d21/d50/d65 39 2024-04-02T04:47:50.968 INFO:tasks.workunit.client.1.smithi179.stdout:8/445: chown d3/d32/d21/d22/d6b/d91/l90 16864 1 2024-04-02T04:47:50.969 INFO:tasks.workunit.client.1.smithi179.stdout:7/561: rename d1/dd/d16/f62 to d1/fc8 0 2024-04-02T04:47:50.969 INFO:tasks.workunit.client.0.smithi070.stdout:5/552: write d5/d17/d50/f5f [2155145,82695] 0 2024-04-02T04:47:50.972 INFO:tasks.workunit.client.1.smithi179.stdout:1/484: read d3/d5/d2f/d36/f67 [25563,18782] 0 2024-04-02T04:47:50.974 INFO:tasks.workunit.client.1.smithi179.stdout:0/517: unlink d1/db/d64/f70 0 2024-04-02T04:47:50.977 INFO:tasks.workunit.client.1.smithi179.stdout:7/562: dwrite d1/dd/f17 [0,4194304] 0 2024-04-02T04:47:50.978 INFO:tasks.workunit.client.0.smithi070.stdout:9/413: mkdir d4/d10/d12/d27/d83 0 2024-04-02T04:47:50.981 INFO:tasks.workunit.client.0.smithi070.stdout:0/540: rmdir d4/d5/d13/d2a/d8d 39 2024-04-02T04:47:50.981 INFO:tasks.workunit.client.1.smithi179.stdout:7/563: write d1/dd/d16/d4d/fb6 [1349142,48592] 0 2024-04-02T04:47:50.982 INFO:tasks.workunit.client.1.smithi179.stdout:8/446: dread d3/d32/d21/d22/f2d [0,4194304] 0 2024-04-02T04:47:50.982 INFO:tasks.workunit.client.0.smithi070.stdout:8/472: mkdir d2/d8/d29/d22/d5a/d94 0 2024-04-02T04:47:50.984 INFO:tasks.workunit.client.0.smithi070.stdout:3/496: rmdir d10/d17 39 2024-04-02T04:47:50.986 INFO:tasks.workunit.client.1.smithi179.stdout:5/432: truncate d8/d1b/f26 2316494 0 2024-04-02T04:47:50.987 INFO:tasks.workunit.client.0.smithi070.stdout:1/488: getdents d0/d9 0 2024-04-02T04:47:50.988 INFO:tasks.workunit.client.0.smithi070.stdout:1/489: dread - d0/d4c/f96 zero size 2024-04-02T04:47:50.989 INFO:tasks.workunit.client.0.smithi070.stdout:1/490: dread - d0/d9/d1d/d22/d2c/d4f/f50 zero size 2024-04-02T04:47:50.990 INFO:tasks.workunit.client.0.smithi070.stdout:2/489: creat d2/f9c x:0 0 0 2024-04-02T04:47:50.990 INFO:tasks.workunit.client.0.smithi070.stdout:1/491: readlink d0/d9/d1d/lb 0 2024-04-02T04:47:50.991 INFO:tasks.workunit.client.0.smithi070.stdout:2/490: write d2/d10/d21/d3d/f47 [1835972,28520] 0 2024-04-02T04:47:50.992 INFO:tasks.workunit.client.1.smithi179.stdout:5/433: dwrite d8/f1a [0,4194304] 0 2024-04-02T04:47:50.993 INFO:tasks.workunit.client.0.smithi070.stdout:5/553: mknod d5/d17/d2b/d7b/cae 0 2024-04-02T04:47:50.994 INFO:tasks.workunit.client.0.smithi070.stdout:5/554: fdatasync d5/f6 0 2024-04-02T04:47:50.994 INFO:tasks.workunit.client.0.smithi070.stdout:5/555: readlink d5/d17/l36 0 2024-04-02T04:47:50.995 INFO:tasks.workunit.client.0.smithi070.stdout:4/402: creat d1/f86 x:0 0 0 2024-04-02T04:47:50.997 INFO:tasks.workunit.client.1.smithi179.stdout:0/518: symlink d1/da/d48/d5c/d83/d97/la0 0 2024-04-02T04:47:50.997 INFO:tasks.workunit.client.0.smithi070.stdout:9/414: dread - d4/d10/d11/f58 zero size 2024-04-02T04:47:50.998 INFO:tasks.workunit.client.1.smithi179.stdout:0/519: read d1/da/d28/d5e/f98 [7671677,15918] 0 2024-04-02T04:47:51.001 INFO:tasks.workunit.client.0.smithi070.stdout:8/473: creat d2/d3/d52/d62/f95 x:0 0 0 2024-04-02T04:47:51.003 INFO:tasks.workunit.client.1.smithi179.stdout:9/472: rename d0/d1/d10/f2b to d0/f90 0 2024-04-02T04:47:51.004 INFO:tasks.workunit.client.1.smithi179.stdout:9/473: stat d0/d19/d27/d48/d6c 0 2024-04-02T04:47:51.005 INFO:tasks.workunit.client.0.smithi070.stdout:3/497: mknod d10/d32/d8a/d8d/cae 0 2024-04-02T04:47:51.008 INFO:tasks.workunit.client.0.smithi070.stdout:1/492: dread d0/d9/d1d/f4e [0,4194304] 0 2024-04-02T04:47:51.009 INFO:tasks.workunit.client.0.smithi070.stdout:2/491: creat d2/d10/d21/d3d/f9d x:0 0 0 2024-04-02T04:47:51.009 INFO:tasks.workunit.client.0.smithi070.stdout:1/493: write d0/d7/d28/f74 [2170135,53608] 0 2024-04-02T04:47:51.011 INFO:tasks.workunit.client.0.smithi070.stdout:2/492: truncate d2/d10/d21/d3d/f9d 879118 0 2024-04-02T04:47:51.013 INFO:tasks.workunit.client.0.smithi070.stdout:5/556: mkdir d5/d17/d2b/daf 0 2024-04-02T04:47:51.014 INFO:tasks.workunit.client.0.smithi070.stdout:4/403: creat d1/d3/d9/d5e/f87 x:0 0 0 2024-04-02T04:47:51.014 INFO:tasks.workunit.client.0.smithi070.stdout:7/455: sync 2024-04-02T04:47:51.014 INFO:tasks.workunit.client.0.smithi070.stdout:6/376: sync 2024-04-02T04:47:51.015 INFO:tasks.workunit.client.1.smithi179.stdout:9/474: write d0/f4c [1306512,56123] 0 2024-04-02T04:47:51.015 INFO:tasks.workunit.client.1.smithi179.stdout:7/564: creat d1/d9/fc9 x:0 0 0 2024-04-02T04:47:51.016 INFO:tasks.workunit.client.0.smithi070.stdout:9/415: mknod d4/d10/d12/d56/c84 0 2024-04-02T04:47:51.017 INFO:tasks.workunit.client.0.smithi070.stdout:6/377: write d0/d3/d19/d31/d46/f68 [1288658,112978] 0 2024-04-02T04:47:51.023 INFO:tasks.workunit.client.0.smithi070.stdout:1/494: creat d0/d4c/fa0 x:0 0 0 2024-04-02T04:47:51.026 INFO:tasks.workunit.client.0.smithi070.stdout:2/493: rename d2/f8 to d2/d12/d3e/d68/d90/f9e 0 2024-04-02T04:47:51.031 INFO:tasks.workunit.client.0.smithi070.stdout:7/456: rmdir d7/dd 39 2024-04-02T04:47:51.041 INFO:tasks.workunit.client.0.smithi070.stdout:6/378: write d0/d43/f51 [2824609,46110] 0 2024-04-02T04:47:51.043 INFO:tasks.workunit.client.0.smithi070.stdout:1/495: mknod d0/d4c/ca1 0 2024-04-02T04:47:51.050 INFO:tasks.workunit.client.1.smithi179.stdout:9/475: dread - d0/d1/d10/d6d/d58/f75 zero size 2024-04-02T04:47:51.051 INFO:tasks.workunit.client.1.smithi179.stdout:9/476: dread - d0/d1/d11/d77/f74 zero size 2024-04-02T04:47:51.051 INFO:tasks.workunit.client.1.smithi179.stdout:7/565: creat d1/dd/d76/fca x:0 0 0 2024-04-02T04:47:51.052 INFO:tasks.workunit.client.1.smithi179.stdout:9/477: fsync d0/d19/d27/f68 0 2024-04-02T04:47:51.052 INFO:tasks.workunit.client.1.smithi179.stdout:8/447: getdents d3/d1b/d2a/d41 0 2024-04-02T04:47:51.053 INFO:tasks.workunit.client.0.smithi070.stdout:0/541: getdents d4/d5/d21 0 2024-04-02T04:47:51.054 INFO:tasks.workunit.client.0.smithi070.stdout:9/416: creat d4/d10/d12/d56/d79/f85 x:0 0 0 2024-04-02T04:47:51.055 INFO:tasks.workunit.client.1.smithi179.stdout:8/448: write d3/d32/d21/d50/d56/d94/f60 [2202513,114276] 0 2024-04-02T04:47:51.056 INFO:tasks.workunit.client.1.smithi179.stdout:2/338: rename dc/f46 to dc/d29/d4a/f67 0 2024-04-02T04:47:51.058 INFO:tasks.workunit.client.0.smithi070.stdout:1/496: mkdir d0/d4c/da2 0 2024-04-02T04:47:51.059 INFO:tasks.workunit.client.1.smithi179.stdout:2/339: readlink dc/d29/l30 0 2024-04-02T04:47:51.059 INFO:tasks.workunit.client.1.smithi179.stdout:7/566: dwrite d1/f15 [0,4194304] 0 2024-04-02T04:47:51.060 INFO:tasks.workunit.client.0.smithi070.stdout:1/497: truncate d0/d7/f6a 350506 0 2024-04-02T04:47:51.060 INFO:tasks.workunit.client.0.smithi070.stdout:4/404: rename d1/d3/f6d to d1/d3/d9/d41/f88 0 2024-04-02T04:47:51.061 INFO:tasks.workunit.client.1.smithi179.stdout:9/478: dwrite d0/d1/d10/f18 [4194304,4194304] 0 2024-04-02T04:47:51.064 INFO:tasks.workunit.client.0.smithi070.stdout:5/557: creat d5/fb0 x:0 0 0 2024-04-02T04:47:51.082 INFO:tasks.workunit.client.0.smithi070.stdout:1/498: creat d0/d7/d1a/d66/fa3 x:0 0 0 2024-04-02T04:47:51.084 INFO:tasks.workunit.client.0.smithi070.stdout:2/494: link d2/d12/d3e/d68/f9b d2/d10/d21/d71/f9f 0 2024-04-02T04:47:51.085 INFO:tasks.workunit.client.0.smithi070.stdout:2/495: chown d2/d10/d21/d2b/l97 0 1 2024-04-02T04:47:51.086 INFO:tasks.workunit.client.0.smithi070.stdout:5/558: fsync d5/f22 0 2024-04-02T04:47:51.088 INFO:tasks.workunit.client.1.smithi179.stdout:8/449: symlink d3/d6c/d96/l9d 0 2024-04-02T04:47:51.094 INFO:tasks.workunit.client.1.smithi179.stdout:9/479: mknod d0/d19/c91 0 2024-04-02T04:47:51.094 INFO:tasks.workunit.client.1.smithi179.stdout:7/567: write d1/f4e [3653038,81244] 0 2024-04-02T04:47:51.095 INFO:tasks.workunit.client.0.smithi070.stdout:7/457: dwrite d7/dd/df/d7a/f64 [0,4194304] 0 2024-04-02T04:47:51.095 INFO:tasks.workunit.client.0.smithi070.stdout:9/417: symlink d4/d10/d12/d27/d68/l86 0 2024-04-02T04:47:51.097 INFO:tasks.workunit.client.1.smithi179.stdout:1/485: rename d3/d5/de/d33/f51 to d3/d5/de/fa9 0 2024-04-02T04:47:51.098 INFO:tasks.workunit.client.0.smithi070.stdout:3/498: link d10/d17/c3e d10/d32/d8a/d8d/caf 0 2024-04-02T04:47:51.100 INFO:tasks.workunit.client.1.smithi179.stdout:2/340: creat dc/f68 x:0 0 0 2024-04-02T04:47:51.102 INFO:tasks.workunit.client.0.smithi070.stdout:9/418: dwrite d4/d10/d12/f62 [0,4194304] 0 2024-04-02T04:47:51.102 INFO:tasks.workunit.client.1.smithi179.stdout:6/470: sync 2024-04-02T04:47:51.103 INFO:tasks.workunit.client.0.smithi070.stdout:0/542: truncate f0 2761588 0 2024-04-02T04:47:51.108 INFO:tasks.workunit.client.1.smithi179.stdout:7/568: rmdir d1/d9/d27/d87 39 2024-04-02T04:47:51.110 INFO:tasks.workunit.client.0.smithi070.stdout:2/496: creat d2/d12/d3e/d68/fa0 x:0 0 0 2024-04-02T04:47:51.110 INFO:tasks.workunit.client.0.smithi070.stdout:9/419: dwrite d4/d13/f37 [4194304,4194304] 0 2024-04-02T04:47:51.112 INFO:tasks.workunit.client.1.smithi179.stdout:1/486: stat d3/d5/d2f/d4f/d35/d49/l9a 0 2024-04-02T04:47:51.115 INFO:tasks.workunit.client.1.smithi179.stdout:7/569: dwrite d1/dd/fc3 [0,4194304] 0 2024-04-02T04:47:51.116 INFO:tasks.workunit.client.0.smithi070.stdout:5/559: rename d5/d9/d23/f42 to d5/d17/d50/fb1 0 2024-04-02T04:47:51.117 INFO:tasks.workunit.client.0.smithi070.stdout:5/560: chown d5/d17/f93 1665927981 1 2024-04-02T04:47:51.122 INFO:tasks.workunit.client.1.smithi179.stdout:6/471: creat d3/d12/d23/d39/d80/fa3 x:0 0 0 2024-04-02T04:47:51.124 INFO:tasks.workunit.client.1.smithi179.stdout:5/434: rename f6 to d8/d1b/d23/d32/d6e/f8a 0 2024-04-02T04:47:51.126 INFO:tasks.workunit.client.1.smithi179.stdout:5/435: write d8/d1b/d23/d32/d4b/f70 [764851,86673] 0 2024-04-02T04:47:51.130 INFO:tasks.workunit.client.1.smithi179.stdout:1/487: creat d3/d5/d2f/d4f/d42/d53/d5b/faa x:0 0 0 2024-04-02T04:47:51.130 INFO:tasks.workunit.client.1.smithi179.stdout:1/488: stat d3/d5/c4d 0 2024-04-02T04:47:51.134 INFO:tasks.workunit.client.1.smithi179.stdout:6/472: mknod d3/d12/d23/d26/d57/ca4 0 2024-04-02T04:47:51.135 INFO:tasks.workunit.client.1.smithi179.stdout:8/450: rename d3/f67 to d3/d31/f9e 0 2024-04-02T04:47:51.138 INFO:tasks.workunit.client.1.smithi179.stdout:1/489: mkdir d3/d5/d2f/d4f/dab 0 2024-04-02T04:47:51.140 INFO:tasks.workunit.client.1.smithi179.stdout:9/480: link d0/d1/c6 d0/d19/d1c/d64/c92 0 2024-04-02T04:47:51.140 INFO:tasks.workunit.client.0.smithi070.stdout:5/561: creat d5/d9/fb2 x:0 0 0 2024-04-02T04:47:51.141 INFO:tasks.workunit.client.0.smithi070.stdout:7/458: creat d7/d4e/d82/d57/f92 x:0 0 0 2024-04-02T04:47:51.142 INFO:tasks.workunit.client.0.smithi070.stdout:7/459: dread - d7/dd/f8d zero size 2024-04-02T04:47:51.142 INFO:tasks.workunit.client.0.smithi070.stdout:3/499: fdatasync d10/d66/fa9 0 2024-04-02T04:47:51.144 INFO:tasks.workunit.client.1.smithi179.stdout:2/341: getdents dc/d29/d4a/d4b 0 2024-04-02T04:47:51.144 INFO:tasks.workunit.client.0.smithi070.stdout:0/543: symlink d4/d5/d13/d2a/d3a/d45/l9a 0 2024-04-02T04:47:51.146 INFO:tasks.workunit.client.1.smithi179.stdout:7/570: dread d1/d9/d35/f3b [0,4194304] 0 2024-04-02T04:47:51.148 INFO:tasks.workunit.client.0.smithi070.stdout:5/562: unlink d5/d17/f93 0 2024-04-02T04:47:51.149 INFO:tasks.workunit.client.1.smithi179.stdout:6/473: mkdir d3/d2d/da5 0 2024-04-02T04:47:51.150 INFO:tasks.workunit.client.0.smithi070.stdout:4/405: truncate d1/fd 9094041 0 2024-04-02T04:47:51.150 INFO:tasks.workunit.client.0.smithi070.stdout:2/497: dread d2/d12/d3e/d68/d90/f9e [0,4194304] 0 2024-04-02T04:47:51.152 INFO:tasks.workunit.client.1.smithi179.stdout:8/451: fsync d3/d31/f7f 0 2024-04-02T04:47:51.154 INFO:tasks.workunit.client.1.smithi179.stdout:2/342: mknod dc/d29/c69 0 2024-04-02T04:47:51.155 INFO:tasks.workunit.client.0.smithi070.stdout:3/500: fsync d10/d17/d51/d73/d8f/fab 0 2024-04-02T04:47:51.156 INFO:tasks.workunit.client.1.smithi179.stdout:2/343: write dc/d29/d4a/f53 [956141,57889] 0 2024-04-02T04:47:51.156 INFO:tasks.workunit.client.0.smithi070.stdout:2/498: dwrite d2/f1c [0,4194304] 0 2024-04-02T04:47:51.158 INFO:tasks.workunit.client.0.smithi070.stdout:1/499: getdents d0/d54/d64 0 2024-04-02T04:47:51.158 INFO:tasks.workunit.client.1.smithi179.stdout:8/452: dwrite d3/d31/f7e [0,4194304] 0 2024-04-02T04:47:51.159 INFO:tasks.workunit.client.0.smithi070.stdout:2/499: write d2/d10/d5c/d7a/f92 [104821,26294] 0 2024-04-02T04:47:51.160 INFO:tasks.workunit.client.1.smithi179.stdout:8/453: fsync d3/d32/d21/d22/d6b/d91/f40 0 2024-04-02T04:47:51.160 INFO:tasks.workunit.client.1.smithi179.stdout:9/481: creat d0/d19/d27/d7a/d84/f93 x:0 0 0 2024-04-02T04:47:51.161 INFO:tasks.workunit.client.1.smithi179.stdout:8/454: fsync d3/d32/d21/d50/d56/f77 0 2024-04-02T04:47:51.166 INFO:tasks.workunit.client.0.smithi070.stdout:9/420: dwrite d4/d10/d11/f4e [4194304,4194304] 0 2024-04-02T04:47:51.176 INFO:tasks.workunit.client.0.smithi070.stdout:3/501: write d10/d32/f42 [1516282,122485] 0 2024-04-02T04:47:51.178 INFO:tasks.workunit.client.0.smithi070.stdout:1/500: unlink d0/d9/c57 0 2024-04-02T04:47:51.180 INFO:tasks.workunit.client.0.smithi070.stdout:2/500: mknod d2/d10/d21/d2b/ca1 0 2024-04-02T04:47:51.181 INFO:tasks.workunit.client.1.smithi179.stdout:1/490: dwrite d3/d5/f21 [0,4194304] 0 2024-04-02T04:47:51.183 INFO:tasks.workunit.client.1.smithi179.stdout:1/491: write d3/d5/d2f/d4f/d42/d53/d5b/f83 [870042,118955] 0 2024-04-02T04:47:51.187 INFO:tasks.workunit.client.1.smithi179.stdout:2/344: link dc/d29/d31/f3a dc/d29/f6a 0 2024-04-02T04:47:51.188 INFO:tasks.workunit.client.1.smithi179.stdout:2/345: write dc/d29/d4a/d1e/f36 [810181,52890] 0 2024-04-02T04:47:51.189 INFO:tasks.workunit.client.0.smithi070.stdout:9/421: symlink d4/d10/d11/d7b/d19/d2f/l87 0 2024-04-02T04:47:51.191 INFO:tasks.workunit.client.0.smithi070.stdout:9/422: chown d4/d10/d11/d7b/d19/d2f/d39/d4f/f77 1 1 2024-04-02T04:47:51.191 INFO:tasks.workunit.client.1.smithi179.stdout:7/571: truncate d1/dd/d76/f86 629386 0 2024-04-02T04:47:51.192 INFO:tasks.workunit.client.0.smithi070.stdout:0/544: write d4/d5/d13/f19 [4438183,98705] 0 2024-04-02T04:47:51.193 INFO:tasks.workunit.client.1.smithi179.stdout:1/492: symlink d3/d5/d2f/d36/lac 0 2024-04-02T04:47:51.194 INFO:tasks.workunit.client.0.smithi070.stdout:0/545: truncate d4/d5/d21/d6b/f7b 314164 0 2024-04-02T04:47:51.194 INFO:tasks.workunit.client.0.smithi070.stdout:5/563: truncate d5/d9/d23/f49 2748851 0 2024-04-02T04:47:51.196 INFO:tasks.workunit.client.0.smithi070.stdout:7/460: link d7/d36/f72 d7/f93 0 2024-04-02T04:47:51.196 INFO:tasks.workunit.client.0.smithi070.stdout:7/461: truncate d7/f93 1348269 0 2024-04-02T04:47:51.197 INFO:tasks.workunit.client.0.smithi070.stdout:3/502: creat d10/d17/d51/fb0 x:0 0 0 2024-04-02T04:47:51.200 INFO:tasks.workunit.client.1.smithi179.stdout:9/482: rename d0/d19/d1c/d64/f85 to d0/d1/d11/d29/f94 0 2024-04-02T04:47:51.201 INFO:tasks.workunit.client.0.smithi070.stdout:2/501: mknod d2/d10/d19/ca2 0 2024-04-02T04:47:51.202 INFO:tasks.workunit.client.1.smithi179.stdout:2/346: creat dc/d29/d31/d57/f6b x:0 0 0 2024-04-02T04:47:51.203 INFO:tasks.workunit.client.0.smithi070.stdout:1/501: read d0/d9/d1d/f60 [960969,128672] 0 2024-04-02T04:47:51.204 INFO:tasks.workunit.client.1.smithi179.stdout:7/572: rmdir d1/d9/d55 39 2024-04-02T04:47:51.204 INFO:tasks.workunit.client.0.smithi070.stdout:1/502: dread d0/d7/f6a [0,4194304] 0 2024-04-02T04:47:51.206 INFO:tasks.workunit.client.0.smithi070.stdout:1/503: write d0/d4c/f72 [1011521,96084] 0 2024-04-02T04:47:51.208 INFO:tasks.workunit.client.0.smithi070.stdout:9/423: rmdir d4/d10/d11/d7b/d19/d53 39 2024-04-02T04:47:51.210 INFO:tasks.workunit.client.1.smithi179.stdout:6/474: getdents d3/d12/d23/d26 0 2024-04-02T04:47:51.211 INFO:tasks.workunit.client.0.smithi070.stdout:4/406: creat d1/d3/f89 x:0 0 0 2024-04-02T04:47:51.212 INFO:tasks.workunit.client.0.smithi070.stdout:5/564: creat d5/d9/d23/d8b/fb3 x:0 0 0 2024-04-02T04:47:51.216 INFO:tasks.workunit.client.1.smithi179.stdout:2/347: mkdir dc/d29/d4a/d1e/d6c 0 2024-04-02T04:47:51.218 INFO:tasks.workunit.client.0.smithi070.stdout:7/462: creat d7/dd/df/f94 x:0 0 0 2024-04-02T04:47:51.219 INFO:tasks.workunit.client.0.smithi070.stdout:3/503: fsync fd 0 2024-04-02T04:47:51.221 INFO:tasks.workunit.client.1.smithi179.stdout:6/475: rename d3/c91 to d3/d2c/d85/ca6 0 2024-04-02T04:47:51.221 INFO:tasks.workunit.client.1.smithi179.stdout:6/476: readlink d3/d12/l58 0 2024-04-02T04:47:51.224 INFO:tasks.workunit.client.1.smithi179.stdout:2/348: creat dc/d29/d4a/d4b/f6d x:0 0 0 2024-04-02T04:47:51.224 INFO:tasks.workunit.client.0.smithi070.stdout:1/504: creat d0/d4c/d5a/fa4 x:0 0 0 2024-04-02T04:47:51.225 INFO:tasks.workunit.client.1.smithi179.stdout:2/349: fdatasync dc/d29/d4a/d1e/f36 0 2024-04-02T04:47:51.228 INFO:tasks.workunit.client.1.smithi179.stdout:7/573: creat d1/d9/d27/d87/d8b/d75/fcb x:0 0 0 2024-04-02T04:47:51.228 INFO:tasks.workunit.client.0.smithi070.stdout:4/407: truncate d1/d3/d9/d41/f48 873664 0 2024-04-02T04:47:51.228 INFO:tasks.workunit.client.1.smithi179.stdout:9/483: dwrite d0/d19/f56 [0,4194304] 0 2024-04-02T04:47:51.229 INFO:tasks.workunit.client.0.smithi070.stdout:0/546: dwrite d4/d5/d13/d2a/d29/f59 [4194304,4194304] 0 2024-04-02T04:47:51.229 INFO:tasks.workunit.client.1.smithi179.stdout:1/493: creat d3/d5/d2f/d4f/d42/d53/fad x:0 0 0 2024-04-02T04:47:51.230 INFO:tasks.workunit.client.0.smithi070.stdout:5/565: creat d5/d9/d23/fb4 x:0 0 0 2024-04-02T04:47:51.230 INFO:tasks.workunit.client.0.smithi070.stdout:5/566: dread - d5/d14/f68 zero size 2024-04-02T04:47:51.231 INFO:tasks.workunit.client.1.smithi179.stdout:1/494: stat d3/d5/d2f/d4f/dab 0 2024-04-02T04:47:51.231 INFO:tasks.workunit.client.0.smithi070.stdout:5/567: readlink d5/d17/d2b/d7b/l95 0 2024-04-02T04:47:51.232 INFO:tasks.workunit.client.1.smithi179.stdout:6/477: creat d3/d2d/d38/d64/fa7 x:0 0 0 2024-04-02T04:47:51.232 INFO:tasks.workunit.client.0.smithi070.stdout:5/568: write d5/d9/d10/d51/faa [827401,7386] 0 2024-04-02T04:47:51.233 INFO:tasks.workunit.client.1.smithi179.stdout:2/350: dwrite dc/d29/d4a/d4b/f6d [0,4194304] 0 2024-04-02T04:47:51.235 INFO:tasks.workunit.client.0.smithi070.stdout:7/463: rmdir d7/dd/df/d79/d46/d65 39 2024-04-02T04:47:51.236 INFO:tasks.workunit.client.0.smithi070.stdout:3/504: rmdir d10/d32/d8a/d8d 39 2024-04-02T04:47:51.236 INFO:tasks.workunit.client.0.smithi070.stdout:6/379: sync 2024-04-02T04:47:51.236 INFO:tasks.workunit.client.0.smithi070.stdout:8/474: sync 2024-04-02T04:47:51.238 INFO:tasks.workunit.client.0.smithi070.stdout:2/502: mknod d2/d10/d21/ca3 0 2024-04-02T04:47:51.239 INFO:tasks.workunit.client.1.smithi179.stdout:7/574: creat d1/d9/d35/d6b/fcc x:0 0 0 2024-04-02T04:47:51.240 INFO:tasks.workunit.client.1.smithi179.stdout:9/484: creat d0/d1/d11/d29/d86/f95 x:0 0 0 2024-04-02T04:47:51.242 INFO:tasks.workunit.client.0.smithi070.stdout:3/505: dwrite d10/f4f [0,4194304] 0 2024-04-02T04:47:51.243 INFO:tasks.workunit.client.1.smithi179.stdout:6/478: unlink d3/d12/d23/d26/f69 0 2024-04-02T04:47:51.243 INFO:tasks.workunit.client.1.smithi179.stdout:2/351: dwrite dc/fe [0,4194304] 0 2024-04-02T04:47:51.244 INFO:tasks.workunit.client.1.smithi179.stdout:6/479: fsync d3/d12/d23/d26/d57/fa2 0 2024-04-02T04:47:51.245 INFO:tasks.workunit.client.0.smithi070.stdout:9/424: creat d4/d10/d12/d27/d60/f88 x:0 0 0 2024-04-02T04:47:51.245 INFO:tasks.workunit.client.1.smithi179.stdout:2/352: stat dc/d29/d4a/d1e/d43/f5f 0 2024-04-02T04:47:51.246 INFO:tasks.workunit.client.1.smithi179.stdout:2/353: dread - dc/d29/d4a/f3f zero size 2024-04-02T04:47:51.246 INFO:tasks.workunit.client.0.smithi070.stdout:1/505: mkdir d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5 0 2024-04-02T04:47:51.252 INFO:tasks.workunit.client.1.smithi179.stdout:6/480: creat d3/d2d/d38/d64/fa8 x:0 0 0 2024-04-02T04:47:51.253 INFO:tasks.workunit.client.1.smithi179.stdout:6/481: stat d3/d12/d6a/f6e 0 2024-04-02T04:47:51.258 INFO:tasks.workunit.client.0.smithi070.stdout:7/464: unlink d7/l7c 0 2024-04-02T04:47:51.259 INFO:tasks.workunit.client.0.smithi070.stdout:0/547: dwrite d4/d5/d13/d2a/d8d/f8f [0,4194304] 0 2024-04-02T04:47:51.260 INFO:tasks.workunit.client.0.smithi070.stdout:7/465: readlink d7/dd/df/l3f 0 2024-04-02T04:47:51.262 INFO:tasks.workunit.client.1.smithi179.stdout:2/354: getdents dc/d29/d4a/d1e/d6c 0 2024-04-02T04:47:51.263 INFO:tasks.workunit.client.0.smithi070.stdout:2/503: write d2/d12/d3e/d68/d90/f9e [3102671,92410] 0 2024-04-02T04:47:51.264 INFO:tasks.workunit.client.1.smithi179.stdout:1/495: getdents d3/d5/d2f/d4f/d42/d53/d72 0 2024-04-02T04:47:51.264 INFO:tasks.workunit.client.0.smithi070.stdout:2/504: chown d2/d10/d21/f33 81273 1 2024-04-02T04:47:51.264 INFO:tasks.workunit.client.0.smithi070.stdout:3/506: fsync d10/d66/fa9 0 2024-04-02T04:47:51.267 INFO:tasks.workunit.client.1.smithi179.stdout:6/482: unlink d3/d12/d23/d39/d4e/d66/d7c/f87 0 2024-04-02T04:47:51.269 INFO:tasks.workunit.client.1.smithi179.stdout:2/355: rename dc/d29/d4a/d1e/f36 to dc/d29/d4a/d1e/d43/f6e 0 2024-04-02T04:47:51.272 INFO:tasks.workunit.client.0.smithi070.stdout:3/507: dread d10/d17/d51/f64 [4194304,4194304] 0 2024-04-02T04:47:51.273 INFO:tasks.workunit.client.1.smithi179.stdout:1/496: dwrite d3/d5/d2f/d4f/d35/f6e [0,4194304] 0 2024-04-02T04:47:51.273 INFO:tasks.workunit.client.0.smithi070.stdout:5/569: mknod d5/d9/d23/d30/da2/cb5 0 2024-04-02T04:47:51.274 INFO:tasks.workunit.client.0.smithi070.stdout:5/570: dread - d5/d9/d23/d8b/fa5 zero size 2024-04-02T04:47:51.277 INFO:tasks.workunit.client.1.smithi179.stdout:9/485: truncate d0/d1/d11/f51 231217 0 2024-04-02T04:47:51.279 INFO:tasks.workunit.client.1.smithi179.stdout:2/356: mknod dc/d29/d4a/d1e/d43/c6f 0 2024-04-02T04:47:51.280 INFO:tasks.workunit.client.0.smithi070.stdout:0/548: creat d4/d5/d21/f9b x:0 0 0 2024-04-02T04:47:51.281 INFO:tasks.workunit.client.1.smithi179.stdout:1/497: chown d3/d5/de/c2e 179 1 2024-04-02T04:47:51.282 INFO:tasks.workunit.client.1.smithi179.stdout:1/498: readlink d3/d5/d2f/d4f/d35/d49/l4b 0 2024-04-02T04:47:51.282 INFO:tasks.workunit.client.0.smithi070.stdout:7/466: symlink d7/d4e/d82/d57/l95 0 2024-04-02T04:47:51.283 INFO:tasks.workunit.client.1.smithi179.stdout:1/499: dread - d3/d5/d2f/d4f/d42/d53/d72/d86/f90 zero size 2024-04-02T04:47:51.283 INFO:tasks.workunit.client.0.smithi070.stdout:8/475: mknod d2/d8/d23/d24/c96 0 2024-04-02T04:47:51.284 INFO:tasks.workunit.client.0.smithi070.stdout:8/476: readlink d2/d8/d23/d24/l45 0 2024-04-02T04:47:51.284 INFO:tasks.workunit.client.0.smithi070.stdout:8/477: fsync d2/d3/d35/f78 0 2024-04-02T04:47:51.285 INFO:tasks.workunit.client.0.smithi070.stdout:6/380: mknod d0/d20/d77/c7c 0 2024-04-02T04:47:51.286 INFO:tasks.workunit.client.1.smithi179.stdout:9/486: creat d0/d1/d10/d6d/d58/f96 x:0 0 0 2024-04-02T04:47:51.287 INFO:tasks.workunit.client.1.smithi179.stdout:2/357: mknod dc/d29/c70 0 2024-04-02T04:47:51.288 INFO:tasks.workunit.client.1.smithi179.stdout:6/483: creat d3/d2d/d38/fa9 x:0 0 0 2024-04-02T04:47:51.289 INFO:tasks.workunit.client.1.smithi179.stdout:1/500: creat d3/d5/d2f/d4f/d42/fae x:0 0 0 2024-04-02T04:47:51.292 INFO:tasks.workunit.client.1.smithi179.stdout:9/487: mknod d0/d19/d1c/c97 0 2024-04-02T04:47:51.295 INFO:tasks.workunit.client.0.smithi070.stdout:1/506: fdatasync d0/d7/f10 0 2024-04-02T04:47:51.295 INFO:tasks.workunit.client.0.smithi070.stdout:4/408: write d1/f12 [639873,54167] 0 2024-04-02T04:47:51.296 INFO:tasks.workunit.client.1.smithi179.stdout:2/358: creat dc/d29/d31/d57/f71 x:0 0 0 2024-04-02T04:47:51.299 INFO:tasks.workunit.client.0.smithi070.stdout:5/571: symlink d5/d57/lb6 0 2024-04-02T04:47:51.300 INFO:tasks.workunit.client.1.smithi179.stdout:1/501: chown d3/d5/de/c3a 19 1 2024-04-02T04:47:51.300 INFO:tasks.workunit.client.0.smithi070.stdout:5/572: write d5/f97 [888221,88616] 0 2024-04-02T04:47:51.301 INFO:tasks.workunit.client.1.smithi179.stdout:1/502: chown d3/d5/d2f/d4f/d42/l71 0 1 2024-04-02T04:47:51.301 INFO:tasks.workunit.client.0.smithi070.stdout:5/573: write d5/d17/d2b/f8d [1314442,86411] 0 2024-04-02T04:47:51.302 INFO:tasks.workunit.client.1.smithi179.stdout:1/503: dread - d3/d5/de/d33/f7b zero size 2024-04-02T04:47:51.303 INFO:tasks.workunit.client.0.smithi070.stdout:7/467: mknod d7/d4e/c96 0 2024-04-02T04:47:51.303 INFO:tasks.workunit.client.1.smithi179.stdout:1/504: write d3/d5/d2f/d4f/d42/d53/d5b/f61 [1682150,9738] 0 2024-04-02T04:47:51.304 INFO:tasks.workunit.client.1.smithi179.stdout:9/488: creat d0/d19/d27/d7a/d82/f98 x:0 0 0 2024-04-02T04:47:51.304 INFO:tasks.workunit.client.0.smithi070.stdout:7/468: write d7/dd/df/d79/d46/f51 [1892279,38592] 0 2024-04-02T04:47:51.310 INFO:tasks.workunit.client.0.smithi070.stdout:9/425: truncate d4/d13/f26 4116940 0 2024-04-02T04:47:51.312 INFO:tasks.workunit.client.1.smithi179.stdout:1/505: mknod d3/d5/caf 0 2024-04-02T04:47:51.318 INFO:tasks.workunit.client.0.smithi070.stdout:1/507: rmdir d0/d9/d1d/d22/d56/d79 39 2024-04-02T04:47:51.319 INFO:tasks.workunit.client.0.smithi070.stdout:1/508: dread - d0/d9/d1d/d22/d2c/d4f/f50 zero size 2024-04-02T04:47:51.319 INFO:tasks.workunit.client.1.smithi179.stdout:6/484: creat d3/d12/d23/d39/d80/faa x:0 0 0 2024-04-02T04:47:51.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:51 smithi179 ceph-mon[36603]: pgmap v14: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 60 MiB/s rd, 280 MiB/s wr, 478 op/s 2024-04-02T04:47:51.324 INFO:tasks.workunit.client.1.smithi179.stdout:0/520: sync 2024-04-02T04:47:51.324 INFO:tasks.workunit.client.1.smithi179.stdout:3/518: sync 2024-04-02T04:47:51.325 INFO:tasks.workunit.client.1.smithi179.stdout:4/553: sync 2024-04-02T04:47:51.325 INFO:tasks.workunit.client.0.smithi070.stdout:6/381: dwrite d0/d3/d19/f1f [0,4194304] 0 2024-04-02T04:47:51.325 INFO:tasks.workunit.client.0.smithi070.stdout:5/574: fdatasync d5/d17/d2b/d7b/f92 0 2024-04-02T04:47:51.326 INFO:tasks.workunit.client.1.smithi179.stdout:9/489: dwrite d0/d1/f44 [0,4194304] 0 2024-04-02T04:47:51.330 INFO:tasks.workunit.client.1.smithi179.stdout:3/519: chown d8/d26/d36/c43 0 1 2024-04-02T04:47:51.330 INFO:tasks.workunit.client.0.smithi070.stdout:8/478: mknod d2/c97 0 2024-04-02T04:47:51.333 INFO:tasks.workunit.client.1.smithi179.stdout:6/485: truncate d3/f10 1203502 0 2024-04-02T04:47:51.335 INFO:tasks.workunit.client.0.smithi070.stdout:9/426: mkdir d4/d13/d89 0 2024-04-02T04:47:51.336 INFO:tasks.workunit.client.0.smithi070.stdout:9/427: fsync d4/d10/d12/f62 0 2024-04-02T04:47:51.337 INFO:tasks.workunit.client.0.smithi070.stdout:0/549: dwrite f0 [0,4194304] 0 2024-04-02T04:47:51.338 INFO:tasks.workunit.client.0.smithi070.stdout:9/428: write d4/d10/d11/f3f [5065258,116488] 0 2024-04-02T04:47:51.340 INFO:tasks.workunit.client.0.smithi070.stdout:2/505: rename d2/d12/d3e/d68 to d2/d10/da4 0 2024-04-02T04:47:51.343 INFO:tasks.workunit.client.0.smithi070.stdout:4/409: symlink d1/d3/l8a 0 2024-04-02T04:47:51.344 INFO:tasks.workunit.client.0.smithi070.stdout:1/509: rmdir d0/d7/d28 39 2024-04-02T04:47:51.345 INFO:tasks.workunit.client.0.smithi070.stdout:1/510: stat d0/d7/d1a/d66/fa3 0 2024-04-02T04:47:51.349 INFO:tasks.workunit.client.1.smithi179.stdout:3/520: link d8/d26/d52/c73 d8/d7a/c9f 0 2024-04-02T04:47:51.349 INFO:tasks.workunit.client.0.smithi070.stdout:6/382: creat d0/d3/d19/d31/d46/d6b/f7d x:0 0 0 2024-04-02T04:47:51.350 INFO:tasks.workunit.client.0.smithi070.stdout:6/383: chown d0/d3/d19/d39 0 1 2024-04-02T04:47:51.350 INFO:tasks.workunit.client.1.smithi179.stdout:4/554: link d1/l3b d1/d3/df/lb7 0 2024-04-02T04:47:51.350 INFO:tasks.workunit.client.0.smithi070.stdout:5/575: truncate d5/d17/f32 5291320 0 2024-04-02T04:47:51.351 INFO:tasks.workunit.client.0.smithi070.stdout:7/469: mkdir d7/dd/df/d79/d46/d97 0 2024-04-02T04:47:51.354 INFO:tasks.workunit.client.1.smithi179.stdout:3/521: rename d8/d26/f2b to d8/d26/d83/fa0 0 2024-04-02T04:47:51.356 INFO:tasks.workunit.client.1.smithi179.stdout:4/555: truncate d1/d15/d6e/f80 448194 0 2024-04-02T04:47:51.359 INFO:tasks.workunit.client.1.smithi179.stdout:9/490: write d0/d19/d1c/d64/f69 [430131,63201] 0 2024-04-02T04:47:51.363 INFO:tasks.workunit.client.1.smithi179.stdout:4/556: rmdir d1/d15/d6e 39 2024-04-02T04:47:51.365 INFO:tasks.workunit.client.0.smithi070.stdout:3/508: rename d10/d17/d4c/c89 to d10/d32/d3f/cb1 0 2024-04-02T04:47:51.365 INFO:tasks.workunit.client.0.smithi070.stdout:3/509: stat d10/d17/d51/d73/d9e/ca2 0 2024-04-02T04:47:51.366 INFO:tasks.workunit.client.1.smithi179.stdout:9/491: symlink d0/d1/d11/d29/d86/l99 0 2024-04-02T04:47:51.368 INFO:tasks.workunit.client.0.smithi070.stdout:2/506: creat d2/d10/d21/d71/fa5 x:0 0 0 2024-04-02T04:47:51.370 INFO:tasks.workunit.client.1.smithi179.stdout:4/557: mkdir d1/d15/d7c/db8 0 2024-04-02T04:47:51.371 INFO:tasks.workunit.client.1.smithi179.stdout:6/486: write d3/d2d/d43/f47 [1042191,24556] 0 2024-04-02T04:47:51.372 INFO:tasks.workunit.client.0.smithi070.stdout:3/510: dwrite d10/d32/f52 [0,4194304] 0 2024-04-02T04:47:51.374 INFO:tasks.workunit.client.0.smithi070.stdout:8/479: dread d2/d3/f49 [0,4194304] 0 2024-04-02T04:47:51.374 INFO:tasks.workunit.client.0.smithi070.stdout:8/480: chown d2/c20 1 1 2024-04-02T04:47:51.376 INFO:tasks.workunit.client.1.smithi179.stdout:6/487: chown d3/d12/d23/d39/d4e/d66/l6d 464 1 2024-04-02T04:47:51.377 INFO:tasks.workunit.client.1.smithi179.stdout:6/488: stat d3/d12/d23/d26/d57/fa2 0 2024-04-02T04:47:51.381 INFO:tasks.workunit.client.0.smithi070.stdout:6/384: creat d0/d20/f7e x:0 0 0 2024-04-02T04:47:51.382 INFO:tasks.workunit.client.0.smithi070.stdout:6/385: write d0/d3/d19/d31/d61/f66 [916744,98188] 0 2024-04-02T04:47:51.382 INFO:tasks.workunit.client.1.smithi179.stdout:4/558: rename d1/d3/dc/d26/d8a/c92 to d1/d3/df/d10/d2f/d60/da6/cb9 0 2024-04-02T04:47:51.384 INFO:tasks.workunit.client.0.smithi070.stdout:0/550: creat d4/d5/d13/d2a/d3a/d80/f9c x:0 0 0 2024-04-02T04:47:51.385 INFO:tasks.workunit.client.1.smithi179.stdout:3/522: write d8/d57/d63/f6b [463615,83216] 0 2024-04-02T04:47:51.386 INFO:tasks.workunit.client.0.smithi070.stdout:9/429: creat d4/d10/d11/d7b/d19/d82/f8a x:0 0 0 2024-04-02T04:47:51.387 INFO:tasks.workunit.client.0.smithi070.stdout:0/551: truncate d4/d5/d13/d2a/d3a/d80/f9c 621848 0 2024-04-02T04:47:51.390 INFO:tasks.workunit.client.0.smithi070.stdout:2/507: creat d2/d12/fa6 x:0 0 0 2024-04-02T04:47:51.393 INFO:tasks.workunit.client.1.smithi179.stdout:4/559: mknod d1/cba 0 2024-04-02T04:47:51.394 INFO:tasks.workunit.client.1.smithi179.stdout:9/492: link d0/d1/d11/f51 d0/d1/d11/d29/d40/f9a 0 2024-04-02T04:47:51.394 INFO:tasks.workunit.client.1.smithi179.stdout:9/493: rename d0 to d0/d1/d11/d29/d80/d9b 22 2024-04-02T04:47:51.395 INFO:tasks.workunit.client.1.smithi179.stdout:9/494: truncate d0/d1/f7b 4757292 0 2024-04-02T04:47:51.396 INFO:tasks.workunit.client.0.smithi070.stdout:8/481: chown d2/d8/d23/d24/d5f/c4a 0 1 2024-04-02T04:47:51.397 INFO:tasks.workunit.client.1.smithi179.stdout:6/489: dread d3/fa [0,4194304] 0 2024-04-02T04:47:51.397 INFO:tasks.workunit.client.0.smithi070.stdout:2/508: dwrite d2/d10/d21/d2b/f6e [0,4194304] 0 2024-04-02T04:47:51.398 INFO:tasks.workunit.client.1.smithi179.stdout:3/523: creat d8/fa1 x:0 0 0 2024-04-02T04:47:51.400 INFO:tasks.workunit.client.0.smithi070.stdout:7/470: creat d7/dd/df/d79/d46/d65/d7b/f98 x:0 0 0 2024-04-02T04:47:51.403 INFO:tasks.workunit.client.1.smithi179.stdout:9/495: unlink d0/d1/d11/d29/f34 0 2024-04-02T04:47:51.405 INFO:tasks.workunit.client.0.smithi070.stdout:7/471: dread d7/dd/df/d7a/f64 [0,4194304] 0 2024-04-02T04:47:51.408 INFO:tasks.workunit.client.0.smithi070.stdout:0/552: creat d4/d5/d13/d2a/d8d/f9d x:0 0 0 2024-04-02T04:47:51.410 INFO:tasks.workunit.client.1.smithi179.stdout:5/436: sync 2024-04-02T04:47:51.411 INFO:tasks.workunit.client.1.smithi179.stdout:5/437: read - d8/d1b/d23/d30/d5f/f77 zero size 2024-04-02T04:47:51.412 INFO:tasks.workunit.client.0.smithi070.stdout:7/472: dwrite d7/dd/df/d7a/f64 [4194304,4194304] 0 2024-04-02T04:47:51.413 INFO:tasks.workunit.client.1.smithi179.stdout:5/438: truncate d8/d1b/d23/d30/d5f/f77 924436 0 2024-04-02T04:47:51.414 INFO:tasks.workunit.client.1.smithi179.stdout:5/439: truncate d8/d1b/d23/f69 1336530 0 2024-04-02T04:47:51.414 INFO:tasks.workunit.client.1.smithi179.stdout:9/496: symlink d0/d1/d11/d3a/l9c 0 2024-04-02T04:47:51.414 INFO:tasks.workunit.client.0.smithi070.stdout:1/511: link d0/d7/c2f d0/d9/d1d/d22/d2c/d4f/ca6 0 2024-04-02T04:47:51.417 INFO:tasks.workunit.client.1.smithi179.stdout:6/490: creat d3/d9a/fab x:0 0 0 2024-04-02T04:47:51.421 INFO:tasks.workunit.client.0.smithi070.stdout:1/512: dwrite d0/f2a [4194304,4194304] 0 2024-04-02T04:47:51.422 INFO:tasks.workunit.client.0.smithi070.stdout:8/482: mkdir d2/d3/d52/d62/d98 0 2024-04-02T04:47:51.427 INFO:tasks.workunit.client.1.smithi179.stdout:9/497: chown d0/d19/d1c/d64/c92 944 1 2024-04-02T04:47:51.429 INFO:tasks.workunit.client.1.smithi179.stdout:6/491: stat d3/d42/c88 0 2024-04-02T04:47:51.431 INFO:tasks.workunit.client.1.smithi179.stdout:6/492: read - d3/d12/d23/d39/d80/faa zero size 2024-04-02T04:47:51.432 INFO:tasks.workunit.client.0.smithi070.stdout:3/511: truncate d10/d17/d4c/fa3 3813132 0 2024-04-02T04:47:51.433 INFO:tasks.workunit.client.1.smithi179.stdout:4/560: dwrite d1/d3/dc/f1c [0,4194304] 0 2024-04-02T04:47:51.433 INFO:tasks.workunit.client.0.smithi070.stdout:9/430: truncate d4/d10/f2c 519239 0 2024-04-02T04:47:51.434 INFO:tasks.workunit.client.1.smithi179.stdout:9/498: dread d0/d1/f7b [0,4194304] 0 2024-04-02T04:47:51.435 INFO:tasks.workunit.client.1.smithi179.stdout:5/440: truncate d8/d1b/d23/d32/d6e/f8a 800056 0 2024-04-02T04:47:51.435 INFO:tasks.workunit.client.1.smithi179.stdout:3/524: write d8/d57/d63/f6c [211801,47664] 0 2024-04-02T04:47:51.438 INFO:tasks.workunit.client.1.smithi179.stdout:5/441: write d8/d1b/d23/d32/d4b/f70 [1861758,65207] 0 2024-04-02T04:47:51.438 INFO:tasks.workunit.client.1.smithi179.stdout:3/525: write d8/d57/d63/f6b [1155232,89869] 0 2024-04-02T04:47:51.438 INFO:tasks.workunit.client.0.smithi070.stdout:4/410: getdents d1 0 2024-04-02T04:47:51.441 INFO:tasks.workunit.client.0.smithi070.stdout:6/386: truncate d0/d3/f3a 3779914 0 2024-04-02T04:47:51.441 INFO:tasks.workunit.client.0.smithi070.stdout:0/553: mknod d4/d5/d13/d2a/d3a/c9e 0 2024-04-02T04:47:51.443 INFO:tasks.workunit.client.1.smithi179.stdout:6/493: mknod d3/d12/d23/d39/d80/cac 0 2024-04-02T04:47:51.444 INFO:tasks.workunit.client.0.smithi070.stdout:7/473: symlink d7/d36/l99 0 2024-04-02T04:47:51.447 INFO:tasks.workunit.client.0.smithi070.stdout:8/483: dread d2/d8/f44 [0,4194304] 0 2024-04-02T04:47:51.453 INFO:tasks.workunit.client.0.smithi070.stdout:7/474: dwrite d7/dd/df/d79/d46/f51 [0,4194304] 0 2024-04-02T04:47:51.453 INFO:tasks.workunit.client.0.smithi070.stdout:4/411: creat d1/d3/d9/f8b x:0 0 0 2024-04-02T04:47:51.453 INFO:tasks.workunit.client.1.smithi179.stdout:3/526: creat d8/d26/d36/fa2 x:0 0 0 2024-04-02T04:47:51.459 INFO:tasks.workunit.client.0.smithi070.stdout:4/412: dwrite d1/f12 [0,4194304] 0 2024-04-02T04:47:51.459 INFO:tasks.workunit.client.0.smithi070.stdout:0/554: rename d4/d5/d13/d2a/d8d/f9d to d4/d5/d13/d2a/d3a/d80/f9f 0 2024-04-02T04:47:51.462 INFO:tasks.workunit.client.1.smithi179.stdout:6/494: rename d3/d2d/d38/d64/ca1 to d3/d2d/d38/d64/cad 0 2024-04-02T04:47:51.467 INFO:tasks.workunit.client.0.smithi070.stdout:0/555: dread d4/d5/d13/d2a/d29/d52/f82 [0,4194304] 0 2024-04-02T04:47:51.471 INFO:tasks.workunit.client.1.smithi179.stdout:4/561: truncate d1/d3/dc/d27/d62/f76 367844 0 2024-04-02T04:47:51.472 INFO:tasks.workunit.client.0.smithi070.stdout:3/512: mkdir d10/d17/db2 0 2024-04-02T04:47:51.474 INFO:tasks.workunit.client.0.smithi070.stdout:3/513: fdatasync d10/d32/d3f/f38 0 2024-04-02T04:47:51.474 INFO:tasks.workunit.client.0.smithi070.stdout:1/513: dwrite d0/d9/d1d/f4e [0,4194304] 0 2024-04-02T04:47:51.475 INFO:tasks.workunit.client.1.smithi179.stdout:5/442: symlink d8/d1b/d23/d36/d53/d89/l8b 0 2024-04-02T04:47:51.475 INFO:tasks.workunit.client.0.smithi070.stdout:3/514: write d10/d32/fa8 [223196,123249] 0 2024-04-02T04:47:51.476 INFO:tasks.workunit.client.0.smithi070.stdout:3/515: chown d10/d32/d3f/f38 61249780 1 2024-04-02T04:47:51.483 INFO:tasks.workunit.client.0.smithi070.stdout:6/387: dread d0/f26 [0,4194304] 0 2024-04-02T04:47:51.485 INFO:tasks.workunit.client.0.smithi070.stdout:8/484: unlink d2/d8/d29/l8c 0 2024-04-02T04:47:51.486 INFO:tasks.workunit.client.1.smithi179.stdout:4/562: dread - d1/f65 zero size 2024-04-02T04:47:51.489 INFO:tasks.workunit.client.0.smithi070.stdout:1/514: symlink d0/d9/d1d/la7 0 2024-04-02T04:47:51.490 INFO:tasks.workunit.client.1.smithi179.stdout:9/499: dwrite d0/d1/d10/d6d/fe [4194304,4194304] 0 2024-04-02T04:47:51.490 INFO:tasks.workunit.client.1.smithi179.stdout:6/495: mkdir d3/d2c/dae 0 2024-04-02T04:47:51.491 INFO:tasks.workunit.client.0.smithi070.stdout:8/485: dread d2/d8/d23/d24/f4b [0,4194304] 0 2024-04-02T04:47:51.491 INFO:tasks.workunit.client.0.smithi070.stdout:6/388: dwrite d0/d3/d19/d31/d37/f5d [0,4194304] 0 2024-04-02T04:47:51.492 INFO:tasks.workunit.client.1.smithi179.stdout:9/500: write d0/d1/d11/d77/f83 [611672,108256] 0 2024-04-02T04:47:51.492 INFO:tasks.workunit.client.1.smithi179.stdout:9/501: stat d0 0 2024-04-02T04:47:51.493 INFO:tasks.workunit.client.0.smithi070.stdout:8/486: chown d2/d8/d29/d22/d30 886 1 2024-04-02T04:47:51.494 INFO:tasks.workunit.client.0.smithi070.stdout:6/389: write d0/d55/d5c/f60 [947811,130461] 0 2024-04-02T04:47:51.495 INFO:tasks.workunit.client.0.smithi070.stdout:8/487: write d2/d3/fd [1450613,100621] 0 2024-04-02T04:47:51.496 INFO:tasks.workunit.client.1.smithi179.stdout:4/563: symlink d1/d3/dc/d27/d62/lbb 0 2024-04-02T04:47:51.501 INFO:tasks.workunit.client.1.smithi179.stdout:6/496: chown d3/c1f 448 1 2024-04-02T04:47:51.503 INFO:tasks.workunit.client.1.smithi179.stdout:6/497: chown d3/d12/d23/d26/f27 949924052 1 2024-04-02T04:47:51.504 INFO:tasks.workunit.client.1.smithi179.stdout:3/527: getdents d8/d6d/d94 0 2024-04-02T04:47:51.505 INFO:tasks.workunit.client.1.smithi179.stdout:5/443: dwrite d8/d1b/d23/f47 [4194304,4194304] 0 2024-04-02T04:47:51.506 INFO:tasks.workunit.client.0.smithi070.stdout:0/556: link d4/d5/d13/d2a/d3a/f8e d4/d5/d13/d2a/d29/d52/fa0 0 2024-04-02T04:47:51.507 INFO:tasks.workunit.client.1.smithi179.stdout:5/444: write d8/d1b/d23/d36/d53/f72 [754023,112650] 0 2024-04-02T04:47:51.507 INFO:tasks.workunit.client.1.smithi179.stdout:9/502: creat d0/d1/d10/d6d/d58/f9d x:0 0 0 2024-04-02T04:47:51.508 INFO:tasks.workunit.client.0.smithi070.stdout:3/516: dwrite d10/d17/d51/f64 [4194304,4194304] 0 2024-04-02T04:47:51.509 INFO:tasks.workunit.client.1.smithi179.stdout:4/564: creat d1/d3/dc/d26/d8a/d5d/fbc x:0 0 0 2024-04-02T04:47:51.509 INFO:tasks.workunit.client.0.smithi070.stdout:3/517: read - d10/d17/d51/fb0 zero size 2024-04-02T04:47:51.511 INFO:tasks.workunit.client.1.smithi179.stdout:6/498: rmdir d3/d2d/d38/d64 39 2024-04-02T04:47:51.511 INFO:tasks.workunit.client.1.smithi179.stdout:5/445: write d8/d1b/d23/d42/d5a/f76 [995890,28928] 0 2024-04-02T04:47:51.512 INFO:tasks.workunit.client.0.smithi070.stdout:6/390: fdatasync d0/d20/f34 0 2024-04-02T04:47:51.513 INFO:tasks.workunit.client.1.smithi179.stdout:9/503: creat d0/d1/d11/d29/d86/f9e x:0 0 0 2024-04-02T04:47:51.514 INFO:tasks.workunit.client.0.smithi070.stdout:4/413: creat d1/d3/f8c x:0 0 0 2024-04-02T04:47:51.515 INFO:tasks.workunit.client.1.smithi179.stdout:4/565: creat d1/d93/fbd x:0 0 0 2024-04-02T04:47:51.515 INFO:tasks.workunit.client.0.smithi070.stdout:8/488: creat d2/d8/d23/d24/d66/f99 x:0 0 0 2024-04-02T04:47:51.517 INFO:tasks.workunit.client.1.smithi179.stdout:6/499: creat d3/d12/d23/d26/faf x:0 0 0 2024-04-02T04:47:51.519 INFO:tasks.workunit.client.1.smithi179.stdout:6/500: truncate d3/d9a/fab 583832 0 2024-04-02T04:47:51.519 INFO:tasks.workunit.client.0.smithi070.stdout:0/557: mkdir d4/d5/d58/da1 0 2024-04-02T04:47:51.520 INFO:tasks.workunit.client.0.smithi070.stdout:0/558: chown d4/d5/d13/f32 4372519 1 2024-04-02T04:47:51.520 INFO:tasks.workunit.client.0.smithi070.stdout:1/515: rmdir d0/d9/d1d/d22/d56/d79 39 2024-04-02T04:47:51.522 INFO:tasks.workunit.client.1.smithi179.stdout:9/504: write d0/d19/d1c/d64/f79 [1399904,121510] 0 2024-04-02T04:47:51.522 INFO:tasks.workunit.client.1.smithi179.stdout:6/501: dread d3/fa [0,4194304] 0 2024-04-02T04:47:51.523 INFO:tasks.workunit.client.0.smithi070.stdout:0/559: write d4/d5/d13/f19 [1875850,16603] 0 2024-04-02T04:47:51.523 INFO:tasks.workunit.client.0.smithi070.stdout:3/518: symlink d10/d17/d51/d73/lb3 0 2024-04-02T04:47:51.523 INFO:tasks.workunit.client.0.smithi070.stdout:6/391: dread d0/d3/d19/d31/d37/f5a [0,4194304] 0 2024-04-02T04:47:51.524 INFO:tasks.workunit.client.1.smithi179.stdout:3/528: link d8/d26/d36/l42 d8/d26/d52/d85/la3 0 2024-04-02T04:47:51.524 INFO:tasks.workunit.client.1.smithi179.stdout:3/529: readlink d8/l18 0 2024-04-02T04:47:51.526 INFO:tasks.workunit.client.1.smithi179.stdout:3/530: write d8/d26/d36/fa2 [216962,92279] 0 2024-04-02T04:47:51.526 INFO:tasks.workunit.client.0.smithi070.stdout:4/414: rmdir d1/d3/d9 39 2024-04-02T04:47:51.527 INFO:tasks.workunit.client.1.smithi179.stdout:3/531: read d8/d57/d63/f6b [863989,89287] 0 2024-04-02T04:47:51.528 INFO:tasks.workunit.client.1.smithi179.stdout:9/505: dread d0/d19/d1c/f6b [0,4194304] 0 2024-04-02T04:47:51.529 INFO:tasks.workunit.client.1.smithi179.stdout:9/506: stat d0/d1/c6 0 2024-04-02T04:47:51.529 INFO:tasks.workunit.client.1.smithi179.stdout:3/532: dread - d8/d26/d36/f90 zero size 2024-04-02T04:47:51.529 INFO:tasks.workunit.client.0.smithi070.stdout:1/516: chown d0/d9/d1d/d22/d56/d79/d89 1644533 1 2024-04-02T04:47:51.530 INFO:tasks.workunit.client.1.smithi179.stdout:3/533: write d8/d26/d36/fa2 [536164,93827] 0 2024-04-02T04:47:51.531 INFO:tasks.workunit.client.1.smithi179.stdout:8/455: sync 2024-04-02T04:47:51.532 INFO:tasks.workunit.client.0.smithi070.stdout:0/560: creat d4/d3c/fa2 x:0 0 0 2024-04-02T04:47:51.533 INFO:tasks.workunit.client.0.smithi070.stdout:0/561: chown d4/d5/d13/d2a/d3a/c9e 1675013 1 2024-04-02T04:47:51.534 INFO:tasks.workunit.client.0.smithi070.stdout:3/519: read d10/d17/f48 [248095,104914] 0 2024-04-02T04:47:51.536 INFO:tasks.workunit.client.0.smithi070.stdout:4/415: write d1/d3/d9/d5e/f77 [962385,24022] 0 2024-04-02T04:47:51.539 INFO:tasks.workunit.client.0.smithi070.stdout:1/517: symlink d0/d54/d64/la8 0 2024-04-02T04:47:51.544 INFO:tasks.workunit.client.1.smithi179.stdout:8/456: fdatasync d3/d31/f9e 0 2024-04-02T04:47:51.545 INFO:tasks.workunit.client.0.smithi070.stdout:3/520: rename d10/c70 to d10/d17/d51/da5/cb4 0 2024-04-02T04:47:51.547 INFO:tasks.workunit.client.1.smithi179.stdout:4/566: truncate d1/d15/d7c/f84 2858333 0 2024-04-02T04:47:51.548 INFO:tasks.workunit.client.1.smithi179.stdout:5/446: getdents d8/d1b/d23/d30/d5f 0 2024-04-02T04:47:51.549 INFO:tasks.workunit.client.1.smithi179.stdout:5/447: chown f2 429513 1 2024-04-02T04:47:51.549 INFO:tasks.workunit.client.1.smithi179.stdout:8/457: dwrite d3/d32/f6e [0,4194304] 0 2024-04-02T04:47:51.550 INFO:tasks.workunit.client.0.smithi070.stdout:6/392: creat d0/d3/d19/f7f x:0 0 0 2024-04-02T04:47:51.552 INFO:tasks.workunit.client.1.smithi179.stdout:9/507: truncate d0/d1/d11/d3a/f81 1822286 0 2024-04-02T04:47:51.553 INFO:tasks.workunit.client.1.smithi179.stdout:5/448: dread d8/d1b/f24 [0,4194304] 0 2024-04-02T04:47:51.553 INFO:tasks.workunit.client.1.smithi179.stdout:9/508: truncate d0/d1/d11/f78 551614 0 2024-04-02T04:47:51.554 INFO:tasks.workunit.client.1.smithi179.stdout:5/449: readlink d8/d1b/d23/d32/d4e/d50/l57 0 2024-04-02T04:47:51.555 INFO:tasks.workunit.client.1.smithi179.stdout:6/502: truncate d3/d12/d23/d2f/f3a 1272456 0 2024-04-02T04:47:51.557 INFO:tasks.workunit.client.0.smithi070.stdout:1/518: rename d0/d7/f8 to d0/d54/d64/fa9 0 2024-04-02T04:47:51.558 INFO:tasks.workunit.client.0.smithi070.stdout:1/519: dread - d0/d4c/d5a/fa4 zero size 2024-04-02T04:47:51.559 INFO:tasks.workunit.client.0.smithi070.stdout:3/521: dread - d10/d32/d8a/d8d/d91/f9f zero size 2024-04-02T04:47:51.560 INFO:tasks.workunit.client.1.smithi179.stdout:1/506: sync 2024-04-02T04:47:51.560 INFO:tasks.workunit.client.1.smithi179.stdout:2/359: sync 2024-04-02T04:47:51.560 INFO:tasks.workunit.client.1.smithi179.stdout:0/521: sync 2024-04-02T04:47:51.560 INFO:tasks.workunit.client.1.smithi179.stdout:7/575: sync 2024-04-02T04:47:51.562 INFO:tasks.workunit.client.0.smithi070.stdout:8/489: link d2/d8/d29/d22/d30/c76 d2/d3/d52/d62/d7c/c9a 0 2024-04-02T04:47:51.566 INFO:tasks.workunit.client.0.smithi070.stdout:0/562: link d4/d3c/l40 d4/d5/d13/d17/d6e/la3 0 2024-04-02T04:47:51.569 INFO:tasks.workunit.client.1.smithi179.stdout:6/503: stat d3/f10 0 2024-04-02T04:47:51.573 INFO:tasks.workunit.client.1.smithi179.stdout:0/522: creat d1/da/d67/fa1 x:0 0 0 2024-04-02T04:47:51.575 INFO:tasks.workunit.client.1.smithi179.stdout:9/509: mkdir d0/d9f 0 2024-04-02T04:47:51.575 INFO:tasks.workunit.client.1.smithi179.stdout:6/504: dwrite d3/d2d/d43/f47 [0,4194304] 0 2024-04-02T04:47:51.576 INFO:tasks.workunit.client.1.smithi179.stdout:1/507: unlink d3/d5/de/l23 0 2024-04-02T04:47:51.578 INFO:tasks.workunit.client.0.smithi070.stdout:1/520: creat d0/d9/d1d/d22/d2c/d4f/d77/d8c/faa x:0 0 0 2024-04-02T04:47:51.578 INFO:tasks.workunit.client.0.smithi070.stdout:3/522: creat d10/d17/d4c/fb5 x:0 0 0 2024-04-02T04:47:51.579 INFO:tasks.workunit.client.0.smithi070.stdout:8/490: dwrite d2/d3/d35/d3a/d4f/d59/f8f [0,4194304] 0 2024-04-02T04:47:51.579 INFO:tasks.workunit.client.0.smithi070.stdout:4/416: link d1/d3/d9/l7f d1/l8d 0 2024-04-02T04:47:51.585 INFO:tasks.workunit.client.1.smithi179.stdout:1/508: dread d3/f2d [0,4194304] 0 2024-04-02T04:47:51.589 INFO:tasks.workunit.client.1.smithi179.stdout:3/534: truncate d8/d26/f4e 3459769 0 2024-04-02T04:47:51.590 INFO:tasks.workunit.client.1.smithi179.stdout:2/360: creat dc/d29/d4a/f72 x:0 0 0 2024-04-02T04:47:51.596 INFO:tasks.workunit.client.1.smithi179.stdout:7/576: link d1/dd/f99 d1/dd/d16/fcd 0 2024-04-02T04:47:51.600 INFO:tasks.workunit.client.1.smithi179.stdout:8/458: getdents d3/d31 0 2024-04-02T04:47:51.600 INFO:tasks.workunit.client.1.smithi179.stdout:8/459: readlink d3/l2e 0 2024-04-02T04:47:51.601 INFO:tasks.workunit.client.0.smithi070.stdout:6/393: getdents d0/d20/d77 0 2024-04-02T04:47:51.602 INFO:tasks.workunit.client.1.smithi179.stdout:0/523: dread d1/da/df/f20 [0,4194304] 0 2024-04-02T04:47:51.602 INFO:tasks.workunit.client.0.smithi070.stdout:6/394: fdatasync d0/d3/d19/d31/f4e 0 2024-04-02T04:47:51.604 INFO:tasks.workunit.client.1.smithi179.stdout:9/510: dread d0/d1/d11/d29/f50 [0,4194304] 0 2024-04-02T04:47:51.607 INFO:tasks.workunit.client.1.smithi179.stdout:6/505: rename d3/f10 to d3/d2d/d38/d64/fb0 0 2024-04-02T04:47:51.608 INFO:tasks.workunit.client.1.smithi179.stdout:8/460: dwrite d3/f82 [0,4194304] 0 2024-04-02T04:47:51.608 INFO:tasks.workunit.client.1.smithi179.stdout:9/511: dread - d0/d19/d27/d7a/d84/f93 zero size 2024-04-02T04:47:51.610 INFO:tasks.workunit.client.1.smithi179.stdout:4/567: write d1/d15/d6e/f80 [530608,66752] 0 2024-04-02T04:47:51.611 INFO:tasks.workunit.client.0.smithi070.stdout:4/417: chown d1/l4f 0 1 2024-04-02T04:47:51.611 INFO:tasks.workunit.client.1.smithi179.stdout:8/461: fdatasync d3/d32/d30/f4e 0 2024-04-02T04:47:51.612 INFO:tasks.workunit.client.1.smithi179.stdout:4/568: fdatasync d1/f42 0 2024-04-02T04:47:51.612 INFO:tasks.workunit.client.1.smithi179.stdout:9/512: chown d0/d1/d10/d6d/d58/f9d 4 1 2024-04-02T04:47:51.614 INFO:tasks.workunit.client.1.smithi179.stdout:5/450: dwrite d8/d1b/f1f [0,4194304] 0 2024-04-02T04:47:51.616 INFO:tasks.workunit.client.1.smithi179.stdout:2/361: mknod dc/d29/c73 0 2024-04-02T04:47:51.616 INFO:tasks.workunit.client.1.smithi179.stdout:4/569: write d1/d3/dc/f1c [3296046,82938] 0 2024-04-02T04:47:51.622 INFO:tasks.workunit.client.0.smithi070.stdout:1/521: creat d0/d9/d1d/d22/fab x:0 0 0 2024-04-02T04:47:51.622 INFO:tasks.workunit.client.0.smithi070.stdout:3/523: write d10/d17/f5d [17802,5433] 0 2024-04-02T04:47:51.622 INFO:tasks.workunit.client.1.smithi179.stdout:3/535: dread d8/d57/d63/f84 [0,4194304] 0 2024-04-02T04:47:51.623 INFO:tasks.workunit.client.0.smithi070.stdout:1/522: write d0/d4c/fa0 [225803,17668] 0 2024-04-02T04:47:51.624 INFO:tasks.workunit.client.1.smithi179.stdout:7/577: mknod d1/d9/d27/d87/d8b/d75/cce 0 2024-04-02T04:47:51.624 INFO:tasks.workunit.client.1.smithi179.stdout:9/513: dwrite d0/d1/f4 [0,4194304] 0 2024-04-02T04:47:51.625 INFO:tasks.workunit.client.0.smithi070.stdout:6/395: fdatasync d0/d20/f34 0 2024-04-02T04:47:51.626 INFO:tasks.workunit.client.1.smithi179.stdout:7/578: truncate d1/d9/fc9 633559 0 2024-04-02T04:47:51.627 INFO:tasks.workunit.client.1.smithi179.stdout:0/524: readlink d1/db/l31 0 2024-04-02T04:47:51.629 INFO:tasks.workunit.client.0.smithi070.stdout:8/491: rmdir d2/d3/d52/d62/d98 0 2024-04-02T04:47:51.630 INFO:tasks.workunit.client.1.smithi179.stdout:9/514: dread d0/d1/f65 [0,4194304] 0 2024-04-02T04:47:51.631 INFO:tasks.workunit.client.0.smithi070.stdout:1/523: dwrite d0/d9/d1d/f13 [4194304,4194304] 0 2024-04-02T04:47:51.634 INFO:tasks.workunit.client.0.smithi070.stdout:0/563: truncate d4/d5/f3f 973659 0 2024-04-02T04:47:51.635 INFO:tasks.workunit.client.0.smithi070.stdout:3/524: creat d10/d17/d51/d73/d8f/fb6 x:0 0 0 2024-04-02T04:47:51.635 INFO:tasks.workunit.client.1.smithi179.stdout:6/506: rmdir d3/d2d/d38/d64 39 2024-04-02T04:47:51.636 INFO:tasks.workunit.client.0.smithi070.stdout:0/564: fdatasync d4/d5/d58/d60/f7c 0 2024-04-02T04:47:51.639 INFO:tasks.workunit.client.0.smithi070.stdout:6/396: mkdir d0/d43/d80 0 2024-04-02T04:47:51.645 INFO:tasks.workunit.client.1.smithi179.stdout:8/462: creat d3/d32/d21/d49/f9f x:0 0 0 2024-04-02T04:47:51.651 INFO:tasks.workunit.client.0.smithi070.stdout:3/525: dread d10/d17/d4c/fa3 [0,4194304] 0 2024-04-02T04:47:51.653 INFO:tasks.workunit.client.0.smithi070.stdout:6/397: symlink d0/d20/d77/l81 0 2024-04-02T04:47:51.653 INFO:tasks.workunit.client.0.smithi070.stdout:6/398: dread - d0/f78 zero size 2024-04-02T04:47:51.654 INFO:tasks.workunit.client.1.smithi179.stdout:5/451: symlink d8/d1b/d23/d32/d46/l8c 0 2024-04-02T04:47:51.656 INFO:tasks.workunit.client.1.smithi179.stdout:2/362: mknod dc/d29/d31/d39/c74 0 2024-04-02T04:47:51.657 INFO:tasks.workunit.client.1.smithi179.stdout:1/509: dwrite d3/d5/f6d [0,4194304] 0 2024-04-02T04:47:51.658 INFO:tasks.workunit.client.0.smithi070.stdout:4/418: mkdir d1/d3/d85/d8e 0 2024-04-02T04:47:51.658 INFO:tasks.workunit.client.0.smithi070.stdout:1/524: write d0/d7/f6a [1366455,37298] 0 2024-04-02T04:47:51.663 INFO:tasks.workunit.client.1.smithi179.stdout:0/525: creat d1/da/df/fa2 x:0 0 0 2024-04-02T04:47:51.664 INFO:tasks.workunit.client.1.smithi179.stdout:0/526: chown d1/f14 10621299 1 2024-04-02T04:47:51.665 INFO:tasks.workunit.client.0.smithi070.stdout:6/399: write d0/d3/d19/d39/f53 [74683,37036] 0 2024-04-02T04:47:51.666 INFO:tasks.workunit.client.1.smithi179.stdout:0/527: read d1/da/d28/d37/f3e [2731955,35310] 0 2024-04-02T04:47:51.668 INFO:tasks.workunit.client.0.smithi070.stdout:6/400: dread d0/d3/d19/d31/d37/f5a [0,4194304] 0 2024-04-02T04:47:51.668 INFO:tasks.workunit.client.1.smithi179.stdout:7/579: rename d1/d9/d27/d87/dc7 to d1/d9/d27/d87/d8b/d75/db9/dcf 0 2024-04-02T04:47:51.669 INFO:tasks.workunit.client.1.smithi179.stdout:7/580: readlink d1/d9/d27/d87/d8b/d75/db9/d8d/la1 0 2024-04-02T04:47:51.672 INFO:tasks.workunit.client.0.smithi070.stdout:4/419: dread d1/d3/d9/f64 [0,4194304] 0 2024-04-02T04:47:51.674 INFO:tasks.workunit.client.0.smithi070.stdout:6/401: creat d0/d55/d5c/f82 x:0 0 0 2024-04-02T04:47:51.674 INFO:tasks.workunit.client.0.smithi070.stdout:8/492: truncate d2/d8/d23/f51 1991356 0 2024-04-02T04:47:51.675 INFO:tasks.workunit.client.0.smithi070.stdout:6/402: fdatasync d0/d3/d19/d31/d61/f66 0 2024-04-02T04:47:51.677 INFO:tasks.workunit.client.0.smithi070.stdout:0/565: truncate d4/d5/d13/d2a/d8d/f8f 1001590 0 2024-04-02T04:47:51.677 INFO:tasks.workunit.client.0.smithi070.stdout:6/403: read d0/d3/f13 [8262794,1714] 0 2024-04-02T04:47:51.678 INFO:tasks.workunit.client.0.smithi070.stdout:6/404: write d0/d3/d19/d31/f4e [3140971,71283] 0 2024-04-02T04:47:51.678 INFO:tasks.workunit.client.1.smithi179.stdout:6/507: creat d3/d12/d23/d39/d4e/d66/d7c/fb1 x:0 0 0 2024-04-02T04:47:51.679 INFO:tasks.workunit.client.0.smithi070.stdout:6/405: dread - d0/d3/d19/d31/d46/d6b/f70 zero size 2024-04-02T04:47:51.681 INFO:tasks.workunit.client.0.smithi070.stdout:6/406: truncate d0/d3/d19/d39/f65 870096 0 2024-04-02T04:47:51.685 INFO:tasks.workunit.client.1.smithi179.stdout:4/570: fdatasync d1/d3/dc/d27/d62/f76 0 2024-04-02T04:47:51.686 INFO:tasks.workunit.client.1.smithi179.stdout:9/515: dread d0/d1/d10/d6d/f20 [0,4194304] 0 2024-04-02T04:47:51.688 INFO:tasks.workunit.client.0.smithi070.stdout:6/407: dwrite d0/d43/f51 [4194304,4194304] 0 2024-04-02T04:47:51.692 INFO:tasks.workunit.client.0.smithi070.stdout:0/566: symlink d4/d3c/la4 0 2024-04-02T04:47:51.699 INFO:tasks.workunit.client.0.smithi070.stdout:4/420: write d1/d3/d9/f2a [11339,117564] 0 2024-04-02T04:47:51.699 INFO:tasks.workunit.client.1.smithi179.stdout:3/536: truncate d8/d57/f5a 581987 0 2024-04-02T04:47:51.699 INFO:tasks.workunit.client.0.smithi070.stdout:4/421: chown d1/d3/d9/f1d 12203079 1 2024-04-02T04:47:51.702 INFO:tasks.workunit.client.0.smithi070.stdout:8/493: write d2/d8/d29/f3b [461063,71023] 0 2024-04-02T04:47:51.703 INFO:tasks.workunit.client.1.smithi179.stdout:2/363: write dc/d29/d4a/d1e/f40 [714259,114588] 0 2024-04-02T04:47:51.704 INFO:tasks.workunit.client.0.smithi070.stdout:8/494: truncate d2/d3/d52/d62/f95 592523 0 2024-04-02T04:47:51.706 INFO:tasks.workunit.client.1.smithi179.stdout:0/528: read d1/da/df/f12 [757303,86036] 0 2024-04-02T04:47:51.707 INFO:tasks.workunit.client.0.smithi070.stdout:8/495: dread d2/d8/d23/d24/f37 [0,4194304] 0 2024-04-02T04:47:51.708 INFO:tasks.workunit.client.1.smithi179.stdout:1/510: dread d3/d5/de/f1a [0,4194304] 0 2024-04-02T04:47:51.709 INFO:tasks.workunit.client.0.smithi070.stdout:4/422: write d1/f12 [4960029,67242] 0 2024-04-02T04:47:51.710 INFO:tasks.workunit.client.1.smithi179.stdout:1/511: write d3/d5/d2f/d4f/d42/d53/d77/f97 [433873,115183] 0 2024-04-02T04:47:51.711 INFO:tasks.workunit.client.0.smithi070.stdout:1/525: getdents d0/d7/d28 0 2024-04-02T04:47:51.712 INFO:tasks.workunit.client.1.smithi179.stdout:8/463: rmdir d3/d32/d21/d50/d65 39 2024-04-02T04:47:51.713 INFO:tasks.workunit.client.0.smithi070.stdout:0/567: mkdir d4/d5/d58/d60/d83/da5 0 2024-04-02T04:47:51.715 INFO:tasks.workunit.client.0.smithi070.stdout:8/496: fsync d2/d8/d29/db/f13 0 2024-04-02T04:47:51.716 INFO:tasks.workunit.client.1.smithi179.stdout:4/571: creat d1/d3/dc/d26/d8a/fbe x:0 0 0 2024-04-02T04:47:51.718 INFO:tasks.workunit.client.1.smithi179.stdout:4/572: read d1/d3/df/d10/d2f/d60/f6c [1061750,72950] 0 2024-04-02T04:47:51.718 INFO:tasks.workunit.client.1.smithi179.stdout:9/516: mknod d0/d1/d11/d29/d86/ca0 0 2024-04-02T04:47:51.719 INFO:tasks.workunit.client.1.smithi179.stdout:9/517: chown d0/d19/f56 122707 1 2024-04-02T04:47:51.720 INFO:tasks.workunit.client.1.smithi179.stdout:9/518: chown d0/d1/d11/d29/d86 0 1 2024-04-02T04:47:51.722 INFO:tasks.workunit.client.0.smithi070.stdout:4/423: chown d1/d3/c3a 30 1 2024-04-02T04:47:51.724 INFO:tasks.workunit.client.1.smithi179.stdout:4/573: dread d1/d3/dc/d27/f7f [0,4194304] 0 2024-04-02T04:47:51.726 INFO:tasks.workunit.client.1.smithi179.stdout:0/529: mknod d1/da/d28/d5e/ca3 0 2024-04-02T04:47:51.729 INFO:tasks.workunit.client.1.smithi179.stdout:5/452: rename d8/d1b/c1c to d8/d1b/d23/d32/d4e/d50/c8d 0 2024-04-02T04:47:51.732 INFO:tasks.workunit.client.0.smithi070.stdout:0/568: unlink d4/d5/d13/d23/c97 0 2024-04-02T04:47:51.734 INFO:tasks.workunit.client.1.smithi179.stdout:8/464: rmdir d3/d32/d21/d45 39 2024-04-02T04:47:51.737 INFO:tasks.workunit.client.1.smithi179.stdout:9/519: fsync d0/d1/d11/d29/f94 0 2024-04-02T04:47:51.737 INFO:tasks.workunit.client.0.smithi070.stdout:0/569: mknod d4/d5/d13/d17/d72/ca6 0 2024-04-02T04:47:51.739 INFO:tasks.workunit.client.0.smithi070.stdout:8/497: creat d2/f9b x:0 0 0 2024-04-02T04:47:51.740 INFO:tasks.workunit.client.0.smithi070.stdout:8/498: chown d2/d3/f9 596 1 2024-04-02T04:47:51.740 INFO:tasks.workunit.client.1.smithi179.stdout:3/537: mknod d8/d26/d83/d89/d9c/ca4 0 2024-04-02T04:47:51.740 INFO:tasks.workunit.client.0.smithi070.stdout:1/526: getdents d0/d54/d64/d6d 0 2024-04-02T04:47:51.743 INFO:tasks.workunit.client.0.smithi070.stdout:0/570: mknod d4/d5/d13/d17/ca7 0 2024-04-02T04:47:51.746 INFO:tasks.workunit.client.1.smithi179.stdout:0/530: truncate d1/da/f1b 946508 0 2024-04-02T04:47:51.746 INFO:tasks.workunit.client.0.smithi070.stdout:8/499: dwrite d2/d8/d29/f3b [0,4194304] 0 2024-04-02T04:47:51.747 INFO:tasks.workunit.client.0.smithi070.stdout:8/500: fsync d2/d3/d52/d62/f95 0 2024-04-02T04:47:51.749 INFO:tasks.workunit.client.0.smithi070.stdout:1/527: write d0/d4c/f51 [896710,57138] 0 2024-04-02T04:47:51.752 INFO:tasks.workunit.client.0.smithi070.stdout:0/571: unlink d4/d5/d13/d2a/d29/f71 0 2024-04-02T04:47:51.753 INFO:tasks.workunit.client.1.smithi179.stdout:7/581: creat d1/d9/fd0 x:0 0 0 2024-04-02T04:47:51.753 INFO:tasks.workunit.client.0.smithi070.stdout:4/424: truncate d1/d3/d9/d5e/f77 310943 0 2024-04-02T04:47:51.754 INFO:tasks.workunit.client.1.smithi179.stdout:7/582: write d1/dd/d76/fca [599699,78218] 0 2024-04-02T04:47:51.756 INFO:tasks.workunit.client.0.smithi070.stdout:1/528: read d0/d7/f10 [3071495,111705] 0 2024-04-02T04:47:51.760 INFO:tasks.workunit.client.1.smithi179.stdout:4/574: dread d1/d3/dc/d26/d8a/f55 [0,4194304] 0 2024-04-02T04:47:51.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:51 smithi070 ceph-mon[29212]: pgmap v14: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 60 MiB/s rd, 280 MiB/s wr, 478 op/s 2024-04-02T04:47:51.789 INFO:tasks.workunit.client.1.smithi179.stdout:6/508: sync 2024-04-02T04:47:51.792 INFO:tasks.workunit.client.1.smithi179.stdout:6/509: dwrite d3/d2d/f65 [0,4194304] 0 2024-04-02T04:47:51.797 INFO:tasks.workunit.client.1.smithi179.stdout:6/510: read d3/d12/d23/d39/d80/f8f [1544820,31902] 0 2024-04-02T04:47:51.801 INFO:tasks.workunit.client.1.smithi179.stdout:8/465: mknod d3/d32/d21/d51/d70/ca0 0 2024-04-02T04:47:51.804 INFO:tasks.workunit.client.0.smithi070.stdout:4/425: rmdir d1/d3/d9/d5e 39 2024-04-02T04:47:51.804 INFO:tasks.workunit.client.1.smithi179.stdout:9/520: symlink d0/d1/d11/d29/d80/la1 0 2024-04-02T04:47:51.805 INFO:tasks.workunit.client.0.smithi070.stdout:8/501: mknod d2/d3/c9c 0 2024-04-02T04:47:51.806 INFO:tasks.workunit.client.0.smithi070.stdout:1/529: truncate d0/d9/f3e 3338482 0 2024-04-02T04:47:51.807 INFO:tasks.workunit.client.1.smithi179.stdout:2/364: creat dc/d29/d4a/f75 x:0 0 0 2024-04-02T04:47:51.810 INFO:tasks.workunit.client.1.smithi179.stdout:5/453: symlink d8/d7e/l8e 0 2024-04-02T04:47:51.810 INFO:tasks.workunit.client.1.smithi179.stdout:5/454: chown d8/d1b/f24 4947 1 2024-04-02T04:47:51.812 INFO:tasks.workunit.client.1.smithi179.stdout:1/512: creat d3/d5/d2f/fb0 x:0 0 0 2024-04-02T04:47:51.821 INFO:tasks.workunit.client.0.smithi070.stdout:0/572: write d4/d5/d13/d2a/d3a/f8e [590289,78921] 0 2024-04-02T04:47:51.823 INFO:tasks.workunit.client.1.smithi179.stdout:8/466: rename d3/d32/d21/d50/d56/d94/f34 to d3/d6c/d96/fa1 0 2024-04-02T04:47:51.826 INFO:tasks.workunit.client.1.smithi179.stdout:9/521: dread d0/d1/d11/d29/d40/f9a [0,4194304] 0 2024-04-02T04:47:51.828 INFO:tasks.workunit.client.1.smithi179.stdout:9/522: write d0/d1/d11/d29/d40/f5d [4228996,60140] 0 2024-04-02T04:47:51.833 INFO:tasks.workunit.client.1.smithi179.stdout:0/531: dwrite d1/da/df/f12 [4194304,4194304] 0 2024-04-02T04:47:51.835 INFO:tasks.workunit.client.1.smithi179.stdout:9/523: dwrite d0/d1/d11/d29/d40/f49 [0,4194304] 0 2024-04-02T04:47:51.889 INFO:tasks.workunit.client.1.smithi179.stdout:5/455: creat d8/d1b/d23/d36/f8f x:0 0 0 2024-04-02T04:47:51.890 INFO:tasks.workunit.client.1.smithi179.stdout:1/513: creat d3/d5/d2f/d4f/d42/fb1 x:0 0 0 2024-04-02T04:47:51.891 INFO:tasks.workunit.client.1.smithi179.stdout:1/514: dread - d3/f95 zero size 2024-04-02T04:47:51.895 INFO:tasks.workunit.client.1.smithi179.stdout:4/575: dwrite d1/d3/f5b [0,4194304] 0 2024-04-02T04:47:51.895 INFO:tasks.workunit.client.1.smithi179.stdout:5/456: dwrite d8/f18 [0,4194304] 0 2024-04-02T04:47:51.896 INFO:tasks.workunit.client.1.smithi179.stdout:4/576: chown d1/d15/f56 207 1 2024-04-02T04:47:51.897 INFO:tasks.workunit.client.1.smithi179.stdout:4/577: stat d1/d3/df/d10/d2f/d60 0 2024-04-02T04:47:51.900 INFO:tasks.workunit.client.1.smithi179.stdout:5/457: dread d8/d1b/d23/f47 [4194304,4194304] 0 2024-04-02T04:47:51.901 INFO:tasks.workunit.client.1.smithi179.stdout:5/458: truncate d8/d1b/d23/d32/d4e/f54 903322 0 2024-04-02T04:47:51.953 INFO:tasks.workunit.client.1.smithi179.stdout:7/583: mknod d1/dd/cd1 0 2024-04-02T04:47:51.960 INFO:tasks.workunit.client.1.smithi179.stdout:3/538: truncate d8/d26/f34 2500422 0 2024-04-02T04:47:51.965 INFO:tasks.workunit.client.0.smithi070.stdout:4/426: dread d1/fb [4194304,4194304] 0 2024-04-02T04:47:51.968 INFO:tasks.workunit.client.1.smithi179.stdout:2/365: mknod dc/d29/d4a/d1e/d6c/c76 0 2024-04-02T04:47:51.969 INFO:tasks.workunit.client.0.smithi070.stdout:0/573: mkdir d4/d5/d13/d2a/d3a/d80/da8 0 2024-04-02T04:47:51.971 INFO:tasks.workunit.client.1.smithi179.stdout:9/524: write d0/f90 [622109,48263] 0 2024-04-02T04:47:51.972 INFO:tasks.workunit.client.1.smithi179.stdout:9/525: chown d0/d1/f7d 509274109 1 2024-04-02T04:47:51.975 INFO:tasks.workunit.client.0.smithi070.stdout:1/530: fsync d0/d7/d1a/f9a 0 2024-04-02T04:47:51.975 INFO:tasks.workunit.client.1.smithi179.stdout:6/511: truncate d3/d12/d23/d26/f37 2509902 0 2024-04-02T04:47:51.976 INFO:tasks.workunit.client.0.smithi070.stdout:1/531: fsync d0/d7/f81 0 2024-04-02T04:47:51.982 INFO:tasks.workunit.client.0.smithi070.stdout:1/532: dwrite d0/d9/d1d/d22/d2c/d4f/f50 [0,4194304] 0 2024-04-02T04:47:51.982 INFO:tasks.workunit.client.1.smithi179.stdout:6/512: dwrite d3/d2d/d38/fa9 [0,4194304] 0 2024-04-02T04:47:51.987 INFO:tasks.workunit.client.1.smithi179.stdout:7/584: fdatasync d1/dd/f1c 0 2024-04-02T04:47:51.989 INFO:tasks.workunit.client.1.smithi179.stdout:3/539: mknod d8/d57/ca5 0 2024-04-02T04:47:51.991 INFO:tasks.workunit.client.1.smithi179.stdout:8/467: link d3/d32/d21/d50/d56/f6a d3/d1b/fa2 0 2024-04-02T04:47:51.992 INFO:tasks.workunit.client.1.smithi179.stdout:8/468: chown d3/d1b/f23 1853348885 1 2024-04-02T04:47:51.993 INFO:tasks.workunit.client.1.smithi179.stdout:2/366: fdatasync dc/d29/d4a/d1e/d43/f6e 0 2024-04-02T04:47:51.994 INFO:tasks.workunit.client.0.smithi070.stdout:1/533: mknod d0/d7/d1a/d66/cac 0 2024-04-02T04:47:51.997 INFO:tasks.workunit.client.0.smithi070.stdout:4/427: rename d1/c8 to d1/d3/d9/d5e/c8f 0 2024-04-02T04:47:51.997 INFO:tasks.workunit.client.1.smithi179.stdout:0/532: truncate d1/f2 2201533 0 2024-04-02T04:47:51.999 INFO:tasks.workunit.client.1.smithi179.stdout:0/533: chown d1/da/df/l55 523621007 1 2024-04-02T04:47:51.999 INFO:tasks.workunit.client.1.smithi179.stdout:2/367: dwrite dc/d29/d31/f41 [0,4194304] 0 2024-04-02T04:47:52.001 INFO:tasks.workunit.client.1.smithi179.stdout:9/526: unlink d0/d19/d1c/f41 0 2024-04-02T04:47:52.003 INFO:tasks.workunit.client.0.smithi070.stdout:8/502: truncate d2/d8/d29/f3b 3635011 0 2024-04-02T04:47:52.006 INFO:tasks.workunit.client.1.smithi179.stdout:5/459: rename d8/f1d to d8/d1b/d23/d32/d6e/f90 0 2024-04-02T04:47:52.007 INFO:tasks.workunit.client.0.smithi070.stdout:8/503: symlink d2/d3/d35/d3a/d4f/d59/l9d 0 2024-04-02T04:47:52.013 INFO:tasks.workunit.client.1.smithi179.stdout:6/513: dread d3/d12/d23/d26/f27 [0,4194304] 0 2024-04-02T04:47:52.017 INFO:tasks.workunit.client.1.smithi179.stdout:1/515: dwrite d3/d5/d2f/d4f/d35/d49/f7e [0,4194304] 0 2024-04-02T04:47:52.017 INFO:tasks.workunit.client.1.smithi179.stdout:5/460: write d8/d1b/d23/d36/d53/f72 [1372579,84915] 0 2024-04-02T04:47:52.023 INFO:tasks.workunit.client.0.smithi070.stdout:4/428: fsync d1/d3/d9/f64 0 2024-04-02T04:47:52.024 INFO:tasks.workunit.client.0.smithi070.stdout:1/534: dwrite d0/d7/d28/f7a [0,4194304] 0 2024-04-02T04:47:52.024 INFO:tasks.workunit.client.1.smithi179.stdout:4/578: dwrite d1/d3/f7 [0,4194304] 0 2024-04-02T04:47:52.025 INFO:tasks.workunit.client.0.smithi070.stdout:4/429: dread - d1/d3/d9/f81 zero size 2024-04-02T04:47:52.025 INFO:tasks.workunit.client.0.smithi070.stdout:1/535: chown d0/d7/d1a/f9a 652341278 1 2024-04-02T04:47:52.028 INFO:tasks.workunit.client.1.smithi179.stdout:8/469: write d3/d32/f5c [2201109,6542] 0 2024-04-02T04:47:52.028 INFO:tasks.workunit.client.1.smithi179.stdout:2/368: dread dc/d29/d4a/f27 [0,4194304] 0 2024-04-02T04:47:52.032 INFO:tasks.workunit.client.1.smithi179.stdout:3/540: rename d8/d26/d36/f47 to d8/d26/d52/d85/fa6 0 2024-04-02T04:47:52.034 INFO:tasks.workunit.client.0.smithi070.stdout:8/504: truncate d2/d3/f9 2578274 0 2024-04-02T04:47:52.035 INFO:tasks.workunit.client.0.smithi070.stdout:1/536: unlink d0/d7/d1a/d66/cac 0 2024-04-02T04:47:52.037 INFO:tasks.workunit.client.1.smithi179.stdout:9/527: dwrite d0/d1/d11/d29/f37 [4194304,4194304] 0 2024-04-02T04:47:52.040 INFO:tasks.workunit.client.1.smithi179.stdout:8/470: dwrite d3/d32/d21/d50/d56/d94/f60 [4194304,4194304] 0 2024-04-02T04:47:52.041 INFO:tasks.workunit.client.0.smithi070.stdout:1/537: dwrite d0/d9/d1d/f42 [0,4194304] 0 2024-04-02T04:47:52.042 INFO:tasks.workunit.client.0.smithi070.stdout:1/538: stat d0/d7/d28/d5c 0 2024-04-02T04:47:52.044 INFO:tasks.workunit.client.1.smithi179.stdout:8/471: write d3/d32/d21/d49/f9f [348126,91400] 0 2024-04-02T04:47:52.045 INFO:tasks.workunit.client.0.smithi070.stdout:4/430: creat d1/d3/d85/d8e/f90 x:0 0 0 2024-04-02T04:47:52.047 INFO:tasks.workunit.client.0.smithi070.stdout:4/431: write d1/d3/d9/f81 [795410,47108] 0 2024-04-02T04:47:52.048 INFO:tasks.workunit.client.0.smithi070.stdout:8/505: fsync d2/f1f 0 2024-04-02T04:47:52.050 INFO:tasks.workunit.client.1.smithi179.stdout:8/472: dwrite d3/d32/d21/d51/d70/f8d [0,4194304] 0 2024-04-02T04:47:52.054 INFO:tasks.workunit.client.0.smithi070.stdout:4/432: chown d1/c14 34 1 2024-04-02T04:47:52.055 INFO:tasks.workunit.client.1.smithi179.stdout:8/473: dread d3/d32/d21/d50/f5f [0,4194304] 0 2024-04-02T04:47:52.055 INFO:tasks.workunit.client.1.smithi179.stdout:8/474: fsync d3/d31/f9e 0 2024-04-02T04:47:52.063 INFO:tasks.workunit.client.0.smithi070.stdout:4/433: dwrite d1/d3/f8c [0,4194304] 0 2024-04-02T04:47:52.095 INFO:tasks.workunit.client.0.smithi070.stdout:2/509: sync 2024-04-02T04:47:52.095 INFO:tasks.workunit.client.0.smithi070.stdout:5/576: sync 2024-04-02T04:47:52.097 INFO:tasks.workunit.client.0.smithi070.stdout:2/510: truncate d2/d10/d21/d71/fa5 940761 0 2024-04-02T04:47:52.122 INFO:tasks.workunit.client.0.smithi070.stdout:4/434: write d1/f46 [3053984,92359] 0 2024-04-02T04:47:52.122 INFO:tasks.workunit.client.0.smithi070.stdout:4/435: chown d1/d3/l8a 83 1 2024-04-02T04:47:52.123 INFO:tasks.workunit.client.0.smithi070.stdout:4/436: chown d1/d3/d9/l49 713 1 2024-04-02T04:47:52.124 INFO:tasks.workunit.client.0.smithi070.stdout:4/437: chown d1/d3/c3a 18221888 1 2024-04-02T04:47:52.131 INFO:tasks.workunit.client.0.smithi070.stdout:1/539: getdents d0/d7/d28 0 2024-04-02T04:47:52.132 INFO:tasks.workunit.client.0.smithi070.stdout:5/577: dwrite d5/d17/d27/f4b [0,4194304] 0 2024-04-02T04:47:52.135 INFO:tasks.workunit.client.0.smithi070.stdout:8/506: getdents d2/d8 0 2024-04-02T04:47:52.138 INFO:tasks.workunit.client.1.smithi179.stdout:0/534: symlink d1/da/d48/d65/la4 0 2024-04-02T04:47:52.139 INFO:tasks.workunit.client.1.smithi179.stdout:0/535: write d1/db/f8e [163253,91269] 0 2024-04-02T04:47:52.150 INFO:tasks.workunit.client.1.smithi179.stdout:7/585: creat d1/fd2 x:0 0 0 2024-04-02T04:47:52.151 INFO:tasks.workunit.client.1.smithi179.stdout:7/586: write d1/fc [418914,47146] 0 2024-04-02T04:47:52.152 INFO:tasks.workunit.client.0.smithi070.stdout:1/540: creat d0/d54/d64/d6d/fad x:0 0 0 2024-04-02T04:47:52.154 INFO:tasks.workunit.client.0.smithi070.stdout:1/541: write d0/d9/d1d/d22/d2c/d4f/d77/d8c/faa [370870,26189] 0 2024-04-02T04:47:52.155 INFO:tasks.workunit.client.0.smithi070.stdout:5/578: chown d5/d17/d2b/c2d 2330068 1 2024-04-02T04:47:52.158 INFO:tasks.workunit.client.1.smithi179.stdout:2/369: creat dc/d29/d31/f77 x:0 0 0 2024-04-02T04:47:52.159 INFO:tasks.workunit.client.0.smithi070.stdout:8/507: mkdir d2/d8/d29/d22/d5a/d9e 0 2024-04-02T04:47:52.161 INFO:tasks.workunit.client.0.smithi070.stdout:4/438: mkdir d1/d3/d78/d91 0 2024-04-02T04:47:52.164 INFO:tasks.workunit.client.1.smithi179.stdout:9/528: mknod d0/d19/d27/d7a/d82/ca2 0 2024-04-02T04:47:52.170 INFO:tasks.workunit.client.0.smithi070.stdout:4/439: creat d1/d3/d85/f92 x:0 0 0 2024-04-02T04:47:52.174 INFO:tasks.workunit.client.0.smithi070.stdout:4/440: chown d1/d3/c59 1710487 1 2024-04-02T04:47:52.176 INFO:tasks.workunit.client.0.smithi070.stdout:5/579: rename d5/d17/d50/f58 to d5/d17/d27/fb7 0 2024-04-02T04:47:52.178 INFO:tasks.workunit.client.1.smithi179.stdout:9/529: dread d0/d1/d10/d6d/f2d [0,4194304] 0 2024-04-02T04:47:52.178 INFO:tasks.workunit.client.0.smithi070.stdout:3/526: sync 2024-04-02T04:47:52.178 INFO:tasks.workunit.client.0.smithi070.stdout:9/431: sync 2024-04-02T04:47:52.179 INFO:tasks.workunit.client.0.smithi070.stdout:7/475: sync 2024-04-02T04:47:52.179 INFO:tasks.workunit.client.0.smithi070.stdout:6/408: sync 2024-04-02T04:47:52.179 INFO:tasks.workunit.client.1.smithi179.stdout:9/530: readlink d0/d1/d11/d29/d86/l99 0 2024-04-02T04:47:52.179 INFO:tasks.workunit.client.0.smithi070.stdout:4/441: dread d1/d3/f54 [0,4194304] 0 2024-04-02T04:47:52.181 INFO:tasks.workunit.client.0.smithi070.stdout:6/409: truncate d0/d20/d5f/f62 1140388 0 2024-04-02T04:47:52.182 INFO:tasks.workunit.client.1.smithi179.stdout:1/516: symlink d3/d5/d2f/d5a/lb2 0 2024-04-02T04:47:52.188 INFO:tasks.workunit.client.0.smithi070.stdout:9/432: dwrite d4/d10/d12/d55/d66/f7c [0,4194304] 0 2024-04-02T04:47:52.189 INFO:tasks.workunit.client.0.smithi070.stdout:7/476: dwrite d7/dd/df/d79/d46/d65/f8c [0,4194304] 0 2024-04-02T04:47:52.190 INFO:tasks.workunit.client.0.smithi070.stdout:8/508: unlink d2/d3/d52/c41 0 2024-04-02T04:47:52.192 INFO:tasks.workunit.client.1.smithi179.stdout:9/531: dread d0/d1/d11/f32 [0,4194304] 0 2024-04-02T04:47:52.196 INFO:tasks.workunit.client.1.smithi179.stdout:3/541: rename d8/d26/d36/l45 to d8/d57/d8f/la7 0 2024-04-02T04:47:52.198 INFO:tasks.workunit.client.0.smithi070.stdout:6/410: rename d0/d55/d5c/c69 to d0/d20/c83 0 2024-04-02T04:47:52.204 INFO:tasks.workunit.client.1.smithi179.stdout:5/461: link c7 d8/d1b/d23/d36/d53/d89/c91 0 2024-04-02T04:47:52.209 INFO:tasks.workunit.client.1.smithi179.stdout:2/370: symlink dc/d29/d4a/l78 0 2024-04-02T04:47:52.210 INFO:tasks.workunit.client.1.smithi179.stdout:2/371: chown dc/d29/d31/d57 418870 1 2024-04-02T04:47:52.211 INFO:tasks.workunit.client.1.smithi179.stdout:9/532: symlink d0/d1/d11/d29/d80/la3 0 2024-04-02T04:47:52.219 INFO:tasks.workunit.client.1.smithi179.stdout:0/536: link d1/f23 d1/da/d48/d65/d7d/d80/fa5 0 2024-04-02T04:47:52.221 INFO:tasks.workunit.client.0.smithi070.stdout:9/433: creat d4/d10/d11/d7b/d19/d53/f8b x:0 0 0 2024-04-02T04:47:52.226 INFO:tasks.workunit.client.1.smithi179.stdout:1/517: creat d3/da8/fb3 x:0 0 0 2024-04-02T04:47:52.230 INFO:tasks.workunit.client.0.smithi070.stdout:9/434: mkdir d4/d10/d11/d7b/d19/d2f/d8c 0 2024-04-02T04:47:52.231 INFO:tasks.workunit.client.1.smithi179.stdout:5/462: dread d8/d1b/f26 [0,4194304] 0 2024-04-02T04:47:52.232 INFO:tasks.workunit.client.1.smithi179.stdout:7/587: rename d1/d9/d7e/f9d to d1/d9/d55/dbf/d9f/fd3 0 2024-04-02T04:47:52.233 INFO:tasks.workunit.client.1.smithi179.stdout:7/588: chown d1/dd/cd1 52 1 2024-04-02T04:47:52.234 INFO:tasks.workunit.client.0.smithi070.stdout:9/435: mknod d4/d10/d12/d55/d66/c8d 0 2024-04-02T04:47:52.235 INFO:tasks.workunit.client.1.smithi179.stdout:0/537: mkdir d1/da/d28/d34/d39/da6 0 2024-04-02T04:47:52.235 INFO:tasks.workunit.client.0.smithi070.stdout:0/574: sync 2024-04-02T04:47:52.236 INFO:tasks.workunit.client.0.smithi070.stdout:7/477: rename d7/d9/c3a to d7/c9a 0 2024-04-02T04:47:52.237 INFO:tasks.workunit.client.1.smithi179.stdout:1/518: dread d3/d5/d2f/d4f/f37 [0,4194304] 0 2024-04-02T04:47:52.237 INFO:tasks.workunit.client.1.smithi179.stdout:2/372: mkdir dc/d79 0 2024-04-02T04:47:52.239 INFO:tasks.workunit.client.0.smithi070.stdout:0/575: creat d4/d5/d13/d2a/d3a/d80/fa9 x:0 0 0 2024-04-02T04:47:52.241 INFO:tasks.workunit.client.1.smithi179.stdout:5/463: rename d8/d1b/d23/d32/d46/l85 to d8/d1b/d23/d32/d4e/d50/l92 0 2024-04-02T04:47:52.242 INFO:tasks.workunit.client.1.smithi179.stdout:5/464: chown d8/d1b/f26 52909 1 2024-04-02T04:47:52.242 INFO:tasks.workunit.client.0.smithi070.stdout:6/411: rename d0/d55/f6f to d0/d20/d5f/f84 0 2024-04-02T04:47:52.245 INFO:tasks.workunit.client.0.smithi070.stdout:9/436: link d4/d10/d11/d7b/f1d d4/d10/d12/d27/d83/f8e 0 2024-04-02T04:47:52.247 INFO:tasks.workunit.client.1.smithi179.stdout:1/519: creat d3/d5/d96/fb4 x:0 0 0 2024-04-02T04:47:52.247 INFO:tasks.workunit.client.0.smithi070.stdout:0/576: chown d4/d5/d13/d2a/d29/c79 12443 1 2024-04-02T04:47:52.248 INFO:tasks.workunit.client.0.smithi070.stdout:0/577: truncate f0 4511342 0 2024-04-02T04:47:52.249 INFO:tasks.workunit.client.1.smithi179.stdout:3/542: getdents d8 0 2024-04-02T04:47:52.249 INFO:tasks.workunit.client.1.smithi179.stdout:6/514: write f2 [2077300,26311] 0 2024-04-02T04:47:52.250 INFO:tasks.workunit.client.1.smithi179.stdout:3/543: chown d8/d26/f69 613228 1 2024-04-02T04:47:52.250 INFO:tasks.workunit.client.1.smithi179.stdout:5/465: creat d8/d1b/d23/d32/d4e/f93 x:0 0 0 2024-04-02T04:47:52.252 INFO:tasks.workunit.client.0.smithi070.stdout:7/478: rename d7/dd/df/d79/d46/d65/d7b/f7d to d7/dd/df/d7a/d6c/f9b 0 2024-04-02T04:47:52.255 INFO:tasks.workunit.client.0.smithi070.stdout:9/437: rmdir d4/d10/d11/d7b/d19/d2f 39 2024-04-02T04:47:52.255 INFO:tasks.workunit.client.1.smithi179.stdout:6/515: dwrite d3/d12/d6a/f6e [0,4194304] 0 2024-04-02T04:47:52.256 INFO:tasks.workunit.client.1.smithi179.stdout:6/516: chown d3/d12/d23/d2f/f56 247219601 1 2024-04-02T04:47:52.258 INFO:tasks.workunit.client.0.smithi070.stdout:0/578: creat d4/d5/d13/d2a/d3a/d80/faa x:0 0 0 2024-04-02T04:47:52.259 INFO:tasks.workunit.client.1.smithi179.stdout:3/544: creat d8/d57/fa8 x:0 0 0 2024-04-02T04:47:52.262 INFO:tasks.workunit.client.1.smithi179.stdout:5/466: unlink d8/d1b/d23/d36/d53/f65 0 2024-04-02T04:47:52.266 INFO:tasks.workunit.client.0.smithi070.stdout:6/412: mkdir d0/d43/d80/d85 0 2024-04-02T04:47:52.269 INFO:tasks.workunit.client.0.smithi070.stdout:6/413: write d0/d3/d19/f1f [443638,107843] 0 2024-04-02T04:47:52.270 INFO:tasks.workunit.client.0.smithi070.stdout:2/511: write d2/d10/d21/d3d/f63 [258509,85839] 0 2024-04-02T04:47:52.270 INFO:tasks.workunit.client.1.smithi179.stdout:6/517: mknod d3/d12/d44/cb2 0 2024-04-02T04:47:52.272 INFO:tasks.workunit.client.1.smithi179.stdout:3/545: mknod d8/d26/d83/d89/d9c/ca9 0 2024-04-02T04:47:52.276 INFO:tasks.workunit.client.0.smithi070.stdout:7/479: unlink d7/l42 0 2024-04-02T04:47:52.277 INFO:tasks.workunit.client.1.smithi179.stdout:1/520: dread d3/d5/de/f32 [0,4194304] 0 2024-04-02T04:47:52.278 INFO:tasks.workunit.client.0.smithi070.stdout:2/512: dwrite d2/d10/d5c/d7a/f92 [0,4194304] 0 2024-04-02T04:47:52.278 INFO:tasks.workunit.client.0.smithi070.stdout:0/579: read d4/d5/da/d2d/f7d [972187,101602] 0 2024-04-02T04:47:52.279 INFO:tasks.workunit.client.1.smithi179.stdout:3/546: dwrite d8/d26/d52/d85/f8b [0,4194304] 0 2024-04-02T04:47:52.279 INFO:tasks.workunit.client.1.smithi179.stdout:1/521: read - d3/d5/d2f/d4f/d42/d53/fad zero size 2024-04-02T04:47:52.280 INFO:tasks.workunit.client.0.smithi070.stdout:0/580: fdatasync d4/d5/d13/d43/f6f 0 2024-04-02T04:47:52.280 INFO:tasks.workunit.client.1.smithi179.stdout:5/467: dread d8/f1a [0,4194304] 0 2024-04-02T04:47:52.280 INFO:tasks.workunit.client.1.smithi179.stdout:1/522: chown d3/d5/de/d33/l43 66096 1 2024-04-02T04:47:52.281 INFO:tasks.workunit.client.0.smithi070.stdout:9/438: creat d4/d10/d12/d27/d83/f8f x:0 0 0 2024-04-02T04:47:52.287 INFO:tasks.workunit.client.0.smithi070.stdout:6/414: dwrite d0/d43/f4a [4194304,4194304] 0 2024-04-02T04:47:52.288 INFO:tasks.workunit.client.1.smithi179.stdout:4/579: sync 2024-04-02T04:47:52.288 INFO:tasks.workunit.client.0.smithi070.stdout:6/415: fdatasync d0/d55/d5c/f60 0 2024-04-02T04:47:52.289 INFO:tasks.workunit.client.0.smithi070.stdout:7/480: creat d7/dd/df/d34/d68/f9c x:0 0 0 2024-04-02T04:47:52.291 INFO:tasks.workunit.client.0.smithi070.stdout:9/439: fdatasync d4/d10/d12/d55/d66/f7c 0 2024-04-02T04:47:52.291 INFO:tasks.workunit.client.0.smithi070.stdout:7/481: write d7/dd/df/d79/d46/d65/f67 [730588,19983] 0 2024-04-02T04:47:52.292 INFO:tasks.workunit.client.1.smithi179.stdout:3/547: mknod d8/d6d/d94/caa 0 2024-04-02T04:47:52.298 INFO:tasks.workunit.client.0.smithi070.stdout:0/581: truncate d4/d5/f3f 1054857 0 2024-04-02T04:47:52.299 INFO:tasks.workunit.client.0.smithi070.stdout:0/582: readlink d4/d5/d21/l35 0 2024-04-02T04:47:52.303 INFO:tasks.workunit.client.1.smithi179.stdout:4/580: unlink d1/d3/df/f57 0 2024-04-02T04:47:52.305 INFO:tasks.workunit.client.1.smithi179.stdout:3/548: creat d8/d26/d83/d89/fab x:0 0 0 2024-04-02T04:47:52.307 INFO:tasks.workunit.client.1.smithi179.stdout:3/549: write d8/d26/d36/f90 [992789,84397] 0 2024-04-02T04:47:52.308 INFO:tasks.workunit.client.1.smithi179.stdout:3/550: chown d8/f11 190 1 2024-04-02T04:47:52.309 INFO:tasks.workunit.client.1.smithi179.stdout:1/523: creat d3/d85/da2/fb5 x:0 0 0 2024-04-02T04:47:52.309 INFO:tasks.workunit.client.1.smithi179.stdout:3/551: dread - d8/d26/d83/d89/fab zero size 2024-04-02T04:47:52.312 INFO:tasks.workunit.client.0.smithi070.stdout:7/482: dread d7/dd/df/d7a/f5a [0,4194304] 0 2024-04-02T04:47:52.316 INFO:tasks.workunit.client.0.smithi070.stdout:2/513: unlink d2/d12/c62 0 2024-04-02T04:47:52.317 INFO:tasks.workunit.client.1.smithi179.stdout:6/518: getdents d3/d12/d23/d26 0 2024-04-02T04:47:52.319 INFO:tasks.workunit.client.0.smithi070.stdout:1/542: symlink d0/lae 0 2024-04-02T04:47:52.319 INFO:tasks.workunit.client.0.smithi070.stdout:9/440: dwrite d4/d10/d11/d7b/d19/f54 [0,4194304] 0 2024-04-02T04:47:52.321 INFO:tasks.workunit.client.1.smithi179.stdout:1/524: mkdir d3/d5/d2f/d4f/d35/d49/db6 0 2024-04-02T04:47:52.324 INFO:tasks.workunit.client.1.smithi179.stdout:3/552: truncate d8/d26/d36/f5e 5206895 0 2024-04-02T04:47:52.325 INFO:tasks.workunit.client.1.smithi179.stdout:3/553: chown d8/d26/f3f 1129801 1 2024-04-02T04:47:52.330 INFO:tasks.workunit.client.0.smithi070.stdout:2/514: creat d2/d12/fa7 x:0 0 0 2024-04-02T04:47:52.331 INFO:tasks.workunit.client.0.smithi070.stdout:2/515: fdatasync d2/d10/d21/d3d/f47 0 2024-04-02T04:47:52.334 INFO:tasks.workunit.client.1.smithi179.stdout:6/519: fsync d3/f72 0 2024-04-02T04:47:52.335 INFO:tasks.workunit.client.1.smithi179.stdout:6/520: dread - d3/d12/d23/d26/f76 zero size 2024-04-02T04:47:52.338 INFO:tasks.workunit.client.1.smithi179.stdout:1/525: mkdir d3/d5/d2f/db7 0 2024-04-02T04:47:52.338 INFO:tasks.workunit.client.1.smithi179.stdout:6/521: read d3/d12/f21 [2558322,85241] 0 2024-04-02T04:47:52.338 INFO:tasks.workunit.client.0.smithi070.stdout:9/441: readlink d4/d13/l46 0 2024-04-02T04:47:52.338 INFO:tasks.workunit.client.1.smithi179.stdout:4/581: getdents d1/d15/d7c/d98 0 2024-04-02T04:47:52.340 INFO:tasks.workunit.client.0.smithi070.stdout:0/583: mkdir d4/d5/da/dab 0 2024-04-02T04:47:52.342 INFO:tasks.workunit.client.1.smithi179.stdout:5/468: rename d8/d1b/d23/d32/d4b/l4c to d8/d1b/d23/d32/l94 0 2024-04-02T04:47:52.342 INFO:tasks.workunit.client.0.smithi070.stdout:5/580: dwrite d5/d9/d23/f48 [4194304,4194304] 0 2024-04-02T04:47:52.343 INFO:tasks.workunit.client.1.smithi179.stdout:5/469: readlink d8/d1b/d23/d36/d53/l6a 0 2024-04-02T04:47:52.344 INFO:tasks.workunit.client.0.smithi070.stdout:0/584: read d4/f9 [1337956,7896] 0 2024-04-02T04:47:52.347 INFO:tasks.workunit.client.1.smithi179.stdout:8/475: mknod d3/d32/d21/d51/ca3 0 2024-04-02T04:47:52.348 INFO:tasks.workunit.client.0.smithi070.stdout:0/585: truncate d4/d5/d58/d60/f81 873552 0 2024-04-02T04:47:52.349 INFO:tasks.workunit.client.1.smithi179.stdout:6/522: chown d3/l19 0 1 2024-04-02T04:47:52.350 INFO:tasks.workunit.client.0.smithi070.stdout:3/527: dwrite d10/d32/f36 [4194304,4194304] 0 2024-04-02T04:47:52.353 INFO:tasks.workunit.client.0.smithi070.stdout:7/483: dread d7/d9/f78 [0,4194304] 0 2024-04-02T04:47:52.355 INFO:tasks.workunit.client.0.smithi070.stdout:3/528: dread d10/d17/d4c/fa3 [0,4194304] 0 2024-04-02T04:47:52.356 INFO:tasks.workunit.client.1.smithi179.stdout:1/526: creat d3/d5/d2f/d4f/d42/d55/fb8 x:0 0 0 2024-04-02T04:47:52.359 INFO:tasks.workunit.client.0.smithi070.stdout:1/543: creat d0/d4c/da2/faf x:0 0 0 2024-04-02T04:47:52.363 INFO:tasks.workunit.client.1.smithi179.stdout:3/554: rename d8/d57/d63/f6c to d8/fac 0 2024-04-02T04:47:52.363 INFO:tasks.workunit.client.0.smithi070.stdout:0/586: dread d4/d5/d13/f19 [4194304,4194304] 0 2024-04-02T04:47:52.364 INFO:tasks.workunit.client.0.smithi070.stdout:9/442: symlink d4/d13/l90 0 2024-04-02T04:47:52.370 INFO:tasks.workunit.client.0.smithi070.stdout:0/587: dwrite d4/d5/d13/d43/f6f [0,4194304] 0 2024-04-02T04:47:52.372 INFO:tasks.workunit.client.1.smithi179.stdout:6/523: creat d3/d12/d44/fb3 x:0 0 0 2024-04-02T04:47:52.375 INFO:tasks.workunit.client.0.smithi070.stdout:5/581: symlink d5/d17/d6c/lb8 0 2024-04-02T04:47:52.375 INFO:tasks.workunit.client.1.smithi179.stdout:4/582: mknod d1/d15/d7c/db8/cbf 0 2024-04-02T04:47:52.380 INFO:tasks.workunit.client.1.smithi179.stdout:1/527: mknod d3/d5/d2f/d65/cb9 0 2024-04-02T04:47:52.381 INFO:tasks.workunit.client.0.smithi070.stdout:0/588: dwrite d4/d5/d13/f8a [0,4194304] 0 2024-04-02T04:47:52.387 INFO:tasks.workunit.client.0.smithi070.stdout:8/509: write d2/f48 [1949659,30598] 0 2024-04-02T04:47:52.393 INFO:tasks.workunit.client.0.smithi070.stdout:3/529: rmdir d10 39 2024-04-02T04:47:52.397 INFO:tasks.workunit.client.1.smithi179.stdout:8/476: rename d3/d32/d21/d22/d6b/d91/f40 to d3/d1b/fa4 0 2024-04-02T04:47:52.398 INFO:tasks.workunit.client.1.smithi179.stdout:8/477: truncate d3/d32/d21/d50/d56/f77 460113 0 2024-04-02T04:47:52.406 INFO:tasks.workunit.client.1.smithi179.stdout:4/583: rmdir d1/d3/df/d10/d2f/d36 39 2024-04-02T04:47:52.407 INFO:tasks.workunit.client.0.smithi070.stdout:6/416: mknod d0/d3/d19/c86 0 2024-04-02T04:47:52.408 INFO:tasks.workunit.client.0.smithi070.stdout:6/417: stat d0/d43/f4a 0 2024-04-02T04:47:52.409 INFO:tasks.workunit.client.0.smithi070.stdout:9/443: dread d4/d10/d11/d7b/f35 [0,4194304] 0 2024-04-02T04:47:52.412 INFO:tasks.workunit.client.0.smithi070.stdout:6/418: read d0/d3/d19/d31/d46/d6b/f6d [82324,79089] 0 2024-04-02T04:47:52.413 INFO:tasks.workunit.client.0.smithi070.stdout:6/419: chown d0/d3/d19/d31/d61/f66 0 1 2024-04-02T04:47:52.415 INFO:tasks.workunit.client.1.smithi179.stdout:8/478: dread d3/f18 [0,4194304] 0 2024-04-02T04:47:52.418 INFO:tasks.workunit.client.1.smithi179.stdout:8/479: dread d3/d32/d21/d22/d6b/d91/f44 [0,4194304] 0 2024-04-02T04:47:52.421 INFO:tasks.workunit.client.1.smithi179.stdout:1/528: truncate d3/d5/de/f3c 1431772 0 2024-04-02T04:47:52.424 INFO:tasks.workunit.client.0.smithi070.stdout:5/582: getdents d5/d9/d10/da9 0 2024-04-02T04:47:52.425 INFO:tasks.workunit.client.1.smithi179.stdout:0/538: dwrite d1/da/d28/d34/f7e [0,4194304] 0 2024-04-02T04:47:52.425 INFO:tasks.workunit.client.1.smithi179.stdout:4/584: mkdir d1/d3/dc0 0 2024-04-02T04:47:52.426 INFO:tasks.workunit.client.1.smithi179.stdout:1/529: dread d3/d5/f6d [0,4194304] 0 2024-04-02T04:47:52.427 INFO:tasks.workunit.client.1.smithi179.stdout:0/539: write d1/f14 [6652802,97187] 0 2024-04-02T04:47:52.429 INFO:tasks.workunit.client.1.smithi179.stdout:0/540: chown d1/db/c25 3369755 1 2024-04-02T04:47:52.430 INFO:tasks.workunit.client.1.smithi179.stdout:0/541: readlink d1/db/l31 0 2024-04-02T04:47:52.431 INFO:tasks.workunit.client.0.smithi070.stdout:6/420: mknod d0/d20/c87 0 2024-04-02T04:47:52.431 INFO:tasks.workunit.client.1.smithi179.stdout:1/530: truncate d3/d5/d2f/d4f/d42/d53/d77/f97 618786 0 2024-04-02T04:47:52.434 INFO:tasks.workunit.client.0.smithi070.stdout:4/442: creat d1/f93 x:0 0 0 2024-04-02T04:47:52.436 INFO:tasks.workunit.client.0.smithi070.stdout:1/544: link d0/d7/d28/c3a d0/d9/d1d/d22/d2c/d4f/d77/d8c/cb0 0 2024-04-02T04:47:52.445 INFO:tasks.workunit.client.1.smithi179.stdout:1/531: rename d3/f2d to d3/d5/d2f/d4f/d35/d49/fba 0 2024-04-02T04:47:52.447 INFO:tasks.workunit.client.0.smithi070.stdout:6/421: rename d0/d43/f4a to d0/d3/d19/d31/d61/f88 0 2024-04-02T04:47:52.449 INFO:tasks.workunit.client.0.smithi070.stdout:6/422: truncate d0/d3/d19/d31/f74 789097 0 2024-04-02T04:47:52.450 INFO:tasks.workunit.client.0.smithi070.stdout:6/423: fsync d0/d3/d19/f7f 0 2024-04-02T04:47:52.451 INFO:tasks.workunit.client.0.smithi070.stdout:6/424: chown d0/d3/d19/d31/d46/f68 1 1 2024-04-02T04:47:52.452 INFO:tasks.workunit.client.0.smithi070.stdout:3/530: mknod d10/cb7 0 2024-04-02T04:47:52.454 INFO:tasks.workunit.client.1.smithi179.stdout:8/480: getdents d3/d6c/d96 0 2024-04-02T04:47:52.455 INFO:tasks.workunit.client.0.smithi070.stdout:5/583: mkdir d5/d9/d10/db9 0 2024-04-02T04:47:52.456 INFO:tasks.workunit.client.1.smithi179.stdout:0/542: rmdir d1/da/d28/d34/d39/d90 0 2024-04-02T04:47:52.457 INFO:tasks.workunit.client.1.smithi179.stdout:0/543: write d1/da/d52/f73 [1351379,104390] 0 2024-04-02T04:47:52.457 INFO:tasks.workunit.client.0.smithi070.stdout:0/589: getdents d4/d5/d13/d2a 0 2024-04-02T04:47:52.458 INFO:tasks.workunit.client.1.smithi179.stdout:0/544: fsync d1/da/d28/d34/d39/d77/f9b 0 2024-04-02T04:47:52.459 INFO:tasks.workunit.client.1.smithi179.stdout:8/481: mkdir d3/d32/d21/d22/da5 0 2024-04-02T04:47:52.462 INFO:tasks.workunit.client.1.smithi179.stdout:0/545: dwrite d1/f2a [0,4194304] 0 2024-04-02T04:47:52.463 INFO:tasks.workunit.client.0.smithi070.stdout:4/443: mknod d1/d3/d9/d5e/c94 0 2024-04-02T04:47:52.466 INFO:tasks.workunit.client.0.smithi070.stdout:3/531: rmdir d10/d32 39 2024-04-02T04:47:52.471 INFO:tasks.workunit.client.0.smithi070.stdout:5/584: write d5/d17/d50/f5a [3573534,75670] 0 2024-04-02T04:47:52.474 INFO:tasks.workunit.client.1.smithi179.stdout:8/482: link d3/c11 d3/d32/d30/ca6 0 2024-04-02T04:47:52.477 INFO:tasks.workunit.client.0.smithi070.stdout:6/425: unlink d0/f15 0 2024-04-02T04:47:52.478 INFO:tasks.workunit.client.0.smithi070.stdout:6/426: write d0/d3/d19/d31/d46/d6b/f70 [341296,67441] 0 2024-04-02T04:47:52.479 INFO:tasks.workunit.client.0.smithi070.stdout:6/427: write d0/f78 [108533,60278] 0 2024-04-02T04:47:52.479 INFO:tasks.workunit.client.1.smithi179.stdout:8/483: rename d3/d32/d21/d45 to d3/d32/d21/d45/d5a/da7 22 2024-04-02T04:47:52.480 INFO:tasks.workunit.client.1.smithi179.stdout:8/484: chown d3/d32/d30/f7d 687 1 2024-04-02T04:47:52.482 INFO:tasks.workunit.client.0.smithi070.stdout:1/545: link d0/d7/l3b d0/d54/d64/lb1 0 2024-04-02T04:47:52.485 INFO:tasks.workunit.client.0.smithi070.stdout:0/590: unlink d4/d5/da/d2d/f7d 0 2024-04-02T04:47:52.487 INFO:tasks.workunit.client.0.smithi070.stdout:3/532: stat d10/d32/d8a/d8d/l98 0 2024-04-02T04:47:52.488 INFO:tasks.workunit.client.1.smithi179.stdout:8/485: dwrite d3/d32/d21/d50/d65/f9b [0,4194304] 0 2024-04-02T04:47:52.489 INFO:tasks.workunit.client.1.smithi179.stdout:8/486: stat d3/d32/d21/d51/d70 0 2024-04-02T04:47:52.490 INFO:tasks.workunit.client.0.smithi070.stdout:5/585: mkdir d5/d9/d10/db9/dba 0 2024-04-02T04:47:52.494 INFO:tasks.workunit.client.0.smithi070.stdout:4/444: creat d1/d3/f95 x:0 0 0 2024-04-02T04:47:52.495 INFO:tasks.workunit.client.1.smithi179.stdout:8/487: readlink d3/d32/d21/l3f 0 2024-04-02T04:47:52.497 INFO:tasks.workunit.client.0.smithi070.stdout:6/428: mknod d0/d3/d19/d31/d61/d7b/c89 0 2024-04-02T04:47:52.499 INFO:tasks.workunit.client.1.smithi179.stdout:8/488: mkdir d3/d32/d21/d22/d6b/d91/da8 0 2024-04-02T04:47:52.501 INFO:tasks.workunit.client.0.smithi070.stdout:2/516: dwrite d2/d10/d21/d2b/f6e [4194304,4194304] 0 2024-04-02T04:47:52.505 INFO:tasks.workunit.client.1.smithi179.stdout:8/489: creat d3/d32/d21/d50/d56/d94/d84/fa9 x:0 0 0 2024-04-02T04:47:52.505 INFO:tasks.workunit.client.0.smithi070.stdout:5/586: dread d5/d17/d2b/f67 [0,4194304] 0 2024-04-02T04:47:52.509 INFO:tasks.workunit.client.0.smithi070.stdout:4/445: unlink d1/d3/f8c 0 2024-04-02T04:47:52.513 INFO:tasks.workunit.client.0.smithi070.stdout:5/587: dwrite d5/d9/d10/d51/f59 [0,4194304] 0 2024-04-02T04:47:52.514 INFO:tasks.workunit.client.1.smithi179.stdout:8/490: unlink d3/f82 0 2024-04-02T04:47:52.515 INFO:tasks.workunit.client.0.smithi070.stdout:5/588: dread - d5/d17/d27/d2e/f83 zero size 2024-04-02T04:47:52.516 INFO:tasks.workunit.client.0.smithi070.stdout:3/533: symlink d10/d32/d8a/d8d/da1/lb8 0 2024-04-02T04:47:52.517 INFO:tasks.workunit.client.0.smithi070.stdout:3/534: stat d10/d32/d4b/l83 0 2024-04-02T04:47:52.523 INFO:tasks.workunit.client.1.smithi179.stdout:3/555: symlink d8/d26/lad 0 2024-04-02T04:47:52.524 INFO:tasks.workunit.client.1.smithi179.stdout:5/470: write d8/d1b/f49 [1266155,123327] 0 2024-04-02T04:47:52.524 INFO:tasks.workunit.client.0.smithi070.stdout:7/484: dwrite d7/dd/df/d79/f48 [0,4194304] 0 2024-04-02T04:47:52.525 INFO:tasks.workunit.client.1.smithi179.stdout:3/556: write d8/d26/d83/d89/fab [20607,118766] 0 2024-04-02T04:47:52.526 INFO:tasks.workunit.client.1.smithi179.stdout:8/491: fdatasync d3/d1b/fa4 0 2024-04-02T04:47:52.532 INFO:tasks.workunit.client.1.smithi179.stdout:3/557: write d8/f22 [2140097,30051] 0 2024-04-02T04:47:52.532 INFO:tasks.workunit.client.1.smithi179.stdout:6/524: dwrite d3/d12/d23/f55 [0,4194304] 0 2024-04-02T04:47:52.534 INFO:tasks.workunit.client.1.smithi179.stdout:8/492: creat d3/d32/d21/d50/d65/faa x:0 0 0 2024-04-02T04:47:52.534 INFO:tasks.workunit.client.1.smithi179.stdout:3/558: chown d8/d26/d52/c61 230 1 2024-04-02T04:47:52.535 INFO:tasks.workunit.client.1.smithi179.stdout:6/525: dread - d3/d12/d23/d26/d57/fa2 zero size 2024-04-02T04:47:52.538 INFO:tasks.workunit.client.0.smithi070.stdout:0/591: link d4/d3c/c96 d4/d5/d58/da1/cac 0 2024-04-02T04:47:52.538 INFO:tasks.workunit.client.0.smithi070.stdout:9/444: write d4/d10/d11/f14 [785768,61646] 0 2024-04-02T04:47:52.540 INFO:tasks.workunit.client.1.smithi179.stdout:3/559: truncate d8/f48 1039994 0 2024-04-02T04:47:52.541 INFO:tasks.workunit.client.0.smithi070.stdout:5/589: truncate d5/f22 5140952 0 2024-04-02T04:47:52.546 INFO:tasks.workunit.client.0.smithi070.stdout:6/429: fdatasync d0/d3/f3a 0 2024-04-02T04:47:52.555 INFO:tasks.workunit.client.0.smithi070.stdout:4/446: rename d1/fb to d1/d3/d78/d91/f96 0 2024-04-02T04:47:52.558 INFO:tasks.workunit.client.1.smithi179.stdout:5/471: getdents d8/d1b/d23/d36/d53 0 2024-04-02T04:47:52.559 INFO:tasks.workunit.client.0.smithi070.stdout:9/445: chown d4/d10/d11/d7b/d19/d2f/d39/d57 128084768 1 2024-04-02T04:47:52.560 INFO:tasks.workunit.client.1.smithi179.stdout:0/546: write d1/da/d28/d37/f72 [954689,45155] 0 2024-04-02T04:47:52.562 INFO:tasks.workunit.client.0.smithi070.stdout:7/485: dread d7/dd/df/d34/d68/f70 [0,4194304] 0 2024-04-02T04:47:52.564 INFO:tasks.workunit.client.1.smithi179.stdout:8/493: link d3/d1b/f9a d3/d32/d21/fab 0 2024-04-02T04:47:52.567 INFO:tasks.workunit.client.0.smithi070.stdout:1/546: write d0/d7/f10 [1653256,36393] 0 2024-04-02T04:47:52.569 INFO:tasks.workunit.client.1.smithi179.stdout:5/472: creat d8/d1b/d23/d32/d6e/f95 x:0 0 0 2024-04-02T04:47:52.570 INFO:tasks.workunit.client.0.smithi070.stdout:6/430: mknod d0/d55/d5c/c8a 0 2024-04-02T04:47:52.571 INFO:tasks.workunit.client.1.smithi179.stdout:8/494: dread d3/d32/d21/f37 [0,4194304] 0 2024-04-02T04:47:52.571 INFO:tasks.workunit.client.0.smithi070.stdout:6/431: dread - d0/d55/d5c/f82 zero size 2024-04-02T04:47:52.573 INFO:tasks.workunit.client.0.smithi070.stdout:2/517: link d2/d10/d5c/d7a/l6a d2/d10/d21/d2b/la8 0 2024-04-02T04:47:52.578 INFO:tasks.workunit.client.1.smithi179.stdout:0/547: dwrite d1/da/d28/d5e/f98 [0,4194304] 0 2024-04-02T04:47:52.580 INFO:tasks.workunit.client.0.smithi070.stdout:0/592: symlink d4/d5/d58/d60/d83/da5/lad 0 2024-04-02T04:47:52.581 INFO:tasks.workunit.client.0.smithi070.stdout:9/446: dread d4/d10/d11/d7b/d19/d53/f59 [0,4194304] 0 2024-04-02T04:47:52.582 INFO:tasks.workunit.client.0.smithi070.stdout:0/593: fdatasync d4/d5/d13/d2a/f68 0 2024-04-02T04:47:52.583 INFO:tasks.workunit.client.1.smithi179.stdout:3/560: getdents d8/d6d 0 2024-04-02T04:47:52.587 INFO:tasks.workunit.client.1.smithi179.stdout:0/548: dwrite d1/da/d28/f41 [0,4194304] 0 2024-04-02T04:47:52.590 INFO:tasks.workunit.client.0.smithi070.stdout:1/547: dread d0/f2a [0,4194304] 0 2024-04-02T04:47:52.595 INFO:tasks.workunit.client.0.smithi070.stdout:1/548: dread d0/d9/d1d/f42 [0,4194304] 0 2024-04-02T04:47:52.596 INFO:tasks.workunit.client.0.smithi070.stdout:1/549: readlink d0/d9/d1d/d22/d2c/l62 0 2024-04-02T04:47:52.602 INFO:tasks.workunit.client.1.smithi179.stdout:6/526: dwrite d3/f7f [0,4194304] 0 2024-04-02T04:47:52.604 INFO:tasks.workunit.client.1.smithi179.stdout:5/473: getdents d8 0 2024-04-02T04:47:52.604 INFO:tasks.workunit.client.1.smithi179.stdout:8/495: write d3/d1b/d2a/f3e [718969,73524] 0 2024-04-02T04:47:52.610 INFO:tasks.workunit.client.1.smithi179.stdout:5/474: truncate d8/d1b/d23/d36/d53/f78 767355 0 2024-04-02T04:47:52.610 INFO:tasks.workunit.client.1.smithi179.stdout:0/549: mknod d1/da/d52/d75/d9a/ca7 0 2024-04-02T04:47:52.610 INFO:tasks.workunit.client.1.smithi179.stdout:5/475: stat d8/d1b/d23/d32/d46/f7d 0 2024-04-02T04:47:52.610 INFO:tasks.workunit.client.0.smithi070.stdout:5/590: write d5/d9/d23/f49 [2869398,79726] 0 2024-04-02T04:47:52.610 INFO:tasks.workunit.client.0.smithi070.stdout:6/432: dread d0/d3/d19/f3f [0,4194304] 0 2024-04-02T04:47:52.611 INFO:tasks.workunit.client.0.smithi070.stdout:5/591: chown d5/d14/f60 31 1 2024-04-02T04:47:52.613 INFO:tasks.workunit.client.0.smithi070.stdout:9/447: symlink d4/d10/d11/d7b/d19/d2f/d39/l91 0 2024-04-02T04:47:52.614 INFO:tasks.workunit.client.1.smithi179.stdout:6/527: mknod d3/d12/d23/d39/d4e/d6c/cb4 0 2024-04-02T04:47:52.616 INFO:tasks.workunit.client.1.smithi179.stdout:3/561: write d8/d26/d36/f5e [1113267,96670] 0 2024-04-02T04:47:52.619 INFO:tasks.workunit.client.0.smithi070.stdout:3/535: link d10/d32/f71 d10/d66/fb9 0 2024-04-02T04:47:52.620 INFO:tasks.workunit.client.0.smithi070.stdout:2/518: dwrite d2/d10/d21/f33 [4194304,4194304] 0 2024-04-02T04:47:52.625 INFO:tasks.workunit.client.1.smithi179.stdout:3/562: mknod d8/d26/d52/cae 0 2024-04-02T04:47:52.627 INFO:tasks.workunit.client.1.smithi179.stdout:8/496: truncate d3/f27 441425 0 2024-04-02T04:47:52.628 INFO:tasks.workunit.client.1.smithi179.stdout:0/550: getdents d1/da/d28/d81/d9d 0 2024-04-02T04:47:52.630 INFO:tasks.workunit.client.0.smithi070.stdout:2/519: dwrite d2/d10/d5c/f8c [0,4194304] 0 2024-04-02T04:47:52.634 INFO:tasks.workunit.client.0.smithi070.stdout:5/592: unlink d5/d17/d27/f45 0 2024-04-02T04:47:52.634 INFO:tasks.workunit.client.0.smithi070.stdout:0/594: dwrite d4/d5/d13/d2a/d29/d52/f82 [0,4194304] 0 2024-04-02T04:47:52.636 INFO:tasks.workunit.client.0.smithi070.stdout:0/595: fsync d4/d5/d58/d60/f7c 0 2024-04-02T04:47:52.636 INFO:tasks.workunit.client.0.smithi070.stdout:5/593: chown d5/d17/d27/d2e/l9e 29027 1 2024-04-02T04:47:52.639 INFO:tasks.workunit.client.1.smithi179.stdout:0/551: creat d1/da/d67/fa8 x:0 0 0 2024-04-02T04:47:52.641 INFO:tasks.workunit.client.1.smithi179.stdout:9/533: sync 2024-04-02T04:47:52.642 INFO:tasks.workunit.client.0.smithi070.stdout:6/433: dread d0/d20/d5f/f6e [0,4194304] 0 2024-04-02T04:47:52.643 INFO:tasks.workunit.client.1.smithi179.stdout:6/528: rename d3/d2c/dae to d3/d2d/d38/db5 0 2024-04-02T04:47:52.645 INFO:tasks.workunit.client.1.smithi179.stdout:3/563: creat d8/d57/d63/faf x:0 0 0 2024-04-02T04:47:52.646 INFO:tasks.workunit.client.0.smithi070.stdout:1/550: dwrite d0/d54/f58 [0,4194304] 0 2024-04-02T04:47:52.646 INFO:tasks.workunit.client.0.smithi070.stdout:3/536: mknod d10/d32/d8a/d8d/da1/cba 0 2024-04-02T04:47:52.647 INFO:tasks.workunit.client.0.smithi070.stdout:3/537: fdatasync d10/d32/d3f/f6d 0 2024-04-02T04:47:52.648 INFO:tasks.workunit.client.1.smithi179.stdout:8/497: symlink d3/d32/d21/d22/da5/lac 0 2024-04-02T04:47:52.648 INFO:tasks.workunit.client.1.smithi179.stdout:0/552: dwrite d1/db/fc [0,4194304] 0 2024-04-02T04:47:52.650 INFO:tasks.workunit.client.1.smithi179.stdout:5/476: getdents d8/d1b/d23/d30/d73 0 2024-04-02T04:47:52.658 INFO:tasks.workunit.client.1.smithi179.stdout:9/534: rmdir d0/d1/d11/d29/d86 39 2024-04-02T04:47:52.658 INFO:tasks.workunit.client.1.smithi179.stdout:5/477: dread d8/d1b/d23/d32/f3d [0,4194304] 0 2024-04-02T04:47:52.660 INFO:tasks.workunit.client.1.smithi179.stdout:9/535: truncate d0/d1/d10/d6d/d58/f96 1039292 0 2024-04-02T04:47:52.661 INFO:tasks.workunit.client.0.smithi070.stdout:9/448: mkdir d4/d10/d11/d7b/d19/d2f/d4d/d92 0 2024-04-02T04:47:52.662 INFO:tasks.workunit.client.1.smithi179.stdout:8/498: mkdir d3/d32/d21/d50/d56/dad 0 2024-04-02T04:47:52.665 INFO:tasks.workunit.client.0.smithi070.stdout:7/486: getdents d7 0 2024-04-02T04:47:52.666 INFO:tasks.workunit.client.1.smithi179.stdout:5/478: fdatasync d8/f1a 0 2024-04-02T04:47:52.668 INFO:tasks.workunit.client.1.smithi179.stdout:9/536: dwrite d0/d1/d11/f78 [0,4194304] 0 2024-04-02T04:47:52.668 INFO:tasks.workunit.client.0.smithi070.stdout:5/594: rename d5/d14/f60 to d5/d17/d6c/fbb 0 2024-04-02T04:47:52.668 INFO:tasks.workunit.client.1.smithi179.stdout:5/479: write d8/d1b/d23/d32/d4e/f54 [753102,105983] 0 2024-04-02T04:47:52.669 INFO:tasks.workunit.client.0.smithi070.stdout:5/595: chown d5/d9/d23/d30/da2/f38 1886628638 1 2024-04-02T04:47:52.669 INFO:tasks.workunit.client.1.smithi179.stdout:9/537: chown d0/d1/d11/f51 99300701 1 2024-04-02T04:47:52.673 INFO:tasks.workunit.client.0.smithi070.stdout:6/434: mkdir d0/d55/d8b 0 2024-04-02T04:47:52.678 INFO:tasks.workunit.client.0.smithi070.stdout:3/538: symlink d10/d17/d51/d73/lbb 0 2024-04-02T04:47:52.680 INFO:tasks.workunit.client.0.smithi070.stdout:4/447: link d1/d3/l55 d1/d3/l97 0 2024-04-02T04:47:52.682 INFO:tasks.workunit.client.0.smithi070.stdout:0/596: dwrite d4/d5/d13/d2a/d29/d52/f82 [4194304,4194304] 0 2024-04-02T04:47:52.684 INFO:tasks.workunit.client.0.smithi070.stdout:2/520: link d2/f9c d2/d12/fa9 0 2024-04-02T04:47:52.684 INFO:tasks.workunit.client.1.smithi179.stdout:6/529: dwrite d3/f72 [0,4194304] 0 2024-04-02T04:47:52.684 INFO:tasks.workunit.client.1.smithi179.stdout:3/564: dwrite d8/d26/f3f [0,4194304] 0 2024-04-02T04:47:52.686 INFO:tasks.workunit.client.0.smithi070.stdout:0/597: read d4/d5/d13/d2a/d29/f59 [5372793,90220] 0 2024-04-02T04:47:52.687 INFO:tasks.workunit.client.1.smithi179.stdout:5/480: symlink d8/d1b/d23/d30/d73/l96 0 2024-04-02T04:47:52.691 INFO:tasks.workunit.client.0.smithi070.stdout:9/449: mknod d4/d34/c93 0 2024-04-02T04:47:52.692 INFO:tasks.workunit.client.0.smithi070.stdout:9/450: write d4/d10/d11/f14 [2086613,91554] 0 2024-04-02T04:47:52.693 INFO:tasks.workunit.client.1.smithi179.stdout:0/553: dwrite d1/da/d48/d65/d7d/f7a [0,4194304] 0 2024-04-02T04:47:52.694 INFO:tasks.workunit.client.0.smithi070.stdout:9/451: rename d4 to d4/d10/d11/d7b/d19/d2f/d4d/d92/d94 22 2024-04-02T04:47:52.695 INFO:tasks.workunit.client.0.smithi070.stdout:9/452: stat d4/d10/d11/d7b/d19/d2f/d39/d4f 0 2024-04-02T04:47:52.696 INFO:tasks.workunit.client.0.smithi070.stdout:7/487: unlink d7/dd/df/d79/d46/f51 0 2024-04-02T04:47:52.699 INFO:tasks.workunit.client.1.smithi179.stdout:1/532: sync 2024-04-02T04:47:52.699 INFO:tasks.workunit.client.1.smithi179.stdout:7/589: sync 2024-04-02T04:47:52.699 INFO:tasks.workunit.client.1.smithi179.stdout:2/373: sync 2024-04-02T04:47:52.699 INFO:tasks.workunit.client.1.smithi179.stdout:4/585: sync 2024-04-02T04:47:52.701 INFO:tasks.workunit.client.1.smithi179.stdout:2/374: fdatasync dc/d29/d4a/d1e/d43/f47 0 2024-04-02T04:47:52.703 INFO:tasks.workunit.client.1.smithi179.stdout:7/590: write d1/dd/fc3 [2768771,101828] 0 2024-04-02T04:47:52.707 INFO:tasks.workunit.client.0.smithi070.stdout:6/435: unlink d0/d20/d5f/f84 0 2024-04-02T04:47:52.709 INFO:tasks.workunit.client.0.smithi070.stdout:1/551: creat d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/fb2 x:0 0 0 2024-04-02T04:47:52.711 INFO:tasks.workunit.client.1.smithi179.stdout:3/565: creat d8/d26/d83/d89/d9c/fb0 x:0 0 0 2024-04-02T04:47:52.711 INFO:tasks.workunit.client.1.smithi179.stdout:9/538: dread d0/d1/d11/d3a/f81 [0,4194304] 0 2024-04-02T04:47:52.712 INFO:tasks.workunit.client.0.smithi070.stdout:3/539: creat d10/d32/d8a/d8d/da1/fbc x:0 0 0 2024-04-02T04:47:52.713 INFO:tasks.workunit.client.1.smithi179.stdout:5/481: mknod d8/d1b/d23/d32/d46/c97 0 2024-04-02T04:47:52.714 INFO:tasks.workunit.client.1.smithi179.stdout:5/482: fsync d8/f3e 0 2024-04-02T04:47:52.716 INFO:tasks.workunit.client.0.smithi070.stdout:1/552: dwrite d0/d7/f81 [0,4194304] 0 2024-04-02T04:47:52.716 INFO:tasks.workunit.client.0.smithi070.stdout:1/553: chown d0/d9 1 1 2024-04-02T04:47:52.717 INFO:tasks.workunit.client.1.smithi179.stdout:5/483: write d8/d1b/d23/d32/d4e/d50/f68 [1530741,33532] 0 2024-04-02T04:47:52.717 INFO:tasks.workunit.client.1.smithi179.stdout:9/539: dread d0/d1/d10/d6d/f20 [4194304,4194304] 0 2024-04-02T04:47:52.717 INFO:tasks.workunit.client.1.smithi179.stdout:5/484: dread - d8/d1b/d23/d36/d53/f7b zero size 2024-04-02T04:47:52.718 INFO:tasks.workunit.client.0.smithi070.stdout:2/521: creat d2/d12/d3e/faa x:0 0 0 2024-04-02T04:47:52.720 INFO:tasks.workunit.client.1.smithi179.stdout:0/554: symlink d1/da/d48/d5c/la9 0 2024-04-02T04:47:52.721 INFO:tasks.workunit.client.0.smithi070.stdout:0/598: mknod d4/d5/d58/d60/d83/cae 0 2024-04-02T04:47:52.722 INFO:tasks.workunit.client.1.smithi179.stdout:0/555: chown d1/da/d28/d5e/f98 10 1 2024-04-02T04:47:52.726 INFO:tasks.workunit.client.1.smithi179.stdout:5/485: dwrite d8/d1b/d23/d36/f8f [0,4194304] 0 2024-04-02T04:47:52.731 INFO:tasks.workunit.client.1.smithi179.stdout:8/499: dwrite d3/f27 [0,4194304] 0 2024-04-02T04:47:52.731 INFO:tasks.workunit.client.1.smithi179.stdout:4/586: truncate d1/d3/d19/f2c 287857 0 2024-04-02T04:47:52.732 INFO:tasks.workunit.client.1.smithi179.stdout:8/500: stat d3/d32/d21/d50/d56/d94/c8a 0 2024-04-02T04:47:52.734 INFO:tasks.workunit.client.1.smithi179.stdout:7/591: creat d1/d9/d27/d87/d8b/fd4 x:0 0 0 2024-04-02T04:47:52.738 INFO:tasks.workunit.client.1.smithi179.stdout:2/375: symlink dc/d29/d4a/d1e/l7a 0 2024-04-02T04:47:52.738 INFO:tasks.workunit.client.1.smithi179.stdout:2/376: readlink dc/d29/l49 0 2024-04-02T04:47:52.740 INFO:tasks.workunit.client.0.smithi070.stdout:4/448: mkdir d1/d98 0 2024-04-02T04:47:52.741 INFO:tasks.workunit.client.1.smithi179.stdout:9/540: dread d0/f90 [0,4194304] 0 2024-04-02T04:47:52.744 INFO:tasks.workunit.client.1.smithi179.stdout:3/566: mknod d8/d26/d36/cb1 0 2024-04-02T04:47:52.747 INFO:tasks.workunit.client.0.smithi070.stdout:1/554: truncate d0/d7/f6c 224608 0 2024-04-02T04:47:52.748 INFO:tasks.workunit.client.0.smithi070.stdout:9/453: dwrite d4/d10/d12/d27/d83/f8e [0,4194304] 0 2024-04-02T04:47:52.748 INFO:tasks.workunit.client.0.smithi070.stdout:2/522: dread - d2/d24/f7e zero size 2024-04-02T04:47:52.750 INFO:tasks.workunit.client.1.smithi179.stdout:0/556: creat d1/db/faa x:0 0 0 2024-04-02T04:47:52.751 INFO:tasks.workunit.client.0.smithi070.stdout:0/599: rmdir d4/d5/d13/d2a/d8d 39 2024-04-02T04:47:52.756 INFO:tasks.workunit.client.0.smithi070.stdout:7/488: symlink d7/dd/l9d 0 2024-04-02T04:47:52.757 INFO:tasks.workunit.client.0.smithi070.stdout:7/489: write d7/dd/f38 [1114424,36143] 0 2024-04-02T04:47:52.758 INFO:tasks.workunit.client.1.smithi179.stdout:0/557: dwrite d1/f2a [4194304,4194304] 0 2024-04-02T04:47:52.758 INFO:tasks.workunit.client.0.smithi070.stdout:3/540: write f0 [2732385,50804] 0 2024-04-02T04:47:52.759 INFO:tasks.workunit.client.1.smithi179.stdout:0/558: truncate d1/f1e 4747281 0 2024-04-02T04:47:52.764 INFO:tasks.workunit.client.1.smithi179.stdout:4/587: readlink d1/d3/df/lb7 0 2024-04-02T04:47:52.766 INFO:tasks.workunit.client.0.smithi070.stdout:3/541: dwrite d10/d32/d8a/d8d/d91/f94 [0,4194304] 0 2024-04-02T04:47:52.768 INFO:tasks.workunit.client.1.smithi179.stdout:8/501: creat d3/d1b/fae x:0 0 0 2024-04-02T04:47:52.769 INFO:tasks.workunit.client.1.smithi179.stdout:8/502: chown d3/d32/d21/d22/f2d 2 1 2024-04-02T04:47:52.771 INFO:tasks.workunit.client.1.smithi179.stdout:7/592: symlink d1/dd/d16/d4d/ld5 0 2024-04-02T04:47:52.771 INFO:tasks.workunit.client.0.smithi070.stdout:4/449: dread - d1/d3/d9/f53 zero size 2024-04-02T04:47:52.771 INFO:tasks.workunit.client.0.smithi070.stdout:3/542: chown d10/d32/d8a/d8d/d91/f9f 15 1 2024-04-02T04:47:52.773 INFO:tasks.workunit.client.1.smithi179.stdout:2/377: creat dc/d29/d4a/d1e/f7b x:0 0 0 2024-04-02T04:47:52.774 INFO:tasks.workunit.client.0.smithi070.stdout:8/510: sync 2024-04-02T04:47:52.776 INFO:tasks.workunit.client.0.smithi070.stdout:9/454: chown d4/d10/d11/d7b/d19/c6c 78081560 1 2024-04-02T04:47:52.777 INFO:tasks.workunit.client.0.smithi070.stdout:2/523: creat d2/d10/d5c/d7a/fab x:0 0 0 2024-04-02T04:47:52.778 INFO:tasks.workunit.client.1.smithi179.stdout:6/530: creat d3/d2d/fb6 x:0 0 0 2024-04-02T04:47:52.778 INFO:tasks.workunit.client.0.smithi070.stdout:2/524: dread - d2/d12/fa7 zero size 2024-04-02T04:47:52.783 INFO:tasks.workunit.client.0.smithi070.stdout:5/596: link d5/d17/f32 d5/d9/d10/d51/fbc 0 2024-04-02T04:47:52.786 INFO:tasks.workunit.client.0.smithi070.stdout:6/436: rename d0/c33 to d0/d3/d19/d31/c8c 0 2024-04-02T04:47:52.787 INFO:tasks.workunit.client.0.smithi070.stdout:7/490: rename d7/dd/df to d7/dd/df/d7a/d6c/d77/d75/d80/d9e 22 2024-04-02T04:47:52.788 INFO:tasks.workunit.client.1.smithi179.stdout:4/588: read d1/d3/df/d10/fa1 [346230,126021] 0 2024-04-02T04:47:52.788 INFO:tasks.workunit.client.0.smithi070.stdout:6/437: write d0/d3/d19/d31/f36 [7534078,123868] 0 2024-04-02T04:47:52.789 INFO:tasks.workunit.client.1.smithi179.stdout:4/589: readlink d1/d3/dc/d27/d62/l81 0 2024-04-02T04:47:52.789 INFO:tasks.workunit.client.0.smithi070.stdout:4/450: chown d1/fa 8962108 1 2024-04-02T04:47:52.796 INFO:tasks.workunit.client.1.smithi179.stdout:2/378: rmdir dc/d29/d4a 39 2024-04-02T04:47:52.799 INFO:tasks.workunit.client.0.smithi070.stdout:9/455: dread d4/d10/d11/d7b/d19/d2f/d39/d4f/f77 [0,4194304] 0 2024-04-02T04:47:52.800 INFO:tasks.workunit.client.1.smithi179.stdout:9/541: rename d0/d1/d11/d29/d86/ca0 to d0/d19/d1c/ca4 0 2024-04-02T04:47:52.801 INFO:tasks.workunit.client.1.smithi179.stdout:9/542: dread - d0/d1/d10/d6d/d58/f75 zero size 2024-04-02T04:47:52.803 INFO:tasks.workunit.client.1.smithi179.stdout:6/531: mknod d3/d12/d23/d39/d4e/d66/cb7 0 2024-04-02T04:47:52.804 INFO:tasks.workunit.client.0.smithi070.stdout:2/525: fdatasync d2/d10/d5c/f81 0 2024-04-02T04:47:52.805 INFO:tasks.workunit.client.1.smithi179.stdout:5/486: creat d8/d1b/f98 x:0 0 0 2024-04-02T04:47:52.805 INFO:tasks.workunit.client.0.smithi070.stdout:2/526: fsync d2/d12/d59/f8e 0 2024-04-02T04:47:52.806 INFO:tasks.workunit.client.0.smithi070.stdout:0/600: chown d4/d5/d13/d2a/f20 20674 1 2024-04-02T04:47:52.806 INFO:tasks.workunit.client.0.smithi070.stdout:2/527: chown d2/d10/d21/d3d/d42/l4f 306958 1 2024-04-02T04:47:52.808 INFO:tasks.workunit.client.0.smithi070.stdout:5/597: rename d5/d17/d27/d2e/f83 to d5/d9/d10/db9/fbd 0 2024-04-02T04:47:52.809 INFO:tasks.workunit.client.1.smithi179.stdout:1/533: getdents d3/d5/d2f/d4f/d42/d53 0 2024-04-02T04:47:52.813 INFO:tasks.workunit.client.0.smithi070.stdout:7/491: mknod d7/dd/df/c9f 0 2024-04-02T04:47:52.814 INFO:tasks.workunit.client.1.smithi179.stdout:0/559: dwrite d1/da/d48/f54 [0,4194304] 0 2024-04-02T04:47:52.814 INFO:tasks.workunit.client.1.smithi179.stdout:0/560: fdatasync d1/f14 0 2024-04-02T04:47:52.820 INFO:tasks.workunit.client.0.smithi070.stdout:3/543: mkdir d10/d17/db2/dbd 0 2024-04-02T04:47:52.823 INFO:tasks.workunit.client.0.smithi070.stdout:1/555: creat d0/d9/d1d/d22/d2c/fb3 x:0 0 0 2024-04-02T04:47:52.824 INFO:tasks.workunit.client.1.smithi179.stdout:9/543: fdatasync d0/d1/d11/f32 0 2024-04-02T04:47:52.826 INFO:tasks.workunit.client.0.smithi070.stdout:9/456: stat d4/d10/d11/d7b/d19/c80 0 2024-04-02T04:47:52.827 INFO:tasks.workunit.client.0.smithi070.stdout:6/438: dwrite d0/f26 [0,4194304] 0 2024-04-02T04:47:52.828 INFO:tasks.workunit.client.1.smithi179.stdout:5/487: rename d8/d1b/f24 to d8/d1b/d23/d32/f99 0 2024-04-02T04:47:52.829 INFO:tasks.workunit.client.0.smithi070.stdout:6/439: fdatasync d0/d3/d19/f7f 0 2024-04-02T04:47:52.837 INFO:tasks.workunit.client.0.smithi070.stdout:1/556: dread d0/d7/d1a/f75 [0,4194304] 0 2024-04-02T04:47:52.837 INFO:tasks.workunit.client.1.smithi179.stdout:8/503: link d3/d1b/l38 d3/d32/d21/d22/d6b/d91/laf 0 2024-04-02T04:47:52.838 INFO:tasks.workunit.client.1.smithi179.stdout:6/532: dread d3/fb [0,4194304] 0 2024-04-02T04:47:52.840 INFO:tasks.workunit.client.1.smithi179.stdout:6/533: read d3/d2d/f65 [8463183,73840] 0 2024-04-02T04:47:52.840 INFO:tasks.workunit.client.1.smithi179.stdout:4/590: truncate d1/f42 1389917 0 2024-04-02T04:47:52.845 INFO:tasks.workunit.client.1.smithi179.stdout:9/544: unlink d0/d1/d10/f18 0 2024-04-02T04:47:52.846 INFO:tasks.workunit.client.1.smithi179.stdout:8/504: dwrite d3/d32/d21/d22/f62 [8388608,4194304] 0 2024-04-02T04:47:52.847 INFO:tasks.workunit.client.1.smithi179.stdout:5/488: chown d8/d1b/d23/d30/f71 32371318 1 2024-04-02T04:47:52.848 INFO:tasks.workunit.client.1.smithi179.stdout:8/505: truncate d3/f61 1223443 0 2024-04-02T04:47:52.852 INFO:tasks.workunit.client.1.smithi179.stdout:3/567: sync 2024-04-02T04:47:52.853 INFO:tasks.workunit.client.1.smithi179.stdout:9/545: dwrite d0/d19/d27/d7a/d84/f93 [0,4194304] 0 2024-04-02T04:47:52.854 INFO:tasks.workunit.client.1.smithi179.stdout:3/568: dread - d8/d57/f9b zero size 2024-04-02T04:47:52.855 INFO:tasks.workunit.client.1.smithi179.stdout:7/593: getdents d1/dd/d23/dab 0 2024-04-02T04:47:52.856 INFO:tasks.workunit.client.1.smithi179.stdout:3/569: dread - d8/d6d/d94/f95 zero size 2024-04-02T04:47:52.863 INFO:tasks.workunit.client.1.smithi179.stdout:2/379: creat dc/f7c x:0 0 0 2024-04-02T04:47:52.866 INFO:tasks.workunit.client.1.smithi179.stdout:7/594: dwrite d1/d9/d27/d87/fbe [0,4194304] 0 2024-04-02T04:47:52.866 INFO:tasks.workunit.client.1.smithi179.stdout:0/561: dread d1/da/df/f33 [0,4194304] 0 2024-04-02T04:47:52.868 INFO:tasks.workunit.client.1.smithi179.stdout:7/595: chown d1/d9/fc9 2 1 2024-04-02T04:47:52.871 INFO:tasks.workunit.client.1.smithi179.stdout:8/506: mkdir d3/d1b/d2a/db0 0 2024-04-02T04:47:52.873 INFO:tasks.workunit.client.0.smithi070.stdout:2/528: rename d2/f1b to d2/d12/d3e/fac 0 2024-04-02T04:47:52.874 INFO:tasks.workunit.client.0.smithi070.stdout:2/529: dread - d2/d10/d21/d71/f9f zero size 2024-04-02T04:47:52.877 INFO:tasks.workunit.client.0.smithi070.stdout:7/492: unlink d7/c91 0 2024-04-02T04:47:52.878 INFO:tasks.workunit.client.1.smithi179.stdout:4/591: dwrite d1/d3/f1b [0,4194304] 0 2024-04-02T04:47:52.879 INFO:tasks.workunit.client.1.smithi179.stdout:4/592: fsync d1/d15/f56 0 2024-04-02T04:47:52.880 INFO:tasks.workunit.client.1.smithi179.stdout:8/507: dwrite d3/d32/d21/f25 [4194304,4194304] 0 2024-04-02T04:47:52.882 INFO:tasks.workunit.client.1.smithi179.stdout:9/546: creat d0/d19/d1c/d45/fa5 x:0 0 0 2024-04-02T04:47:52.883 INFO:tasks.workunit.client.0.smithi070.stdout:5/598: dwrite d5/d57/f5d [0,4194304] 0 2024-04-02T04:47:52.884 INFO:tasks.workunit.client.1.smithi179.stdout:8/508: write d3/d32/d21/d51/d70/f8d [482517,117021] 0 2024-04-02T04:47:52.885 INFO:tasks.workunit.client.0.smithi070.stdout:5/599: chown d5/d17/l36 28740758 1 2024-04-02T04:47:52.893 INFO:tasks.workunit.client.0.smithi070.stdout:6/440: mkdir d0/d55/d79/d8d 0 2024-04-02T04:47:52.894 INFO:tasks.workunit.client.0.smithi070.stdout:6/441: read d0/f26 [1917443,85623] 0 2024-04-02T04:47:52.895 INFO:tasks.workunit.client.0.smithi070.stdout:6/442: readlink d0/d3/d19/d39/l63 0 2024-04-02T04:47:52.897 INFO:tasks.workunit.client.1.smithi179.stdout:1/534: getdents d3/d5/d2f/d4f/d35 0 2024-04-02T04:47:52.898 INFO:tasks.workunit.client.0.smithi070.stdout:1/557: fsync d0/f2a 0 2024-04-02T04:47:52.899 INFO:tasks.workunit.client.1.smithi179.stdout:4/593: dread d1/d3/df/f51 [0,4194304] 0 2024-04-02T04:47:52.900 INFO:tasks.workunit.client.1.smithi179.stdout:0/562: truncate d1/da/f46 1800040 0 2024-04-02T04:47:52.901 INFO:tasks.workunit.client.1.smithi179.stdout:4/594: chown d1/d15/d7c/db8 172674125 1 2024-04-02T04:47:52.901 INFO:tasks.workunit.client.1.smithi179.stdout:0/563: read - d1/da/f4a zero size 2024-04-02T04:47:52.901 INFO:tasks.workunit.client.1.smithi179.stdout:0/564: chown d1/f3b 11 1 2024-04-02T04:47:52.904 INFO:tasks.workunit.client.0.smithi070.stdout:0/601: mknod d4/d5/d13/d17/d98/caf 0 2024-04-02T04:47:52.912 INFO:tasks.workunit.client.1.smithi179.stdout:5/489: dwrite d8/d1b/d23/d42/f5d [0,4194304] 0 2024-04-02T04:47:52.912 INFO:tasks.workunit.client.0.smithi070.stdout:2/530: creat d2/d10/d21/d3d/d42/fad x:0 0 0 2024-04-02T04:47:52.918 INFO:tasks.workunit.client.1.smithi179.stdout:3/570: fsync d8/d26/f4e 0 2024-04-02T04:47:52.921 INFO:tasks.workunit.client.1.smithi179.stdout:4/595: dread d1/d3/df/d10/d2f/fa7 [0,4194304] 0 2024-04-02T04:47:52.922 INFO:tasks.workunit.client.1.smithi179.stdout:9/547: fsync d0/d1/d11/f32 0 2024-04-02T04:47:52.925 INFO:tasks.workunit.client.1.smithi179.stdout:8/509: mkdir d3/d32/d21/d22/d6b/db1 0 2024-04-02T04:47:52.926 INFO:tasks.workunit.client.0.smithi070.stdout:8/511: link d2/c12 d2/d8/d29/d22/c9f 0 2024-04-02T04:47:52.927 INFO:tasks.workunit.client.0.smithi070.stdout:8/512: stat d2/d3/d35/d3a/d4f/c85 0 2024-04-02T04:47:52.928 INFO:tasks.workunit.client.1.smithi179.stdout:1/535: readlink d3/d5/l2c 0 2024-04-02T04:47:52.929 INFO:tasks.workunit.client.1.smithi179.stdout:1/536: fsync d3/d5/de/d33/f7b 0 2024-04-02T04:47:52.934 INFO:tasks.workunit.client.0.smithi070.stdout:8/513: dwrite d2/f9b [0,4194304] 0 2024-04-02T04:47:52.936 INFO:tasks.workunit.client.0.smithi070.stdout:4/451: truncate d1/f2d 2049425 0 2024-04-02T04:47:52.937 INFO:tasks.workunit.client.0.smithi070.stdout:4/452: read d1/d3/d9/f64 [1953737,93565] 0 2024-04-02T04:47:52.942 INFO:tasks.workunit.client.0.smithi070.stdout:0/602: truncate d4/d5/da/d2d/f66 4783032 0 2024-04-02T04:47:52.944 INFO:tasks.workunit.client.1.smithi179.stdout:3/571: symlink d8/d26/d52/d85/lb2 0 2024-04-02T04:47:52.944 INFO:tasks.workunit.client.0.smithi070.stdout:9/457: write d4/d13/f26 [1374126,115640] 0 2024-04-02T04:47:52.944 INFO:tasks.workunit.client.0.smithi070.stdout:2/531: truncate d2/d10/da4/f9b 299735 0 2024-04-02T04:47:52.948 INFO:tasks.workunit.client.1.smithi179.stdout:6/534: rename d3/d2d/d43/c92 to d3/d12/d23/d39/cb8 0 2024-04-02T04:47:52.950 INFO:tasks.workunit.client.0.smithi070.stdout:7/493: chown d7/dd/df/d7a/d6c 135843108 1 2024-04-02T04:47:52.950 INFO:tasks.workunit.client.0.smithi070.stdout:5/600: dread d5/d9/d23/d30/da2/d75/f80 [0,4194304] 0 2024-04-02T04:47:52.950 INFO:tasks.workunit.client.0.smithi070.stdout:6/443: write d0/d3/f13 [5216334,59505] 0 2024-04-02T04:47:52.953 INFO:tasks.workunit.client.0.smithi070.stdout:1/558: write d0/d9/d1d/f60 [703708,25458] 0 2024-04-02T04:47:52.960 INFO:tasks.workunit.client.1.smithi179.stdout:3/572: dread d8/d26/d52/d85/fa6 [0,4194304] 0 2024-04-02T04:47:52.962 INFO:tasks.workunit.client.1.smithi179.stdout:1/537: symlink d3/d5/d2f/d4f/d35/d49/lbb 0 2024-04-02T04:47:52.963 INFO:tasks.workunit.client.1.smithi179.stdout:1/538: chown d3/d5/d2f/d4f/d35/d49/fba 0 1 2024-04-02T04:47:52.964 INFO:tasks.workunit.client.1.smithi179.stdout:1/539: chown d3/d5/d2f/d4f/d6a/la4 446748255 1 2024-04-02T04:47:52.965 INFO:tasks.workunit.client.1.smithi179.stdout:1/540: truncate d3/d5/d2f/d4f/d42/d53/d72/d86/fa6 1652341 0 2024-04-02T04:47:52.966 INFO:tasks.workunit.client.0.smithi070.stdout:4/453: mkdir d1/d3/d85/d8e/d99 0 2024-04-02T04:47:52.968 INFO:tasks.workunit.client.1.smithi179.stdout:4/596: mkdir d1/d3/dc/dc1 0 2024-04-02T04:47:52.968 INFO:tasks.workunit.client.0.smithi070.stdout:2/532: stat d2/d10/d53 0 2024-04-02T04:47:52.969 INFO:tasks.workunit.client.0.smithi070.stdout:2/533: chown d2/d12/fa9 515478158 1 2024-04-02T04:47:52.971 INFO:tasks.workunit.client.1.smithi179.stdout:9/548: mknod d0/d19/ca6 0 2024-04-02T04:47:52.972 INFO:tasks.workunit.client.1.smithi179.stdout:9/549: chown d0/d1/d10/c16 377908 1 2024-04-02T04:47:52.973 INFO:tasks.workunit.client.0.smithi070.stdout:7/494: write d7/d9/f21 [225985,118810] 0 2024-04-02T04:47:52.973 INFO:tasks.workunit.client.0.smithi070.stdout:3/544: link d10/d17/c8c d10/d17/d51/d73/cbe 0 2024-04-02T04:47:52.974 INFO:tasks.workunit.client.0.smithi070.stdout:3/545: write d10/d17/d4c/f62 [2969517,28275] 0 2024-04-02T04:47:52.975 INFO:tasks.workunit.client.0.smithi070.stdout:6/444: symlink d0/d43/d80/l8e 0 2024-04-02T04:47:52.975 INFO:tasks.workunit.client.0.smithi070.stdout:3/546: dread - d10/fad zero size 2024-04-02T04:47:52.976 INFO:tasks.workunit.client.1.smithi179.stdout:5/490: creat d8/d1b/f9a x:0 0 0 2024-04-02T04:47:52.976 INFO:tasks.workunit.client.1.smithi179.stdout:5/491: stat d8/d7e 0 2024-04-02T04:47:52.977 INFO:tasks.workunit.client.1.smithi179.stdout:5/492: fdatasync d8/f6c 0 2024-04-02T04:47:52.977 INFO:tasks.workunit.client.0.smithi070.stdout:6/445: chown d0/d3/d19/d39/l63 4 1 2024-04-02T04:47:52.977 INFO:tasks.workunit.client.0.smithi070.stdout:5/601: mknod d5/d9/d23/d8b/cbe 0 2024-04-02T04:47:52.978 INFO:tasks.workunit.client.1.smithi179.stdout:1/541: creat d3/d5/d2f/d4f/d42/d53/d72/fbc x:0 0 0 2024-04-02T04:47:52.979 INFO:tasks.workunit.client.1.smithi179.stdout:1/542: chown d3/d5/d2f/d4f/d42/d55/c8f 2848 1 2024-04-02T04:47:52.979 INFO:tasks.workunit.client.1.smithi179.stdout:2/380: rename dc/d29/d31/c5c to dc/d29/d31/d66/c7d 0 2024-04-02T04:47:52.981 INFO:tasks.workunit.client.1.smithi179.stdout:4/597: creat d1/d15/d7c/fc2 x:0 0 0 2024-04-02T04:47:52.983 INFO:tasks.workunit.client.1.smithi179.stdout:4/598: stat d1/d3/dc/d27/f7f 0 2024-04-02T04:47:52.986 INFO:tasks.workunit.client.1.smithi179.stdout:4/599: chown d1/d15/l40 536 1 2024-04-02T04:47:52.986 INFO:tasks.workunit.client.1.smithi179.stdout:2/381: dread dc/f17 [0,4194304] 0 2024-04-02T04:47:52.988 INFO:tasks.workunit.client.1.smithi179.stdout:4/600: read - d1/d3/dc/d26/d8a/fbe zero size 2024-04-02T04:47:52.989 INFO:tasks.workunit.client.1.smithi179.stdout:7/596: rename d1/d9/d27/d87/daf to d1/d9/d27/d87/d8b/d75/dd6 0 2024-04-02T04:47:52.989 INFO:tasks.workunit.client.1.smithi179.stdout:1/543: dwrite d3/d5/d2f/d4f/d42/d53/d72/d86/f90 [0,4194304] 0 2024-04-02T04:47:52.990 INFO:tasks.workunit.client.1.smithi179.stdout:8/510: getdents d3/d32/d21/d51 0 2024-04-02T04:47:52.991 INFO:tasks.workunit.client.1.smithi179.stdout:4/601: fsync d1/d3/dc/d26/d8a/d5d/fbc 0 2024-04-02T04:47:52.992 INFO:tasks.workunit.client.1.smithi179.stdout:4/602: chown d1/d93 4972 1 2024-04-02T04:47:52.995 INFO:tasks.workunit.client.1.smithi179.stdout:5/493: fdatasync d8/d1b/d23/d32/d6e/f90 0 2024-04-02T04:47:52.997 INFO:tasks.workunit.client.1.smithi179.stdout:5/494: write d8/d1b/d23/d32/d6e/f95 [42833,129976] 0 2024-04-02T04:47:52.997 INFO:tasks.workunit.client.1.smithi179.stdout:7/597: creat d1/d9/d55/fd7 x:0 0 0 2024-04-02T04:47:52.998 INFO:tasks.workunit.client.1.smithi179.stdout:1/544: creat d3/d5/d2f/d4f/d42/fbd x:0 0 0 2024-04-02T04:47:53.001 INFO:tasks.workunit.client.1.smithi179.stdout:6/535: rename d3/l4c to d3/d12/d23/d39/d4e/lb9 0 2024-04-02T04:47:53.002 INFO:tasks.workunit.client.1.smithi179.stdout:5/495: chown d8/d1b/d23/d42/c58 74459 1 2024-04-02T04:47:53.006 INFO:tasks.workunit.client.1.smithi179.stdout:3/573: write d8/d26/f34 [1801430,117294] 0 2024-04-02T04:47:53.007 INFO:tasks.workunit.client.1.smithi179.stdout:2/382: mknod dc/d29/d4a/c7e 0 2024-04-02T04:47:53.008 INFO:tasks.workunit.client.1.smithi179.stdout:2/383: read - dc/f5a zero size 2024-04-02T04:47:53.009 INFO:tasks.workunit.client.1.smithi179.stdout:4/603: creat d1/fc3 x:0 0 0 2024-04-02T04:47:53.009 INFO:tasks.workunit.client.1.smithi179.stdout:6/536: truncate d3/d2d/d38/d64/fa7 614658 0 2024-04-02T04:47:53.009 INFO:tasks.workunit.client.1.smithi179.stdout:5/496: rmdir d8/d1b/d23/d32/d4b 39 2024-04-02T04:47:53.010 INFO:tasks.workunit.client.1.smithi179.stdout:5/497: dread - d8/d1b/d23/d36/d53/d62/f75 zero size 2024-04-02T04:47:53.011 INFO:tasks.workunit.client.1.smithi179.stdout:1/545: mkdir d3/dbe 0 2024-04-02T04:47:53.011 INFO:tasks.workunit.client.1.smithi179.stdout:3/574: creat d8/d57/fb3 x:0 0 0 2024-04-02T04:47:53.012 INFO:tasks.workunit.client.1.smithi179.stdout:5/498: read - d8/d1b/d23/d30/f34 zero size 2024-04-02T04:47:53.014 INFO:tasks.workunit.client.1.smithi179.stdout:1/546: unlink d3/d5/de/f32 0 2024-04-02T04:47:53.014 INFO:tasks.workunit.client.1.smithi179.stdout:5/499: symlink d8/d1b/d23/d30/l9b 0 2024-04-02T04:47:53.015 INFO:tasks.workunit.client.1.smithi179.stdout:3/575: truncate d8/d6d/d94/f95 793892 0 2024-04-02T04:47:53.016 INFO:tasks.workunit.client.1.smithi179.stdout:3/576: write d8/d57/d63/faf [606030,31799] 0 2024-04-02T04:47:53.022 INFO:tasks.workunit.client.1.smithi179.stdout:1/547: creat d3/d5/d2f/d4f/d35/fbf x:0 0 0 2024-04-02T04:47:53.022 INFO:tasks.workunit.client.1.smithi179.stdout:6/537: link d3/d42/l90 d3/d12/d23/d39/d80/d93/lba 0 2024-04-02T04:47:53.022 INFO:tasks.workunit.client.1.smithi179.stdout:6/538: chown d3/d12/d23/d39/f71 780568 1 2024-04-02T04:47:53.025 INFO:tasks.workunit.client.1.smithi179.stdout:5/500: truncate d8/d1b/d23/d30/f71 616124 0 2024-04-02T04:47:53.026 INFO:tasks.workunit.client.1.smithi179.stdout:3/577: creat d8/d26/d52/fb4 x:0 0 0 2024-04-02T04:47:53.026 INFO:tasks.workunit.client.1.smithi179.stdout:5/501: write d8/d1b/f88 [116099,90979] 0 2024-04-02T04:47:53.027 INFO:tasks.workunit.client.1.smithi179.stdout:1/548: chown d3/d5/de/f3c 216 1 2024-04-02T04:47:53.027 INFO:tasks.workunit.client.1.smithi179.stdout:6/539: creat d3/d12/d44/fbb x:0 0 0 2024-04-02T04:47:53.028 INFO:tasks.workunit.client.1.smithi179.stdout:3/578: write d8/d26/f56 [2760963,72541] 0 2024-04-02T04:47:53.029 INFO:tasks.workunit.client.1.smithi179.stdout:3/579: write d8/d26/f62 [754530,48] 0 2024-04-02T04:47:53.031 INFO:tasks.workunit.client.1.smithi179.stdout:6/540: mkdir d3/d42/dbc 0 2024-04-02T04:47:53.032 INFO:tasks.workunit.client.1.smithi179.stdout:1/549: creat d3/d5/de/d33/fc0 x:0 0 0 2024-04-02T04:47:53.032 INFO:tasks.workunit.client.1.smithi179.stdout:5/502: creat d8/d1b/d23/d32/d4e/d51/f9c x:0 0 0 2024-04-02T04:47:53.035 INFO:tasks.workunit.client.1.smithi179.stdout:3/580: write d8/d57/f9b [889244,121957] 0 2024-04-02T04:47:53.035 INFO:tasks.workunit.client.1.smithi179.stdout:6/541: write d3/d12/d23/f55 [2301491,48669] 0 2024-04-02T04:47:53.036 INFO:tasks.workunit.client.1.smithi179.stdout:1/550: creat d3/d5/d2f/d4f/fc1 x:0 0 0 2024-04-02T04:47:53.037 INFO:tasks.workunit.client.1.smithi179.stdout:2/384: dread dc/d29/d4a/d4b/f6d [0,4194304] 0 2024-04-02T04:47:53.037 INFO:tasks.workunit.client.1.smithi179.stdout:3/581: write d8/d57/f9b [827144,47361] 0 2024-04-02T04:47:53.039 INFO:tasks.workunit.client.1.smithi179.stdout:6/542: creat d3/d2d/da5/fbd x:0 0 0 2024-04-02T04:47:53.040 INFO:tasks.workunit.client.1.smithi179.stdout:2/385: truncate dc/d29/d4a/f72 320856 0 2024-04-02T04:47:53.040 INFO:tasks.workunit.client.1.smithi179.stdout:3/582: symlink d8/d26/d83/d89/d9c/lb5 0 2024-04-02T04:47:53.041 INFO:tasks.workunit.client.1.smithi179.stdout:6/543: write d3/d2d/d38/d64/fa8 [604280,104314] 0 2024-04-02T04:47:53.041 INFO:tasks.workunit.client.1.smithi179.stdout:3/583: chown d8/d26/f4e 0 1 2024-04-02T04:47:53.042 INFO:tasks.workunit.client.1.smithi179.stdout:6/544: stat d3/d2d/f65 0 2024-04-02T04:47:53.042 INFO:tasks.workunit.client.1.smithi179.stdout:6/545: readlink d3/d12/d23/l89 0 2024-04-02T04:47:53.043 INFO:tasks.workunit.client.1.smithi179.stdout:3/584: creat d8/d6d/fb6 x:0 0 0 2024-04-02T04:47:53.044 INFO:tasks.workunit.client.1.smithi179.stdout:3/585: rename d8 to d8/d26/d52/d85/d9a/db7 22 2024-04-02T04:47:53.049 INFO:tasks.workunit.client.1.smithi179.stdout:6/546: dread d3/ff [0,4194304] 0 2024-04-02T04:47:53.050 INFO:tasks.workunit.client.0.smithi070.stdout:6/446: dwrite d0/d3/d19/d31/d46/d6b/f70 [0,4194304] 0 2024-04-02T04:47:53.055 INFO:tasks.workunit.client.0.smithi070.stdout:8/514: truncate d2/d8/f44 1497567 0 2024-04-02T04:47:53.055 INFO:tasks.workunit.client.0.smithi070.stdout:4/454: mknod d1/c9a 0 2024-04-02T04:47:53.057 INFO:tasks.workunit.client.1.smithi179.stdout:6/547: mkdir d3/d42/dbe 0 2024-04-02T04:47:53.057 INFO:tasks.workunit.client.1.smithi179.stdout:5/503: read d8/d1b/f56 [2909929,113556] 0 2024-04-02T04:47:53.061 INFO:tasks.workunit.client.0.smithi070.stdout:1/559: rename d0/d4c/d5a to d0/db4 0 2024-04-02T04:47:53.062 INFO:tasks.workunit.client.0.smithi070.stdout:6/447: rename d0 to d0/d55/d79/d8d/d8f 22 2024-04-02T04:47:53.063 INFO:tasks.workunit.client.0.smithi070.stdout:7/495: dread d7/d36/f72 [0,4194304] 0 2024-04-02T04:47:53.064 INFO:tasks.workunit.client.0.smithi070.stdout:5/602: truncate d5/d17/d6c/f41 1164323 0 2024-04-02T04:47:53.065 INFO:tasks.workunit.client.0.smithi070.stdout:7/496: stat d7/dd/df/d79/d46/d65/d7b 0 2024-04-02T04:47:53.065 INFO:tasks.workunit.client.0.smithi070.stdout:5/603: dread - d5/d17/d50/fa6 zero size 2024-04-02T04:47:53.065 INFO:tasks.workunit.client.0.smithi070.stdout:7/497: dread - d7/dd/df/f94 zero size 2024-04-02T04:47:53.066 INFO:tasks.workunit.client.0.smithi070.stdout:5/604: write d5/d14/f68 [273687,76039] 0 2024-04-02T04:47:53.067 INFO:tasks.workunit.client.1.smithi179.stdout:6/548: rename d3/d12/f25 to d3/d12/d23/d39/d4e/fbf 0 2024-04-02T04:47:53.073 INFO:tasks.workunit.client.1.smithi179.stdout:9/550: truncate d0/f4c 2160703 0 2024-04-02T04:47:53.074 INFO:tasks.workunit.client.0.smithi070.stdout:0/603: dwrite d4/d5/d13/d43/f53 [0,4194304] 0 2024-04-02T04:47:53.074 INFO:tasks.workunit.client.1.smithi179.stdout:9/551: dread - d0/d19/d1c/d45/fa5 zero size 2024-04-02T04:47:53.076 INFO:tasks.workunit.client.0.smithi070.stdout:9/458: dwrite d4/d10/d11/d7b/d19/f48 [0,4194304] 0 2024-04-02T04:47:53.077 INFO:tasks.workunit.client.0.smithi070.stdout:6/448: symlink d0/d55/l90 0 2024-04-02T04:47:53.078 INFO:tasks.workunit.client.0.smithi070.stdout:9/459: write d4/d10/d12/d27/d83/f8f [795524,110498] 0 2024-04-02T04:47:53.082 INFO:tasks.workunit.client.1.smithi179.stdout:6/549: truncate d3/d12/d44/f52 958894 0 2024-04-02T04:47:53.083 INFO:tasks.workunit.client.1.smithi179.stdout:9/552: dread d0/d1/f7d [0,4194304] 0 2024-04-02T04:47:53.084 INFO:tasks.workunit.client.0.smithi070.stdout:1/560: dwrite d0/d9/d1d/d22/d2c/f76 [0,4194304] 0 2024-04-02T04:47:53.087 INFO:tasks.workunit.client.1.smithi179.stdout:0/565: sync 2024-04-02T04:47:53.087 INFO:tasks.workunit.client.1.smithi179.stdout:8/511: truncate d3/d1b/d2a/f3e 583629 0 2024-04-02T04:47:53.091 INFO:tasks.workunit.client.1.smithi179.stdout:6/550: mkdir d3/d12/d6a/dc0 0 2024-04-02T04:47:53.093 INFO:tasks.workunit.client.0.smithi070.stdout:1/561: dwrite d0/d7/f81 [4194304,4194304] 0 2024-04-02T04:47:53.094 INFO:tasks.workunit.client.0.smithi070.stdout:1/562: fdatasync d0/db4/f9b 0 2024-04-02T04:47:53.094 INFO:tasks.workunit.client.1.smithi179.stdout:7/598: truncate d1/f7b 3139171 0 2024-04-02T04:47:53.097 INFO:tasks.workunit.client.1.smithi179.stdout:9/553: dwrite d0/d19/d1c/f31 [0,4194304] 0 2024-04-02T04:47:53.098 INFO:tasks.workunit.client.0.smithi070.stdout:0/604: truncate d4/d5/d13/d2a/d29/d52/f69 613144 0 2024-04-02T04:47:53.099 INFO:tasks.workunit.client.1.smithi179.stdout:9/554: truncate d0/d19/d27/d7a/d82/f98 789048 0 2024-04-02T04:47:53.099 INFO:tasks.workunit.client.0.smithi070.stdout:0/605: chown d4/d5/da/d2d/f4c 38 1 2024-04-02T04:47:53.100 INFO:tasks.workunit.client.0.smithi070.stdout:6/449: mknod d0/d3/d19/d31/d46/d6b/c91 0 2024-04-02T04:47:53.103 INFO:tasks.workunit.client.0.smithi070.stdout:6/450: read d0/d3/d19/d39/f65 [379200,39506] 0 2024-04-02T04:47:53.105 INFO:tasks.workunit.client.0.smithi070.stdout:6/451: fsync d0/d3/d19/d39/f6c 0 2024-04-02T04:47:53.105 INFO:tasks.workunit.client.0.smithi070.stdout:7/498: mknod d7/d4e/ca0 0 2024-04-02T04:47:53.106 INFO:tasks.workunit.client.1.smithi179.stdout:0/566: dwrite d1/da/d28/d37/f95 [0,4194304] 0 2024-04-02T04:47:53.106 INFO:tasks.workunit.client.1.smithi179.stdout:6/551: truncate d3/fa 1124575 0 2024-04-02T04:47:53.108 INFO:tasks.workunit.client.0.smithi070.stdout:5/605: mknod d5/d17/d27/cbf 0 2024-04-02T04:47:53.108 INFO:tasks.workunit.client.1.smithi179.stdout:7/599: read d1/d9/d27/f5f [779352,10403] 0 2024-04-02T04:47:53.110 INFO:tasks.workunit.client.1.smithi179.stdout:9/555: rmdir d0/d19 39 2024-04-02T04:47:53.115 INFO:tasks.workunit.client.1.smithi179.stdout:5/504: truncate d8/d1b/d23/d32/d4e/f59 456217 0 2024-04-02T04:47:53.116 INFO:tasks.workunit.client.0.smithi070.stdout:1/563: creat d0/d4c/da2/fb5 x:0 0 0 2024-04-02T04:47:53.117 INFO:tasks.workunit.client.1.smithi179.stdout:1/551: write d3/d5/d2f/d4f/d35/d49/f8b [1390951,17645] 0 2024-04-02T04:47:53.118 INFO:tasks.workunit.client.1.smithi179.stdout:1/552: dread - d3/d5/d96/fb4 zero size 2024-04-02T04:47:53.121 INFO:tasks.workunit.client.0.smithi070.stdout:9/460: mknod d4/d10/c95 0 2024-04-02T04:47:53.126 INFO:tasks.workunit.client.1.smithi179.stdout:7/600: symlink d1/d9/d27/d87/d8b/d75/db9/d8d/ld8 0 2024-04-02T04:47:53.129 INFO:tasks.workunit.client.0.smithi070.stdout:5/606: unlink d5/d9/d23/l76 0 2024-04-02T04:47:53.130 INFO:tasks.workunit.client.0.smithi070.stdout:5/607: write d5/d9/d23/d8b/fb3 [874920,100108] 0 2024-04-02T04:47:53.133 INFO:tasks.workunit.client.0.smithi070.stdout:1/564: mkdir d0/d54/d64/d6d/db6 0 2024-04-02T04:47:53.134 INFO:tasks.workunit.client.1.smithi179.stdout:5/505: symlink d8/d1b/d23/d42/l9d 0 2024-04-02T04:47:53.134 INFO:tasks.workunit.client.1.smithi179.stdout:5/506: chown d8/d1b/d23/d32/f7f 26707 1 2024-04-02T04:47:53.135 INFO:tasks.workunit.client.0.smithi070.stdout:0/606: symlink d4/d5/lb0 0 2024-04-02T04:47:53.135 INFO:tasks.workunit.client.1.smithi179.stdout:5/507: write d8/f1a [1213126,36496] 0 2024-04-02T04:47:53.136 INFO:tasks.workunit.client.1.smithi179.stdout:5/508: chown d8/d1b/d23/d32/d4e/f61 104952 1 2024-04-02T04:47:53.141 INFO:tasks.workunit.client.1.smithi179.stdout:6/552: truncate d3/f18 66550 0 2024-04-02T04:47:53.141 INFO:tasks.workunit.client.1.smithi179.stdout:6/553: chown d3/d2d/da5/fbd 700 1 2024-04-02T04:47:53.142 INFO:tasks.workunit.client.1.smithi179.stdout:6/554: write d3/d2d/fb6 [504014,32269] 0 2024-04-02T04:47:53.146 INFO:tasks.workunit.client.1.smithi179.stdout:9/556: creat d0/d19/d27/fa7 x:0 0 0 2024-04-02T04:47:53.146 INFO:tasks.workunit.client.1.smithi179.stdout:9/557: stat d0/d1/f4 0 2024-04-02T04:47:53.147 INFO:tasks.workunit.client.1.smithi179.stdout:3/586: write d8/d26/f4e [3141342,98800] 0 2024-04-02T04:47:53.147 INFO:tasks.workunit.client.1.smithi179.stdout:9/558: chown d0/d1/d10/l28 812765351 1 2024-04-02T04:47:53.148 INFO:tasks.workunit.client.1.smithi179.stdout:3/587: readlink d8/d26/lad 0 2024-04-02T04:47:53.148 INFO:tasks.workunit.client.0.smithi070.stdout:9/461: mknod d4/d10/c96 0 2024-04-02T04:47:53.149 INFO:tasks.workunit.client.1.smithi179.stdout:3/588: readlink d8/d26/d36/l8c 0 2024-04-02T04:47:53.149 INFO:tasks.workunit.client.1.smithi179.stdout:3/589: dread - d8/d57/fa8 zero size 2024-04-02T04:47:53.151 INFO:tasks.workunit.client.1.smithi179.stdout:8/512: link d3/d32/d21/d51/ca3 d3/d32/d21/cb2 0 2024-04-02T04:47:53.152 INFO:tasks.workunit.client.0.smithi070.stdout:1/565: dread d0/d9/fc [0,4194304] 0 2024-04-02T04:47:53.152 INFO:tasks.workunit.client.0.smithi070.stdout:3/547: write d10/d17/f6c [950607,23322] 0 2024-04-02T04:47:53.158 INFO:tasks.workunit.client.0.smithi070.stdout:9/462: fsync d4/d10/d12/d56/f5b 0 2024-04-02T04:47:53.158 INFO:tasks.workunit.client.0.smithi070.stdout:8/515: write d2/d8/d23/d24/f37 [239680,10082] 0 2024-04-02T04:47:53.160 INFO:tasks.workunit.client.1.smithi179.stdout:6/555: creat d3/d12/d23/d39/d4e/d66/fc1 x:0 0 0 2024-04-02T04:47:53.160 INFO:tasks.workunit.client.1.smithi179.stdout:6/556: stat d3/d12/d23/d39/d80/cac 0 2024-04-02T04:47:53.161 INFO:tasks.workunit.client.0.smithi070.stdout:7/499: getdents d7/dd/df/d7a/d6c/d77 0 2024-04-02T04:47:53.165 INFO:tasks.workunit.client.0.smithi070.stdout:4/455: dwrite d1/d3/d9/f11 [0,4194304] 0 2024-04-02T04:47:53.166 INFO:tasks.workunit.client.1.smithi179.stdout:7/601: mkdir d1/d9/dd9 0 2024-04-02T04:47:53.168 INFO:tasks.workunit.client.0.smithi070.stdout:3/548: mkdir d10/d17/db2/dbf 0 2024-04-02T04:47:53.171 INFO:tasks.workunit.client.1.smithi179.stdout:3/590: mkdir d8/d26/d52/d7e/db8 0 2024-04-02T04:47:53.174 INFO:tasks.workunit.client.1.smithi179.stdout:8/513: creat d3/d32/d21/d22/d6b/fb3 x:0 0 0 2024-04-02T04:47:53.175 INFO:tasks.workunit.client.1.smithi179.stdout:8/514: stat d3/d32/d21/d50/d56/f77 0 2024-04-02T04:47:53.176 INFO:tasks.workunit.client.1.smithi179.stdout:4/604: sync 2024-04-02T04:47:53.177 INFO:tasks.workunit.client.1.smithi179.stdout:2/386: sync 2024-04-02T04:47:53.178 INFO:tasks.workunit.client.1.smithi179.stdout:3/591: dwrite d8/d26/f34 [0,4194304] 0 2024-04-02T04:47:53.179 INFO:tasks.workunit.client.0.smithi070.stdout:9/463: creat d4/d10/d12/d27/f97 x:0 0 0 2024-04-02T04:47:53.180 INFO:tasks.workunit.client.1.smithi179.stdout:3/592: write d8/d26/f62 [539192,8842] 0 2024-04-02T04:47:53.182 INFO:tasks.workunit.client.0.smithi070.stdout:1/566: dread d0/d9/d1d/f21 [0,4194304] 0 2024-04-02T04:47:53.185 INFO:tasks.workunit.client.0.smithi070.stdout:1/567: truncate d0/d9/d1d/d22/d2c/d4f/f50 4822257 0 2024-04-02T04:47:53.185 INFO:tasks.workunit.client.0.smithi070.stdout:7/500: symlink d7/d36/la1 0 2024-04-02T04:47:53.186 INFO:tasks.workunit.client.0.smithi070.stdout:7/501: readlink d7/d36/l99 0 2024-04-02T04:47:53.187 INFO:tasks.workunit.client.0.smithi070.stdout:1/568: chown d0/d9/d1d/d22/d2c/d4f/f61 1 1 2024-04-02T04:47:53.189 INFO:tasks.workunit.client.0.smithi070.stdout:7/502: write d7/dd/df/d79/d46/d65/f71 [4594,113170] 0 2024-04-02T04:47:53.189 INFO:tasks.workunit.client.0.smithi070.stdout:1/569: chown d0/d9/d1d/d22/d2c/d4f/d77 181626 1 2024-04-02T04:47:53.190 INFO:tasks.workunit.client.1.smithi179.stdout:7/602: rename d1/dd/d16/d4d/db1 to d1/d9/d55/dbf/d9f/da8/dda 0 2024-04-02T04:47:53.191 INFO:tasks.workunit.client.1.smithi179.stdout:7/603: chown d1/d9/d35/d6b/cc6 0 1 2024-04-02T04:47:53.191 INFO:tasks.workunit.client.0.smithi070.stdout:4/456: dread d1/d3/d9/f5f [0,4194304] 0 2024-04-02T04:47:53.192 INFO:tasks.workunit.client.1.smithi179.stdout:7/604: fsync d1/d9/d55/dbf/d9f/da8/dda/fc0 0 2024-04-02T04:47:53.193 INFO:tasks.workunit.client.0.smithi070.stdout:3/549: rename d10/d17/d51/da5 to d10/d32/d3f/dc0 0 2024-04-02T04:47:53.193 INFO:tasks.workunit.client.0.smithi070.stdout:3/550: stat fb 0 2024-04-02T04:47:53.194 INFO:tasks.workunit.client.0.smithi070.stdout:3/551: chown d10/d17/c3e 92 1 2024-04-02T04:47:53.197 INFO:tasks.workunit.client.1.smithi179.stdout:8/515: dread d3/d31/f9e [0,4194304] 0 2024-04-02T04:47:53.198 INFO:tasks.workunit.client.0.smithi070.stdout:9/464: read d4/d10/d12/d27/f5c [1030960,30156] 0 2024-04-02T04:47:53.199 INFO:tasks.workunit.client.1.smithi179.stdout:0/567: write d1/db/f2c [135176,1474] 0 2024-04-02T04:47:53.203 INFO:tasks.workunit.client.1.smithi179.stdout:3/593: rmdir d8/d26/d36 39 2024-04-02T04:47:53.206 INFO:tasks.workunit.client.1.smithi179.stdout:3/594: dread d8/d6d/d94/f95 [0,4194304] 0 2024-04-02T04:47:53.208 INFO:tasks.workunit.client.1.smithi179.stdout:9/559: rmdir d0/d19/d27/d48/d6c 0 2024-04-02T04:47:53.210 INFO:tasks.workunit.client.1.smithi179.stdout:0/568: dread d1/da/d28/d5e/f98 [4194304,4194304] 0 2024-04-02T04:47:53.210 INFO:tasks.workunit.client.0.smithi070.stdout:9/465: dread d4/d13/f37 [4194304,4194304] 0 2024-04-02T04:47:53.213 INFO:tasks.workunit.client.1.smithi179.stdout:2/387: rename dc/d29/d4a/f27 to dc/d29/d4a/d1e/d43/f7f 0 2024-04-02T04:47:53.214 INFO:tasks.workunit.client.0.smithi070.stdout:3/552: creat d10/d32/d8a/d8d/fc1 x:0 0 0 2024-04-02T04:47:53.214 INFO:tasks.workunit.client.0.smithi070.stdout:4/457: dread d1/d3/f4c [0,4194304] 0 2024-04-02T04:47:53.215 INFO:tasks.workunit.client.1.smithi179.stdout:1/553: write d3/d5/d2f/d4f/d42/d53/f5e [749524,47066] 0 2024-04-02T04:47:53.215 INFO:tasks.workunit.client.1.smithi179.stdout:9/560: dwrite d0/d1/d10/d6d/d58/f9d [0,4194304] 0 2024-04-02T04:47:53.215 INFO:tasks.workunit.client.1.smithi179.stdout:7/605: fdatasync d1/dd/d16/f2a 0 2024-04-02T04:47:53.216 INFO:tasks.workunit.client.1.smithi179.stdout:2/388: chown dc/d29/d31/d39/f4f 135 1 2024-04-02T04:47:53.217 INFO:tasks.workunit.client.0.smithi070.stdout:3/553: chown d10/d32/d8a/d8d/da1/lb8 6 1 2024-04-02T04:47:53.218 INFO:tasks.workunit.client.1.smithi179.stdout:7/606: chown d1/d9/d35/c4f 21367995 1 2024-04-02T04:47:53.218 INFO:tasks.workunit.client.0.smithi070.stdout:9/466: dwrite d4/d10/d11/f4e [0,4194304] 0 2024-04-02T04:47:53.219 INFO:tasks.workunit.client.0.smithi070.stdout:5/608: write d5/d17/d2b/d7b/f92 [3944246,21066] 0 2024-04-02T04:47:53.223 INFO:tasks.workunit.client.0.smithi070.stdout:7/503: unlink d7/l58 0 2024-04-02T04:47:53.230 INFO:tasks.workunit.client.0.smithi070.stdout:0/607: dwrite d4/d3c/f6a [0,4194304] 0 2024-04-02T04:47:53.233 INFO:tasks.workunit.client.1.smithi179.stdout:5/509: write f1 [3501124,102510] 0 2024-04-02T04:47:53.236 INFO:tasks.workunit.client.1.smithi179.stdout:0/569: creat d1/da/d28/d81/fab x:0 0 0 2024-04-02T04:47:53.242 INFO:tasks.workunit.client.1.smithi179.stdout:9/561: chown d0/d19/d1c/f52 16183130 1 2024-04-02T04:47:53.244 INFO:tasks.workunit.client.1.smithi179.stdout:1/554: symlink d3/d5/d2f/d4f/d6a/lc2 0 2024-04-02T04:47:53.246 INFO:tasks.workunit.client.1.smithi179.stdout:2/389: rmdir dc/d29/d4a/d4b 39 2024-04-02T04:47:53.251 INFO:tasks.workunit.client.0.smithi070.stdout:7/504: dwrite d7/dd/df/f83 [0,4194304] 0 2024-04-02T04:47:53.252 INFO:tasks.workunit.client.1.smithi179.stdout:0/570: dread d1/db/d64/f6a [0,4194304] 0 2024-04-02T04:47:53.252 INFO:tasks.workunit.client.0.smithi070.stdout:8/516: dwrite d2/d8/d29/d22/d5a/f7e [0,4194304] 0 2024-04-02T04:47:53.252 INFO:tasks.workunit.client.1.smithi179.stdout:0/571: chown d1/da/d48/d5c/la9 1 1 2024-04-02T04:47:53.255 INFO:tasks.workunit.client.1.smithi179.stdout:4/605: write d1/d3/dc/d26/d8a/f82 [2892354,112003] 0 2024-04-02T04:47:53.263 INFO:tasks.workunit.client.1.smithi179.stdout:3/595: rename d8/d26/f69 to d8/d26/d36/fb9 0 2024-04-02T04:47:53.265 INFO:tasks.workunit.client.0.smithi070.stdout:1/570: getdents d0/d9/d1d 0 2024-04-02T04:47:53.269 INFO:tasks.workunit.client.1.smithi179.stdout:8/516: write d3/d1b/fa2 [901794,29745] 0 2024-04-02T04:47:53.269 INFO:tasks.workunit.client.0.smithi070.stdout:0/608: chown d4/d5/d13/d2a/d3a/l76 26457 1 2024-04-02T04:47:53.271 INFO:tasks.workunit.client.1.smithi179.stdout:9/562: symlink d0/d19/d27/la8 0 2024-04-02T04:47:53.272 INFO:tasks.workunit.client.1.smithi179.stdout:5/510: dread f2 [0,4194304] 0 2024-04-02T04:47:53.273 INFO:tasks.workunit.client.0.smithi070.stdout:4/458: creat d1/d3/d9/f9b x:0 0 0 2024-04-02T04:47:53.275 INFO:tasks.workunit.client.1.smithi179.stdout:1/555: unlink d3/d5/d2f/d4f/d42/d55/l76 0 2024-04-02T04:47:53.275 INFO:tasks.workunit.client.1.smithi179.stdout:5/511: dread d8/d1b/f88 [0,4194304] 0 2024-04-02T04:47:53.276 INFO:tasks.workunit.client.0.smithi070.stdout:8/517: rename d2/d8/d23/f92 to d2/d8/d23/d24/d79/fa0 0 2024-04-02T04:47:53.278 INFO:tasks.workunit.client.0.smithi070.stdout:9/467: link d4/f8 d4/d13/f98 0 2024-04-02T04:47:53.279 INFO:tasks.workunit.client.1.smithi179.stdout:8/517: dwrite d3/d32/d21/d50/d65/faa [0,4194304] 0 2024-04-02T04:47:53.280 INFO:tasks.workunit.client.1.smithi179.stdout:2/390: rmdir dc/d29/d4a/d1e/d6c 39 2024-04-02T04:47:53.281 INFO:tasks.workunit.client.1.smithi179.stdout:9/563: dwrite d0/d19/d1c/d64/f79 [0,4194304] 0 2024-04-02T04:47:53.281 INFO:tasks.workunit.client.0.smithi070.stdout:0/609: symlink d4/d5/d58/d60/d83/da5/lb1 0 2024-04-02T04:47:53.283 INFO:tasks.workunit.client.1.smithi179.stdout:7/607: truncate d1/dd/d76/f86 1325157 0 2024-04-02T04:47:53.285 INFO:tasks.workunit.client.0.smithi070.stdout:5/609: write d5/d14/f16 [5956928,69914] 0 2024-04-02T04:47:53.285 INFO:tasks.workunit.client.1.smithi179.stdout:6/557: sync 2024-04-02T04:47:53.286 INFO:tasks.workunit.client.0.smithi070.stdout:4/459: dread d1/d3/d9/d41/d65/f6a [0,4194304] 0 2024-04-02T04:47:53.286 INFO:tasks.workunit.client.0.smithi070.stdout:5/610: fsync d5/d17/d50/fa6 0 2024-04-02T04:47:53.287 INFO:tasks.workunit.client.0.smithi070.stdout:5/611: write d5/d9/d23/f49 [1627464,31930] 0 2024-04-02T04:47:53.288 INFO:tasks.workunit.client.1.smithi179.stdout:4/606: fsync d1/d3/dc/d27/d62/f69 0 2024-04-02T04:47:53.289 INFO:tasks.workunit.client.1.smithi179.stdout:4/607: dread - d1/d93/fbd zero size 2024-04-02T04:47:53.521 INFO:tasks.workunit.client.0.smithi070.stdout:9/468: creat d4/d13/f99 x:0 0 0 2024-04-02T04:47:53.522 INFO:tasks.workunit.client.0.smithi070.stdout:9/469: fdatasync d4/d10/d11/d7b/d19/d2f/f3c 0 2024-04-02T04:47:53.522 INFO:tasks.workunit.client.0.smithi070.stdout:7/505: rmdir d7/dd/df/d79/d46/d97 0 2024-04-02T04:47:53.527 INFO:tasks.workunit.client.0.smithi070.stdout:7/506: dwrite d7/dd/df/d7a/f85 [0,4194304] 0 2024-04-02T04:47:53.528 INFO:tasks.workunit.client.0.smithi070.stdout:7/507: dread - d7/dd/df/d7a/d6c/f9b zero size 2024-04-02T04:47:53.545 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:53 smithi179 ceph-mon[36603]: pgmap v15: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 59 MiB/s rd, 231 MiB/s wr, 407 op/s 2024-04-02T04:47:53.576 INFO:tasks.workunit.client.1.smithi179.stdout:1/556: mknod d3/d5/d2f/d36/cc3 0 2024-04-02T04:47:53.577 INFO:tasks.workunit.client.0.smithi070.stdout:8/518: mknod d2/d3/ca1 0 2024-04-02T04:47:53.577 INFO:tasks.workunit.client.1.smithi179.stdout:1/557: chown d3/d5/d96/fa7 318 1 2024-04-02T04:47:53.578 INFO:tasks.workunit.client.1.smithi179.stdout:8/518: mknod d3/d32/d21/d50/d56/cb4 0 2024-04-02T04:47:53.580 INFO:tasks.workunit.client.0.smithi070.stdout:8/519: dread d2/d3/d35/d3a/d4f/d59/f8f [4194304,4194304] 0 2024-04-02T04:47:53.583 INFO:tasks.workunit.client.1.smithi179.stdout:9/564: unlink d0/d19/d1c/l59 0 2024-04-02T04:47:53.589 INFO:tasks.workunit.client.1.smithi179.stdout:6/558: creat d3/d12/d6a/fc2 x:0 0 0 2024-04-02T04:47:53.593 INFO:tasks.workunit.client.1.smithi179.stdout:7/608: dwrite d1/d9/d27/d87/d8b/d75/db9/f92 [0,4194304] 0 2024-04-02T04:47:53.593 INFO:tasks.workunit.client.0.smithi070.stdout:5/612: dwrite d5/d9/d10/f19 [4194304,4194304] 0 2024-04-02T04:47:53.593 INFO:tasks.workunit.client.1.smithi179.stdout:4/608: symlink d1/d3/df/d10/d2f/d60/da6/lc4 0 2024-04-02T04:47:53.596 INFO:tasks.workunit.client.1.smithi179.stdout:3/596: creat d8/d26/d52/d7e/db8/fba x:0 0 0 2024-04-02T04:47:53.598 INFO:tasks.workunit.client.0.smithi070.stdout:7/508: mknod d7/d36/ca2 0 2024-04-02T04:47:53.599 INFO:tasks.workunit.client.1.smithi179.stdout:0/572: write d1/da/d52/f5d [839662,12985] 0 2024-04-02T04:47:53.600 INFO:tasks.workunit.client.1.smithi179.stdout:0/573: chown d1/da/df/l1f 227734597 1 2024-04-02T04:47:53.601 INFO:tasks.workunit.client.0.smithi070.stdout:9/470: dwrite d4/d10/d11/d7b/d19/d53/f59 [4194304,4194304] 0 2024-04-02T04:47:53.602 INFO:tasks.workunit.client.1.smithi179.stdout:0/574: read - d1/da/d28/d34/d39/d77/f9b zero size 2024-04-02T04:47:53.603 INFO:tasks.workunit.client.0.smithi070.stdout:7/509: dread - d7/dd/df/d34/d68/f9c zero size 2024-04-02T04:47:53.605 INFO:tasks.workunit.client.0.smithi070.stdout:4/460: rename d1/d3/d9/d41 to d1/d9c 0 2024-04-02T04:47:53.605 INFO:tasks.workunit.client.1.smithi179.stdout:5/512: readlink d8/d1b/d23/d32/d4b/l64 0 2024-04-02T04:47:53.606 INFO:tasks.workunit.client.0.smithi070.stdout:9/471: read d4/d13/f26 [3781105,83542] 0 2024-04-02T04:47:53.608 INFO:tasks.workunit.client.0.smithi070.stdout:9/472: truncate d4/d10/d12/d55/f7e 8419778 0 2024-04-02T04:47:53.610 INFO:tasks.workunit.client.1.smithi179.stdout:8/519: write d3/d32/d21/d50/d56/f9c [4106785,30113] 0 2024-04-02T04:47:53.613 INFO:tasks.workunit.client.0.smithi070.stdout:5/613: creat d5/d9/d10/d6a/fc0 x:0 0 0 2024-04-02T04:47:53.613 INFO:tasks.workunit.client.1.smithi179.stdout:8/520: chown d3/d32/d21/f25 31580114 1 2024-04-02T04:47:53.614 INFO:tasks.workunit.client.0.smithi070.stdout:7/510: mknod d7/dd/df/ca3 0 2024-04-02T04:47:53.614 INFO:tasks.workunit.client.1.smithi179.stdout:5/513: dwrite d8/d1b/d23/d42/f5d [0,4194304] 0 2024-04-02T04:47:53.615 INFO:tasks.workunit.client.0.smithi070.stdout:7/511: chown d7/dd/df/c40 941 1 2024-04-02T04:47:53.615 INFO:tasks.workunit.client.1.smithi179.stdout:3/597: dread d8/d26/d83/fa0 [4194304,4194304] 0 2024-04-02T04:47:53.616 INFO:tasks.workunit.client.1.smithi179.stdout:6/559: dread - d3/d12/f62 zero size 2024-04-02T04:47:53.617 INFO:tasks.workunit.client.0.smithi070.stdout:8/520: rename d2/d8/d29/d22/d30 to d2/d8/d29/d22/d5a/d94/da2 0 2024-04-02T04:47:53.617 INFO:tasks.workunit.client.1.smithi179.stdout:3/598: write d8/d26/d36/fa2 [772875,63628] 0 2024-04-02T04:47:53.617 INFO:tasks.workunit.client.1.smithi179.stdout:6/560: stat d3/d42/c4d 0 2024-04-02T04:47:53.619 INFO:tasks.workunit.client.0.smithi070.stdout:9/473: mknod d4/d10/d11/d7b/d19/d2f/d4d/c9a 0 2024-04-02T04:47:53.623 INFO:tasks.workunit.client.1.smithi179.stdout:7/609: creat d1/dd/d23/dab/fdb x:0 0 0 2024-04-02T04:47:53.623 INFO:tasks.workunit.client.0.smithi070.stdout:8/521: dwrite d2/d3/d35/d3a/d4f/f87 [0,4194304] 0 2024-04-02T04:47:53.627 INFO:tasks.workunit.client.0.smithi070.stdout:4/461: symlink d1/l9d 0 2024-04-02T04:47:53.631 INFO:tasks.workunit.client.1.smithi179.stdout:0/575: mknod d1/da/d67/cac 0 2024-04-02T04:47:53.632 INFO:tasks.workunit.client.1.smithi179.stdout:0/576: write d1/da/d67/fa1 [696439,76148] 0 2024-04-02T04:47:53.636 INFO:tasks.workunit.client.1.smithi179.stdout:2/391: rename dc/d29/d31/d39/l55 to dc/d29/d4a/d1e/l80 0 2024-04-02T04:47:53.640 INFO:tasks.workunit.client.1.smithi179.stdout:1/558: dwrite d3/d5/de/f16 [4194304,4194304] 0 2024-04-02T04:47:53.640 INFO:tasks.workunit.client.1.smithi179.stdout:5/514: fdatasync d8/d1b/f37 0 2024-04-02T04:47:53.640 INFO:tasks.workunit.client.0.smithi070.stdout:8/522: mknod d2/d8/d29/d22/d5a/d94/ca3 0 2024-04-02T04:47:53.642 INFO:tasks.workunit.client.0.smithi070.stdout:8/523: write d2/f7f [1801740,31951] 0 2024-04-02T04:47:53.642 INFO:tasks.workunit.client.1.smithi179.stdout:8/521: creat d3/d32/d21/d45/d5a/fb5 x:0 0 0 2024-04-02T04:47:53.643 INFO:tasks.workunit.client.0.smithi070.stdout:4/462: readlink d1/d3/l34 0 2024-04-02T04:47:53.654 INFO:tasks.workunit.client.1.smithi179.stdout:2/392: dread dc/f50 [0,4194304] 0 2024-04-02T04:47:53.654 INFO:tasks.workunit.client.1.smithi179.stdout:2/393: write dc/f44 [4077929,71149] 0 2024-04-02T04:47:53.731 INFO:tasks.workunit.client.0.smithi070.stdout:7/512: rmdir d7/d4e/d82/d57 39 2024-04-02T04:47:53.733 INFO:tasks.workunit.client.1.smithi179.stdout:3/599: mkdir d8/d26/d52/d85/dbb 0 2024-04-02T04:47:53.734 INFO:tasks.workunit.client.0.smithi070.stdout:9/474: mknod d4/d10/d11/d7b/c9b 0 2024-04-02T04:47:53.736 INFO:tasks.workunit.client.1.smithi179.stdout:3/600: dread d8/f48 [0,4194304] 0 2024-04-02T04:47:53.736 INFO:tasks.workunit.client.1.smithi179.stdout:6/561: unlink d3/d12/d23/d26/d57/f99 0 2024-04-02T04:47:53.739 INFO:tasks.workunit.client.0.smithi070.stdout:5/614: dwrite d5/d9/d23/f6e [0,4194304] 0 2024-04-02T04:47:53.743 INFO:tasks.workunit.client.0.smithi070.stdout:9/475: dwrite d4/d10/d11/d7b/d19/d53/f8b [0,4194304] 0 2024-04-02T04:47:53.746 INFO:tasks.workunit.client.0.smithi070.stdout:8/524: rename d2/d8/f5e to d2/d3/d52/d62/d7c/fa4 0 2024-04-02T04:47:53.749 INFO:tasks.workunit.client.1.smithi179.stdout:0/577: mkdir d1/db/dad 0 2024-04-02T04:47:53.750 INFO:tasks.workunit.client.1.smithi179.stdout:9/565: mkdir d0/d1/d11/d29/da9 0 2024-04-02T04:47:53.753 INFO:tasks.workunit.client.1.smithi179.stdout:1/559: mknod d3/d5/d2f/d36/cc4 0 2024-04-02T04:47:53.755 INFO:tasks.workunit.client.1.smithi179.stdout:5/515: dread - d8/d1b/d23/d30/f5c zero size 2024-04-02T04:47:53.758 INFO:tasks.workunit.client.0.smithi070.stdout:5/615: dwrite d5/d17/d50/fb1 [0,4194304] 0 2024-04-02T04:47:53.758 INFO:tasks.workunit.client.0.smithi070.stdout:9/476: mkdir d4/d10/d11/d7b/d19/d53/d9c 0 2024-04-02T04:47:53.762 INFO:tasks.workunit.client.1.smithi179.stdout:7/610: dread d1/d9/d55/f64 [0,4194304] 0 2024-04-02T04:47:53.762 INFO:tasks.workunit.client.1.smithi179.stdout:3/601: mkdir d8/d26/d83/dbc 0 2024-04-02T04:47:53.762 INFO:tasks.workunit.client.1.smithi179.stdout:6/562: creat d3/d12/d23/d26/d57/fc3 x:0 0 0 2024-04-02T04:47:53.766 INFO:tasks.workunit.client.1.smithi179.stdout:9/566: unlink d0/d19/c91 0 2024-04-02T04:47:53.768 INFO:tasks.workunit.client.1.smithi179.stdout:8/522: dwrite d3/d32/d21/f37 [4194304,4194304] 0 2024-04-02T04:47:53.769 INFO:tasks.workunit.client.0.smithi070.stdout:9/477: symlink d4/d10/l9d 0 2024-04-02T04:47:53.769 INFO:tasks.workunit.client.1.smithi179.stdout:3/602: write d8/d26/d83/d89/fab [1125059,30339] 0 2024-04-02T04:47:53.771 INFO:tasks.workunit.client.1.smithi179.stdout:1/560: rmdir d3/d5/d2f/d4f/d42/d53/d77 39 2024-04-02T04:47:53.772 INFO:tasks.workunit.client.1.smithi179.stdout:2/394: dwrite dc/d29/d4a/f67 [0,4194304] 0 2024-04-02T04:47:53.775 INFO:tasks.workunit.client.0.smithi070.stdout:7/513: write d7/dd/df/d7a/d6c/f6d [2130688,63155] 0 2024-04-02T04:47:53.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:53 smithi070 ceph-mon[29212]: pgmap v15: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 59 MiB/s rd, 231 MiB/s wr, 407 op/s 2024-04-02T04:47:53.781 INFO:tasks.workunit.client.0.smithi070.stdout:9/478: dread d4/d13/f37 [4194304,4194304] 0 2024-04-02T04:47:53.782 INFO:tasks.workunit.client.1.smithi179.stdout:2/395: chown dc/d29/d4a/d1e/c35 4022 1 2024-04-02T04:47:53.784 INFO:tasks.workunit.client.0.smithi070.stdout:7/514: creat d7/dd/df/d79/d46/d65/fa4 x:0 0 0 2024-04-02T04:47:53.785 INFO:tasks.workunit.client.1.smithi179.stdout:6/563: dwrite d3/d12/d23/d26/d57/f9e [0,4194304] 0 2024-04-02T04:47:53.785 INFO:tasks.workunit.client.0.smithi070.stdout:7/515: fdatasync d7/dd/df/f94 0 2024-04-02T04:47:53.788 INFO:tasks.workunit.client.0.smithi070.stdout:8/525: fsync d2/d8/d23/d24/d66/f6a 0 2024-04-02T04:47:53.788 INFO:tasks.workunit.client.0.smithi070.stdout:4/463: write d1/fd [221702,59203] 0 2024-04-02T04:47:53.790 INFO:tasks.workunit.client.0.smithi070.stdout:9/479: truncate d4/d13/f26 818644 0 2024-04-02T04:47:53.792 INFO:tasks.workunit.client.1.smithi179.stdout:4/609: dwrite d1/d3/dc/f18 [0,4194304] 0 2024-04-02T04:47:53.792 INFO:tasks.workunit.client.0.smithi070.stdout:2/534: sync 2024-04-02T04:47:53.792 INFO:tasks.workunit.client.1.smithi179.stdout:3/603: dwrite d8/d26/d36/f90 [0,4194304] 0 2024-04-02T04:47:53.798 INFO:tasks.workunit.client.1.smithi179.stdout:4/610: chown d1/fb 48 1 2024-04-02T04:47:53.801 INFO:tasks.workunit.client.0.smithi070.stdout:5/616: truncate d5/d9/d10/d51/fa3 132778 0 2024-04-02T04:47:53.809 INFO:tasks.workunit.client.0.smithi070.stdout:9/480: rename d4/d10/d11/d7b/d19/d2f/d39/d4f to d4/d10/d12/d27/d60/d9e 0 2024-04-02T04:47:53.811 INFO:tasks.workunit.client.0.smithi070.stdout:9/481: write d4/d10/f18 [411594,15304] 0 2024-04-02T04:47:53.811 INFO:tasks.workunit.client.0.smithi070.stdout:8/526: dread d2/d8/d23/d24/f5b [0,4194304] 0 2024-04-02T04:47:53.813 INFO:tasks.workunit.client.0.smithi070.stdout:2/535: creat d2/d10/d21/d71/fae x:0 0 0 2024-04-02T04:47:53.813 INFO:tasks.workunit.client.0.smithi070.stdout:8/527: read - d2/d8/d23/d24/d79/fa0 zero size 2024-04-02T04:47:53.814 INFO:tasks.workunit.client.1.smithi179.stdout:0/578: dread d1/da/d48/d65/d7d/d80/fa5 [0,4194304] 0 2024-04-02T04:47:53.815 INFO:tasks.workunit.client.0.smithi070.stdout:8/528: chown d2/d3/d35/d8e 61740651 1 2024-04-02T04:47:53.815 INFO:tasks.workunit.client.1.smithi179.stdout:0/579: fsync d1/da/d67/fa8 0 2024-04-02T04:47:53.816 INFO:tasks.workunit.client.0.smithi070.stdout:9/482: read d4/d10/d12/d27/d83/f8f [733823,117189] 0 2024-04-02T04:47:53.819 INFO:tasks.workunit.client.1.smithi179.stdout:7/611: truncate d1/dd/d23/dab/fba 305489 0 2024-04-02T04:47:53.820 INFO:tasks.workunit.client.1.smithi179.stdout:8/523: getdents d3/d32/d21/d50/d56/d94/d99 0 2024-04-02T04:47:53.821 INFO:tasks.workunit.client.0.smithi070.stdout:5/617: fdatasync d5/fa 0 2024-04-02T04:47:53.822 INFO:tasks.workunit.client.0.smithi070.stdout:5/618: chown d5/d17/d50/fb1 10147 1 2024-04-02T04:47:53.823 INFO:tasks.workunit.client.1.smithi179.stdout:8/524: chown d3/d32/d21/d50/d56 14829998 1 2024-04-02T04:47:53.824 INFO:tasks.workunit.client.1.smithi179.stdout:0/580: dwrite d1/da/d67/fa8 [0,4194304] 0 2024-04-02T04:47:53.825 INFO:tasks.workunit.client.1.smithi179.stdout:8/525: fdatasync d3/d32/d21/d50/d65/f9b 0 2024-04-02T04:47:53.827 INFO:tasks.workunit.client.1.smithi179.stdout:8/526: write d3/f76 [6712982,129115] 0 2024-04-02T04:47:53.829 INFO:tasks.workunit.client.1.smithi179.stdout:5/516: write d8/d1b/d23/d32/d4e/f59 [1060502,78787] 0 2024-04-02T04:47:53.831 INFO:tasks.workunit.client.1.smithi179.stdout:1/561: unlink d3/d5/de/c10 0 2024-04-02T04:47:53.833 INFO:tasks.workunit.client.1.smithi179.stdout:1/562: readlink d3/d5/d2f/d4f/l7a 0 2024-04-02T04:47:53.834 INFO:tasks.workunit.client.1.smithi179.stdout:9/567: creat d0/d1/d11/d29/d40/faa x:0 0 0 2024-04-02T04:47:53.838 INFO:tasks.workunit.client.1.smithi179.stdout:9/568: fdatasync d0/d1/f44 0 2024-04-02T04:47:53.838 INFO:tasks.workunit.client.1.smithi179.stdout:5/517: dwrite d8/d1b/d23/d36/d53/f78 [0,4194304] 0 2024-04-02T04:47:53.839 INFO:tasks.workunit.client.1.smithi179.stdout:7/612: read d1/dd/d16/f48 [4059268,119794] 0 2024-04-02T04:47:53.843 INFO:tasks.workunit.client.1.smithi179.stdout:2/396: truncate dc/d29/d4a/d1e/d43/f47 2843493 0 2024-04-02T04:47:53.849 INFO:tasks.workunit.client.0.smithi070.stdout:7/516: dread d7/d9/f1b [0,4194304] 0 2024-04-02T04:47:53.855 INFO:tasks.workunit.client.0.smithi070.stdout:7/517: dwrite d7/dd/df/f83 [0,4194304] 0 2024-04-02T04:47:53.877 INFO:tasks.workunit.client.0.smithi070.stdout:6/452: sync 2024-04-02T04:47:53.877 INFO:tasks.workunit.client.0.smithi070.stdout:0/610: sync 2024-04-02T04:47:53.877 INFO:tasks.workunit.client.0.smithi070.stdout:1/571: sync 2024-04-02T04:47:53.877 INFO:tasks.workunit.client.0.smithi070.stdout:3/554: sync 2024-04-02T04:47:53.877 INFO:tasks.workunit.client.0.smithi070.stdout:2/536: readlink d2/d10/d5c/d7a/l6a 0 2024-04-02T04:47:53.879 INFO:tasks.workunit.client.0.smithi070.stdout:4/464: creat d1/d3/d9/f9e x:0 0 0 2024-04-02T04:47:53.881 INFO:tasks.workunit.client.0.smithi070.stdout:5/619: creat d5/d57/fc1 x:0 0 0 2024-04-02T04:47:53.882 INFO:tasks.workunit.client.0.smithi070.stdout:2/537: write d2/d10/d21/d2b/f7d [561053,15465] 0 2024-04-02T04:47:53.882 INFO:tasks.workunit.client.0.smithi070.stdout:8/529: write d2/d8/f6d [1045551,44596] 0 2024-04-02T04:47:53.885 INFO:tasks.workunit.client.0.smithi070.stdout:7/518: write d7/dd/df/d7a/f64 [3788790,115875] 0 2024-04-02T04:47:53.885 INFO:tasks.workunit.client.0.smithi070.stdout:0/611: dread d4/d5/d21/d6b/f7b [0,4194304] 0 2024-04-02T04:47:53.888 INFO:tasks.workunit.client.0.smithi070.stdout:6/453: dread d0/d55/d5c/f60 [0,4194304] 0 2024-04-02T04:47:53.889 INFO:tasks.workunit.client.0.smithi070.stdout:1/572: creat d0/d4c/fb7 x:0 0 0 2024-04-02T04:47:53.892 INFO:tasks.workunit.client.0.smithi070.stdout:3/555: dread f0 [0,4194304] 0 2024-04-02T04:47:53.892 INFO:tasks.workunit.client.0.smithi070.stdout:8/530: rename d2/d8 to d2/d8/da5 22 2024-04-02T04:47:53.893 INFO:tasks.workunit.client.0.smithi070.stdout:4/465: creat d1/d9c/d65/d7a/f9f x:0 0 0 2024-04-02T04:47:53.895 INFO:tasks.workunit.client.0.smithi070.stdout:2/538: mknod d2/d10/d21/d71/caf 0 2024-04-02T04:47:53.898 INFO:tasks.workunit.client.0.smithi070.stdout:5/620: truncate d5/d17/d2b/f89 272988 0 2024-04-02T04:47:53.901 INFO:tasks.workunit.client.0.smithi070.stdout:6/454: truncate d0/d20/f34 938096 0 2024-04-02T04:47:53.901 INFO:tasks.workunit.client.0.smithi070.stdout:4/466: dwrite d1/f93 [0,4194304] 0 2024-04-02T04:47:53.901 INFO:tasks.workunit.client.0.smithi070.stdout:6/455: stat d0/l12 0 2024-04-02T04:47:53.902 INFO:tasks.workunit.client.0.smithi070.stdout:4/467: fdatasync d1/d9c/d65/d7a/f7d 0 2024-04-02T04:47:53.904 INFO:tasks.workunit.client.0.smithi070.stdout:3/556: fsync d10/d17/d4c/f84 0 2024-04-02T04:47:53.904 INFO:tasks.workunit.client.0.smithi070.stdout:4/468: chown d1/l9d 22 1 2024-04-02T04:47:53.906 INFO:tasks.workunit.client.0.smithi070.stdout:4/469: dread - d1/d3/f95 zero size 2024-04-02T04:47:53.909 INFO:tasks.workunit.client.0.smithi070.stdout:9/483: getdents d4/d10/d11/d7b/d19/d53 0 2024-04-02T04:47:53.912 INFO:tasks.workunit.client.0.smithi070.stdout:5/621: unlink d5/d9/d23/d8b/cbe 0 2024-04-02T04:47:53.913 INFO:tasks.workunit.client.0.smithi070.stdout:9/484: write d4/d10/d11/d7b/d19/f54 [2396945,73291] 0 2024-04-02T04:47:53.913 INFO:tasks.workunit.client.0.smithi070.stdout:1/573: symlink d0/d9/d1d/d22/d2c/d4d/lb8 0 2024-04-02T04:47:53.914 INFO:tasks.workunit.client.0.smithi070.stdout:6/456: dwrite d0/d20/f7e [0,4194304] 0 2024-04-02T04:47:53.916 INFO:tasks.workunit.client.0.smithi070.stdout:5/622: write d5/d17/d50/f5a [8034757,66679] 0 2024-04-02T04:47:53.918 INFO:tasks.workunit.client.0.smithi070.stdout:3/557: mknod d10/d17/d51/cc2 0 2024-04-02T04:47:53.918 INFO:tasks.workunit.client.1.smithi179.stdout:2/397: fdatasync dc/fe 0 2024-04-02T04:47:53.919 INFO:tasks.workunit.client.1.smithi179.stdout:3/604: mknod d8/d26/d52/d85/dbb/cbd 0 2024-04-02T04:47:53.922 INFO:tasks.workunit.client.1.smithi179.stdout:5/518: creat d8/f9e x:0 0 0 2024-04-02T04:47:53.923 INFO:tasks.workunit.client.1.smithi179.stdout:5/519: chown d8/d7e 86156978 1 2024-04-02T04:47:53.924 INFO:tasks.workunit.client.1.smithi179.stdout:7/613: read d1/d9/d35/d6b/f73 [2380374,36409] 0 2024-04-02T04:47:53.924 INFO:tasks.workunit.client.0.smithi070.stdout:8/531: write d2/d3/f53 [742943,68560] 0 2024-04-02T04:47:53.926 INFO:tasks.workunit.client.1.smithi179.stdout:0/581: fsync d1/da/df/f21 0 2024-04-02T04:47:53.926 INFO:tasks.workunit.client.0.smithi070.stdout:0/612: dwrite d4/d5/d58/d60/f92 [0,4194304] 0 2024-04-02T04:47:53.927 INFO:tasks.workunit.client.1.smithi179.stdout:7/614: chown d1/dd/d23/c7d 61864544 1 2024-04-02T04:47:53.927 INFO:tasks.workunit.client.0.smithi070.stdout:6/457: dread d0/d20/d5f/f6e [0,4194304] 0 2024-04-02T04:47:53.927 INFO:tasks.workunit.client.1.smithi179.stdout:3/605: creat d8/d26/d52/d85/dbb/fbe x:0 0 0 2024-04-02T04:47:53.928 INFO:tasks.workunit.client.1.smithi179.stdout:7/615: fdatasync d1/d9/d27/d87/d8b/d75/db9/f92 0 2024-04-02T04:47:53.930 INFO:tasks.workunit.client.1.smithi179.stdout:2/398: unlink dc/d29/d4a/d1e/c1f 0 2024-04-02T04:47:53.935 INFO:tasks.workunit.client.1.smithi179.stdout:0/582: dwrite d1/da/d48/d5c/d83/f8b [4194304,4194304] 0 2024-04-02T04:47:53.935 INFO:tasks.workunit.client.1.smithi179.stdout:8/527: creat d3/d32/d21/d50/d56/d94/d99/fb6 x:0 0 0 2024-04-02T04:47:53.936 INFO:tasks.workunit.client.1.smithi179.stdout:9/569: mkdir d0/d1/d11/dab 0 2024-04-02T04:47:53.937 INFO:tasks.workunit.client.1.smithi179.stdout:4/611: dread d1/d3/dc/d27/d62/f69 [0,4194304] 0 2024-04-02T04:47:53.937 INFO:tasks.workunit.client.1.smithi179.stdout:0/583: chown d1/da/df/d56/d84 0 1 2024-04-02T04:47:53.939 INFO:tasks.workunit.client.0.smithi070.stdout:1/574: rename d0/d9/d1d/d22/d2c/d4d to d0/d7/d1a/db9 0 2024-04-02T04:47:53.940 INFO:tasks.workunit.client.1.smithi179.stdout:4/612: dread - d1/d3/df/d10/d2f/d60/da6/fb6 zero size 2024-04-02T04:47:53.940 INFO:tasks.workunit.client.1.smithi179.stdout:9/570: chown d0/d1/d11/d29/d40/f63 4 1 2024-04-02T04:47:53.945 INFO:tasks.workunit.client.0.smithi070.stdout:2/539: dread d2/d10/d21/d71/f76 [0,4194304] 0 2024-04-02T04:47:53.946 INFO:tasks.workunit.client.0.smithi070.stdout:2/540: fdatasync d2/d10/d21/d3d/d42/fad 0 2024-04-02T04:47:53.948 INFO:tasks.workunit.client.0.smithi070.stdout:8/532: chown d2/d8/d29/d22/d5a/d77 7216 1 2024-04-02T04:47:53.950 INFO:tasks.workunit.client.1.smithi179.stdout:5/520: dread d8/d1b/d23/d36/f41 [0,4194304] 0 2024-04-02T04:47:53.950 INFO:tasks.workunit.client.0.smithi070.stdout:2/541: fdatasync d2/d10/d5c/d7a/f92 0 2024-04-02T04:47:53.951 INFO:tasks.workunit.client.1.smithi179.stdout:3/606: rmdir d8/d26/d52/d7e/db8 39 2024-04-02T04:47:53.952 INFO:tasks.workunit.client.1.smithi179.stdout:3/607: fdatasync d8/d57/fb3 0 2024-04-02T04:47:53.952 INFO:tasks.workunit.client.0.smithi070.stdout:4/470: dwrite d1/d3/f54 [0,4194304] 0 2024-04-02T04:47:53.952 INFO:tasks.workunit.client.0.smithi070.stdout:9/485: fdatasync d4/d10/d11/d7b/d19/f24 0 2024-04-02T04:47:53.955 INFO:tasks.workunit.client.1.smithi179.stdout:7/616: creat d1/d9/d35/fdc x:0 0 0 2024-04-02T04:47:53.956 INFO:tasks.workunit.client.1.smithi179.stdout:7/617: chown d1/dd/d16/l79 109619828 1 2024-04-02T04:47:53.957 INFO:tasks.workunit.client.0.smithi070.stdout:4/471: dread d1/d3/d9/f81 [0,4194304] 0 2024-04-02T04:47:53.957 INFO:tasks.workunit.client.0.smithi070.stdout:4/472: readlink d1/d3/l8a 0 2024-04-02T04:47:53.959 INFO:tasks.workunit.client.0.smithi070.stdout:3/558: symlink d10/d17/db2/dbf/lc3 0 2024-04-02T04:47:53.960 INFO:tasks.workunit.client.0.smithi070.stdout:3/559: write d10/d32/f59 [4602445,128920] 0 2024-04-02T04:47:53.963 INFO:tasks.workunit.client.1.smithi179.stdout:0/584: creat d1/da/d28/d81/fae x:0 0 0 2024-04-02T04:47:53.964 INFO:tasks.workunit.client.1.smithi179.stdout:0/585: write d1/f2a [5333494,105727] 0 2024-04-02T04:47:53.967 INFO:tasks.workunit.client.1.smithi179.stdout:9/571: unlink d0/d19/d27/l3b 0 2024-04-02T04:47:53.973 INFO:tasks.workunit.client.0.smithi070.stdout:2/542: dread d2/d10/d21/d3d/f47 [0,4194304] 0 2024-04-02T04:47:53.974 INFO:tasks.workunit.client.0.smithi070.stdout:0/613: creat d4/d5/d13/d2a/d8d/fb2 x:0 0 0 2024-04-02T04:47:53.976 INFO:tasks.workunit.client.1.smithi179.stdout:5/521: rename d8/l16 to d8/d1b/d23/d36/d53/d89/l9f 0 2024-04-02T04:47:53.983 INFO:tasks.workunit.client.0.smithi070.stdout:5/623: creat d5/d9/d10/fc2 x:0 0 0 2024-04-02T04:47:53.985 INFO:tasks.workunit.client.0.smithi070.stdout:5/624: read - d5/d9/d10/d6a/fc0 zero size 2024-04-02T04:47:53.985 INFO:tasks.workunit.client.0.smithi070.stdout:4/473: rmdir d1/d9c/d65/d7a 39 2024-04-02T04:47:53.987 INFO:tasks.workunit.client.0.smithi070.stdout:5/625: truncate d5/d9/d23/d8b/fa5 48861 0 2024-04-02T04:47:53.987 INFO:tasks.workunit.client.1.smithi179.stdout:0/586: fdatasync d1/da/d28/d37/f3f 0 2024-04-02T04:47:53.989 INFO:tasks.workunit.client.1.smithi179.stdout:9/572: mknod d0/d1/d10/d6d/d58/cac 0 2024-04-02T04:47:53.991 INFO:tasks.workunit.client.0.smithi070.stdout:3/560: dread d10/d17/d51/f64 [4194304,4194304] 0 2024-04-02T04:47:53.993 INFO:tasks.workunit.client.1.smithi179.stdout:0/587: dwrite d1/db/f2c [0,4194304] 0 2024-04-02T04:47:53.994 INFO:tasks.workunit.client.1.smithi179.stdout:3/608: rename d8/d26/d36/cb1 to d8/d26/d52/d85/d9a/cbf 0 2024-04-02T04:47:53.996 INFO:tasks.workunit.client.1.smithi179.stdout:0/588: write d1/da/d28/d81/fae [986383,60785] 0 2024-04-02T04:47:53.996 INFO:tasks.workunit.client.1.smithi179.stdout:3/609: fdatasync d8/d26/d52/d85/dbb/fbe 0 2024-04-02T04:47:53.997 INFO:tasks.workunit.client.1.smithi179.stdout:7/618: unlink d1/d9/f11 0 2024-04-02T04:47:53.998 INFO:tasks.workunit.client.1.smithi179.stdout:7/619: stat d1/c60 0 2024-04-02T04:47:54.001 INFO:tasks.workunit.client.1.smithi179.stdout:5/522: creat d8/d1b/d23/d32/d4e/fa0 x:0 0 0 2024-04-02T04:47:54.003 INFO:tasks.workunit.client.1.smithi179.stdout:3/610: dwrite d8/d26/d36/f5c [0,4194304] 0 2024-04-02T04:47:54.003 INFO:tasks.workunit.client.0.smithi070.stdout:2/543: mkdir d2/d10/d5c/d7a/db0 0 2024-04-02T04:47:54.006 INFO:tasks.workunit.client.0.smithi070.stdout:0/614: unlink d4/d5/d58/d60/f7c 0 2024-04-02T04:47:54.007 INFO:tasks.workunit.client.0.smithi070.stdout:9/486: write d4/fc [1617706,115859] 0 2024-04-02T04:47:54.008 INFO:tasks.workunit.client.1.smithi179.stdout:8/528: link d3/d32/d21/d49/l81 d3/d32/d21/d51/d70/lb7 0 2024-04-02T04:47:54.009 INFO:tasks.workunit.client.1.smithi179.stdout:8/529: chown d3/d1b/fae 60 1 2024-04-02T04:47:54.019 INFO:tasks.workunit.client.0.smithi070.stdout:5/626: mkdir d5/d9/d23/dc3 0 2024-04-02T04:47:54.019 INFO:tasks.workunit.client.0.smithi070.stdout:4/474: dread d1/f26 [0,4194304] 0 2024-04-02T04:47:54.020 INFO:tasks.workunit.client.0.smithi070.stdout:8/533: write d2/d3/f7 [400936,95299] 0 2024-04-02T04:47:54.021 INFO:tasks.workunit.client.0.smithi070.stdout:5/627: chown d5/d17/d27/c39 67863 1 2024-04-02T04:47:54.022 INFO:tasks.workunit.client.0.smithi070.stdout:8/534: chown d2/d8/d29/d22/d5a/d94/ca3 15353979 1 2024-04-02T04:47:54.022 INFO:tasks.workunit.client.0.smithi070.stdout:5/628: dread - d5/d9/d10/d6a/fc0 zero size 2024-04-02T04:47:54.023 INFO:tasks.workunit.client.1.smithi179.stdout:9/573: rename d0/d1/d11/d29/da9 to d0/d19/d27/dad 0 2024-04-02T04:47:54.024 INFO:tasks.workunit.client.0.smithi070.stdout:3/561: mknod d10/d66/cc4 0 2024-04-02T04:47:54.027 INFO:tasks.workunit.client.0.smithi070.stdout:6/458: getdents d0/d20/d5f 0 2024-04-02T04:47:54.028 INFO:tasks.workunit.client.0.smithi070.stdout:6/459: write d0/d3/d19/d31/d61/f66 [3345468,92618] 0 2024-04-02T04:47:54.029 INFO:tasks.workunit.client.1.smithi179.stdout:7/620: mkdir d1/d9/d27/d87/d8b/d75/db9/ddd 0 2024-04-02T04:47:54.031 INFO:tasks.workunit.client.1.smithi179.stdout:4/613: truncate d1/d3/df/d10/d2f/d36/f58 3235108 0 2024-04-02T04:47:54.033 INFO:tasks.workunit.client.0.smithi070.stdout:2/544: creat d2/d10/d21/d3d/d42/fb1 x:0 0 0 2024-04-02T04:47:54.034 INFO:tasks.workunit.client.0.smithi070.stdout:2/545: stat d2/d10/d5c/c84 0 2024-04-02T04:47:54.035 INFO:tasks.workunit.client.1.smithi179.stdout:2/399: getdents dc/d29/d31 0 2024-04-02T04:47:54.039 INFO:tasks.workunit.client.0.smithi070.stdout:1/575: getdents d0/d9/d1d/d22/d2c/d4f 0 2024-04-02T04:47:54.040 INFO:tasks.workunit.client.1.smithi179.stdout:0/589: dwrite d1/f62 [0,4194304] 0 2024-04-02T04:47:54.040 INFO:tasks.workunit.client.0.smithi070.stdout:4/475: dread - d1/d3/d9/f43 zero size 2024-04-02T04:47:54.043 INFO:tasks.workunit.client.0.smithi070.stdout:0/615: dwrite d4/d5/d13/d2a/f68 [0,4194304] 0 2024-04-02T04:47:54.043 INFO:tasks.workunit.client.1.smithi179.stdout:2/400: dwrite dc/d29/d4a/d1e/f63 [0,4194304] 0 2024-04-02T04:47:54.044 INFO:tasks.workunit.client.1.smithi179.stdout:2/401: write dc/d29/d4a/f3f [415328,37687] 0 2024-04-02T04:47:54.045 INFO:tasks.workunit.client.0.smithi070.stdout:0/616: write d4/d5/d13/d2a/d3a/d80/f9c [1417011,46138] 0 2024-04-02T04:47:54.046 INFO:tasks.workunit.client.0.smithi070.stdout:0/617: chown d4/d5/d13/d2a/d3a/c3e 7003 1 2024-04-02T04:47:54.048 INFO:tasks.workunit.client.0.smithi070.stdout:5/629: unlink d5/d9/d10/d6a/d7d/c94 0 2024-04-02T04:47:54.057 INFO:tasks.workunit.client.0.smithi070.stdout:9/487: dwrite d4/d10/d11/d7b/d19/f24 [0,4194304] 0 2024-04-02T04:47:54.064 INFO:tasks.workunit.client.0.smithi070.stdout:8/535: dwrite d2/d3/f61 [4194304,4194304] 0 2024-04-02T04:47:54.067 INFO:tasks.workunit.client.0.smithi070.stdout:0/618: mkdir d4/d5/d13/d2a/d3a/db3 0 2024-04-02T04:47:54.072 INFO:tasks.workunit.client.0.smithi070.stdout:6/460: symlink d0/d55/d8b/l92 0 2024-04-02T04:47:54.074 INFO:tasks.workunit.client.0.smithi070.stdout:3/562: write d10/f29 [24922,37753] 0 2024-04-02T04:47:54.077 INFO:tasks.workunit.client.0.smithi070.stdout:2/546: rename d2/d10/d21/d3d/d42/fb1 to d2/d12/d59/fb2 0 2024-04-02T04:47:54.079 INFO:tasks.workunit.client.0.smithi070.stdout:1/576: unlink d0/db4/l93 0 2024-04-02T04:47:54.081 INFO:tasks.workunit.client.0.smithi070.stdout:8/536: symlink d2/d8/d23/d24/d79/la6 0 2024-04-02T04:47:54.082 INFO:tasks.workunit.client.0.smithi070.stdout:0/619: mknod d4/d5/d21/cb4 0 2024-04-02T04:47:54.085 INFO:tasks.workunit.client.0.smithi070.stdout:6/461: chown d0/d20/c83 1528179 1 2024-04-02T04:47:54.086 INFO:tasks.workunit.client.0.smithi070.stdout:9/488: fsync d4/f8 0 2024-04-02T04:47:54.087 INFO:tasks.workunit.client.0.smithi070.stdout:3/563: mkdir d10/d32/d4b/dc5 0 2024-04-02T04:47:54.089 INFO:tasks.workunit.client.0.smithi070.stdout:2/547: truncate d2/d10/d19/f39 635812 0 2024-04-02T04:47:54.092 INFO:tasks.workunit.client.1.smithi179.stdout:3/611: mknod d8/d26/cc0 0 2024-04-02T04:47:54.093 INFO:tasks.workunit.client.0.smithi070.stdout:8/537: symlink d2/d8/d23/d24/d66/la7 0 2024-04-02T04:47:54.094 INFO:tasks.workunit.client.0.smithi070.stdout:8/538: readlink d2/d8/d29/l89 0 2024-04-02T04:47:54.095 INFO:tasks.workunit.client.1.smithi179.stdout:8/530: truncate d3/d32/d21/d50/d65/f9b 2704114 0 2024-04-02T04:47:54.096 INFO:tasks.workunit.client.1.smithi179.stdout:8/531: chown d3/d32/d21/d22/d6b 6 1 2024-04-02T04:47:54.101 INFO:tasks.workunit.client.1.smithi179.stdout:2/402: mkdir dc/d29/d31/d81 0 2024-04-02T04:47:54.102 INFO:tasks.workunit.client.1.smithi179.stdout:2/403: truncate dc/d29/d31/f41 4202087 0 2024-04-02T04:47:54.103 INFO:tasks.workunit.client.0.smithi070.stdout:2/548: rmdir d2/d10/d21/d3d 39 2024-04-02T04:47:54.107 INFO:tasks.workunit.client.0.smithi070.stdout:8/539: mknod d2/d3/d35/ca8 0 2024-04-02T04:47:54.108 INFO:tasks.workunit.client.0.smithi070.stdout:8/540: write d2/f9b [3683788,101890] 0 2024-04-02T04:47:54.109 INFO:tasks.workunit.client.0.smithi070.stdout:8/541: write d2/d3/d35/f78 [1561896,118522] 0 2024-04-02T04:47:54.110 INFO:tasks.workunit.client.1.smithi179.stdout:7/621: symlink d1/d9/d27/d87/lde 0 2024-04-02T04:47:54.112 INFO:tasks.workunit.client.1.smithi179.stdout:6/564: sync 2024-04-02T04:47:54.112 INFO:tasks.workunit.client.0.smithi070.stdout:5/630: write d5/d17/f32 [3202848,27576] 0 2024-04-02T04:47:54.113 INFO:tasks.workunit.client.0.smithi070.stdout:5/631: chown d5/d9/d10 401554821 1 2024-04-02T04:47:54.114 INFO:tasks.workunit.client.1.smithi179.stdout:3/612: unlink d8/f48 0 2024-04-02T04:47:54.121 INFO:tasks.workunit.client.1.smithi179.stdout:2/404: symlink dc/d29/d4a/d1e/d43/l82 0 2024-04-02T04:47:54.122 INFO:tasks.workunit.client.0.smithi070.stdout:9/489: write d4/d34/f4a [457472,61877] 0 2024-04-02T04:47:54.123 INFO:tasks.workunit.client.0.smithi070.stdout:9/490: chown d4/d10/d12/c38 1376321 1 2024-04-02T04:47:54.123 INFO:tasks.workunit.client.0.smithi070.stdout:5/632: mkdir d5/d9/d10/d6a/d7d/dc4 0 2024-04-02T04:47:54.126 INFO:tasks.workunit.client.1.smithi179.stdout:0/590: dwrite d1/db/f2f [0,4194304] 0 2024-04-02T04:47:54.127 INFO:tasks.workunit.client.0.smithi070.stdout:6/462: getdents d0/d3/d19/d31/d61/d7b 0 2024-04-02T04:47:54.129 INFO:tasks.workunit.client.0.smithi070.stdout:6/463: fsync d0/d3/d19/d31/d46/d6b/f70 0 2024-04-02T04:47:54.129 INFO:tasks.workunit.client.0.smithi070.stdout:9/491: creat d4/f9f x:0 0 0 2024-04-02T04:47:54.130 INFO:tasks.workunit.client.0.smithi070.stdout:0/620: truncate d4/d5/da/d2d/f66 1132335 0 2024-04-02T04:47:54.130 INFO:tasks.workunit.client.0.smithi070.stdout:1/577: dwrite d0/d7/f6c [0,4194304] 0 2024-04-02T04:47:54.133 INFO:tasks.workunit.client.1.smithi179.stdout:4/614: write d1/f42 [1038006,108247] 0 2024-04-02T04:47:54.137 INFO:tasks.workunit.client.1.smithi179.stdout:8/532: symlink d3/lb8 0 2024-04-02T04:47:54.139 INFO:tasks.workunit.client.0.smithi070.stdout:0/621: dwrite d4/d5/d13/d2a/d8d/fb2 [0,4194304] 0 2024-04-02T04:47:54.139 INFO:tasks.workunit.client.0.smithi070.stdout:8/542: write d2/d8/d23/d24/f4b [316601,119164] 0 2024-04-02T04:47:54.140 INFO:tasks.workunit.client.1.smithi179.stdout:1/563: sync 2024-04-02T04:47:54.142 INFO:tasks.workunit.client.1.smithi179.stdout:2/405: mkdir dc/d29/d31/d83 0 2024-04-02T04:47:54.142 INFO:tasks.workunit.client.1.smithi179.stdout:1/564: write d3/d5/de/d33/fc0 [327726,2710] 0 2024-04-02T04:47:54.144 INFO:tasks.workunit.client.0.smithi070.stdout:1/578: dread d0/d7/f6a [0,4194304] 0 2024-04-02T04:47:54.145 INFO:tasks.workunit.client.1.smithi179.stdout:5/523: rename d8/d1b/d23/c39 to d8/d1b/ca1 0 2024-04-02T04:47:54.146 INFO:tasks.workunit.client.1.smithi179.stdout:5/524: read d8/f3e [2736447,81156] 0 2024-04-02T04:47:54.154 INFO:tasks.workunit.client.1.smithi179.stdout:3/613: symlink d8/d26/d52/d85/lc1 0 2024-04-02T04:47:54.155 INFO:tasks.workunit.client.1.smithi179.stdout:7/622: dwrite d1/dd/d16/f2a [0,4194304] 0 2024-04-02T04:47:54.157 INFO:tasks.workunit.client.0.smithi070.stdout:5/633: rename d5/d9/d10/d6a/fc0 to d5/d17/d27/fc5 0 2024-04-02T04:47:54.159 INFO:tasks.workunit.client.0.smithi070.stdout:0/622: symlink d4/d5/lb5 0 2024-04-02T04:47:54.159 INFO:tasks.workunit.client.0.smithi070.stdout:9/492: dwrite d4/f8 [0,4194304] 0 2024-04-02T04:47:54.162 INFO:tasks.workunit.client.0.smithi070.stdout:0/623: dread - d4/d5/d13/d2a/d3a/d80/f9f zero size 2024-04-02T04:47:54.162 INFO:tasks.workunit.client.0.smithi070.stdout:9/493: dread - d4/d10/d12/d27/d60/f88 zero size 2024-04-02T04:47:54.163 INFO:tasks.workunit.client.0.smithi070.stdout:8/543: symlink d2/d3/d35/d8e/la9 0 2024-04-02T04:47:54.164 INFO:tasks.workunit.client.0.smithi070.stdout:8/544: readlink d2/d3/d35/d3a/d4f/d59/l6f 0 2024-04-02T04:47:54.167 INFO:tasks.workunit.client.1.smithi179.stdout:9/574: getdents d0/d1/d11/d29/d86 0 2024-04-02T04:47:54.168 INFO:tasks.workunit.client.0.smithi070.stdout:9/494: dread d4/d10/d11/d7b/d19/d53/f5a [0,4194304] 0 2024-04-02T04:47:54.168 INFO:tasks.workunit.client.0.smithi070.stdout:1/579: creat d0/d54/d64/d6d/db6/fba x:0 0 0 2024-04-02T04:47:54.171 INFO:tasks.workunit.client.0.smithi070.stdout:6/464: rename d0/d3/d19/d31/d37/f5a to d0/d3/d19/d31/d46/f93 0 2024-04-02T04:47:54.172 INFO:tasks.workunit.client.1.smithi179.stdout:0/591: mknod d1/da/d67/d9f/caf 0 2024-04-02T04:47:54.173 INFO:tasks.workunit.client.0.smithi070.stdout:5/634: symlink d5/d17/lc6 0 2024-04-02T04:47:54.177 INFO:tasks.workunit.client.0.smithi070.stdout:9/495: write d4/d10/d11/d7b/f35 [3660902,64503] 0 2024-04-02T04:47:54.180 INFO:tasks.workunit.client.1.smithi179.stdout:6/565: dwrite d3/d12/d23/d39/d80/f8f [0,4194304] 0 2024-04-02T04:47:54.181 INFO:tasks.workunit.client.1.smithi179.stdout:8/533: link d3/d32/d21/d50/d56/f77 d3/d32/d21/d49/fb9 0 2024-04-02T04:47:54.182 INFO:tasks.workunit.client.1.smithi179.stdout:1/565: truncate d3/d5/de/f4e 2793454 0 2024-04-02T04:47:54.186 INFO:tasks.workunit.client.1.smithi179.stdout:5/525: unlink c7 0 2024-04-02T04:47:54.187 INFO:tasks.workunit.client.1.smithi179.stdout:5/526: fdatasync d8/d1b/d23/d32/d4e/f54 0 2024-04-02T04:47:54.191 INFO:tasks.workunit.client.1.smithi179.stdout:4/615: creat d1/d3/fc5 x:0 0 0 2024-04-02T04:47:54.192 INFO:tasks.workunit.client.0.smithi070.stdout:1/580: symlink d0/d7/d28/d83/lbb 0 2024-04-02T04:47:54.195 INFO:tasks.workunit.client.0.smithi070.stdout:9/496: mknod d4/d10/d12/d27/d68/ca0 0 2024-04-02T04:47:54.197 INFO:tasks.workunit.client.1.smithi179.stdout:6/566: rmdir d3/d12/d23/d39/d4e/d66/d7c 39 2024-04-02T04:47:54.199 INFO:tasks.workunit.client.1.smithi179.stdout:1/566: fdatasync d3/d5/f25 0 2024-04-02T04:47:54.202 INFO:tasks.workunit.client.0.smithi070.stdout:9/497: symlink d4/d10/d12/d55/la1 0 2024-04-02T04:47:54.203 INFO:tasks.workunit.client.0.smithi070.stdout:6/465: write d0/d20/f45 [4719789,69732] 0 2024-04-02T04:47:54.203 INFO:tasks.workunit.client.0.smithi070.stdout:8/545: dwrite d2/d8/d23/d24/d66/f6a [0,4194304] 0 2024-04-02T04:47:54.204 INFO:tasks.workunit.client.1.smithi179.stdout:2/406: write dc/d29/d4a/f48 [597826,107121] 0 2024-04-02T04:47:54.208 INFO:tasks.workunit.client.1.smithi179.stdout:7/623: dwrite d1/d9/d27/f4b [0,4194304] 0 2024-04-02T04:47:54.210 INFO:tasks.workunit.client.1.smithi179.stdout:1/567: dwrite d3/d5/d2f/fb0 [0,4194304] 0 2024-04-02T04:47:54.233 INFO:tasks.workunit.client.1.smithi179.stdout:5/527: creat d8/d1b/d23/d36/d53/d89/fa2 x:0 0 0 2024-04-02T04:47:54.235 INFO:tasks.workunit.client.1.smithi179.stdout:9/575: dwrite d0/d1/d11/d3a/f81 [0,4194304] 0 2024-04-02T04:47:54.237 INFO:tasks.workunit.client.0.smithi070.stdout:6/466: chown d0/d3/d19/d31/c8c 0 1 2024-04-02T04:47:54.237 INFO:tasks.workunit.client.0.smithi070.stdout:9/498: dread d4/d10/d11/f14 [0,4194304] 0 2024-04-02T04:47:54.241 INFO:tasks.workunit.client.0.smithi070.stdout:8/546: dread - d2/d8/d29/d22/f60 zero size 2024-04-02T04:47:54.241 INFO:tasks.workunit.client.0.smithi070.stdout:5/635: dwrite d5/d9/d23/d30/f79 [0,4194304] 0 2024-04-02T04:47:54.243 INFO:tasks.workunit.client.0.smithi070.stdout:6/467: read d0/d20/f45 [3568839,3181] 0 2024-04-02T04:47:54.243 INFO:tasks.workunit.client.1.smithi179.stdout:3/614: unlink d8/d7a/c9f 0 2024-04-02T04:47:54.243 INFO:tasks.workunit.client.0.smithi070.stdout:1/581: write d0/d7/f43 [417273,19634] 0 2024-04-02T04:47:54.246 INFO:tasks.workunit.client.1.smithi179.stdout:6/567: unlink d3/d2d/d38/d64/fa7 0 2024-04-02T04:47:54.249 INFO:tasks.workunit.client.1.smithi179.stdout:0/592: truncate d1/f1e 3239741 0 2024-04-02T04:47:54.251 INFO:tasks.workunit.client.0.smithi070.stdout:6/468: dwrite d0/d55/d5c/f60 [0,4194304] 0 2024-04-02T04:47:54.254 INFO:tasks.workunit.client.0.smithi070.stdout:1/582: mkdir d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc 0 2024-04-02T04:47:54.256 INFO:tasks.workunit.client.1.smithi179.stdout:7/624: mknod d1/d9/d7e/cdf 0 2024-04-02T04:47:54.259 INFO:tasks.workunit.client.1.smithi179.stdout:1/568: mkdir d3/d5/d2f/d4f/d42/d55/dc5 0 2024-04-02T04:47:54.264 INFO:tasks.workunit.client.1.smithi179.stdout:5/528: mknod d8/d1b/d23/d32/d6e/ca3 0 2024-04-02T04:47:54.265 INFO:tasks.workunit.client.0.smithi070.stdout:5/636: mkdir d5/d9/d10/dc7 0 2024-04-02T04:47:54.267 INFO:tasks.workunit.client.1.smithi179.stdout:9/576: rename d0/d1/d11/d29/d40/f9a to d0/d19/d27/dad/fae 0 2024-04-02T04:47:54.269 INFO:tasks.workunit.client.0.smithi070.stdout:6/469: rename d0/d3/d19/d39 to d0/d55/d8b/d94 0 2024-04-02T04:47:54.271 INFO:tasks.workunit.client.0.smithi070.stdout:6/470: write d0/d3/d19/f1f [222350,96705] 0 2024-04-02T04:47:54.277 INFO:tasks.workunit.client.0.smithi070.stdout:9/499: creat d4/d10/d11/d7b/fa2 x:0 0 0 2024-04-02T04:47:54.278 INFO:tasks.workunit.client.0.smithi070.stdout:6/471: dwrite d0/d20/d5f/f62 [0,4194304] 0 2024-04-02T04:47:54.280 INFO:tasks.workunit.client.0.smithi070.stdout:8/547: write d2/d8/f44 [1498716,85172] 0 2024-04-02T04:47:54.283 INFO:tasks.workunit.client.0.smithi070.stdout:5/637: creat d5/d9/d10/d6a/d7d/fc8 x:0 0 0 2024-04-02T04:47:54.284 INFO:tasks.workunit.client.1.smithi179.stdout:6/568: rmdir d3/d12/d23/d39/d4e/d6c 39 2024-04-02T04:47:54.285 INFO:tasks.workunit.client.0.smithi070.stdout:1/583: dread d0/f2a [0,4194304] 0 2024-04-02T04:47:54.289 INFO:tasks.workunit.client.0.smithi070.stdout:8/548: dwrite d2/d8/f6d [0,4194304] 0 2024-04-02T04:47:54.294 INFO:tasks.workunit.client.0.smithi070.stdout:6/472: fsync d0/d20/d5f/f6e 0 2024-04-02T04:47:54.296 INFO:tasks.workunit.client.0.smithi070.stdout:9/500: mknod d4/d10/d11/d7b/d19/d2f/d39/ca3 0 2024-04-02T04:47:54.297 INFO:tasks.workunit.client.1.smithi179.stdout:0/593: fsync d1/f22 0 2024-04-02T04:47:54.302 INFO:tasks.workunit.client.0.smithi070.stdout:5/638: dread d5/d17/d50/fb1 [4194304,4194304] 0 2024-04-02T04:47:54.303 INFO:tasks.workunit.client.1.smithi179.stdout:5/529: rename d8/ff to d8/d1b/d23/d42/fa4 0 2024-04-02T04:47:54.305 INFO:tasks.workunit.client.1.smithi179.stdout:0/594: dwrite d1/da/d52/f5d [0,4194304] 0 2024-04-02T04:47:54.306 INFO:tasks.workunit.client.1.smithi179.stdout:5/530: write d8/d1b/d23/d30/d5f/f79 [993523,36267] 0 2024-04-02T04:47:54.309 INFO:tasks.workunit.client.1.smithi179.stdout:8/534: getdents d3/d32/d21/d50/d56/d94 0 2024-04-02T04:47:54.314 INFO:tasks.workunit.client.0.smithi070.stdout:5/639: link d5/d17/d6c/f41 d5/d9/fc9 0 2024-04-02T04:47:54.317 INFO:tasks.workunit.client.1.smithi179.stdout:4/616: getdents d1/d3/df/d10/d2f/d60 0 2024-04-02T04:47:54.317 INFO:tasks.workunit.client.0.smithi070.stdout:9/501: write d4/d10/f25 [1447357,125577] 0 2024-04-02T04:47:54.319 INFO:tasks.workunit.client.0.smithi070.stdout:5/640: rename d5/f97 to d5/d9/d10/d6a/d7d/fca 0 2024-04-02T04:47:54.319 INFO:tasks.workunit.client.1.smithi179.stdout:4/617: read d1/fb [4412121,73751] 0 2024-04-02T04:47:54.321 INFO:tasks.workunit.client.0.smithi070.stdout:8/549: truncate d2/d8/d23/d24/f5b 4623475 0 2024-04-02T04:47:54.321 INFO:tasks.workunit.client.1.smithi179.stdout:1/569: dwrite d3/d5/f4a [0,4194304] 0 2024-04-02T04:47:54.321 INFO:tasks.workunit.client.1.smithi179.stdout:4/618: write d1/d3/df/d10/d2f/d7b/fa2 [1385673,125061] 0 2024-04-02T04:47:54.323 INFO:tasks.workunit.client.1.smithi179.stdout:3/615: link d8/d6d/f76 d8/d26/d83/d89/d9c/fc2 0 2024-04-02T04:47:54.327 INFO:tasks.workunit.client.1.smithi179.stdout:5/531: mknod d8/d1b/d23/d36/d53/d62/ca5 0 2024-04-02T04:47:54.328 INFO:tasks.workunit.client.1.smithi179.stdout:9/577: truncate d0/d1/d10/d6d/f2d 4061874 0 2024-04-02T04:47:54.329 INFO:tasks.workunit.client.1.smithi179.stdout:9/578: write d0/d1/f44 [1987104,49836] 0 2024-04-02T04:47:54.330 INFO:tasks.workunit.client.1.smithi179.stdout:9/579: chown d0 52526 1 2024-04-02T04:47:54.330 INFO:tasks.workunit.client.1.smithi179.stdout:0/595: rename d1/db/faa to d1/da/d48/d65/d7d/d80/fb0 0 2024-04-02T04:47:54.331 INFO:tasks.workunit.client.1.smithi179.stdout:8/535: chown d3/d31/l54 1871473 1 2024-04-02T04:47:54.332 INFO:tasks.workunit.client.1.smithi179.stdout:2/407: link dc/d29/l30 dc/d29/l84 0 2024-04-02T04:47:54.333 INFO:tasks.workunit.client.1.smithi179.stdout:2/408: readlink dc/d29/d4a/d1e/l59 0 2024-04-02T04:47:54.334 INFO:tasks.workunit.client.1.smithi179.stdout:7/625: creat d1/d9/d27/d87/fe0 x:0 0 0 2024-04-02T04:47:54.334 INFO:tasks.workunit.client.0.smithi070.stdout:5/641: getdents d5/d9/d10/d6a 0 2024-04-02T04:47:54.335 INFO:tasks.workunit.client.1.smithi179.stdout:7/626: dread - d1/dd/f90 zero size 2024-04-02T04:47:54.337 INFO:tasks.workunit.client.0.smithi070.stdout:8/550: rename d2/d8/l2e to d2/d8/laa 0 2024-04-02T04:47:54.339 INFO:tasks.workunit.client.0.smithi070.stdout:8/551: read d2/f48 [1069398,23942] 0 2024-04-02T04:47:54.340 INFO:tasks.workunit.client.1.smithi179.stdout:2/409: dwrite dc/d29/f34 [0,4194304] 0 2024-04-02T04:47:54.342 INFO:tasks.workunit.client.1.smithi179.stdout:2/410: write dc/d29/d31/d57/f71 [862209,85144] 0 2024-04-02T04:47:54.346 INFO:tasks.workunit.client.1.smithi179.stdout:4/619: unlink d1/d3/df/l78 0 2024-04-02T04:47:54.347 INFO:tasks.workunit.client.1.smithi179.stdout:9/580: dread d0/d1/f36 [0,4194304] 0 2024-04-02T04:47:54.347 INFO:tasks.workunit.client.0.smithi070.stdout:5/642: mkdir d5/d9/d23/dc3/dcb 0 2024-04-02T04:47:54.348 INFO:tasks.workunit.client.1.smithi179.stdout:9/581: chown d0/d19/d27/d7a/d84 321047799 1 2024-04-02T04:47:54.352 INFO:tasks.workunit.client.1.smithi179.stdout:3/616: rmdir d8/d26/d52/d85 39 2024-04-02T04:47:54.354 INFO:tasks.workunit.client.1.smithi179.stdout:6/569: creat d3/d12/fc4 x:0 0 0 2024-04-02T04:47:54.358 INFO:tasks.workunit.client.0.smithi070.stdout:8/552: chown d2/d8/d23/d24/d66/f70 66260 1 2024-04-02T04:47:54.358 INFO:tasks.workunit.client.1.smithi179.stdout:5/532: fdatasync d8/d1b/d23/d32/d4e/d50/f6f 0 2024-04-02T04:47:54.362 INFO:tasks.workunit.client.0.smithi070.stdout:5/643: dwrite d5/d17/d27/fc5 [0,4194304] 0 2024-04-02T04:47:54.362 INFO:tasks.workunit.client.1.smithi179.stdout:8/536: chown d3/c8 31281002 1 2024-04-02T04:47:54.362 INFO:tasks.workunit.client.1.smithi179.stdout:8/537: stat d3/d6c/d96 0 2024-04-02T04:47:54.365 INFO:tasks.workunit.client.1.smithi179.stdout:0/596: rename d1/da/df/f21 to d1/da/d48/d5c/d83/fb1 0 2024-04-02T04:47:54.367 INFO:tasks.workunit.client.0.smithi070.stdout:8/553: creat d2/d3/d35/d3a/d4f/fab x:0 0 0 2024-04-02T04:47:54.369 INFO:tasks.workunit.client.0.smithi070.stdout:8/554: fsync d2/d3/d52/d62/d7c/fa4 0 2024-04-02T04:47:54.371 INFO:tasks.workunit.client.1.smithi179.stdout:2/411: mknod dc/d29/d4a/d1e/d43/c85 0 2024-04-02T04:47:54.372 INFO:tasks.workunit.client.0.smithi070.stdout:8/555: fdatasync d2/d8/f6d 0 2024-04-02T04:47:54.374 INFO:tasks.workunit.client.1.smithi179.stdout:1/570: mknod d3/d5/d2f/d4f/d6a/da0/cc6 0 2024-04-02T04:47:54.374 INFO:tasks.workunit.client.1.smithi179.stdout:2/412: write dc/d29/d4a/f45 [668370,91807] 0 2024-04-02T04:47:54.374 INFO:tasks.workunit.client.1.smithi179.stdout:0/597: dread d1/db/d64/f79 [0,4194304] 0 2024-04-02T04:47:54.374 INFO:tasks.workunit.client.1.smithi179.stdout:7/627: dread d1/d9/f85 [0,4194304] 0 2024-04-02T04:47:54.376 INFO:tasks.workunit.client.1.smithi179.stdout:7/628: fsync d1/d9/fd0 0 2024-04-02T04:47:54.376 INFO:tasks.workunit.client.1.smithi179.stdout:6/570: rmdir d3/d12/d23/d2f 39 2024-04-02T04:47:54.377 INFO:tasks.workunit.client.0.smithi070.stdout:8/556: mknod d2/cac 0 2024-04-02T04:47:54.378 INFO:tasks.workunit.client.1.smithi179.stdout:5/533: fsync d8/d1b/d23/d32/d4e/d51/f66 0 2024-04-02T04:47:54.379 INFO:tasks.workunit.client.1.smithi179.stdout:3/617: write d8/d57/fb3 [332479,118845] 0 2024-04-02T04:47:54.380 INFO:tasks.workunit.client.1.smithi179.stdout:5/534: readlink d8/d1b/d23/d42/l63 0 2024-04-02T04:47:54.381 INFO:tasks.workunit.client.1.smithi179.stdout:8/538: unlink d3/d32/d21/d50/d56/c68 0 2024-04-02T04:47:54.383 INFO:tasks.workunit.client.0.smithi070.stdout:8/557: creat d2/d8/d29/d22/d5a/d94/da2/fad x:0 0 0 2024-04-02T04:47:54.384 INFO:tasks.workunit.client.1.smithi179.stdout:2/413: dwrite dc/d29/d4a/d1e/f40 [0,4194304] 0 2024-04-02T04:47:54.385 INFO:tasks.workunit.client.0.smithi070.stdout:5/644: truncate d5/d57/f5d 1887508 0 2024-04-02T04:47:54.388 INFO:tasks.workunit.client.1.smithi179.stdout:4/620: write d1/d3/df/d10/d2f/d7b/f91 [1035006,62995] 0 2024-04-02T04:47:54.388 INFO:tasks.workunit.client.0.smithi070.stdout:8/558: mknod d2/d8/d23/d24/d66/cae 0 2024-04-02T04:47:54.389 INFO:tasks.workunit.client.0.smithi070.stdout:8/559: chown d2/d3/c10 2363118 1 2024-04-02T04:47:54.390 INFO:tasks.workunit.client.1.smithi179.stdout:1/571: mknod d3/d5/de/d33/cc7 0 2024-04-02T04:47:54.391 INFO:tasks.workunit.client.1.smithi179.stdout:4/621: read d1/d3/f28 [4120831,97923] 0 2024-04-02T04:47:54.392 INFO:tasks.workunit.client.1.smithi179.stdout:2/414: dwrite dc/d29/d4a/d1e/d43/f6e [0,4194304] 0 2024-04-02T04:47:54.393 INFO:tasks.workunit.client.1.smithi179.stdout:2/415: write dc/f5d [1026607,63343] 0 2024-04-02T04:47:54.394 INFO:tasks.workunit.client.1.smithi179.stdout:2/416: chown dc/f50 3365178 1 2024-04-02T04:47:54.395 INFO:tasks.workunit.client.1.smithi179.stdout:0/598: mknod d1/da/d67/cb2 0 2024-04-02T04:47:54.396 INFO:tasks.workunit.client.0.smithi070.stdout:5/645: symlink d5/lcc 0 2024-04-02T04:47:54.397 INFO:tasks.workunit.client.0.smithi070.stdout:5/646: chown d5/d9/d10/db9 671487809 1 2024-04-02T04:47:54.399 INFO:tasks.workunit.client.1.smithi179.stdout:7/629: symlink d1/d9/d55/dbf/d9f/le1 0 2024-04-02T04:47:54.399 INFO:tasks.workunit.client.1.smithi179.stdout:7/630: dread - d1/fd2 zero size 2024-04-02T04:47:54.401 INFO:tasks.workunit.client.1.smithi179.stdout:7/631: write d1/dd/fc3 [2152048,118479] 0 2024-04-02T04:47:54.403 INFO:tasks.workunit.client.0.smithi070.stdout:5/647: unlink d5/d17/d50/fa6 0 2024-04-02T04:47:54.408 INFO:tasks.workunit.client.1.smithi179.stdout:9/582: dwrite d0/d1/d10/d6d/f25 [0,4194304] 0 2024-04-02T04:47:54.409 INFO:tasks.workunit.client.1.smithi179.stdout:9/583: write d0/d19/d1c/d64/f69 [154319,29469] 0 2024-04-02T04:47:54.410 INFO:tasks.workunit.client.0.smithi070.stdout:5/648: creat d5/d17/d27/fcd x:0 0 0 2024-04-02T04:47:54.411 INFO:tasks.workunit.client.0.smithi070.stdout:5/649: chown d5/d17/d27/d2e 2880008 1 2024-04-02T04:47:54.412 INFO:tasks.workunit.client.0.smithi070.stdout:5/650: write d5/d9/d23/f49 [397558,97950] 0 2024-04-02T04:47:54.413 INFO:tasks.workunit.client.1.smithi179.stdout:1/572: rmdir d3/d85/da2 39 2024-04-02T04:47:54.414 INFO:tasks.workunit.client.1.smithi179.stdout:0/599: symlink d1/da/df/lb3 0 2024-04-02T04:47:54.415 INFO:tasks.workunit.client.0.smithi070.stdout:5/651: creat d5/d17/d6c/fce x:0 0 0 2024-04-02T04:47:54.420 INFO:tasks.workunit.client.0.smithi070.stdout:5/652: symlink d5/lcf 0 2024-04-02T04:47:54.421 INFO:tasks.workunit.client.1.smithi179.stdout:3/618: write d8/d26/d52/d85/fa6 [1646141,3459] 0 2024-04-02T04:47:54.424 INFO:tasks.workunit.client.1.smithi179.stdout:8/539: symlink d3/d32/d21/d50/lba 0 2024-04-02T04:47:54.426 INFO:tasks.workunit.client.1.smithi179.stdout:3/619: dread d8/d26/d36/f8d [0,4194304] 0 2024-04-02T04:47:54.427 INFO:tasks.workunit.client.0.smithi070.stdout:5/653: getdents d5/d9/d10/d51 0 2024-04-02T04:47:54.427 INFO:tasks.workunit.client.1.smithi179.stdout:3/620: chown d8/d26/c74 163407 1 2024-04-02T04:47:54.428 INFO:tasks.workunit.client.1.smithi179.stdout:0/600: symlink d1/da/d28/d81/lb4 0 2024-04-02T04:47:54.429 INFO:tasks.workunit.client.0.smithi070.stdout:5/654: truncate d5/f56 5150865 0 2024-04-02T04:47:54.429 INFO:tasks.workunit.client.0.smithi070.stdout:5/655: readlink d5/d9/d10/d6a/l9c 0 2024-04-02T04:47:54.430 INFO:tasks.workunit.client.0.smithi070.stdout:5/656: fdatasync d5/ff 0 2024-04-02T04:47:54.434 INFO:tasks.workunit.client.0.smithi070.stdout:5/657: truncate d5/d17/d27/d2e/f3b 985617 0 2024-04-02T04:47:54.435 INFO:tasks.workunit.client.1.smithi179.stdout:9/584: write d0/d1/f7d [1098929,15581] 0 2024-04-02T04:47:54.437 INFO:tasks.workunit.client.1.smithi179.stdout:1/573: stat d3/d5/d2f/d4f/d42/d53/d5b/c64 0 2024-04-02T04:47:54.438 INFO:tasks.workunit.client.1.smithi179.stdout:3/621: unlink d8/d6d/f76 0 2024-04-02T04:47:54.445 INFO:tasks.workunit.client.1.smithi179.stdout:8/540: dwrite d3/d31/f69 [0,4194304] 0 2024-04-02T04:47:54.449 INFO:tasks.workunit.client.1.smithi179.stdout:0/601: creat d1/da/df/fb5 x:0 0 0 2024-04-02T04:47:54.450 INFO:tasks.workunit.client.0.smithi070.stdout:5/658: rmdir d5/d9/d23/dc3/dcb 0 2024-04-02T04:47:54.452 INFO:tasks.workunit.client.1.smithi179.stdout:6/571: rename d3/d12/d23/l32 to d3/d2d/lc5 0 2024-04-02T04:47:54.453 INFO:tasks.workunit.client.0.smithi070.stdout:7/519: sync 2024-04-02T04:47:54.454 INFO:tasks.workunit.client.0.smithi070.stdout:7/520: fdatasync d7/dd/f8d 0 2024-04-02T04:47:54.455 INFO:tasks.workunit.client.1.smithi179.stdout:7/632: link d1/d9/d27/d87/d8b/d75/db9/d8d/f8f d1/fe2 0 2024-04-02T04:47:54.455 INFO:tasks.workunit.client.0.smithi070.stdout:7/521: rename d7/dd/df to d7/dd/df/d79/d46/d65/da5 22 2024-04-02T04:47:54.456 INFO:tasks.workunit.client.1.smithi179.stdout:0/602: dwrite d1/da/d67/fa8 [0,4194304] 0 2024-04-02T04:47:54.456 INFO:tasks.workunit.client.0.smithi070.stdout:5/659: creat d5/d9/d23/dc3/fd0 x:0 0 0 2024-04-02T04:47:54.457 INFO:tasks.workunit.client.1.smithi179.stdout:5/535: getdents d8/d7e 0 2024-04-02T04:47:54.462 INFO:tasks.workunit.client.1.smithi179.stdout:6/572: dwrite d3/d12/d23/d39/d80/faa [0,4194304] 0 2024-04-02T04:47:54.463 INFO:tasks.workunit.client.1.smithi179.stdout:6/573: truncate d3/d9a/f9f 722920 0 2024-04-02T04:47:54.464 INFO:tasks.workunit.client.1.smithi179.stdout:6/574: read - d3/d12/d23/d39/d4e/d66/fc1 zero size 2024-04-02T04:47:54.465 INFO:tasks.workunit.client.0.smithi070.stdout:7/522: rename d7/dd/f2e to d7/d8e/fa6 0 2024-04-02T04:47:54.468 INFO:tasks.workunit.client.1.smithi179.stdout:3/622: mkdir d8/d57/d63/dc3 0 2024-04-02T04:47:54.468 INFO:tasks.workunit.client.0.smithi070.stdout:7/523: dread d7/dd/df/f4d [0,4194304] 0 2024-04-02T04:47:54.469 INFO:tasks.workunit.client.0.smithi070.stdout:5/660: mknod d5/d9/d10/db9/dba/cd1 0 2024-04-02T04:47:54.469 INFO:tasks.workunit.client.1.smithi179.stdout:2/417: link dc/d29/d4a/c7e dc/d29/d4a/d1e/d6c/c86 0 2024-04-02T04:47:54.469 INFO:tasks.workunit.client.0.smithi070.stdout:7/524: dread - d7/dd/f8d zero size 2024-04-02T04:47:54.477 INFO:tasks.workunit.client.1.smithi179.stdout:3/623: dread d8/d26/d83/d89/fab [0,4194304] 0 2024-04-02T04:47:54.477 INFO:tasks.workunit.client.0.smithi070.stdout:5/661: dread d5/d17/d27/d2e/f91 [0,4194304] 0 2024-04-02T04:47:54.478 INFO:tasks.workunit.client.0.smithi070.stdout:7/525: dwrite d7/d9/f21 [0,4194304] 0 2024-04-02T04:47:54.479 INFO:tasks.workunit.client.1.smithi179.stdout:0/603: creat d1/da/d48/d7b/fb6 x:0 0 0 2024-04-02T04:47:54.480 INFO:tasks.workunit.client.0.smithi070.stdout:5/662: write d5/d9/d23/f74 [4421505,34397] 0 2024-04-02T04:47:54.480 INFO:tasks.workunit.client.1.smithi179.stdout:3/624: dread d8/d57/fb3 [0,4194304] 0 2024-04-02T04:47:54.482 INFO:tasks.workunit.client.0.smithi070.stdout:5/663: write d5/d9/d23/d8b/fa5 [1074533,43546] 0 2024-04-02T04:47:54.482 INFO:tasks.workunit.client.1.smithi179.stdout:5/536: creat d8/d1b/d23/d32/d46/fa6 x:0 0 0 2024-04-02T04:47:54.483 INFO:tasks.workunit.client.1.smithi179.stdout:5/537: read d8/d1b/d23/d30/d5f/f77 [63375,97892] 0 2024-04-02T04:47:54.485 INFO:tasks.workunit.client.1.smithi179.stdout:5/538: write d8/d1b/d23/d32/d4e/f93 [606592,115965] 0 2024-04-02T04:47:54.485 INFO:tasks.workunit.client.1.smithi179.stdout:5/539: dread - d8/d1b/d23/d30/f5c zero size 2024-04-02T04:47:54.493 INFO:tasks.workunit.client.0.smithi070.stdout:7/526: mknod d7/dd/df/d7a/d6c/d77/d75/d80/ca7 0 2024-04-02T04:47:54.494 INFO:tasks.workunit.client.0.smithi070.stdout:7/527: fsync d7/dd/df/d34/d68/f9c 0 2024-04-02T04:47:54.494 INFO:tasks.workunit.client.0.smithi070.stdout:7/528: stat d7/d8e 0 2024-04-02T04:47:54.495 INFO:tasks.workunit.client.0.smithi070.stdout:5/664: mknod d5/cd2 0 2024-04-02T04:47:54.500 INFO:tasks.workunit.client.0.smithi070.stdout:7/529: mknod d7/dd/df/d79/ca8 0 2024-04-02T04:47:54.502 INFO:tasks.workunit.client.0.smithi070.stdout:7/530: write d7/dd/df/d7a/f64 [5214482,126067] 0 2024-04-02T04:47:54.503 INFO:tasks.workunit.client.0.smithi070.stdout:5/665: fdatasync d5/d57/f5d 0 2024-04-02T04:47:54.509 INFO:tasks.workunit.client.0.smithi070.stdout:5/666: dwrite d5/fb0 [0,4194304] 0 2024-04-02T04:47:54.513 INFO:tasks.workunit.client.0.smithi070.stdout:5/667: creat d5/d9/d23/d30/da2/d75/fd3 x:0 0 0 2024-04-02T04:47:54.514 INFO:tasks.workunit.client.0.smithi070.stdout:5/668: write d5/d9/d10/d6a/d7d/fc8 [93279,94481] 0 2024-04-02T04:47:54.515 INFO:tasks.workunit.client.0.smithi070.stdout:5/669: stat d5/d9/d23/f48 0 2024-04-02T04:47:54.517 INFO:tasks.workunit.client.0.smithi070.stdout:5/670: creat d5/d9/fd4 x:0 0 0 2024-04-02T04:47:54.523 INFO:tasks.workunit.client.0.smithi070.stdout:5/671: dwrite d5/d9/d23/d8b/fb3 [0,4194304] 0 2024-04-02T04:47:54.524 INFO:tasks.workunit.client.0.smithi070.stdout:7/531: dread d7/dd/df/d7a/f64 [4194304,4194304] 0 2024-04-02T04:47:54.530 INFO:tasks.workunit.client.0.smithi070.stdout:7/532: rename d7/dd/df/d79/d46/d65/d7b/f98 to d7/dd/df/fa9 0 2024-04-02T04:47:54.530 INFO:tasks.workunit.client.0.smithi070.stdout:5/672: dwrite d5/d9/d10/d6a/d7d/fc8 [0,4194304] 0 2024-04-02T04:47:54.532 INFO:tasks.workunit.client.0.smithi070.stdout:7/533: write d7/dd/df/d79/d46/d65/f67 [1073667,95706] 0 2024-04-02T04:47:54.538 INFO:tasks.workunit.client.0.smithi070.stdout:5/673: dwrite d5/d9/d10/d6a/f81 [4194304,4194304] 0 2024-04-02T04:47:54.543 INFO:tasks.workunit.client.0.smithi070.stdout:5/674: getdents d5/d17/d2b/d7b 0 2024-04-02T04:47:54.545 INFO:tasks.workunit.client.0.smithi070.stdout:5/675: write d5/d9/d23/d30/da2/d75/f72 [532948,281] 0 2024-04-02T04:47:54.546 INFO:tasks.workunit.client.0.smithi070.stdout:5/676: stat d5/d9/d10/c1b 0 2024-04-02T04:47:54.548 INFO:tasks.workunit.client.0.smithi070.stdout:5/677: creat d5/d9/d10/d6a/fd5 x:0 0 0 2024-04-02T04:47:54.552 INFO:tasks.workunit.client.0.smithi070.stdout:5/678: read d5/f56 [3334526,48296] 0 2024-04-02T04:47:54.553 INFO:tasks.workunit.client.0.smithi070.stdout:5/679: write d5/d9/d10/d6a/d7d/fc8 [1706128,100327] 0 2024-04-02T04:47:54.554 INFO:tasks.workunit.client.0.smithi070.stdout:5/680: fsync d5/d17/d27/fcd 0 2024-04-02T04:47:54.556 INFO:tasks.workunit.client.0.smithi070.stdout:5/681: symlink d5/d9/d10/db9/ld6 0 2024-04-02T04:47:54.558 INFO:tasks.workunit.client.0.smithi070.stdout:5/682: chown d5/d9/d23/d30/c44 15072 1 2024-04-02T04:47:54.558 INFO:tasks.workunit.client.0.smithi070.stdout:7/534: dread d7/dd/df/d79/d46/d65/f67 [0,4194304] 0 2024-04-02T04:47:54.560 INFO:tasks.workunit.client.0.smithi070.stdout:7/535: creat d7/dd/df/d7a/d6c/d77/d75/d80/faa x:0 0 0 2024-04-02T04:47:54.561 INFO:tasks.workunit.client.0.smithi070.stdout:7/536: stat d7/c9a 0 2024-04-02T04:47:54.564 INFO:tasks.workunit.client.0.smithi070.stdout:7/537: creat d7/dd/df/d79/d46/d65/d7b/fab x:0 0 0 2024-04-02T04:47:54.565 INFO:tasks.workunit.client.0.smithi070.stdout:5/683: dwrite d5/d57/f5d [0,4194304] 0 2024-04-02T04:47:54.566 INFO:tasks.workunit.client.0.smithi070.stdout:5/684: fdatasync d5/ff 0 2024-04-02T04:47:54.569 INFO:tasks.workunit.client.0.smithi070.stdout:5/685: rmdir d5/d17/d2b 39 2024-04-02T04:47:54.570 INFO:tasks.workunit.client.0.smithi070.stdout:5/686: write d5/d14/f16 [3301929,83057] 0 2024-04-02T04:47:54.571 INFO:tasks.workunit.client.0.smithi070.stdout:5/687: truncate d5/d9/fb2 705853 0 2024-04-02T04:47:54.571 INFO:tasks.workunit.client.0.smithi070.stdout:5/688: fdatasync d5/ff 0 2024-04-02T04:47:54.574 INFO:tasks.workunit.client.0.smithi070.stdout:7/538: truncate d7/d9/f78 126399 0 2024-04-02T04:47:54.577 INFO:tasks.workunit.client.0.smithi070.stdout:7/539: symlink d7/dd/df/d7a/d6c/d77/d54/lac 0 2024-04-02T04:47:54.578 INFO:tasks.workunit.client.0.smithi070.stdout:7/540: rmdir d7/d8e 39 2024-04-02T04:47:54.581 INFO:tasks.workunit.client.0.smithi070.stdout:7/541: symlink d7/dd/df/d79/d46/d65/lad 0 2024-04-02T04:47:54.582 INFO:tasks.workunit.client.0.smithi070.stdout:5/689: truncate d5/d9/d23/d30/f79 318283 0 2024-04-02T04:47:54.583 INFO:tasks.workunit.client.0.smithi070.stdout:7/542: creat d7/d36/fae x:0 0 0 2024-04-02T04:47:54.584 INFO:tasks.workunit.client.1.smithi179.stdout:2/418: rename dc/f7c to dc/d29/d4a/d1e/d6c/f87 0 2024-04-02T04:47:54.584 INFO:tasks.workunit.client.1.smithi179.stdout:8/541: mknod d3/d32/d21/d50/d56/dad/cbb 0 2024-04-02T04:47:54.584 INFO:tasks.workunit.client.0.smithi070.stdout:5/690: creat d5/d17/d50/fd7 x:0 0 0 2024-04-02T04:47:54.585 INFO:tasks.workunit.client.1.smithi179.stdout:7/633: truncate d1/dd/d16/d4d/fb6 1112447 0 2024-04-02T04:47:54.586 INFO:tasks.workunit.client.1.smithi179.stdout:1/574: creat d3/d5/fc8 x:0 0 0 2024-04-02T04:47:54.586 INFO:tasks.workunit.client.1.smithi179.stdout:8/542: dread - d3/d32/d21/d50/d56/d94/d99/fb6 zero size 2024-04-02T04:47:54.586 INFO:tasks.workunit.client.1.smithi179.stdout:5/540: symlink d8/d1b/d23/d42/la7 0 2024-04-02T04:47:54.587 INFO:tasks.workunit.client.0.smithi070.stdout:5/691: mkdir d5/d14/dd8 0 2024-04-02T04:47:54.587 INFO:tasks.workunit.client.1.smithi179.stdout:7/634: readlink d1/dd/l88 0 2024-04-02T04:47:54.588 INFO:tasks.workunit.client.0.smithi070.stdout:5/692: chown d5/d9/d23/d30/da2/d75/f72 2 1 2024-04-02T04:47:54.590 INFO:tasks.workunit.client.0.smithi070.stdout:5/693: unlink d5/d17/d27/c39 0 2024-04-02T04:47:54.593 INFO:tasks.workunit.client.0.smithi070.stdout:7/543: link d7/dd/df/d79/c3d d7/dd/caf 0 2024-04-02T04:47:54.601 INFO:tasks.workunit.client.0.smithi070.stdout:7/544: dwrite d7/dd/df/d7a/d6c/f9b [0,4194304] 0 2024-04-02T04:47:54.605 INFO:tasks.workunit.client.0.smithi070.stdout:7/545: creat d7/dd/df/d7a/d6c/d77/fb0 x:0 0 0 2024-04-02T04:47:54.608 INFO:tasks.workunit.client.0.smithi070.stdout:7/546: dread d7/dd/df/d79/d46/d65/f67 [0,4194304] 0 2024-04-02T04:47:54.610 INFO:tasks.workunit.client.0.smithi070.stdout:7/547: read d7/dd/df/d79/d46/d65/f8c [2357093,69535] 0 2024-04-02T04:47:54.612 INFO:tasks.workunit.client.0.smithi070.stdout:7/548: mkdir d7/dd/db1 0 2024-04-02T04:47:54.613 INFO:tasks.workunit.client.0.smithi070.stdout:5/694: dread d5/d9/d23/f48 [0,4194304] 0 2024-04-02T04:47:54.613 INFO:tasks.workunit.client.0.smithi070.stdout:7/549: read - d7/d36/fae zero size 2024-04-02T04:47:54.620 INFO:tasks.workunit.client.0.smithi070.stdout:7/550: write d7/dd/df/d34/f4a [3280585,107754] 0 2024-04-02T04:47:54.621 INFO:tasks.workunit.client.0.smithi070.stdout:7/551: fsync d7/dd/df/d79/d46/d65/d7b/fab 0 2024-04-02T04:47:54.624 INFO:tasks.workunit.client.0.smithi070.stdout:7/552: creat d7/dd/df/d79/d46/fb2 x:0 0 0 2024-04-02T04:47:54.626 INFO:tasks.workunit.client.0.smithi070.stdout:7/553: readlink d7/dd/df/d7a/d6c/d77/d54/l6b 0 2024-04-02T04:47:54.629 INFO:tasks.workunit.client.0.smithi070.stdout:7/554: creat d7/dd/df/d7a/d6c/d77/d75/fb3 x:0 0 0 2024-04-02T04:47:54.633 INFO:tasks.workunit.client.0.smithi070.stdout:5/695: dread d5/d17/d2b/f8d [0,4194304] 0 2024-04-02T04:47:54.633 INFO:tasks.workunit.client.0.smithi070.stdout:7/555: rename d7/dd/df/d34/f4a to d7/dd/df/d79/d46/d65/d7b/fb4 0 2024-04-02T04:47:54.635 INFO:tasks.workunit.client.0.smithi070.stdout:5/696: mkdir d5/d9/d10/db9/dd9 0 2024-04-02T04:47:54.636 INFO:tasks.workunit.client.0.smithi070.stdout:7/556: mkdir d7/dd/db5 0 2024-04-02T04:47:54.636 INFO:tasks.workunit.client.0.smithi070.stdout:5/697: creat d5/d14/fda x:0 0 0 2024-04-02T04:47:54.637 INFO:tasks.workunit.client.0.smithi070.stdout:5/698: write d5/d9/d23/fb4 [572114,82276] 0 2024-04-02T04:47:54.638 INFO:tasks.workunit.client.0.smithi070.stdout:7/557: truncate d7/d36/f72 621505 0 2024-04-02T04:47:54.641 INFO:tasks.workunit.client.0.smithi070.stdout:5/699: getdents d5/d9/d10/db9/dd9 0 2024-04-02T04:47:54.643 INFO:tasks.workunit.client.0.smithi070.stdout:5/700: mkdir d5/d14/dd8/ddb 0 2024-04-02T04:47:54.645 INFO:tasks.workunit.client.0.smithi070.stdout:5/701: creat d5/d14/dd8/ddb/fdc x:0 0 0 2024-04-02T04:47:54.646 INFO:tasks.workunit.client.0.smithi070.stdout:5/702: fdatasync d5/fb0 0 2024-04-02T04:47:54.647 INFO:tasks.workunit.client.0.smithi070.stdout:5/703: chown d5/d9/d23/c4a 46163 1 2024-04-02T04:47:54.647 INFO:tasks.workunit.client.1.smithi179.stdout:3/625: write d8/f1b [4072825,44171] 0 2024-04-02T04:47:54.647 INFO:tasks.workunit.client.1.smithi179.stdout:8/543: fsync d3/d6c/f89 0 2024-04-02T04:47:54.652 INFO:tasks.workunit.client.1.smithi179.stdout:2/419: dwrite dc/d29/d4a/f53 [0,4194304] 0 2024-04-02T04:47:54.653 INFO:tasks.workunit.client.1.smithi179.stdout:1/575: dread d3/d5/f4a [0,4194304] 0 2024-04-02T04:47:54.656 INFO:tasks.workunit.client.0.smithi070.stdout:7/558: dwrite d7/dd/df/d79/f5e [0,4194304] 0 2024-04-02T04:47:54.657 INFO:tasks.workunit.client.1.smithi179.stdout:8/544: mkdir d3/d32/d21/d50/d56/dad/dbc 0 2024-04-02T04:47:54.658 INFO:tasks.workunit.client.1.smithi179.stdout:0/604: dread d1/f1e [0,4194304] 0 2024-04-02T04:47:54.659 INFO:tasks.workunit.client.0.smithi070.stdout:5/704: creat d5/d9/d23/d30/da2/fdd x:0 0 0 2024-04-02T04:47:54.664 INFO:tasks.workunit.client.0.smithi070.stdout:7/559: mknod d7/dd/df/cb6 0 2024-04-02T04:47:54.664 INFO:tasks.workunit.client.1.smithi179.stdout:2/420: write dc/d29/d4a/d4b/f61 [907912,120060] 0 2024-04-02T04:47:54.666 INFO:tasks.workunit.client.1.smithi179.stdout:0/605: chown d1/da/f46 0 1 2024-04-02T04:47:54.667 INFO:tasks.workunit.client.0.smithi070.stdout:7/560: chown d7/dd/df/d7a/d6c/d77/fb0 0 1 2024-04-02T04:47:54.668 INFO:tasks.workunit.client.1.smithi179.stdout:7/635: creat d1/d9/d27/d87/fe3 x:0 0 0 2024-04-02T04:47:54.668 INFO:tasks.workunit.client.0.smithi070.stdout:7/561: readlink d7/dd/df/d7a/d6c/d77/l7f 0 2024-04-02T04:47:54.669 INFO:tasks.workunit.client.1.smithi179.stdout:1/576: symlink d3/lc9 0 2024-04-02T04:47:54.670 INFO:tasks.workunit.client.0.smithi070.stdout:5/705: dwrite d5/d9/d10/d51/faa [0,4194304] 0 2024-04-02T04:47:54.670 INFO:tasks.workunit.client.1.smithi179.stdout:7/636: write d1/d9/d27/d87/fe0 [912830,116321] 0 2024-04-02T04:47:54.671 INFO:tasks.workunit.client.0.smithi070.stdout:7/562: write d7/dd/df/d79/d46/d65/f69 [2472995,86756] 0 2024-04-02T04:47:54.672 INFO:tasks.workunit.client.1.smithi179.stdout:5/541: dread d8/d1b/d23/d42/d5a/f6b [0,4194304] 0 2024-04-02T04:47:54.672 INFO:tasks.workunit.client.1.smithi179.stdout:3/626: rename d8/d26/d36/l71 to d8/d26/d52/d7e/lc4 0 2024-04-02T04:47:54.677 INFO:tasks.workunit.client.1.smithi179.stdout:1/577: creat d3/d5/d2f/d4f/fca x:0 0 0 2024-04-02T04:47:54.678 INFO:tasks.workunit.client.1.smithi179.stdout:1/578: chown d3/d5/d2f/d4f/d35/fbf 1 1 2024-04-02T04:47:54.678 INFO:tasks.workunit.client.0.smithi070.stdout:5/706: mkdir d5/d9/d23/d30/dde 0 2024-04-02T04:47:54.680 INFO:tasks.workunit.client.1.smithi179.stdout:7/637: creat d1/dd/d16/d4d/fe4 x:0 0 0 2024-04-02T04:47:54.683 INFO:tasks.workunit.client.1.smithi179.stdout:8/545: rename d3/d32/d21/d50/d56/f77 to d3/d32/d21/d50/d56/d94/d84/fbd 0 2024-04-02T04:47:54.684 INFO:tasks.workunit.client.0.smithi070.stdout:5/707: mkdir d5/d9/d23/d30/da2/d75/ddf 0 2024-04-02T04:47:54.685 INFO:tasks.workunit.client.1.smithi179.stdout:8/546: fdatasync d3/d32/d21/d22/f62 0 2024-04-02T04:47:54.685 INFO:tasks.workunit.client.1.smithi179.stdout:3/627: unlink d8/d26/d36/c4c 0 2024-04-02T04:47:54.686 INFO:tasks.workunit.client.0.smithi070.stdout:7/563: truncate d7/f93 368551 0 2024-04-02T04:47:54.687 INFO:tasks.workunit.client.0.smithi070.stdout:5/708: dread d5/d17/d2b/f67 [0,4194304] 0 2024-04-02T04:47:54.687 INFO:tasks.workunit.client.1.smithi179.stdout:5/542: symlink d8/d1b/d23/d36/d53/d62/la8 0 2024-04-02T04:47:54.687 INFO:tasks.workunit.client.1.smithi179.stdout:5/543: dread - d8/d1b/f98 zero size 2024-04-02T04:47:54.688 INFO:tasks.workunit.client.0.smithi070.stdout:5/709: write d5/d17/d2b/f6f [584758,114058] 0 2024-04-02T04:47:54.690 INFO:tasks.workunit.client.0.smithi070.stdout:5/710: write d5/d9/d10/d6a/d7d/fc8 [758158,62542] 0 2024-04-02T04:47:54.691 INFO:tasks.workunit.client.1.smithi179.stdout:2/421: rename dc/d29/d4a/d1e/c2c to dc/d29/d4a/d1e/d6c/c88 0 2024-04-02T04:47:54.692 INFO:tasks.workunit.client.1.smithi179.stdout:0/606: dread d1/da/d28/d37/f49 [0,4194304] 0 2024-04-02T04:47:54.693 INFO:tasks.workunit.client.1.smithi179.stdout:2/422: write dc/d29/d31/d57/f6b [657621,64225] 0 2024-04-02T04:47:54.694 INFO:tasks.workunit.client.1.smithi179.stdout:3/628: creat d8/d6d/d94/fc5 x:0 0 0 2024-04-02T04:47:54.695 INFO:tasks.workunit.client.0.smithi070.stdout:7/564: mknod d7/dd/db5/cb7 0 2024-04-02T04:47:54.696 INFO:tasks.workunit.client.1.smithi179.stdout:8/547: truncate d3/d32/d21/d22/d6b/d91/f44 1590676 0 2024-04-02T04:47:54.698 INFO:tasks.workunit.client.0.smithi070.stdout:5/711: symlink d5/d9/d23/d30/da2/d75/le0 0 2024-04-02T04:47:54.700 INFO:tasks.workunit.client.1.smithi179.stdout:1/579: mknod d3/d5/d2f/d4f/d42/d55/dc5/ccb 0 2024-04-02T04:47:54.703 INFO:tasks.workunit.client.0.smithi070.stdout:7/565: read d7/dd/df/d79/f3b [2097661,72119] 0 2024-04-02T04:47:54.706 INFO:tasks.workunit.client.1.smithi179.stdout:7/638: write d1/d9/d27/d87/d8b/fa2 [2142415,22926] 0 2024-04-02T04:47:54.707 INFO:tasks.workunit.client.0.smithi070.stdout:5/712: dwrite d5/d17/d27/d2e/f9f [0,4194304] 0 2024-04-02T04:47:54.707 INFO:tasks.workunit.client.1.smithi179.stdout:7/639: write d1/dd/f17 [1803211,33622] 0 2024-04-02T04:47:54.709 INFO:tasks.workunit.client.1.smithi179.stdout:0/607: mknod d1/da/df/d56/d84/cb7 0 2024-04-02T04:47:54.710 INFO:tasks.workunit.client.0.smithi070.stdout:7/566: creat d7/dd/df/d79/d46/d65/d7b/fb8 x:0 0 0 2024-04-02T04:47:54.711 INFO:tasks.workunit.client.1.smithi179.stdout:0/608: write d1/da/df/fb5 [636431,86729] 0 2024-04-02T04:47:54.712 INFO:tasks.workunit.client.0.smithi070.stdout:5/713: mkdir d5/d9/d10/db9/dba/de1 0 2024-04-02T04:47:54.712 INFO:tasks.workunit.client.1.smithi179.stdout:3/629: creat d8/d26/d52/d85/dbb/fc6 x:0 0 0 2024-04-02T04:47:54.713 INFO:tasks.workunit.client.0.smithi070.stdout:5/714: chown d5/d9/d10/d6a/d7d/fc8 168737 1 2024-04-02T04:47:54.714 INFO:tasks.workunit.client.0.smithi070.stdout:7/567: chown d7/c26 0 1 2024-04-02T04:47:54.716 INFO:tasks.workunit.client.1.smithi179.stdout:7/640: dwrite d1/dd/d23/f2f [0,4194304] 0 2024-04-02T04:47:54.716 INFO:tasks.workunit.client.0.smithi070.stdout:7/568: dread - d7/dd/df/d79/d46/fb2 zero size 2024-04-02T04:47:54.718 INFO:tasks.workunit.client.1.smithi179.stdout:1/580: unlink d3/d5/d2f/d4f/c66 0 2024-04-02T04:47:54.719 INFO:tasks.workunit.client.1.smithi179.stdout:1/581: chown d3/d5/d2f/f81 62 1 2024-04-02T04:47:54.722 INFO:tasks.workunit.client.0.smithi070.stdout:7/569: rename d7/dd/df/d7a/f85 to d7/d36/fb9 0 2024-04-02T04:47:54.726 INFO:tasks.workunit.client.1.smithi179.stdout:0/609: creat d1/da/d48/d65/d7d/fb8 x:0 0 0 2024-04-02T04:47:54.728 INFO:tasks.workunit.client.1.smithi179.stdout:8/548: creat d3/d32/d21/d22/d6b/db1/fbe x:0 0 0 2024-04-02T04:47:54.728 INFO:tasks.workunit.client.1.smithi179.stdout:3/630: dread - d8/d26/d36/f49 zero size 2024-04-02T04:47:54.729 INFO:tasks.workunit.client.0.smithi070.stdout:7/570: link d7/dd/df/d79/f31 d7/dd/df/d34/fba 0 2024-04-02T04:47:54.730 INFO:tasks.workunit.client.1.smithi179.stdout:7/641: mknod d1/dd/d16/d4d/ce5 0 2024-04-02T04:47:54.731 INFO:tasks.workunit.client.1.smithi179.stdout:2/423: creat dc/f89 x:0 0 0 2024-04-02T04:47:54.733 INFO:tasks.workunit.client.1.smithi179.stdout:0/610: dwrite d1/db/f8e [0,4194304] 0 2024-04-02T04:47:54.734 INFO:tasks.workunit.client.1.smithi179.stdout:3/631: mknod d8/d6d/cc7 0 2024-04-02T04:47:54.734 INFO:tasks.workunit.client.1.smithi179.stdout:4/622: sync 2024-04-02T04:47:54.735 INFO:tasks.workunit.client.1.smithi179.stdout:8/549: mknod d3/d1b/d2a/d41/cbf 0 2024-04-02T04:47:54.737 INFO:tasks.workunit.client.1.smithi179.stdout:5/544: truncate f1 1412130 0 2024-04-02T04:47:54.738 INFO:tasks.workunit.client.1.smithi179.stdout:1/582: mkdir d3/d5/d2f/d5a/dcc 0 2024-04-02T04:47:54.740 INFO:tasks.workunit.client.1.smithi179.stdout:2/424: unlink dc/d29/d4a/d1e/f63 0 2024-04-02T04:47:54.740 INFO:tasks.workunit.client.1.smithi179.stdout:2/425: readlink dc/l13 0 2024-04-02T04:47:54.745 INFO:tasks.workunit.client.1.smithi179.stdout:8/550: creat d3/d32/d21/d22/d6b/d91/fc0 x:0 0 0 2024-04-02T04:47:54.745 INFO:tasks.workunit.client.1.smithi179.stdout:8/551: rename d3/d32/d21 to d3/d32/d21/d50/d56/dad/dc1 22 2024-04-02T04:47:54.747 INFO:tasks.workunit.client.1.smithi179.stdout:5/545: mkdir d8/d1b/d23/d36/d53/d62/da9 0 2024-04-02T04:47:54.749 INFO:tasks.workunit.client.1.smithi179.stdout:2/426: rmdir dc/d29/d4a/d1e/d6c 39 2024-04-02T04:47:54.749 INFO:tasks.workunit.client.0.smithi070.stdout:7/571: write d7/d36/f62 [1899358,84385] 0 2024-04-02T04:47:54.750 INFO:tasks.workunit.client.0.smithi070.stdout:7/572: chown d7 17 1 2024-04-02T04:47:54.751 INFO:tasks.workunit.client.1.smithi179.stdout:0/611: link d1/da/d28/d37/f3e d1/da/d48/d5c/fb9 0 2024-04-02T04:47:54.753 INFO:tasks.workunit.client.1.smithi179.stdout:8/552: rmdir d3/d32/d21/d50/d56/d94/d84 39 2024-04-02T04:47:54.755 INFO:tasks.workunit.client.1.smithi179.stdout:5/546: creat d8/d7e/faa x:0 0 0 2024-04-02T04:47:54.755 INFO:tasks.workunit.client.1.smithi179.stdout:3/632: dwrite d8/d26/d83/d89/d9c/fb0 [0,4194304] 0 2024-04-02T04:47:54.757 INFO:tasks.workunit.client.1.smithi179.stdout:1/583: mkdir d3/d5/d2f/d4f/dab/dcd 0 2024-04-02T04:47:54.757 INFO:tasks.workunit.client.1.smithi179.stdout:3/633: write d8/d26/d52/d85/f8b [2093268,116847] 0 2024-04-02T04:47:54.760 INFO:tasks.workunit.client.1.smithi179.stdout:3/634: fdatasync d8/d26/d36/f38 0 2024-04-02T04:47:54.763 INFO:tasks.workunit.client.0.smithi070.stdout:7/573: link d7/dd/df/d7a/d6c/d77/d75/fb3 d7/dd/df/d79/fbb 0 2024-04-02T04:47:54.765 INFO:tasks.workunit.client.1.smithi179.stdout:7/642: getdents d1/d9/d27/d87 0 2024-04-02T04:47:54.767 INFO:tasks.workunit.client.1.smithi179.stdout:8/553: dwrite d3/d32/d21/d51/f55 [0,4194304] 0 2024-04-02T04:47:54.768 INFO:tasks.workunit.client.1.smithi179.stdout:5/547: creat d8/d1b/d23/d32/d4e/fab x:0 0 0 2024-04-02T04:47:54.771 INFO:tasks.workunit.client.1.smithi179.stdout:3/635: fdatasync d8/f98 0 2024-04-02T04:47:54.771 INFO:tasks.workunit.client.1.smithi179.stdout:1/584: mknod d3/d5/d2f/d4f/d6a/da0/cce 0 2024-04-02T04:47:54.772 INFO:tasks.workunit.client.1.smithi179.stdout:4/623: dread d1/d3/df/d10/d2f/d36/f58 [0,4194304] 0 2024-04-02T04:47:54.775 INFO:tasks.workunit.client.1.smithi179.stdout:5/548: chown d8/d1b/d23/d32/d46/c97 0 1 2024-04-02T04:47:54.777 INFO:tasks.workunit.client.1.smithi179.stdout:0/612: rename d1/da/df/d56/d84/cb7 to d1/da/cba 0 2024-04-02T04:47:54.777 INFO:tasks.workunit.client.1.smithi179.stdout:7/643: dwrite d1/d9/d55/dbf/d9f/da8/dda/fc0 [0,4194304] 0 2024-04-02T04:47:54.778 INFO:tasks.workunit.client.1.smithi179.stdout:2/427: dread dc/d29/d4a/f24 [0,4194304] 0 2024-04-02T04:47:54.779 INFO:tasks.workunit.client.1.smithi179.stdout:7/644: fdatasync d1/d9/d27/f4b 0 2024-04-02T04:47:54.784 INFO:tasks.workunit.client.1.smithi179.stdout:1/585: creat d3/d5/d2f/d4f/d42/d55/fcf x:0 0 0 2024-04-02T04:47:54.786 INFO:tasks.workunit.client.1.smithi179.stdout:8/554: write d3/d32/d21/d50/f5f [669326,112753] 0 2024-04-02T04:47:54.789 INFO:tasks.workunit.client.1.smithi179.stdout:7/645: creat d1/d9/d27/d87/d8b/d75/db9/fe6 x:0 0 0 2024-04-02T04:47:54.792 INFO:tasks.workunit.client.1.smithi179.stdout:5/549: write d8/d1b/f26 [7004,19048] 0 2024-04-02T04:47:54.793 INFO:tasks.workunit.client.1.smithi179.stdout:2/428: mkdir dc/d29/d4a/d1e/d8a 0 2024-04-02T04:47:54.795 INFO:tasks.workunit.client.1.smithi179.stdout:8/555: rename d3/d32/d21/d45/d5a/l7c to d3/d32/d30/lc2 0 2024-04-02T04:47:54.797 INFO:tasks.workunit.client.1.smithi179.stdout:5/550: chown d8/d1b/d23/d36/l87 873735944 1 2024-04-02T04:47:54.799 INFO:tasks.workunit.client.1.smithi179.stdout:1/586: unlink d3/d5/d2f/d4f/d42/d53/fad 0 2024-04-02T04:47:54.799 INFO:tasks.workunit.client.1.smithi179.stdout:8/556: dread - d3/d32/d21/d22/d6b/fb3 zero size 2024-04-02T04:47:54.800 INFO:tasks.workunit.client.1.smithi179.stdout:4/624: dwrite d1/d3/dc/d27/d62/f54 [0,4194304] 0 2024-04-02T04:47:54.801 INFO:tasks.workunit.client.1.smithi179.stdout:7/646: dwrite d1/dd/fc3 [0,4194304] 0 2024-04-02T04:47:54.812 INFO:tasks.workunit.client.1.smithi179.stdout:1/587: symlink d3/d85/ld0 0 2024-04-02T04:47:54.815 INFO:tasks.workunit.client.1.smithi179.stdout:2/429: dwrite dc/d29/d4a/d1e/f7b [0,4194304] 0 2024-04-02T04:47:54.817 INFO:tasks.workunit.client.1.smithi179.stdout:5/551: truncate d8/d1b/d23/d42/fa4 708384 0 2024-04-02T04:47:54.818 INFO:tasks.workunit.client.1.smithi179.stdout:2/430: chown dc/d29/d4a/d1e 126 1 2024-04-02T04:47:54.818 INFO:tasks.workunit.client.1.smithi179.stdout:1/588: rmdir d3/d5/d96 39 2024-04-02T04:47:54.818 INFO:tasks.workunit.client.1.smithi179.stdout:0/613: dwrite d1/da/f1b [0,4194304] 0 2024-04-02T04:47:54.820 INFO:tasks.workunit.client.1.smithi179.stdout:2/431: chown dc/d29/d31/d39/c74 35284 1 2024-04-02T04:47:54.820 INFO:tasks.workunit.client.1.smithi179.stdout:0/614: write d1/da/d67/fa8 [202498,80225] 0 2024-04-02T04:47:54.822 INFO:tasks.workunit.client.1.smithi179.stdout:3/636: dread d8/f22 [0,4194304] 0 2024-04-02T04:47:54.822 INFO:tasks.workunit.client.1.smithi179.stdout:0/615: write d1/da/d67/fa1 [783051,24580] 0 2024-04-02T04:47:54.823 INFO:tasks.workunit.client.1.smithi179.stdout:0/616: chown d1/da/d48/d65 45771 1 2024-04-02T04:47:54.828 INFO:tasks.workunit.client.1.smithi179.stdout:1/589: mkdir d3/d85/da2/dd1 0 2024-04-02T04:47:54.830 INFO:tasks.workunit.client.1.smithi179.stdout:0/617: dwrite d1/da/df/fb5 [0,4194304] 0 2024-04-02T04:47:54.830 INFO:tasks.workunit.client.1.smithi179.stdout:1/590: truncate d3/d5/fc8 748870 0 2024-04-02T04:47:54.836 INFO:tasks.workunit.client.1.smithi179.stdout:0/618: write d1/da/d28/d37/f72 [1049761,54255] 0 2024-04-02T04:47:54.838 INFO:tasks.workunit.client.1.smithi179.stdout:0/619: readlink d1/l45 0 2024-04-02T04:47:54.839 INFO:tasks.workunit.client.1.smithi179.stdout:7/647: dwrite d1/dd/d16/fcd [0,4194304] 0 2024-04-02T04:47:54.839 INFO:tasks.workunit.client.1.smithi179.stdout:5/552: mkdir d8/d1b/d23/d36/d53/d89/dac 0 2024-04-02T04:47:54.843 INFO:tasks.workunit.client.1.smithi179.stdout:8/557: dwrite d3/f4 [0,4194304] 0 2024-04-02T04:47:54.845 INFO:tasks.workunit.client.1.smithi179.stdout:8/558: chown d3/d32/d21/d45/l64 15 1 2024-04-02T04:47:54.845 INFO:tasks.workunit.client.1.smithi179.stdout:2/432: creat dc/d29/f8b x:0 0 0 2024-04-02T04:47:54.846 INFO:tasks.workunit.client.1.smithi179.stdout:8/559: write d3/d32/d21/d51/d70/f8d [3217931,45972] 0 2024-04-02T04:47:54.847 INFO:tasks.workunit.client.1.smithi179.stdout:5/553: mkdir d8/d1b/d23/d30/d5f/dad 0 2024-04-02T04:47:54.850 INFO:tasks.workunit.client.1.smithi179.stdout:7/648: unlink d1/d9/d55/dbf/d9f/da8/dda/lb3 0 2024-04-02T04:47:54.851 INFO:tasks.workunit.client.1.smithi179.stdout:5/554: dread d8/f6c [0,4194304] 0 2024-04-02T04:47:54.853 INFO:tasks.workunit.client.1.smithi179.stdout:1/591: rename d3/c58 to d3/cd2 0 2024-04-02T04:47:54.857 INFO:tasks.workunit.client.1.smithi179.stdout:0/620: link d1/db/f2c d1/da/d52/fbb 0 2024-04-02T04:47:54.859 INFO:tasks.workunit.client.1.smithi179.stdout:8/560: mkdir d3/d31/dc3 0 2024-04-02T04:47:54.859 INFO:tasks.workunit.client.1.smithi179.stdout:7/649: write d1/d9/d7e/f8a [876826,23809] 0 2024-04-02T04:47:54.859 INFO:tasks.workunit.client.1.smithi179.stdout:5/555: mkdir d8/d1b/d23/d32/d4e/d50/dae 0 2024-04-02T04:47:54.859 INFO:tasks.workunit.client.1.smithi179.stdout:8/561: write d3/d1b/d2a/f8b [479079,83566] 0 2024-04-02T04:47:54.860 INFO:tasks.workunit.client.1.smithi179.stdout:5/556: fdatasync d8/d1b/d23/d36/d53/d62/f75 0 2024-04-02T04:47:54.861 INFO:tasks.workunit.client.1.smithi179.stdout:0/621: symlink d1/da/d48/d7b/lbc 0 2024-04-02T04:47:54.863 INFO:tasks.workunit.client.1.smithi179.stdout:2/433: dread dc/d29/d31/d57/f62 [0,4194304] 0 2024-04-02T04:47:54.866 INFO:tasks.workunit.client.1.smithi179.stdout:5/557: dread d8/d1b/d23/d32/d4e/d50/f6f [0,4194304] 0 2024-04-02T04:47:54.869 INFO:tasks.workunit.client.1.smithi179.stdout:7/650: rename d1/f7b to d1/d9/d55/dbf/d9f/fe7 0 2024-04-02T04:47:54.871 INFO:tasks.workunit.client.1.smithi179.stdout:8/562: rename d3/l2e to d3/d32/d21/d45/d5a/lc4 0 2024-04-02T04:47:54.873 INFO:tasks.workunit.client.1.smithi179.stdout:5/558: dread d8/d1b/d23/d32/f3d [0,4194304] 0 2024-04-02T04:47:54.874 INFO:tasks.workunit.client.1.smithi179.stdout:2/434: link dc/d29/d31/f3a dc/d29/d4a/f8c 0 2024-04-02T04:47:54.878 INFO:tasks.workunit.client.1.smithi179.stdout:5/559: mkdir d8/d1b/d23/d36/d53/d89/dac/daf 0 2024-04-02T04:47:54.882 INFO:tasks.workunit.client.1.smithi179.stdout:1/592: truncate d3/d5/d2f/d4f/f4c 1661828 0 2024-04-02T04:47:54.882 INFO:tasks.workunit.client.1.smithi179.stdout:0/622: dread d1/da/d28/d34/f7e [0,4194304] 0 2024-04-02T04:47:54.882 INFO:tasks.workunit.client.1.smithi179.stdout:2/435: dwrite dc/f68 [0,4194304] 0 2024-04-02T04:47:54.884 INFO:tasks.workunit.client.1.smithi179.stdout:1/593: write d3/d5/f21 [2423899,33954] 0 2024-04-02T04:47:54.888 INFO:tasks.workunit.client.1.smithi179.stdout:5/560: dread d8/d1b/d23/f2f [0,4194304] 0 2024-04-02T04:47:54.889 INFO:tasks.workunit.client.1.smithi179.stdout:2/436: dread dc/d29/d4a/f24 [0,4194304] 0 2024-04-02T04:47:54.890 INFO:tasks.workunit.client.1.smithi179.stdout:1/594: getdents d3/d5/d2f/d4f/d35/d49/db6 0 2024-04-02T04:47:54.892 INFO:tasks.workunit.client.1.smithi179.stdout:0/623: write d1/da/d28/d34/f35 [2077962,61143] 0 2024-04-02T04:47:54.893 INFO:tasks.workunit.client.1.smithi179.stdout:1/595: mknod d3/d5/de/d33/cd3 0 2024-04-02T04:47:54.895 INFO:tasks.workunit.client.1.smithi179.stdout:2/437: write dc/d29/d4a/d1e/d43/f6e [5059072,69930] 0 2024-04-02T04:47:54.895 INFO:tasks.workunit.client.1.smithi179.stdout:8/563: dread d3/d31/f9e [0,4194304] 0 2024-04-02T04:47:54.898 INFO:tasks.workunit.client.1.smithi179.stdout:8/564: readlink d3/l3a 0 2024-04-02T04:47:54.898 INFO:tasks.workunit.client.1.smithi179.stdout:5/561: dwrite d8/d1b/d23/d32/d6e/f95 [0,4194304] 0 2024-04-02T04:47:54.900 INFO:tasks.workunit.client.1.smithi179.stdout:8/565: chown d3/d32/d21/d49 29414912 1 2024-04-02T04:47:54.903 INFO:tasks.workunit.client.1.smithi179.stdout:2/438: write dc/d29/d4a/d1e/d43/f5f [1243026,42685] 0 2024-04-02T04:47:54.909 INFO:tasks.workunit.client.1.smithi179.stdout:2/439: rmdir dc/d29/d31/d81 0 2024-04-02T04:47:54.909 INFO:tasks.workunit.client.1.smithi179.stdout:5/562: dread d8/d1b/d23/d32/d4e/f93 [0,4194304] 0 2024-04-02T04:47:54.912 INFO:tasks.workunit.client.1.smithi179.stdout:5/563: unlink d8/d1b/d23/d36/f8f 0 2024-04-02T04:47:54.912 INFO:tasks.workunit.client.1.smithi179.stdout:2/440: mkdir dc/d29/d31/d83/d8d 0 2024-04-02T04:47:54.913 INFO:tasks.workunit.client.1.smithi179.stdout:2/441: stat dc/d29/d4a/d1e 0 2024-04-02T04:47:54.913 INFO:tasks.workunit.client.1.smithi179.stdout:5/564: fdatasync d8/d1b/d23/d36/d53/f78 0 2024-04-02T04:47:54.913 INFO:tasks.workunit.client.1.smithi179.stdout:0/624: dread d1/da/df/f12 [0,4194304] 0 2024-04-02T04:47:54.915 INFO:tasks.workunit.client.1.smithi179.stdout:2/442: symlink dc/d29/d31/l8e 0 2024-04-02T04:47:54.916 INFO:tasks.workunit.client.1.smithi179.stdout:5/565: creat d8/d1b/d23/d32/d4b/fb0 x:0 0 0 2024-04-02T04:47:54.916 INFO:tasks.workunit.client.1.smithi179.stdout:0/625: truncate d1/da/d48/d65/d7d/f61 211347 0 2024-04-02T04:47:54.917 INFO:tasks.workunit.client.1.smithi179.stdout:2/443: mknod dc/d79/c8f 0 2024-04-02T04:47:54.918 INFO:tasks.workunit.client.1.smithi179.stdout:5/566: mkdir d8/d1b/d23/db1 0 2024-04-02T04:47:54.919 INFO:tasks.workunit.client.1.smithi179.stdout:0/626: unlink d1/da/df/c1c 0 2024-04-02T04:47:54.920 INFO:tasks.workunit.client.1.smithi179.stdout:5/567: mkdir d8/d1b/d23/d36/d53/d62/db2 0 2024-04-02T04:47:54.921 INFO:tasks.workunit.client.1.smithi179.stdout:2/444: symlink dc/d29/l90 0 2024-04-02T04:47:54.921 INFO:tasks.workunit.client.1.smithi179.stdout:9/585: sync 2024-04-02T04:47:54.921 INFO:tasks.workunit.client.1.smithi179.stdout:6/575: sync 2024-04-02T04:47:54.922 INFO:tasks.workunit.client.1.smithi179.stdout:2/445: chown dc/d29/d31/d39 808947 1 2024-04-02T04:47:54.923 INFO:tasks.workunit.client.1.smithi179.stdout:5/568: mkdir d8/d1b/d23/d30/d73/db3 0 2024-04-02T04:47:54.924 INFO:tasks.workunit.client.1.smithi179.stdout:8/566: write d3/d6c/f89 [602490,103043] 0 2024-04-02T04:47:54.925 INFO:tasks.workunit.client.1.smithi179.stdout:0/627: creat d1/db/dad/fbd x:0 0 0 2024-04-02T04:47:54.928 INFO:tasks.workunit.client.1.smithi179.stdout:0/628: chown d1/da 2445 1 2024-04-02T04:47:54.929 INFO:tasks.workunit.client.1.smithi179.stdout:2/446: symlink dc/d29/d4a/d1e/l91 0 2024-04-02T04:47:54.929 INFO:tasks.workunit.client.1.smithi179.stdout:9/586: dread d0/d19/d27/d7a/d82/f98 [0,4194304] 0 2024-04-02T04:47:54.930 INFO:tasks.workunit.client.1.smithi179.stdout:1/596: dwrite d3/d5/d2f/f47 [0,4194304] 0 2024-04-02T04:47:54.934 INFO:tasks.workunit.client.1.smithi179.stdout:0/629: dread d1/db/d64/f79 [0,4194304] 0 2024-04-02T04:47:54.938 INFO:tasks.workunit.client.1.smithi179.stdout:9/587: mkdir d0/d19/d27/d7a/d87/daf 0 2024-04-02T04:47:54.938 INFO:tasks.workunit.client.1.smithi179.stdout:8/567: dwrite d3/d6c/d96/fa1 [0,4194304] 0 2024-04-02T04:47:54.941 INFO:tasks.workunit.client.1.smithi179.stdout:2/447: rename dc/d29/d4a/d1e/l91 to dc/d29/d4a/d4b/l92 0 2024-04-02T04:47:54.942 INFO:tasks.workunit.client.1.smithi179.stdout:2/448: readlink dc/d29/d31/l8e 0 2024-04-02T04:47:54.944 INFO:tasks.workunit.client.1.smithi179.stdout:9/588: dread d0/d19/d1c/f52 [0,4194304] 0 2024-04-02T04:47:54.944 INFO:tasks.workunit.client.1.smithi179.stdout:2/449: write dc/f68 [4939591,77558] 0 2024-04-02T04:47:54.945 INFO:tasks.workunit.client.1.smithi179.stdout:6/576: dread d3/d12/d23/d26/f27 [0,4194304] 0 2024-04-02T04:47:54.946 INFO:tasks.workunit.client.1.smithi179.stdout:9/589: fsync d0/d1/f44 0 2024-04-02T04:47:54.948 INFO:tasks.workunit.client.1.smithi179.stdout:2/450: creat dc/d79/f93 x:0 0 0 2024-04-02T04:47:54.949 INFO:tasks.workunit.client.1.smithi179.stdout:6/577: rmdir d3/d12/d23/d39/d4e/d66 39 2024-04-02T04:47:54.950 INFO:tasks.workunit.client.1.smithi179.stdout:6/578: chown d3/d2d/fb6 5490 1 2024-04-02T04:47:54.951 INFO:tasks.workunit.client.1.smithi179.stdout:9/590: mkdir d0/d1/d11/d29/d86/db0 0 2024-04-02T04:47:54.955 INFO:tasks.workunit.client.1.smithi179.stdout:6/579: readlink d3/d12/d44/l67 0 2024-04-02T04:47:54.955 INFO:tasks.workunit.client.1.smithi179.stdout:9/591: mknod d0/d19/d1c/cb1 0 2024-04-02T04:47:54.956 INFO:tasks.workunit.client.1.smithi179.stdout:6/580: chown d3/d2d/f65 24237 1 2024-04-02T04:47:54.957 INFO:tasks.workunit.client.1.smithi179.stdout:0/630: write d1/da/df/f33 [4677598,24351] 0 2024-04-02T04:47:54.961 INFO:tasks.workunit.client.1.smithi179.stdout:6/581: unlink d3/d42/c9c 0 2024-04-02T04:47:54.961 INFO:tasks.workunit.client.1.smithi179.stdout:1/597: dread d3/d5/de/f57 [0,4194304] 0 2024-04-02T04:47:54.962 INFO:tasks.workunit.client.1.smithi179.stdout:2/451: rename dc/d29/d4a/d1e/l80 to dc/d29/d4a/d1e/d6c/l94 0 2024-04-02T04:47:54.964 INFO:tasks.workunit.client.1.smithi179.stdout:0/631: creat d1/da/d28/d5e/fbe x:0 0 0 2024-04-02T04:47:54.964 INFO:tasks.workunit.client.1.smithi179.stdout:8/568: truncate d3/d32/d21/d50/d56/f6a 32482 0 2024-04-02T04:47:54.964 INFO:tasks.workunit.client.1.smithi179.stdout:8/569: chown d3/ca 494374745 1 2024-04-02T04:47:54.965 INFO:tasks.workunit.client.1.smithi179.stdout:6/582: fdatasync d3/d12/d23/d26/f27 0 2024-04-02T04:47:54.966 INFO:tasks.workunit.client.1.smithi179.stdout:1/598: mkdir d3/d5/d2f/d4f/d6a/da0/dd4 0 2024-04-02T04:47:54.968 INFO:tasks.workunit.client.1.smithi179.stdout:2/452: write dc/d29/d4a/d4b/f6d [3920868,54363] 0 2024-04-02T04:47:54.970 INFO:tasks.workunit.client.1.smithi179.stdout:1/599: fdatasync d3/d5/f13 0 2024-04-02T04:47:54.973 INFO:tasks.workunit.client.1.smithi179.stdout:8/570: unlink d3/d32/d21/d51/d70/l71 0 2024-04-02T04:47:54.974 INFO:tasks.workunit.client.1.smithi179.stdout:8/571: readlink d3/d32/d21/d22/d6b/d91/l90 0 2024-04-02T04:47:54.978 INFO:tasks.workunit.client.1.smithi179.stdout:8/572: symlink d3/d1b/lc5 0 2024-04-02T04:47:54.979 INFO:tasks.workunit.client.1.smithi179.stdout:0/632: getdents d1/da/d52/d75/d9a 0 2024-04-02T04:47:54.981 INFO:tasks.workunit.client.1.smithi179.stdout:8/573: creat d3/d31/dc3/fc6 x:0 0 0 2024-04-02T04:47:54.982 INFO:tasks.workunit.client.1.smithi179.stdout:0/633: read d1/da/d28/d37/f95 [1904369,70442] 0 2024-04-02T04:47:54.982 INFO:tasks.workunit.client.1.smithi179.stdout:8/574: stat d3/d6c/f89 0 2024-04-02T04:47:54.984 INFO:tasks.workunit.client.1.smithi179.stdout:6/583: rename d3/d12/d44/f52 to d3/d12/d23/d39/d4e/d66/fc6 0 2024-04-02T04:47:54.984 INFO:tasks.workunit.client.1.smithi179.stdout:8/575: read - d3/d32/d21/d22/d6b/d91/fc0 zero size 2024-04-02T04:47:54.985 INFO:tasks.workunit.client.1.smithi179.stdout:9/592: dwrite d0/d1/d11/d29/f94 [0,4194304] 0 2024-04-02T04:47:54.986 INFO:tasks.workunit.client.1.smithi179.stdout:0/634: symlink d1/db/d64/lbf 0 2024-04-02T04:47:54.987 INFO:tasks.workunit.client.1.smithi179.stdout:8/576: write d3/d32/d21/d22/d6b/db1/fbe [843474,90731] 0 2024-04-02T04:47:54.987 INFO:tasks.workunit.client.0.smithi070.stdout:4/476: sync 2024-04-02T04:47:54.987 INFO:tasks.workunit.client.0.smithi070.stdout:3/564: sync 2024-04-02T04:47:54.988 INFO:tasks.workunit.client.0.smithi070.stdout:3/565: fdatasync d10/d17/d51/d73/d8f/f9c 0 2024-04-02T04:47:54.991 INFO:tasks.workunit.client.1.smithi179.stdout:6/584: fsync d3/fa 0 2024-04-02T04:47:54.994 INFO:tasks.workunit.client.0.smithi070.stdout:3/566: truncate d10/d66/fb9 440421 0 2024-04-02T04:47:54.995 INFO:tasks.workunit.client.1.smithi179.stdout:1/600: write d3/d5/f1d [4643163,86648] 0 2024-04-02T04:47:54.995 INFO:tasks.workunit.client.1.smithi179.stdout:6/585: write d3/d12/d23/f55 [4936984,71808] 0 2024-04-02T04:47:54.996 INFO:tasks.workunit.client.1.smithi179.stdout:0/635: dwrite d1/da/f93 [0,4194304] 0 2024-04-02T04:47:54.999 INFO:tasks.workunit.client.0.smithi070.stdout:3/567: mknod d10/d66/cc6 0 2024-04-02T04:47:55.000 INFO:tasks.workunit.client.1.smithi179.stdout:2/453: dwrite dc/d29/d4a/f24 [0,4194304] 0 2024-04-02T04:47:55.000 INFO:tasks.workunit.client.0.smithi070.stdout:3/568: stat d10/d66/cc6 0 2024-04-02T04:47:55.002 INFO:tasks.workunit.client.0.smithi070.stdout:3/569: truncate d10/d17/d51/fb0 131379 0 2024-04-02T04:47:55.007 INFO:tasks.workunit.client.0.smithi070.stdout:3/570: rmdir d10/d17/db2/dbf 39 2024-04-02T04:47:55.008 INFO:tasks.workunit.client.0.smithi070.stdout:3/571: chown d10/d32/d4b/dc5 17968082 1 2024-04-02T04:47:55.009 INFO:tasks.workunit.client.1.smithi179.stdout:1/601: dread d3/d85/f89 [0,4194304] 0 2024-04-02T04:47:55.009 INFO:tasks.workunit.client.1.smithi179.stdout:0/636: rmdir d1/da/d28/d34 39 2024-04-02T04:47:55.010 INFO:tasks.workunit.client.0.smithi070.stdout:3/572: mknod d10/d66/cc7 0 2024-04-02T04:47:55.012 INFO:tasks.workunit.client.1.smithi179.stdout:2/454: dwrite dc/d29/d31/f41 [0,4194304] 0 2024-04-02T04:47:55.013 INFO:tasks.workunit.client.0.smithi070.stdout:3/573: readlink d10/d17/l44 0 2024-04-02T04:47:55.014 INFO:tasks.workunit.client.1.smithi179.stdout:8/577: write d3/d32/d21/d50/d56/f6a [679974,129865] 0 2024-04-02T04:47:55.014 INFO:tasks.workunit.client.1.smithi179.stdout:1/602: dread - d3/d5/d2f/d4f/d42/d55/fb8 zero size 2024-04-02T04:47:55.015 INFO:tasks.workunit.client.1.smithi179.stdout:0/637: mkdir d1/da/d52/d75/dc0 0 2024-04-02T04:47:55.016 INFO:tasks.workunit.client.0.smithi070.stdout:4/477: dwrite d1/d3/d9/d5e/f77 [0,4194304] 0 2024-04-02T04:47:55.018 INFO:tasks.workunit.client.1.smithi179.stdout:9/593: dwrite d0/d19/d27/dad/fae [0,4194304] 0 2024-04-02T04:47:55.020 INFO:tasks.workunit.client.1.smithi179.stdout:9/594: chown d0/d1/d11/d29/d40/f49 1806 1 2024-04-02T04:47:55.021 INFO:tasks.workunit.client.0.smithi070.stdout:3/574: rename d10/d32/d3f/cb1 to d10/d32/d8a/d8d/d91/cc8 0 2024-04-02T04:47:55.021 INFO:tasks.workunit.client.1.smithi179.stdout:8/578: dread - d3/d32/d21/d22/d6b/d91/fc0 zero size 2024-04-02T04:47:55.021 INFO:tasks.workunit.client.1.smithi179.stdout:0/638: chown d1/db/d64/f6a 44522 1 2024-04-02T04:47:55.024 INFO:tasks.workunit.client.0.smithi070.stdout:3/575: creat d10/d17/d51/fc9 x:0 0 0 2024-04-02T04:47:55.025 INFO:tasks.workunit.client.0.smithi070.stdout:3/576: chown d10/d17/d51/cc2 13 1 2024-04-02T04:47:55.026 INFO:tasks.workunit.client.0.smithi070.stdout:3/577: readlink d10/d32/d4b/l68 0 2024-04-02T04:47:55.032 INFO:tasks.workunit.client.1.smithi179.stdout:1/603: truncate d3/d5/f27 4944013 0 2024-04-02T04:47:55.032 INFO:tasks.workunit.client.1.smithi179.stdout:8/579: mknod d3/d32/d21/d50/d65/cc7 0 2024-04-02T04:47:55.033 INFO:tasks.workunit.client.1.smithi179.stdout:1/604: chown d3/d85/da2/fb5 240688 1 2024-04-02T04:47:55.035 INFO:tasks.workunit.client.1.smithi179.stdout:6/586: dread d3/d2d/d43/f4b [0,4194304] 0 2024-04-02T04:47:55.039 INFO:tasks.workunit.client.1.smithi179.stdout:2/455: write dc/f52 [201008,43410] 0 2024-04-02T04:47:55.039 INFO:tasks.workunit.client.1.smithi179.stdout:6/587: dwrite d3/d42/f7e [0,4194304] 0 2024-04-02T04:47:55.042 INFO:tasks.workunit.client.1.smithi179.stdout:4/625: sync 2024-04-02T04:47:55.044 INFO:tasks.workunit.client.1.smithi179.stdout:6/588: dread d3/d9a/f9f [0,4194304] 0 2024-04-02T04:47:55.046 INFO:tasks.workunit.client.1.smithi179.stdout:0/639: rename d1/da/d28/d37/f72 to d1/fc1 0 2024-04-02T04:47:55.046 INFO:tasks.workunit.client.1.smithi179.stdout:2/456: dwrite dc/d29/d31/f77 [0,4194304] 0 2024-04-02T04:47:55.048 INFO:tasks.workunit.client.0.smithi070.stdout:4/478: dwrite d1/f2d [0,4194304] 0 2024-04-02T04:47:55.048 INFO:tasks.workunit.client.1.smithi179.stdout:2/457: write dc/d29/d4a/d4b/f61 [938381,107183] 0 2024-04-02T04:47:55.052 INFO:tasks.workunit.client.1.smithi179.stdout:0/640: dread d1/db/d64/f79 [0,4194304] 0 2024-04-02T04:47:55.052 INFO:tasks.workunit.client.1.smithi179.stdout:0/641: readlink d1/da/l19 0 2024-04-02T04:47:55.053 INFO:tasks.workunit.client.1.smithi179.stdout:2/458: write dc/d29/d4a/f67 [8837,33901] 0 2024-04-02T04:47:55.053 INFO:tasks.workunit.client.1.smithi179.stdout:9/595: getdents d0/d19/d27/d7a/d87 0 2024-04-02T04:47:55.054 INFO:tasks.workunit.client.1.smithi179.stdout:8/580: mknod d3/d32/cc8 0 2024-04-02T04:47:55.057 INFO:tasks.workunit.client.1.smithi179.stdout:2/459: chown dc/d29/d4a/f15 45 1 2024-04-02T04:47:55.058 INFO:tasks.workunit.client.1.smithi179.stdout:4/626: mkdir d1/d3/dc/dc6 0 2024-04-02T04:47:55.059 INFO:tasks.workunit.client.1.smithi179.stdout:8/581: chown d3/d1b/l95 331268 1 2024-04-02T04:47:55.060 INFO:tasks.workunit.client.1.smithi179.stdout:0/642: rename d1/da/l44 to d1/da/d28/d81/d9d/lc2 0 2024-04-02T04:47:55.066 INFO:tasks.workunit.client.1.smithi179.stdout:8/582: mknod d3/d32/d21/d51/d70/cc9 0 2024-04-02T04:47:55.066 INFO:tasks.workunit.client.1.smithi179.stdout:4/627: symlink d1/d3/df/d10/lc7 0 2024-04-02T04:47:55.067 INFO:tasks.workunit.client.1.smithi179.stdout:2/460: symlink dc/d29/l95 0 2024-04-02T04:47:55.068 INFO:tasks.workunit.client.1.smithi179.stdout:6/589: getdents d3 0 2024-04-02T04:47:55.069 INFO:tasks.workunit.client.1.smithi179.stdout:8/583: truncate d3/d1b/d2a/f4c 914338 0 2024-04-02T04:47:55.073 INFO:tasks.workunit.client.1.smithi179.stdout:4/628: rename d1/d15/f79 to d1/d3/dc/d7a/fc8 0 2024-04-02T04:47:55.074 INFO:tasks.workunit.client.1.smithi179.stdout:2/461: symlink dc/d29/l96 0 2024-04-02T04:47:55.075 INFO:tasks.workunit.client.1.smithi179.stdout:8/584: mknod d3/d32/d21/d51/d70/cca 0 2024-04-02T04:47:55.075 INFO:tasks.workunit.client.1.smithi179.stdout:2/462: chown dc/d29/d4a/d1e/l7a 235037470 1 2024-04-02T04:47:55.077 INFO:tasks.workunit.client.1.smithi179.stdout:8/585: truncate d3/d32/d21/d50/d56/f6a 1759517 0 2024-04-02T04:47:55.079 INFO:tasks.workunit.client.1.smithi179.stdout:2/463: dread - dc/d29/d31/d39/f4f zero size 2024-04-02T04:47:55.079 INFO:tasks.workunit.client.0.smithi070.stdout:4/479: dread d1/d3/d9/f29 [0,4194304] 0 2024-04-02T04:47:55.079 INFO:tasks.workunit.client.1.smithi179.stdout:4/629: symlink d1/d3/dc/lc9 0 2024-04-02T04:47:55.081 INFO:tasks.workunit.client.1.smithi179.stdout:0/643: dread d1/da/f17 [0,4194304] 0 2024-04-02T04:47:55.081 INFO:tasks.workunit.client.1.smithi179.stdout:9/596: dread d0/d1/d10/d6d/fe [4194304,4194304] 0 2024-04-02T04:47:55.081 INFO:tasks.workunit.client.0.smithi070.stdout:4/480: symlink d1/la0 0 2024-04-02T04:47:55.082 INFO:tasks.workunit.client.1.smithi179.stdout:0/644: chown d1/da/d48/d65/d7d/d80 28 1 2024-04-02T04:47:55.082 INFO:tasks.workunit.client.1.smithi179.stdout:8/586: creat d3/d32/d21/d22/d6b/db1/fcb x:0 0 0 2024-04-02T04:47:55.083 INFO:tasks.workunit.client.1.smithi179.stdout:0/645: chown d1/da/d28/l71 0 1 2024-04-02T04:47:55.085 INFO:tasks.workunit.client.1.smithi179.stdout:0/646: truncate d1/da/d28/d5e/fbe 363487 0 2024-04-02T04:47:55.086 INFO:tasks.workunit.client.1.smithi179.stdout:2/464: dwrite dc/d29/d4a/f53 [0,4194304] 0 2024-04-02T04:47:55.088 INFO:tasks.workunit.client.1.smithi179.stdout:4/630: mknod d1/d3/cca 0 2024-04-02T04:47:55.088 INFO:tasks.workunit.client.1.smithi179.stdout:0/647: stat d1/da/d28/d5e/f98 0 2024-04-02T04:47:55.090 INFO:tasks.workunit.client.0.smithi070.stdout:4/481: dwrite d1/f86 [0,4194304] 0 2024-04-02T04:47:55.090 INFO:tasks.workunit.client.1.smithi179.stdout:2/465: mkdir dc/d29/d31/d57/d97 0 2024-04-02T04:47:55.091 INFO:tasks.workunit.client.1.smithi179.stdout:0/648: chown d1/db/c89 3 1 2024-04-02T04:47:55.092 INFO:tasks.workunit.client.1.smithi179.stdout:8/587: mkdir d3/d32/d21/d50/d56/d94/d84/dcc 0 2024-04-02T04:47:55.096 INFO:tasks.workunit.client.1.smithi179.stdout:8/588: dread d3/d31/f69 [0,4194304] 0 2024-04-02T04:47:55.098 INFO:tasks.workunit.client.0.smithi070.stdout:4/482: unlink d1/d3/l4e 0 2024-04-02T04:47:55.102 INFO:tasks.workunit.client.1.smithi179.stdout:4/631: mkdir d1/d15/dcb 0 2024-04-02T04:47:55.103 INFO:tasks.workunit.client.1.smithi179.stdout:2/466: truncate dc/d29/d4a/f22 1278317 0 2024-04-02T04:47:55.105 INFO:tasks.workunit.client.0.smithi070.stdout:4/483: dwrite d1/d3/d9/f9e [0,4194304] 0 2024-04-02T04:47:55.109 INFO:tasks.workunit.client.0.smithi070.stdout:4/484: chown d1/l4f 544 1 2024-04-02T04:47:55.111 INFO:tasks.workunit.client.1.smithi179.stdout:6/590: dwrite d3/d12/d23/d2f/f3c [0,4194304] 0 2024-04-02T04:47:55.111 INFO:tasks.workunit.client.1.smithi179.stdout:4/632: mkdir d1/d3/dc/d26/d8a/dcc 0 2024-04-02T04:47:55.116 INFO:tasks.workunit.client.0.smithi070.stdout:4/485: dread - d1/d3/d9/f75 zero size 2024-04-02T04:47:55.116 INFO:tasks.workunit.client.1.smithi179.stdout:8/589: link d3/f18 d3/d32/d21/d50/d56/fcd 0 2024-04-02T04:47:55.117 INFO:tasks.workunit.client.1.smithi179.stdout:8/590: chown d3/d32/cc8 1376112 1 2024-04-02T04:47:55.119 INFO:tasks.workunit.client.1.smithi179.stdout:2/467: dread dc/f50 [0,4194304] 0 2024-04-02T04:47:55.119 INFO:tasks.workunit.client.1.smithi179.stdout:9/597: dwrite d0/d1/d10/f6f [0,4194304] 0 2024-04-02T04:47:55.120 INFO:tasks.workunit.client.0.smithi070.stdout:4/486: chown d1/d3/l7 1722609 1 2024-04-02T04:47:55.122 INFO:tasks.workunit.client.1.smithi179.stdout:0/649: write d1/da/f17 [4071690,44442] 0 2024-04-02T04:47:55.123 INFO:tasks.workunit.client.0.smithi070.stdout:4/487: chown d1/fe 19623 1 2024-04-02T04:47:55.124 INFO:tasks.workunit.client.1.smithi179.stdout:0/650: read - d1/da/df/fa2 zero size 2024-04-02T04:47:55.127 INFO:tasks.workunit.client.1.smithi179.stdout:8/591: mknod d3/d32/d21/d51/d70/cce 0 2024-04-02T04:47:55.127 INFO:tasks.workunit.client.1.smithi179.stdout:0/651: read d1/db/fc [1276523,100392] 0 2024-04-02T04:47:55.128 INFO:tasks.workunit.client.1.smithi179.stdout:0/652: write d1/da/d52/f73 [877078,112723] 0 2024-04-02T04:47:55.131 INFO:tasks.workunit.client.0.smithi070.stdout:4/488: mkdir d1/d9c/d65/d7a/da1 0 2024-04-02T04:47:55.132 INFO:tasks.workunit.client.1.smithi179.stdout:9/598: dread d0/d19/d1c/f47 [0,4194304] 0 2024-04-02T04:47:55.133 INFO:tasks.workunit.client.1.smithi179.stdout:6/591: symlink d3/d12/d23/d39/d4e/d66/d7c/lc7 0 2024-04-02T04:47:55.133 INFO:tasks.workunit.client.1.smithi179.stdout:2/468: read dc/d29/d4a/f8c [2644589,129280] 0 2024-04-02T04:47:55.134 INFO:tasks.workunit.client.0.smithi070.stdout:4/489: mknod d1/d3/d78/d91/ca2 0 2024-04-02T04:47:55.135 INFO:tasks.workunit.client.1.smithi179.stdout:4/633: write d1/d15/d7c/f84 [456235,91489] 0 2024-04-02T04:47:55.137 INFO:tasks.workunit.client.0.smithi070.stdout:4/490: mkdir d1/d3/d78/da3 0 2024-04-02T04:47:55.139 INFO:tasks.workunit.client.1.smithi179.stdout:9/599: creat d0/d1/d11/d29/d86/fb2 x:0 0 0 2024-04-02T04:47:55.140 INFO:tasks.workunit.client.0.smithi070.stdout:4/491: read d1/d3/d9/f81 [442602,130357] 0 2024-04-02T04:47:55.141 INFO:tasks.workunit.client.1.smithi179.stdout:6/592: write d3/d12/d23/d39/d4e/fbf [9365863,49700] 0 2024-04-02T04:47:55.141 INFO:tasks.workunit.client.1.smithi179.stdout:2/469: creat dc/d29/d4a/d1e/d43/f98 x:0 0 0 2024-04-02T04:47:55.142 INFO:tasks.workunit.client.1.smithi179.stdout:4/634: chown d1/c9 181 1 2024-04-02T04:47:55.142 INFO:tasks.workunit.client.1.smithi179.stdout:2/470: write f6 [4062827,15708] 0 2024-04-02T04:47:55.145 INFO:tasks.workunit.client.1.smithi179.stdout:0/653: mkdir d1/da/dc3 0 2024-04-02T04:47:55.146 INFO:tasks.workunit.client.1.smithi179.stdout:0/654: truncate d1/da/d67/fa8 4696409 0 2024-04-02T04:47:55.146 INFO:tasks.workunit.client.1.smithi179.stdout:0/655: chown d1/da/d48/d7b/lbc 79218598 1 2024-04-02T04:47:55.148 INFO:tasks.workunit.client.1.smithi179.stdout:4/635: creat d1/d3/dc/d26/d8a/fcd x:0 0 0 2024-04-02T04:47:55.153 INFO:tasks.workunit.client.0.smithi070.stdout:4/492: getdents d1/d3/d85/d8e 0 2024-04-02T04:47:55.153 INFO:tasks.workunit.client.1.smithi179.stdout:7/651: sync 2024-04-02T04:47:55.153 INFO:tasks.workunit.client.1.smithi179.stdout:3/637: sync 2024-04-02T04:47:55.153 INFO:tasks.workunit.client.1.smithi179.stdout:5/569: sync 2024-04-02T04:47:55.156 INFO:tasks.workunit.client.0.smithi070.stdout:4/493: creat d1/d3/d78/d91/fa4 x:0 0 0 2024-04-02T04:47:55.157 INFO:tasks.workunit.client.0.smithi070.stdout:4/494: chown d1/d3/d9/f81 429 1 2024-04-02T04:47:55.159 INFO:tasks.workunit.client.0.smithi070.stdout:2/549: sync 2024-04-02T04:47:55.160 INFO:tasks.workunit.client.0.smithi070.stdout:6/473: sync 2024-04-02T04:47:55.160 INFO:tasks.workunit.client.0.smithi070.stdout:1/584: sync 2024-04-02T04:47:55.160 INFO:tasks.workunit.client.0.smithi070.stdout:7/574: sync 2024-04-02T04:47:55.160 INFO:tasks.workunit.client.0.smithi070.stdout:9/502: sync 2024-04-02T04:47:55.160 INFO:tasks.workunit.client.0.smithi070.stdout:8/560: sync 2024-04-02T04:47:55.160 INFO:tasks.workunit.client.0.smithi070.stdout:0/624: sync 2024-04-02T04:47:55.160 INFO:tasks.workunit.client.0.smithi070.stdout:5/715: sync 2024-04-02T04:47:55.162 INFO:tasks.workunit.client.0.smithi070.stdout:5/716: write d5/d9/d10/d51/fbc [3135977,15068] 0 2024-04-02T04:47:55.165 INFO:tasks.workunit.client.0.smithi070.stdout:1/585: creat d0/d7/d28/d83/fbd x:0 0 0 2024-04-02T04:47:55.165 INFO:tasks.workunit.client.0.smithi070.stdout:2/550: symlink d2/d12/d3e/lb3 0 2024-04-02T04:47:55.167 INFO:tasks.workunit.client.1.smithi179.stdout:2/471: symlink dc/d29/d31/d57/l99 0 2024-04-02T04:47:55.167 INFO:tasks.workunit.client.1.smithi179.stdout:4/636: write d1/d3/dc/d26/d8a/d5d/f89 [885314,92964] 0 2024-04-02T04:47:55.168 INFO:tasks.workunit.client.1.smithi179.stdout:9/600: symlink d0/d1/d10/d6d/lb3 0 2024-04-02T04:47:55.169 INFO:tasks.workunit.client.1.smithi179.stdout:8/592: truncate d3/d32/d30/f7d 4011031 0 2024-04-02T04:47:55.177 INFO:tasks.workunit.client.1.smithi179.stdout:7/652: mknod d1/d9/d35/d6b/ce8 0 2024-04-02T04:47:55.178 INFO:tasks.workunit.client.1.smithi179.stdout:0/656: dread d1/da/df/f33 [0,4194304] 0 2024-04-02T04:47:55.179 INFO:tasks.workunit.client.1.smithi179.stdout:5/570: fsync d8/d1b/d23/d30/f71 0 2024-04-02T04:47:55.180 INFO:tasks.workunit.client.1.smithi179.stdout:2/472: symlink dc/d29/d4a/d1e/l9a 0 2024-04-02T04:47:55.182 INFO:tasks.workunit.client.1.smithi179.stdout:0/657: truncate d1/da/d48/d65/d7d/fb8 75772 0 2024-04-02T04:47:55.183 INFO:tasks.workunit.client.1.smithi179.stdout:9/601: creat d0/d19/d27/d7a/d87/daf/fb4 x:0 0 0 2024-04-02T04:47:55.183 INFO:tasks.workunit.client.1.smithi179.stdout:7/653: dread d1/dd/f99 [0,4194304] 0 2024-04-02T04:47:55.184 INFO:tasks.workunit.client.1.smithi179.stdout:2/473: mknod dc/d29/d31/d66/c9b 0 2024-04-02T04:47:55.184 INFO:tasks.workunit.client.0.smithi070.stdout:7/575: mkdir d7/dd/df/d7a/dbc 0 2024-04-02T04:47:55.185 INFO:tasks.workunit.client.0.smithi070.stdout:0/625: mkdir d4/d5/d13/d2a/d3a/d80/db6 0 2024-04-02T04:47:55.187 INFO:tasks.workunit.client.1.smithi179.stdout:8/593: rename d3/d32/d21/d50/d56/d94/d84/dcc to d3/d32/d21/d50/dcf 0 2024-04-02T04:47:55.187 INFO:tasks.workunit.client.0.smithi070.stdout:1/586: symlink d0/d9/d1d/d22/d2c/d4f/lbe 0 2024-04-02T04:47:55.188 INFO:tasks.workunit.client.0.smithi070.stdout:5/717: chown d5/d17/d2b/d7b/cad 6 1 2024-04-02T04:47:55.189 INFO:tasks.workunit.client.0.smithi070.stdout:9/503: write d4/d10/f2c [949248,9818] 0 2024-04-02T04:47:55.189 INFO:tasks.workunit.client.1.smithi179.stdout:3/638: getdents d8/d7a 0 2024-04-02T04:47:55.189 INFO:tasks.workunit.client.1.smithi179.stdout:2/474: dread dc/d29/d31/d57/f71 [0,4194304] 0 2024-04-02T04:47:55.191 INFO:tasks.workunit.client.1.smithi179.stdout:8/594: dread d3/f61 [0,4194304] 0 2024-04-02T04:47:55.193 INFO:tasks.workunit.client.0.smithi070.stdout:4/495: dwrite d1/d9c/f48 [0,4194304] 0 2024-04-02T04:47:55.193 INFO:tasks.workunit.client.1.smithi179.stdout:9/602: chown d0/d1/d11/d29/f50 771 1 2024-04-02T04:47:55.194 INFO:tasks.workunit.client.0.smithi070.stdout:7/576: read - d7/d4e/d82/d57/f92 zero size 2024-04-02T04:47:55.194 INFO:tasks.workunit.client.1.smithi179.stdout:4/637: write d1/d3/d19/f2c [212993,27459] 0 2024-04-02T04:47:55.195 INFO:tasks.workunit.client.0.smithi070.stdout:7/577: chown d7/d36/fae 1886194 1 2024-04-02T04:47:55.195 INFO:tasks.workunit.client.1.smithi179.stdout:8/595: read d3/d32/d21/d50/d65/faa [2492904,4354] 0 2024-04-02T04:47:55.196 INFO:tasks.workunit.client.0.smithi070.stdout:7/578: dread - d7/dd/df/f94 zero size 2024-04-02T04:47:55.197 INFO:tasks.workunit.client.1.smithi179.stdout:2/475: dwrite dc/f68 [4194304,4194304] 0 2024-04-02T04:47:55.198 INFO:tasks.workunit.client.0.smithi070.stdout:0/626: mknod d4/d5/d13/d2a/d3a/d80/cb7 0 2024-04-02T04:47:55.199 INFO:tasks.workunit.client.1.smithi179.stdout:2/476: write dc/d79/f93 [1023043,11941] 0 2024-04-02T04:47:55.201 INFO:tasks.workunit.client.0.smithi070.stdout:2/551: dread d2/f4 [0,4194304] 0 2024-04-02T04:47:55.202 INFO:tasks.workunit.client.0.smithi070.stdout:6/474: getdents d0 0 2024-04-02T04:47:55.203 INFO:tasks.workunit.client.1.smithi179.stdout:7/654: dread - d1/d9/d27/d87/d8b/f6f zero size 2024-04-02T04:47:55.203 INFO:tasks.workunit.client.0.smithi070.stdout:6/475: stat d0/d3/l49 0 2024-04-02T04:47:55.204 INFO:tasks.workunit.client.0.smithi070.stdout:9/504: chown d4/d10/d11/d7b/d19/f50 0 1 2024-04-02T04:47:55.204 INFO:tasks.workunit.client.1.smithi179.stdout:7/655: fsync d1/d9/d55/fd7 0 2024-04-02T04:47:55.206 INFO:tasks.workunit.client.0.smithi070.stdout:5/718: rename d5/d14/l82 to d5/d9/d23/dc3/le2 0 2024-04-02T04:47:55.207 INFO:tasks.workunit.client.1.smithi179.stdout:7/656: fdatasync d1/d9/d27/d87/d8b/d75/db9/fe6 0 2024-04-02T04:47:55.208 INFO:tasks.workunit.client.1.smithi179.stdout:2/477: dwrite dc/f5d [0,4194304] 0 2024-04-02T04:47:55.210 INFO:tasks.workunit.client.0.smithi070.stdout:7/579: creat d7/dd/df/d7a/d6c/d77/d54/fbd x:0 0 0 2024-04-02T04:47:55.210 INFO:tasks.workunit.client.1.smithi179.stdout:5/571: creat d8/d1b/d23/fb4 x:0 0 0 2024-04-02T04:47:55.211 INFO:tasks.workunit.client.0.smithi070.stdout:0/627: mknod d4/d5/d58/d60/d83/da5/cb8 0 2024-04-02T04:47:55.213 INFO:tasks.workunit.client.0.smithi070.stdout:2/552: creat d2/d12/d3e/fb4 x:0 0 0 2024-04-02T04:47:55.216 INFO:tasks.workunit.client.1.smithi179.stdout:9/603: mknod d0/d19/d27/d7a/d84/cb5 0 2024-04-02T04:47:55.216 INFO:tasks.workunit.client.1.smithi179.stdout:9/604: chown d0/d1 28900092 1 2024-04-02T04:47:55.217 INFO:tasks.workunit.client.1.smithi179.stdout:4/638: mkdir d1/d3/df/d10/d2f/d60/d71/dce 0 2024-04-02T04:47:55.217 INFO:tasks.workunit.client.1.smithi179.stdout:9/605: chown d0/d1/d11/d29/d86/f9e 51190302 1 2024-04-02T04:47:55.222 INFO:tasks.workunit.client.0.smithi070.stdout:5/719: mkdir d5/d17/d27/de3 0 2024-04-02T04:47:55.223 INFO:tasks.workunit.client.0.smithi070.stdout:5/720: write d5/d14/f16 [697832,57693] 0 2024-04-02T04:47:55.223 INFO:tasks.workunit.client.1.smithi179.stdout:5/572: fdatasync d8/d1b/d23/d32/d4e/f61 0 2024-04-02T04:47:55.225 INFO:tasks.workunit.client.0.smithi070.stdout:6/476: rmdir d0/d43/d80/d85 0 2024-04-02T04:47:55.226 INFO:tasks.workunit.client.0.smithi070.stdout:6/477: chown d0/d3/d19/f47 202453 1 2024-04-02T04:47:55.226 INFO:tasks.workunit.client.1.smithi179.stdout:9/606: creat d0/d19/d27/d7a/d84/fb6 x:0 0 0 2024-04-02T04:47:55.227 INFO:tasks.workunit.client.1.smithi179.stdout:9/607: chown d0/d19/d1c/l43 56557 1 2024-04-02T04:47:55.228 INFO:tasks.workunit.client.0.smithi070.stdout:1/587: rename d0/d54/f7f to d0/d9/d1d/d22/d56/fbf 0 2024-04-02T04:47:55.229 INFO:tasks.workunit.client.1.smithi179.stdout:2/478: mknod dc/d29/d31/d57/d97/c9c 0 2024-04-02T04:47:55.229 INFO:tasks.workunit.client.1.smithi179.stdout:2/479: readlink dc/d29/d4a/l78 0 2024-04-02T04:47:55.230 INFO:tasks.workunit.client.1.smithi179.stdout:2/480: write dc/f52 [715869,15304] 0 2024-04-02T04:47:55.231 INFO:tasks.workunit.client.0.smithi070.stdout:4/496: write d1/d3/d9/f53 [843071,75652] 0 2024-04-02T04:47:55.233 INFO:tasks.workunit.client.1.smithi179.stdout:5/573: readlink d8/d1b/d23/d32/l94 0 2024-04-02T04:47:55.235 INFO:tasks.workunit.client.1.smithi179.stdout:3/639: dwrite d8/d26/d36/f8d [0,4194304] 0 2024-04-02T04:47:55.236 INFO:tasks.workunit.client.1.smithi179.stdout:0/658: truncate d1/da/f93 409878 0 2024-04-02T04:47:55.238 INFO:tasks.workunit.client.1.smithi179.stdout:8/596: dread d3/d32/d21/d50/d65/faa [0,4194304] 0 2024-04-02T04:47:55.238 INFO:tasks.workunit.client.1.smithi179.stdout:3/640: write d8/d26/d52/d85/f97 [331176,114780] 0 2024-04-02T04:47:55.239 INFO:tasks.workunit.client.1.smithi179.stdout:9/608: creat d0/d19/d1c/d64/fb7 x:0 0 0 2024-04-02T04:47:55.242 INFO:tasks.workunit.client.0.smithi070.stdout:6/478: truncate d0/d20/d5f/f6e 2940118 0 2024-04-02T04:47:55.246 INFO:tasks.workunit.client.0.smithi070.stdout:6/479: stat d0/d3/d19/d31/d46/d6b/f7d 0 2024-04-02T04:47:55.246 INFO:tasks.workunit.client.1.smithi179.stdout:7/657: getdents d1/d9/d7e 0 2024-04-02T04:47:55.248 INFO:tasks.workunit.client.0.smithi070.stdout:5/721: creat d5/d17/d27/fe4 x:0 0 0 2024-04-02T04:47:55.248 INFO:tasks.workunit.client.0.smithi070.stdout:0/628: dwrite d4/d5/da/f6d [0,4194304] 0 2024-04-02T04:47:55.248 INFO:tasks.workunit.client.1.smithi179.stdout:1/605: sync 2024-04-02T04:47:55.249 INFO:tasks.workunit.client.1.smithi179.stdout:4/639: dwrite d1/d3/df/f48 [0,4194304] 0 2024-04-02T04:47:55.250 INFO:tasks.workunit.client.1.smithi179.stdout:4/640: chown d1/d3/df/d10/d2f/d7b 69630 1 2024-04-02T04:47:55.251 INFO:tasks.workunit.client.0.smithi070.stdout:2/553: dwrite d2/d24/f6c [0,4194304] 0 2024-04-02T04:47:55.256 INFO:tasks.workunit.client.0.smithi070.stdout:9/505: rename d4/d10/d12/d27/d68/l86 to d4/d10/d11/d7b/d19/la4 0 2024-04-02T04:47:55.260 INFO:tasks.workunit.client.0.smithi070.stdout:4/497: dread d1/d3/d9/f1d [0,4194304] 0 2024-04-02T04:47:55.260 INFO:tasks.workunit.client.1.smithi179.stdout:5/574: mknod d8/d1b/d23/d30/d5f/dad/cb5 0 2024-04-02T04:47:55.262 INFO:tasks.workunit.client.0.smithi070.stdout:6/480: mkdir d0/d43/d95 0 2024-04-02T04:47:55.263 INFO:tasks.workunit.client.0.smithi070.stdout:3/578: sync 2024-04-02T04:47:55.263 INFO:tasks.workunit.client.0.smithi070.stdout:3/579: readlink d10/d17/d4c/l5c 0 2024-04-02T04:47:55.270 INFO:tasks.workunit.client.1.smithi179.stdout:8/597: write d3/d32/d21/f66 [271270,74664] 0 2024-04-02T04:47:55.271 INFO:tasks.workunit.client.0.smithi070.stdout:0/629: dwrite d4/d5/d13/d2a/d3a/d80/f9f [0,4194304] 0 2024-04-02T04:47:55.271 INFO:tasks.workunit.client.1.smithi179.stdout:8/598: write d3/f76 [2495831,55097] 0 2024-04-02T04:47:55.273 INFO:tasks.workunit.client.0.smithi070.stdout:9/506: fsync d4/d10/d11/d7b/f2b 0 2024-04-02T04:47:55.276 INFO:tasks.workunit.client.0.smithi070.stdout:4/498: creat d1/d9c/fa5 x:0 0 0 2024-04-02T04:47:55.279 INFO:tasks.workunit.client.1.smithi179.stdout:7/658: symlink d1/dd/le9 0 2024-04-02T04:47:55.279 INFO:tasks.workunit.client.1.smithi179.stdout:7/659: stat d1/dd/d76/fca 0 2024-04-02T04:47:55.280 INFO:tasks.workunit.client.0.smithi070.stdout:3/580: creat d10/d32/d3f/fca x:0 0 0 2024-04-02T04:47:55.281 INFO:tasks.workunit.client.1.smithi179.stdout:1/606: symlink d3/d5/ld5 0 2024-04-02T04:47:55.281 INFO:tasks.workunit.client.0.smithi070.stdout:3/581: write d10/d32/f36 [6777193,69551] 0 2024-04-02T04:47:55.281 INFO:tasks.workunit.client.0.smithi070.stdout:5/722: mknod d5/d9/d23/d30/da2/d75/ddf/ce5 0 2024-04-02T04:47:55.284 INFO:tasks.workunit.client.1.smithi179.stdout:3/641: truncate d8/d26/d83/d89/d9c/fc2 2204133 0 2024-04-02T04:47:55.285 INFO:tasks.workunit.client.1.smithi179.stdout:9/609: getdents d0/d19/d27/d7a/d87 0 2024-04-02T04:47:55.286 INFO:tasks.workunit.client.1.smithi179.stdout:9/610: truncate d0/d19/d1c/d45/fa5 982981 0 2024-04-02T04:47:55.287 INFO:tasks.workunit.client.1.smithi179.stdout:8/599: rename d3/d32/d30/f4e to d3/d32/d21/d50/d56/dad/dbc/fd0 0 2024-04-02T04:47:55.288 INFO:tasks.workunit.client.0.smithi070.stdout:0/630: symlink d4/d5/d13/d2a/d3a/lb9 0 2024-04-02T04:47:55.289 INFO:tasks.workunit.client.1.smithi179.stdout:5/575: symlink d8/d1b/d23/d30/d73/db3/lb6 0 2024-04-02T04:47:55.290 INFO:tasks.workunit.client.0.smithi070.stdout:9/507: mkdir d4/d10/d11/d7b/d19/d2f/d39/da5 0 2024-04-02T04:47:55.292 INFO:tasks.workunit.client.1.smithi179.stdout:7/660: mkdir d1/d9/d55/dbf/d9f/da8/dda/dea 0 2024-04-02T04:47:55.292 INFO:tasks.workunit.client.0.smithi070.stdout:6/481: mknod d0/d43/d95/c96 0 2024-04-02T04:47:55.295 INFO:tasks.workunit.client.1.smithi179.stdout:4/641: rmdir d1/d15/dcb 0 2024-04-02T04:47:55.296 INFO:tasks.workunit.client.1.smithi179.stdout:5/576: dwrite d8/d1b/d23/d36/d53/f78 [0,4194304] 0 2024-04-02T04:47:55.296 INFO:tasks.workunit.client.0.smithi070.stdout:4/499: dwrite d1/f46 [0,4194304] 0 2024-04-02T04:47:55.296 INFO:tasks.workunit.client.1.smithi179.stdout:5/577: chown d8/d1b/d23 0 1 2024-04-02T04:47:55.297 INFO:tasks.workunit.client.0.smithi070.stdout:4/500: write d1/d3/d85/f92 [981546,125353] 0 2024-04-02T04:47:55.300 INFO:tasks.workunit.client.0.smithi070.stdout:2/554: dread d2/d12/f38 [0,4194304] 0 2024-04-02T04:47:55.304 INFO:tasks.workunit.client.1.smithi179.stdout:9/611: rmdir d0/d19/d27/dad 39 2024-04-02T04:47:55.309 INFO:tasks.workunit.client.1.smithi179.stdout:1/607: link d3/d5/d2f/d4f/fc1 d3/d5/d2f/d4f/d42/d55/fd6 0 2024-04-02T04:47:55.310 INFO:tasks.workunit.client.1.smithi179.stdout:7/661: rename d1/d9/d35/la7 to d1/dd/d23/dab/leb 0 2024-04-02T04:47:55.312 INFO:tasks.workunit.client.1.smithi179.stdout:3/642: dread d8/f1b [0,4194304] 0 2024-04-02T04:47:55.313 INFO:tasks.workunit.client.0.smithi070.stdout:7/580: rename d7/d4e/d82/c87 to d7/dd/df/d79/d46/cbe 0 2024-04-02T04:47:55.313 INFO:tasks.workunit.client.1.smithi179.stdout:3/643: write d8/d26/d36/f90 [2887624,22790] 0 2024-04-02T04:47:55.314 INFO:tasks.workunit.client.0.smithi070.stdout:7/581: write d7/dd/df/d79/d46/fb2 [148501,41612] 0 2024-04-02T04:47:55.316 INFO:tasks.workunit.client.1.smithi179.stdout:4/642: mknod d1/d3/df/d10/d2f/d60/d71/dce/ccf 0 2024-04-02T04:47:55.316 INFO:tasks.workunit.client.0.smithi070.stdout:2/555: dread d2/d10/d21/d3d/f47 [0,4194304] 0 2024-04-02T04:47:55.317 INFO:tasks.workunit.client.0.smithi070.stdout:3/582: creat d10/d32/fcb x:0 0 0 2024-04-02T04:47:55.317 INFO:tasks.workunit.client.0.smithi070.stdout:3/583: stat d10/d32/d3f/l26 0 2024-04-02T04:47:55.320 INFO:tasks.workunit.client.0.smithi070.stdout:5/723: rename d5/d9/d23/d30/da2/f38 to d5/d17/d6c/fe6 0 2024-04-02T04:47:55.321 INFO:tasks.workunit.client.1.smithi179.stdout:5/578: rename d8/d1b/d23/d30/d5f to d8/d1b/d23/d36/d53/d89/dac/daf/db7 0 2024-04-02T04:47:55.322 INFO:tasks.workunit.client.0.smithi070.stdout:7/582: write d7/dd/df/d34/d68/f70 [4743117,124680] 0 2024-04-02T04:47:55.322 INFO:tasks.workunit.client.0.smithi070.stdout:5/724: write d5/d14/f68 [1135743,27961] 0 2024-04-02T04:47:55.323 INFO:tasks.workunit.client.0.smithi070.stdout:3/584: dwrite d10/d17/d51/d73/d8f/fab [0,4194304] 0 2024-04-02T04:47:55.325 INFO:tasks.workunit.client.0.smithi070.stdout:5/725: chown d5/d9/d10/d51/fbc 86 1 2024-04-02T04:47:55.326 INFO:tasks.workunit.client.0.smithi070.stdout:3/585: chown d10/d17/d4c/l5c 136675938 1 2024-04-02T04:47:55.326 INFO:tasks.workunit.client.0.smithi070.stdout:5/726: chown d5/d17/d2b/f67 1 1 2024-04-02T04:47:55.327 INFO:tasks.workunit.client.0.smithi070.stdout:5/727: chown d5/d9/d10/da9 4926 1 2024-04-02T04:47:55.327 INFO:tasks.workunit.client.0.smithi070.stdout:3/586: write d10/d32/d8a/d8d/d91/f9f [31666,130308] 0 2024-04-02T04:47:55.330 INFO:tasks.workunit.client.1.smithi179.stdout:5/579: creat d8/d1b/d23/d32/d4e/fb8 x:0 0 0 2024-04-02T04:47:55.332 INFO:tasks.workunit.client.1.smithi179.stdout:9/612: link d0/d1/f7b d0/d19/d27/d7a/d7e/fb8 0 2024-04-02T04:47:55.332 INFO:tasks.workunit.client.1.smithi179.stdout:9/613: dread - d0/d1/d10/d6d/d58/f75 zero size 2024-04-02T04:47:55.333 INFO:tasks.workunit.client.1.smithi179.stdout:9/614: write d0/d1/d10/d6d/d58/f9d [4758188,85037] 0 2024-04-02T04:47:55.334 INFO:tasks.workunit.client.1.smithi179.stdout:4/643: mknod d1/cd0 0 2024-04-02T04:47:55.335 INFO:tasks.workunit.client.0.smithi070.stdout:4/501: link d1/d3/c52 d1/d9c/d65/d7a/ca6 0 2024-04-02T04:47:55.336 INFO:tasks.workunit.client.0.smithi070.stdout:4/502: dread - d1/d3/d78/d91/fa4 zero size 2024-04-02T04:47:55.337 INFO:tasks.workunit.client.1.smithi179.stdout:1/608: link d3/d5/c54 d3/da8/cd7 0 2024-04-02T04:47:55.337 INFO:tasks.workunit.client.0.smithi070.stdout:4/503: chown d1/c9a 344946 1 2024-04-02T04:47:55.337 INFO:tasks.workunit.client.0.smithi070.stdout:4/504: dread - d1/d3/f95 zero size 2024-04-02T04:47:55.340 INFO:tasks.workunit.client.0.smithi070.stdout:6/482: rmdir d0/d43 39 2024-04-02T04:47:55.342 INFO:tasks.workunit.client.1.smithi179.stdout:1/609: dwrite d3/d5/fc8 [0,4194304] 0 2024-04-02T04:47:55.342 INFO:tasks.workunit.client.0.smithi070.stdout:2/556: creat d2/fb5 x:0 0 0 2024-04-02T04:47:55.343 INFO:tasks.workunit.client.1.smithi179.stdout:1/610: write d3/d5/d2f/fb0 [3824970,36499] 0 2024-04-02T04:47:55.344 INFO:tasks.workunit.client.1.smithi179.stdout:1/611: fsync d3/d5/d2f/d4f/d35/d49/f8b 0 2024-04-02T04:47:55.345 INFO:tasks.workunit.client.0.smithi070.stdout:4/505: unlink d1/d3/d9/l2e 0 2024-04-02T04:47:55.349 INFO:tasks.workunit.client.1.smithi179.stdout:8/600: dwrite d3/d32/d21/fab [0,4194304] 0 2024-04-02T04:47:55.350 INFO:tasks.workunit.client.0.smithi070.stdout:7/583: creat d7/dd/df/d79/fbf x:0 0 0 2024-04-02T04:47:55.351 INFO:tasks.workunit.client.0.smithi070.stdout:7/584: chown d7/dd/df/d79 51903 1 2024-04-02T04:47:55.352 INFO:tasks.workunit.client.0.smithi070.stdout:9/508: truncate d4/f8 3765262 0 2024-04-02T04:47:55.354 INFO:tasks.workunit.client.0.smithi070.stdout:2/557: creat d2/d10/d21/d2b/fb6 x:0 0 0 2024-04-02T04:47:55.358 INFO:tasks.workunit.client.0.smithi070.stdout:7/585: readlink d7/d9/l4c 0 2024-04-02T04:47:55.361 INFO:tasks.workunit.client.0.smithi070.stdout:7/586: dread d7/dd/df/d79/d46/fb2 [0,4194304] 0 2024-04-02T04:47:55.361 INFO:tasks.workunit.client.0.smithi070.stdout:9/509: mkdir d4/d10/d11/d7b/d19/d2f/d39/d57/da6 0 2024-04-02T04:47:55.363 INFO:tasks.workunit.client.0.smithi070.stdout:2/558: truncate d2/d10/d5c/f81 448467 0 2024-04-02T04:47:55.366 INFO:tasks.workunit.client.1.smithi179.stdout:1/612: fdatasync d3/d5/d2f/d4f/d42/d53/d72/d86/f90 0 2024-04-02T04:47:55.367 INFO:tasks.workunit.client.0.smithi070.stdout:2/559: creat d2/d10/d5c/d7a/fb7 x:0 0 0 2024-04-02T04:47:55.368 INFO:tasks.workunit.client.1.smithi179.stdout:8/601: creat d3/d31/dc3/fd1 x:0 0 0 2024-04-02T04:47:55.371 INFO:tasks.workunit.client.0.smithi070.stdout:7/587: dwrite d7/dd/df/d34/f2a [0,4194304] 0 2024-04-02T04:47:55.375 INFO:tasks.workunit.client.1.smithi179.stdout:5/580: getdents d8/d1b/d23/d36/d53/d89 0 2024-04-02T04:47:55.375 INFO:tasks.workunit.client.0.smithi070.stdout:7/588: dread d7/d9/f78 [0,4194304] 0 2024-04-02T04:47:55.376 INFO:tasks.workunit.client.1.smithi179.stdout:5/581: write d8/d1b/d23/f69 [564657,95773] 0 2024-04-02T04:47:55.376 INFO:tasks.workunit.client.1.smithi179.stdout:5/582: write d8/f6c [550960,4089] 0 2024-04-02T04:47:55.379 INFO:tasks.workunit.client.1.smithi179.stdout:5/583: creat d8/d1b/d23/d32/d4e/d50/fb9 x:0 0 0 2024-04-02T04:47:55.381 INFO:tasks.workunit.client.0.smithi070.stdout:7/589: dwrite d7/dd/df/d34/f2a [0,4194304] 0 2024-04-02T04:47:55.387 INFO:tasks.workunit.client.0.smithi070.stdout:7/590: dwrite d7/dd/df/f94 [0,4194304] 0 2024-04-02T04:47:55.387 INFO:tasks.workunit.client.1.smithi179.stdout:5/584: dwrite d8/d1b/d23/d32/d4e/fab [0,4194304] 0 2024-04-02T04:47:55.388 INFO:tasks.workunit.client.0.smithi070.stdout:7/591: read d7/dd/df/d34/d68/f70 [4866398,85536] 0 2024-04-02T04:47:55.393 INFO:tasks.workunit.client.0.smithi070.stdout:7/592: creat d7/dd/fc0 x:0 0 0 2024-04-02T04:47:55.395 INFO:tasks.workunit.client.0.smithi070.stdout:7/593: dread - d7/dd/df/d34/d68/f9c zero size 2024-04-02T04:47:55.397 INFO:tasks.workunit.client.1.smithi179.stdout:5/585: dread d8/d1b/d23/d36/d53/f78 [0,4194304] 0 2024-04-02T04:47:55.401 INFO:tasks.workunit.client.1.smithi179.stdout:5/586: dread d8/d1b/d23/d30/f71 [0,4194304] 0 2024-04-02T04:47:55.402 INFO:tasks.workunit.client.0.smithi070.stdout:7/594: dwrite d7/dd/df/f94 [0,4194304] 0 2024-04-02T04:47:55.406 INFO:tasks.workunit.client.0.smithi070.stdout:7/595: read d7/dd/df/f32 [4090023,25818] 0 2024-04-02T04:47:55.409 INFO:tasks.workunit.client.0.smithi070.stdout:3/587: dwrite fb [0,4194304] 0 2024-04-02T04:47:55.410 INFO:tasks.workunit.client.1.smithi179.stdout:4/644: dwrite d1/d3/dc/d26/d8a/f94 [0,4194304] 0 2024-04-02T04:47:55.410 INFO:tasks.workunit.client.1.smithi179.stdout:5/587: mkdir d8/d1b/d23/d30/d73/dba 0 2024-04-02T04:47:55.411 INFO:tasks.workunit.client.1.smithi179.stdout:4/645: stat d1/c30 0 2024-04-02T04:47:55.416 INFO:tasks.workunit.client.0.smithi070.stdout:4/506: write d1/d3/d9/f6f [259418,111492] 0 2024-04-02T04:47:55.418 INFO:tasks.workunit.client.1.smithi179.stdout:9/615: truncate d0/d1/d11/f78 3822812 0 2024-04-02T04:47:55.419 INFO:tasks.workunit.client.0.smithi070.stdout:7/596: fsync d7/d9/fa 0 2024-04-02T04:47:55.421 INFO:tasks.workunit.client.1.smithi179.stdout:4/646: creat d1/d3/df/d10/d3a/fd1 x:0 0 0 2024-04-02T04:47:55.428 INFO:tasks.workunit.client.1.smithi179.stdout:5/588: rename d8/d1b/d23/d32/d4e/d50/c8d to d8/d1b/d23/d32/d4e/cbb 0 2024-04-02T04:47:55.429 INFO:tasks.workunit.client.1.smithi179.stdout:5/589: chown d8/d1b/d23/fb4 115 1 2024-04-02T04:47:55.429 INFO:tasks.workunit.client.1.smithi179.stdout:5/590: stat d8/d7e/faa 0 2024-04-02T04:47:55.429 INFO:tasks.workunit.client.1.smithi179.stdout:9/616: mkdir d0/d19/d27/d7a/d7e/db9 0 2024-04-02T04:47:55.431 INFO:tasks.workunit.client.1.smithi179.stdout:1/613: write d3/d5/de/f3c [2128136,119128] 0 2024-04-02T04:47:55.432 INFO:tasks.workunit.client.1.smithi179.stdout:4/647: rename d1/f65 to d1/d3/dc/dc1/fd2 0 2024-04-02T04:47:55.433 INFO:tasks.workunit.client.1.smithi179.stdout:9/617: chown d0/d1/d11/d29/f37 85 1 2024-04-02T04:47:55.434 INFO:tasks.workunit.client.0.smithi070.stdout:7/597: mknod d7/d4e/d82/cc1 0 2024-04-02T04:47:55.434 INFO:tasks.workunit.client.0.smithi070.stdout:2/560: write d2/d10/da4/f9b [1004974,109928] 0 2024-04-02T04:47:55.435 INFO:tasks.workunit.client.0.smithi070.stdout:2/561: fdatasync d2/d10/d5c/d7a/fab 0 2024-04-02T04:47:55.437 INFO:tasks.workunit.client.0.smithi070.stdout:3/588: creat d10/d17/fcc x:0 0 0 2024-04-02T04:47:55.437 INFO:tasks.workunit.client.1.smithi179.stdout:8/602: write d3/d32/d21/d49/fb9 [387951,123496] 0 2024-04-02T04:47:55.441 INFO:tasks.workunit.client.1.smithi179.stdout:9/618: mkdir d0/d19/d27/d7a/dba 0 2024-04-02T04:47:55.446 INFO:tasks.workunit.client.1.smithi179.stdout:4/648: dread d1/d3/df/d10/d3a/faf [0,4194304] 0 2024-04-02T04:47:55.447 INFO:tasks.workunit.client.1.smithi179.stdout:5/591: getdents d8/d1b/d23/d32/d4e/d51 0 2024-04-02T04:47:55.447 INFO:tasks.workunit.client.1.smithi179.stdout:8/603: mknod d3/d1b/d2a/db0/cd2 0 2024-04-02T04:47:55.450 INFO:tasks.workunit.client.1.smithi179.stdout:5/592: rmdir d8/d1b/d23/d42/d5a 39 2024-04-02T04:47:55.451 INFO:tasks.workunit.client.1.smithi179.stdout:5/593: dread - d8/d1b/d23/d32/d4e/d51/f9c zero size 2024-04-02T04:47:55.451 INFO:tasks.workunit.client.1.smithi179.stdout:8/604: creat d3/d32/d21/d51/fd3 x:0 0 0 2024-04-02T04:47:55.454 INFO:tasks.workunit.client.1.smithi179.stdout:4/649: rename d1/d15/d7c/d98/c9b to d1/d3/dc/d7a/cd3 0 2024-04-02T04:47:55.455 INFO:tasks.workunit.client.1.smithi179.stdout:5/594: chown d8/d1b/l21 4 1 2024-04-02T04:47:55.455 INFO:tasks.workunit.client.1.smithi179.stdout:9/619: dread d0/d1/d11/d29/d40/f5d [0,4194304] 0 2024-04-02T04:47:55.456 INFO:tasks.workunit.client.1.smithi179.stdout:5/595: read d8/d1b/f26 [414727,102926] 0 2024-04-02T04:47:55.457 INFO:tasks.workunit.client.1.smithi179.stdout:8/605: creat d3/d31/d8f/fd4 x:0 0 0 2024-04-02T04:47:55.458 INFO:tasks.workunit.client.1.smithi179.stdout:5/596: chown d8/d1b/d23/d42 4976185 1 2024-04-02T04:47:55.458 INFO:tasks.workunit.client.1.smithi179.stdout:8/606: write d3/d32/d21/d45/d5a/fb5 [68711,52178] 0 2024-04-02T04:47:55.459 INFO:tasks.workunit.client.1.smithi179.stdout:8/607: chown d3/d32/d21/d50/d56/d94/d84 0 1 2024-04-02T04:47:55.460 INFO:tasks.workunit.client.1.smithi179.stdout:4/650: rename d1/d3/d19/c1e to d1/d3/df/d10/d2f/d60/cd4 0 2024-04-02T04:47:55.463 INFO:tasks.workunit.client.0.smithi070.stdout:4/507: creat d1/d3/d78/d91/fa7 x:0 0 0 2024-04-02T04:47:55.465 INFO:tasks.workunit.client.1.smithi179.stdout:5/597: creat d8/d7e/fbc x:0 0 0 2024-04-02T04:47:55.466 INFO:tasks.workunit.client.1.smithi179.stdout:6/593: sync 2024-04-02T04:47:55.467 INFO:tasks.workunit.client.1.smithi179.stdout:8/608: dwrite d3/d32/d21/d50/d56/f6a [0,4194304] 0 2024-04-02T04:47:55.469 INFO:tasks.workunit.client.0.smithi070.stdout:4/508: write d1/f93 [4335791,37616] 0 2024-04-02T04:47:55.469 INFO:tasks.workunit.client.1.smithi179.stdout:9/620: creat d0/d1/d11/dab/fbb x:0 0 0 2024-04-02T04:47:55.471 INFO:tasks.workunit.client.1.smithi179.stdout:8/609: fdatasync d3/d32/d21/f25 0 2024-04-02T04:47:55.472 INFO:tasks.workunit.client.1.smithi179.stdout:6/594: creat d3/d42/fc8 x:0 0 0 2024-04-02T04:47:55.473 INFO:tasks.workunit.client.0.smithi070.stdout:2/562: dwrite d2/d24/f8f [0,4194304] 0 2024-04-02T04:47:55.474 INFO:tasks.workunit.client.0.smithi070.stdout:3/589: dwrite d10/d32/d3f/f1f [0,4194304] 0 2024-04-02T04:47:55.475 INFO:tasks.workunit.client.1.smithi179.stdout:4/651: dwrite d1/d3/df/f48 [4194304,4194304] 0 2024-04-02T04:47:55.475 INFO:tasks.workunit.client.1.smithi179.stdout:1/614: dwrite d3/d5/de/f1a [0,4194304] 0 2024-04-02T04:47:55.477 INFO:tasks.workunit.client.1.smithi179.stdout:6/595: write d3/d12/d23/f55 [4415515,4686] 0 2024-04-02T04:47:55.477 INFO:tasks.workunit.client.0.smithi070.stdout:4/509: mkdir d1/d3/d85/da8 0 2024-04-02T04:47:55.478 INFO:tasks.workunit.client.1.smithi179.stdout:5/598: fsync d8/d1b/d23/d32/f7a 0 2024-04-02T04:47:55.478 INFO:tasks.workunit.client.1.smithi179.stdout:1/615: write d3/d5/de/f16 [8844215,61801] 0 2024-04-02T04:47:55.480 INFO:tasks.workunit.client.1.smithi179.stdout:5/599: chown d8/d1b/d23/d32/d4e 77 1 2024-04-02T04:47:55.481 INFO:tasks.workunit.client.1.smithi179.stdout:9/621: rmdir d0/d19/d27/d7a/d82 39 2024-04-02T04:47:55.487 INFO:tasks.workunit.client.0.smithi070.stdout:4/510: rmdir d1/d9c 39 2024-04-02T04:47:55.493 INFO:tasks.workunit.client.0.smithi070.stdout:4/511: stat d1/d9c/f45 0 2024-04-02T04:47:55.494 INFO:tasks.workunit.client.0.smithi070.stdout:2/563: dread d2/d10/d5c/d7a/f92 [0,4194304] 0 2024-04-02T04:47:55.498 INFO:tasks.workunit.client.0.smithi070.stdout:4/512: creat d1/d9c/d65/d7a/fa9 x:0 0 0 2024-04-02T04:47:55.500 INFO:tasks.workunit.client.1.smithi179.stdout:8/610: link l2 d3/d32/d21/d45/ld5 0 2024-04-02T04:47:55.501 INFO:tasks.workunit.client.1.smithi179.stdout:8/611: write d3/d1b/fa4 [7839327,42206] 0 2024-04-02T04:47:55.502 INFO:tasks.workunit.client.0.smithi070.stdout:2/564: mknod d2/d10/cb8 0 2024-04-02T04:47:55.502 INFO:tasks.workunit.client.1.smithi179.stdout:8/612: write d3/d32/d21/fab [290282,106838] 0 2024-04-02T04:47:55.504 INFO:tasks.workunit.client.0.smithi070.stdout:2/565: rmdir d2/d10/d19 39 2024-04-02T04:47:55.505 INFO:tasks.workunit.client.0.smithi070.stdout:2/566: chown d2/d24/f96 3769243 1 2024-04-02T04:47:55.508 INFO:tasks.workunit.client.1.smithi179.stdout:8/613: link d3/d32/d21/d50/d56/d94/c8a d3/d32/d21/d50/d56/dad/cd6 0 2024-04-02T04:47:55.508 INFO:tasks.workunit.client.0.smithi070.stdout:4/513: getdents d1/d9c/d65 0 2024-04-02T04:47:55.509 INFO:tasks.workunit.client.1.smithi179.stdout:2/481: sync 2024-04-02T04:47:55.512 INFO:tasks.workunit.client.0.smithi070.stdout:2/567: truncate d2/d10/d21/d71/f76 2625072 0 2024-04-02T04:47:55.513 INFO:tasks.workunit.client.1.smithi179.stdout:8/614: dwrite d3/d32/d21/d50/d56/f74 [0,4194304] 0 2024-04-02T04:47:55.513 INFO:tasks.workunit.client.0.smithi070.stdout:4/514: rename d1/f46 to d1/d3/d78/d91/faa 0 2024-04-02T04:47:55.514 INFO:tasks.workunit.client.1.smithi179.stdout:2/482: dwrite dc/d29/d4a/d1e/d43/f6e [0,4194304] 0 2024-04-02T04:47:55.515 INFO:tasks.workunit.client.1.smithi179.stdout:8/615: dread - d3/d32/d21/d22/d6b/d91/fc0 zero size 2024-04-02T04:47:55.518 INFO:tasks.workunit.client.1.smithi179.stdout:2/483: creat dc/d29/d31/d83/f9d x:0 0 0 2024-04-02T04:47:55.518 INFO:tasks.workunit.client.1.smithi179.stdout:8/616: rmdir d3/d1b/d2a/db0 39 2024-04-02T04:47:55.519 INFO:tasks.workunit.client.0.smithi070.stdout:4/515: link d1/d3/d9/l23 d1/d3/d78/d91/lab 0 2024-04-02T04:47:55.519 INFO:tasks.workunit.client.0.smithi070.stdout:4/516: chown d1/d3/f89 9 1 2024-04-02T04:47:55.520 INFO:tasks.workunit.client.1.smithi179.stdout:2/484: fsync dc/d29/d4a/d1e/d6c/f87 0 2024-04-02T04:47:55.521 INFO:tasks.workunit.client.1.smithi179.stdout:2/485: stat dc/d29/d4a/f56 0 2024-04-02T04:47:55.522 INFO:tasks.workunit.client.1.smithi179.stdout:2/486: write dc/f44 [2406917,123590] 0 2024-04-02T04:47:55.523 INFO:tasks.workunit.client.0.smithi070.stdout:4/517: creat d1/d3/d85/d8e/fac x:0 0 0 2024-04-02T04:47:55.524 INFO:tasks.workunit.client.1.smithi179.stdout:2/487: write dc/d29/d4a/d1e/d43/f7f [5201516,102338] 0 2024-04-02T04:47:55.529 INFO:tasks.workunit.client.1.smithi179.stdout:2/488: dwrite dc/f50 [0,4194304] 0 2024-04-02T04:47:55.530 INFO:tasks.workunit.client.1.smithi179.stdout:2/489: write dc/d29/d31/d83/f9d [500305,26610] 0 2024-04-02T04:47:55.533 INFO:tasks.workunit.client.1.smithi179.stdout:2/490: dread dc/f5d [0,4194304] 0 2024-04-02T04:47:55.536 INFO:tasks.workunit.client.1.smithi179.stdout:2/491: creat dc/d29/d4a/d1e/d6c/f9e x:0 0 0 2024-04-02T04:47:55.544 INFO:tasks.workunit.client.1.smithi179.stdout:8/617: dread d3/d32/d21/d50/d56/d94/d84/fbd [0,4194304] 0 2024-04-02T04:47:55.547 INFO:tasks.workunit.client.1.smithi179.stdout:8/618: symlink d3/d32/d21/d45/d5a/ld7 0 2024-04-02T04:47:55.552 INFO:tasks.workunit.client.1.smithi179.stdout:8/619: dwrite d3/d6c/d96/fa1 [0,4194304] 0 2024-04-02T04:47:55.555 INFO:tasks.workunit.client.1.smithi179.stdout:8/620: fsync d3/d32/d21/d49/f52 0 2024-04-02T04:47:55.560 INFO:tasks.workunit.client.1.smithi179.stdout:8/621: link d3/d31/d8f/fd4 d3/d1b/d2a/fd8 0 2024-04-02T04:47:55.563 INFO:tasks.workunit.client.1.smithi179.stdout:8/622: symlink d3/d31/ld9 0 2024-04-02T04:47:55.563 INFO:tasks.workunit.client.1.smithi179.stdout:4/652: truncate d1/d3/df/d10/f2d 3556887 0 2024-04-02T04:47:55.564 INFO:tasks.workunit.client.1.smithi179.stdout:8/623: write d3/d31/dc3/fd1 [787208,41465] 0 2024-04-02T04:47:55.565 INFO:tasks.workunit.client.1.smithi179.stdout:5/600: truncate d8/d1b/d23/d42/d5a/f76 850884 0 2024-04-02T04:47:55.570 INFO:tasks.workunit.client.1.smithi179.stdout:4/653: chown d1/d3/df/d10/f2d 1012177 1 2024-04-02T04:47:55.570 INFO:tasks.workunit.client.1.smithi179.stdout:6/596: dwrite d3/d12/f62 [0,4194304] 0 2024-04-02T04:47:55.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:55 smithi179 ceph-mon[36603]: pgmap v16: 65 pgs: 65 active+clean; 2.0 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 68 MiB/s rd, 254 MiB/s wr, 437 op/s 2024-04-02T04:47:55.574 INFO:tasks.workunit.client.1.smithi179.stdout:5/601: mknod d8/d1b/d23/d36/d53/d62/db2/cbd 0 2024-04-02T04:47:55.575 INFO:tasks.workunit.client.1.smithi179.stdout:1/616: dwrite d3/d5/d2f/d4f/f4c [0,4194304] 0 2024-04-02T04:47:55.575 INFO:tasks.workunit.client.0.smithi070.stdout:4/518: write d1/d3/f39 [3471210,68458] 0 2024-04-02T04:47:55.578 INFO:tasks.workunit.client.0.smithi070.stdout:4/519: mkdir d1/d3/d78/dad 0 2024-04-02T04:47:55.579 INFO:tasks.workunit.client.1.smithi179.stdout:1/617: readlink d3/d5/l2b 0 2024-04-02T04:47:55.580 INFO:tasks.workunit.client.1.smithi179.stdout:6/597: mknod d3/d42/dbc/cc9 0 2024-04-02T04:47:55.582 INFO:tasks.workunit.client.1.smithi179.stdout:1/618: readlink d3/d5/l2b 0 2024-04-02T04:47:55.583 INFO:tasks.workunit.client.1.smithi179.stdout:8/624: dwrite d3/d32/d21/d45/d5a/fb5 [0,4194304] 0 2024-04-02T04:47:55.583 INFO:tasks.workunit.client.1.smithi179.stdout:6/598: chown d3/d42/l7b 52 1 2024-04-02T04:47:55.586 INFO:tasks.workunit.client.0.smithi070.stdout:4/520: rename d1/d3/d9/c76 to d1/d3/cae 0 2024-04-02T04:47:55.589 INFO:tasks.workunit.client.1.smithi179.stdout:2/492: dwrite dc/d29/d4a/f8c [0,4194304] 0 2024-04-02T04:47:55.592 INFO:tasks.workunit.client.1.smithi179.stdout:8/625: creat d3/d32/d21/d49/fda x:0 0 0 2024-04-02T04:47:55.594 INFO:tasks.workunit.client.0.smithi070.stdout:4/521: read d1/fe [151393,19279] 0 2024-04-02T04:47:55.596 INFO:tasks.workunit.client.1.smithi179.stdout:6/599: creat d3/d42/dbe/fca x:0 0 0 2024-04-02T04:47:55.596 INFO:tasks.workunit.client.1.smithi179.stdout:1/619: mkdir d3/d5/d2f/d4f/d35/d49/db6/dd8 0 2024-04-02T04:47:55.597 INFO:tasks.workunit.client.0.smithi070.stdout:4/522: creat d1/d3/d78/d91/faf x:0 0 0 2024-04-02T04:47:55.599 INFO:tasks.workunit.client.1.smithi179.stdout:5/602: rename d8/d1b/d23/d36/d53/d89/dac to d8/d1b/d23/d32/dbe 0 2024-04-02T04:47:55.601 INFO:tasks.workunit.client.0.smithi070.stdout:4/523: rmdir d1/d3/d85 39 2024-04-02T04:47:55.603 INFO:tasks.workunit.client.1.smithi179.stdout:6/600: fsync d3/d12/d23/d39/f71 0 2024-04-02T04:47:55.606 INFO:tasks.workunit.client.1.smithi179.stdout:1/620: rename d3/d5/d2f/d4f/d35/d49/l75 to d3/d5/d2f/d36/ld9 0 2024-04-02T04:47:55.606 INFO:tasks.workunit.client.1.smithi179.stdout:4/654: truncate d1/d3/d19/f2c 144808 0 2024-04-02T04:47:55.607 INFO:tasks.workunit.client.1.smithi179.stdout:1/621: chown d3/d5/d2f/d4f/d42/d53/d72/d86 518870817 1 2024-04-02T04:47:55.607 INFO:tasks.workunit.client.0.smithi070.stdout:4/524: mknod d1/d3/d78/da3/cb0 0 2024-04-02T04:47:55.609 INFO:tasks.workunit.client.1.smithi179.stdout:5/603: fsync d8/d1b/d23/d32/d6e/f90 0 2024-04-02T04:47:55.612 INFO:tasks.workunit.client.1.smithi179.stdout:8/626: link d3/d31/f7e d3/d1b/d2a/fdb 0 2024-04-02T04:47:55.612 INFO:tasks.workunit.client.0.smithi070.stdout:4/525: truncate d1/f32 1770207 0 2024-04-02T04:47:55.614 INFO:tasks.workunit.client.1.smithi179.stdout:8/627: write d3/d32/d21/d22/d6b/fb3 [329061,48710] 0 2024-04-02T04:47:55.614 INFO:tasks.workunit.client.1.smithi179.stdout:2/493: getdents dc/d29/d4a/d1e 0 2024-04-02T04:47:55.618 INFO:tasks.workunit.client.1.smithi179.stdout:8/628: dread d3/d32/d21/d50/d65/faa [0,4194304] 0 2024-04-02T04:47:55.619 INFO:tasks.workunit.client.1.smithi179.stdout:8/629: readlink d3/d32/d21/d22/da5/lac 0 2024-04-02T04:47:55.619 INFO:tasks.workunit.client.0.smithi070.stdout:4/526: mkdir d1/d3/d85/d8e/d99/db1 0 2024-04-02T04:47:55.622 INFO:tasks.workunit.client.1.smithi179.stdout:4/655: truncate d1/d3/dc/dc1/fd2 1038141 0 2024-04-02T04:47:55.623 INFO:tasks.workunit.client.0.smithi070.stdout:4/527: write d1/d9c/f79 [749686,123586] 0 2024-04-02T04:47:55.624 INFO:tasks.workunit.client.1.smithi179.stdout:1/622: rename d3/l7f to d3/lda 0 2024-04-02T04:47:55.625 INFO:tasks.workunit.client.1.smithi179.stdout:4/656: dread d1/d3/dc/d26/d8a/f55 [4194304,4194304] 0 2024-04-02T04:47:55.625 INFO:tasks.workunit.client.1.smithi179.stdout:8/630: mknod d3/d32/d21/d22/d6b/d91/cdc 0 2024-04-02T04:47:55.626 INFO:tasks.workunit.client.1.smithi179.stdout:6/601: dread d3/d12/f97 [0,4194304] 0 2024-04-02T04:47:55.628 INFO:tasks.workunit.client.1.smithi179.stdout:4/657: mknod d1/d93/cd5 0 2024-04-02T04:47:55.629 INFO:tasks.workunit.client.1.smithi179.stdout:2/494: creat dc/f9f x:0 0 0 2024-04-02T04:47:55.636 INFO:tasks.workunit.client.1.smithi179.stdout:8/631: write d3/d32/d21/d49/f52 [85117,130550] 0 2024-04-02T04:47:55.637 INFO:tasks.workunit.client.1.smithi179.stdout:4/658: symlink d1/d3/df/d10/d2f/ld6 0 2024-04-02T04:47:55.639 INFO:tasks.workunit.client.1.smithi179.stdout:8/632: read d3/d1b/d2a/f8b [359150,93331] 0 2024-04-02T04:47:55.639 INFO:tasks.workunit.client.0.smithi070.stdout:4/528: dwrite d1/d9c/f6e [0,4194304] 0 2024-04-02T04:47:55.640 INFO:tasks.workunit.client.1.smithi179.stdout:1/623: rename d3/d5/d2f/d4f/d42/d53/f9f to d3/d5/d2f/d4f/d42/d53/fdb 0 2024-04-02T04:47:55.642 INFO:tasks.workunit.client.1.smithi179.stdout:6/602: rename d3/d12/d23/d39/cb8 to d3/d12/d23/ccb 0 2024-04-02T04:47:55.643 INFO:tasks.workunit.client.1.smithi179.stdout:6/603: fdatasync d3/d12/d23/f55 0 2024-04-02T04:47:55.643 INFO:tasks.workunit.client.1.smithi179.stdout:4/659: symlink d1/d3/df/d10/d3a/ld7 0 2024-04-02T04:47:55.645 INFO:tasks.workunit.client.1.smithi179.stdout:8/633: fdatasync d3/d31/f7f 0 2024-04-02T04:47:55.646 INFO:tasks.workunit.client.1.smithi179.stdout:1/624: fsync d3/d5/f25 0 2024-04-02T04:47:55.647 INFO:tasks.workunit.client.1.smithi179.stdout:8/634: read d3/d32/d21/d50/d56/f6a [3521225,47982] 0 2024-04-02T04:47:55.648 INFO:tasks.workunit.client.1.smithi179.stdout:2/495: dread dc/d29/d4a/d1e/f7b [0,4194304] 0 2024-04-02T04:47:55.649 INFO:tasks.workunit.client.1.smithi179.stdout:4/660: mknod d1/d3/dc/d27/cd8 0 2024-04-02T04:47:55.654 INFO:tasks.workunit.client.0.smithi070.stdout:4/529: dread d1/d3/f62 [0,4194304] 0 2024-04-02T04:47:55.654 INFO:tasks.workunit.client.1.smithi179.stdout:4/661: readlink d1/d3/d19/la9 0 2024-04-02T04:47:55.654 INFO:tasks.workunit.client.1.smithi179.stdout:4/662: chown d1/cd0 7782 1 2024-04-02T04:47:55.655 INFO:tasks.workunit.client.1.smithi179.stdout:2/496: dwrite dc/d29/f8b [0,4194304] 0 2024-04-02T04:47:55.657 INFO:tasks.workunit.client.1.smithi179.stdout:8/635: symlink d3/d32/d21/d50/d56/d94/d84/ldd 0 2024-04-02T04:47:55.658 INFO:tasks.workunit.client.1.smithi179.stdout:8/636: stat d3/c1f 0 2024-04-02T04:47:55.659 INFO:tasks.workunit.client.1.smithi179.stdout:1/625: mknod d3/d85/da2/dd1/cdc 0 2024-04-02T04:47:55.660 INFO:tasks.workunit.client.1.smithi179.stdout:2/497: dread dc/d29/d31/d57/f6b [0,4194304] 0 2024-04-02T04:47:55.661 INFO:tasks.workunit.client.1.smithi179.stdout:1/626: stat d3/d5/d2f/d36/f94 0 2024-04-02T04:47:55.661 INFO:tasks.workunit.client.1.smithi179.stdout:4/663: symlink d1/d15/d6e/ld9 0 2024-04-02T04:47:55.661 INFO:tasks.workunit.client.1.smithi179.stdout:1/627: stat d3/d5/f6d 0 2024-04-02T04:47:55.662 INFO:tasks.workunit.client.0.smithi070.stdout:4/530: creat d1/d3/fb2 x:0 0 0 2024-04-02T04:47:55.664 INFO:tasks.workunit.client.1.smithi179.stdout:6/604: getdents d3/d12/d23/d39/d80/d93 0 2024-04-02T04:47:55.666 INFO:tasks.workunit.client.0.smithi070.stdout:4/531: chown d1/f32 10 1 2024-04-02T04:47:55.666 INFO:tasks.workunit.client.0.smithi070.stdout:8/561: sync 2024-04-02T04:47:55.667 INFO:tasks.workunit.client.1.smithi179.stdout:2/498: rmdir dc/d29/d31/d66 39 2024-04-02T04:47:55.667 INFO:tasks.workunit.client.1.smithi179.stdout:2/499: fdatasync dc/d29/d4a/f53 0 2024-04-02T04:47:55.667 INFO:tasks.workunit.client.0.smithi070.stdout:8/562: chown d2/d3/d35/d3a/d4f/c85 8347 1 2024-04-02T04:47:55.668 INFO:tasks.workunit.client.1.smithi179.stdout:2/500: read dc/fe [4188927,79466] 0 2024-04-02T04:47:55.669 INFO:tasks.workunit.client.1.smithi179.stdout:4/664: rename d1/d3/f5b to d1/d3/df/d10/d2f/d60/fda 0 2024-04-02T04:47:55.670 INFO:tasks.workunit.client.0.smithi070.stdout:4/532: mknod d1/d3/d85/d8e/cb3 0 2024-04-02T04:47:55.673 INFO:tasks.workunit.client.0.smithi070.stdout:8/563: creat d2/d3/d52/faf x:0 0 0 2024-04-02T04:47:55.674 INFO:tasks.workunit.client.0.smithi070.stdout:8/564: chown d2/d3/f9 698 1 2024-04-02T04:47:55.674 INFO:tasks.workunit.client.1.smithi179.stdout:8/637: symlink d3/lde 0 2024-04-02T04:47:55.675 INFO:tasks.workunit.client.1.smithi179.stdout:1/628: write d3/d5/f6d [477926,13680] 0 2024-04-02T04:47:55.676 INFO:tasks.workunit.client.1.smithi179.stdout:6/605: rmdir d3/d12/d44 39 2024-04-02T04:47:55.678 INFO:tasks.workunit.client.1.smithi179.stdout:6/606: dread - d3/d12/d23/d26/faf zero size 2024-04-02T04:47:55.683 INFO:tasks.workunit.client.0.smithi070.stdout:4/533: dread d1/f28 [0,4194304] 0 2024-04-02T04:47:55.683 INFO:tasks.workunit.client.1.smithi179.stdout:4/665: dread d1/d3/dc/d27/f7f [0,4194304] 0 2024-04-02T04:47:55.684 INFO:tasks.workunit.client.1.smithi179.stdout:8/638: creat d3/d1b/fdf x:0 0 0 2024-04-02T04:47:55.685 INFO:tasks.workunit.client.1.smithi179.stdout:1/629: fdatasync d3/d5/d2f/f81 0 2024-04-02T04:47:55.685 INFO:tasks.workunit.client.0.smithi070.stdout:8/565: write d2/d8/d23/d24/f5b [1998001,39156] 0 2024-04-02T04:47:55.689 INFO:tasks.workunit.client.0.smithi070.stdout:8/566: readlink d2/d3/d52/l11 0 2024-04-02T04:47:55.691 INFO:tasks.workunit.client.1.smithi179.stdout:1/630: dwrite d3/d5/f60 [4194304,4194304] 0 2024-04-02T04:47:55.693 INFO:tasks.workunit.client.1.smithi179.stdout:4/666: creat d1/d3/df/d10/d2f/d36/fdb x:0 0 0 2024-04-02T04:47:55.693 INFO:tasks.workunit.client.0.smithi070.stdout:8/567: creat d2/d8/d23/d24/d5f/fb0 x:0 0 0 2024-04-02T04:47:55.696 INFO:tasks.workunit.client.0.smithi070.stdout:4/534: dwrite d1/d3/d9/f81 [0,4194304] 0 2024-04-02T04:47:55.696 INFO:tasks.workunit.client.1.smithi179.stdout:6/607: dread d3/d12/d23/d2f/f3a [0,4194304] 0 2024-04-02T04:47:55.697 INFO:tasks.workunit.client.1.smithi179.stdout:6/608: fsync d3/d12/fc4 0 2024-04-02T04:47:55.697 INFO:tasks.workunit.client.0.smithi070.stdout:4/535: dread - d1/d3/d78/d91/fa7 zero size 2024-04-02T04:47:55.698 INFO:tasks.workunit.client.1.smithi179.stdout:4/667: mknod d1/d3/df/d10/d2f/d60/d71/cdc 0 2024-04-02T04:47:55.698 INFO:tasks.workunit.client.1.smithi179.stdout:1/631: symlink d3/d5/d96/ldd 0 2024-04-02T04:47:55.700 INFO:tasks.workunit.client.1.smithi179.stdout:4/668: write d1/d3/dc/d26/d8a/d5d/fbc [544472,96118] 0 2024-04-02T04:47:55.701 INFO:tasks.workunit.client.0.smithi070.stdout:8/568: mknod d2/cb1 0 2024-04-02T04:47:55.701 INFO:tasks.workunit.client.1.smithi179.stdout:4/669: fsync d1/d3/df/d10/d2f/d7b/f91 0 2024-04-02T04:47:55.701 INFO:tasks.workunit.client.1.smithi179.stdout:4/670: stat d1/cd0 0 2024-04-02T04:47:55.702 INFO:tasks.workunit.client.0.smithi070.stdout:4/536: dread d1/d3/d9/f2a [0,4194304] 0 2024-04-02T04:47:55.702 INFO:tasks.workunit.client.0.smithi070.stdout:8/569: fdatasync d2/d8/d29/d22/d5a/d94/da2/fad 0 2024-04-02T04:47:55.704 INFO:tasks.workunit.client.1.smithi179.stdout:7/662: sync 2024-04-02T04:47:55.704 INFO:tasks.workunit.client.1.smithi179.stdout:0/659: sync 2024-04-02T04:47:55.704 INFO:tasks.workunit.client.1.smithi179.stdout:3/644: sync 2024-04-02T04:47:55.704 INFO:tasks.workunit.client.1.smithi179.stdout:1/632: dread - d3/d5/f8d zero size 2024-04-02T04:47:55.705 INFO:tasks.workunit.client.1.smithi179.stdout:0/660: chown d1/db/d64/c69 99204923 1 2024-04-02T04:47:55.706 INFO:tasks.workunit.client.0.smithi070.stdout:4/537: creat d1/d9c/d65/fb4 x:0 0 0 2024-04-02T04:47:55.707 INFO:tasks.workunit.client.0.smithi070.stdout:8/570: mknod d2/d3/d35/d3a/d4f/cb2 0 2024-04-02T04:47:55.708 INFO:tasks.workunit.client.0.smithi070.stdout:8/571: chown d2/c19 7561412 1 2024-04-02T04:47:55.709 INFO:tasks.workunit.client.1.smithi179.stdout:4/671: creat d1/d3/df/d10/d2f/d60/fdd x:0 0 0 2024-04-02T04:47:55.709 INFO:tasks.workunit.client.0.smithi070.stdout:4/538: mknod d1/d3/d78/d91/cb5 0 2024-04-02T04:47:55.711 INFO:tasks.workunit.client.0.smithi070.stdout:8/572: rename d2/d8/d29/d22/f60 to d2/d3/d35/fb3 0 2024-04-02T04:47:55.712 INFO:tasks.workunit.client.1.smithi179.stdout:1/633: creat d3/d5/d2f/d4f/dab/fde x:0 0 0 2024-04-02T04:47:55.715 INFO:tasks.workunit.client.1.smithi179.stdout:1/634: write d3/da8/fb3 [851843,127328] 0 2024-04-02T04:47:55.715 INFO:tasks.workunit.client.1.smithi179.stdout:0/661: mknod d1/da/d67/cc4 0 2024-04-02T04:47:55.716 INFO:tasks.workunit.client.1.smithi179.stdout:8/639: link d3/d32/d21/d50/d56/dad/cd6 d3/d31/d8f/ce0 0 2024-04-02T04:47:55.717 INFO:tasks.workunit.client.0.smithi070.stdout:8/573: dwrite d2/d3/d35/f78 [0,4194304] 0 2024-04-02T04:47:55.717 INFO:tasks.workunit.client.1.smithi179.stdout:0/662: stat d1/da/d48/d7b/fb6 0 2024-04-02T04:47:55.720 INFO:tasks.workunit.client.1.smithi179.stdout:4/672: dwrite d1/d3/df/d10/d2f/d36/fdb [0,4194304] 0 2024-04-02T04:47:55.721 INFO:tasks.workunit.client.1.smithi179.stdout:0/663: chown d1/da/d48/d5c/d83/d97 10 1 2024-04-02T04:47:55.722 INFO:tasks.workunit.client.1.smithi179.stdout:1/635: truncate d3/d5/d2f/d4f/fc1 170401 0 2024-04-02T04:47:55.723 INFO:tasks.workunit.client.1.smithi179.stdout:3/645: dread d8/fac [0,4194304] 0 2024-04-02T04:47:55.726 INFO:tasks.workunit.client.1.smithi179.stdout:8/640: mknod d3/d32/d21/ce1 0 2024-04-02T04:47:55.726 INFO:tasks.workunit.client.0.smithi070.stdout:4/539: fdatasync d1/d9c/d65/d7a/f7d 0 2024-04-02T04:47:55.731 INFO:tasks.workunit.client.1.smithi179.stdout:6/609: dread d3/d12/d23/d39/d4e/d6c/f50 [0,4194304] 0 2024-04-02T04:47:55.731 INFO:tasks.workunit.client.1.smithi179.stdout:7/663: write d1/d9/d27/d87/d8b/d75/db9/d8d/f8f [692285,31842] 0 2024-04-02T04:47:55.732 INFO:tasks.workunit.client.0.smithi070.stdout:4/540: rmdir d1/d9c/d65/d7a/da1 0 2024-04-02T04:47:55.733 INFO:tasks.workunit.client.1.smithi179.stdout:4/673: creat d1/d3/df/d10/d2f/d60/d71/fde x:0 0 0 2024-04-02T04:47:55.737 INFO:tasks.workunit.client.1.smithi179.stdout:0/664: dwrite d1/db/f8e [0,4194304] 0 2024-04-02T04:47:55.737 INFO:tasks.workunit.client.0.smithi070.stdout:8/574: dwrite d2/d3/d35/d3a/d4f/d59/f8f [0,4194304] 0 2024-04-02T04:47:55.739 INFO:tasks.workunit.client.1.smithi179.stdout:6/610: creat d3/d12/d23/d26/fcc x:0 0 0 2024-04-02T04:47:55.739 INFO:tasks.workunit.client.1.smithi179.stdout:1/636: dwrite d3/d5/d96/fb4 [0,4194304] 0 2024-04-02T04:47:55.741 INFO:tasks.workunit.client.1.smithi179.stdout:0/665: dread - d1/da/df/fa2 zero size 2024-04-02T04:47:55.741 INFO:tasks.workunit.client.1.smithi179.stdout:8/641: rename d3/d31/dc3/fd1 to d3/d32/d21/d50/d56/fe2 0 2024-04-02T04:47:55.742 INFO:tasks.workunit.client.0.smithi070.stdout:4/541: dwrite d1/fd [0,4194304] 0 2024-04-02T04:47:55.750 INFO:tasks.workunit.client.1.smithi179.stdout:6/611: unlink d3/d12/d23/d26/f27 0 2024-04-02T04:47:55.751 INFO:tasks.workunit.client.0.smithi070.stdout:4/542: fsync d1/fa 0 2024-04-02T04:47:55.753 INFO:tasks.workunit.client.1.smithi179.stdout:3/646: getdents d8/d26/d83/d89/d9c 0 2024-04-02T04:47:55.754 INFO:tasks.workunit.client.1.smithi179.stdout:6/612: chown d3/d12/d23/d39/d4e/d66/d7c/lc7 460326 1 2024-04-02T04:47:55.754 INFO:tasks.workunit.client.1.smithi179.stdout:3/647: fsync d8/d6d/fb6 0 2024-04-02T04:47:55.755 INFO:tasks.workunit.client.1.smithi179.stdout:0/666: creat d1/da/df/d56/d84/fc5 x:0 0 0 2024-04-02T04:47:55.757 INFO:tasks.workunit.client.1.smithi179.stdout:8/642: read d3/d1b/fa2 [1715592,39152] 0 2024-04-02T04:47:55.758 INFO:tasks.workunit.client.0.smithi070.stdout:4/543: dwrite d1/f93 [0,4194304] 0 2024-04-02T04:47:55.758 INFO:tasks.workunit.client.1.smithi179.stdout:1/637: truncate d3/d5/de/f57 4147376 0 2024-04-02T04:47:55.763 INFO:tasks.workunit.client.1.smithi179.stdout:3/648: dwrite d8/d26/d36/f90 [4194304,4194304] 0 2024-04-02T04:47:55.765 INFO:tasks.workunit.client.1.smithi179.stdout:0/667: truncate d1/da/d28/d34/d39/d77/f9b 20518 0 2024-04-02T04:47:55.767 INFO:tasks.workunit.client.1.smithi179.stdout:8/643: getdents d3/d32/d30 0 2024-04-02T04:47:55.775 INFO:tasks.workunit.client.1.smithi179.stdout:1/638: dread d3/d5/d2f/d4f/f37 [0,4194304] 0 2024-04-02T04:47:55.777 INFO:tasks.workunit.client.1.smithi179.stdout:8/644: dread d3/d1b/fa2 [0,4194304] 0 2024-04-02T04:47:55.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:55 smithi070 ceph-mon[29212]: pgmap v16: 65 pgs: 65 active+clean; 2.0 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 68 MiB/s rd, 254 MiB/s wr, 437 op/s 2024-04-02T04:47:55.781 INFO:tasks.workunit.client.1.smithi179.stdout:1/639: getdents d3/d5/d2f/d4f/d35/d49/db6 0 2024-04-02T04:47:55.781 INFO:tasks.workunit.client.1.smithi179.stdout:4/674: dread d1/d3/fa [0,4194304] 0 2024-04-02T04:47:55.781 INFO:tasks.workunit.client.1.smithi179.stdout:8/645: truncate d3/d32/d21/d22/f2d 46324 0 2024-04-02T04:47:55.782 INFO:tasks.workunit.client.0.smithi070.stdout:8/575: dwrite d2/d3/d35/d3a/d4f/d59/f90 [0,4194304] 0 2024-04-02T04:47:55.782 INFO:tasks.workunit.client.1.smithi179.stdout:7/664: truncate d1/f15 2258499 0 2024-04-02T04:47:55.785 INFO:tasks.workunit.client.1.smithi179.stdout:0/668: dread d1/f36 [0,4194304] 0 2024-04-02T04:47:55.786 INFO:tasks.workunit.client.1.smithi179.stdout:8/646: mknod d3/d32/d21/d50/d56/ce3 0 2024-04-02T04:47:55.788 INFO:tasks.workunit.client.0.smithi070.stdout:8/576: mknod d2/d8/cb4 0 2024-04-02T04:47:55.788 INFO:tasks.workunit.client.0.smithi070.stdout:4/544: write d1/fa [1690012,72664] 0 2024-04-02T04:47:55.788 INFO:tasks.workunit.client.1.smithi179.stdout:9/622: sync 2024-04-02T04:47:55.789 INFO:tasks.workunit.client.0.smithi070.stdout:8/577: fdatasync d2/d3/d35/d3a/d4f/f87 0 2024-04-02T04:47:55.789 INFO:tasks.workunit.client.1.smithi179.stdout:7/665: write d1/dd/d16/fcd [2954693,16408] 0 2024-04-02T04:47:55.790 INFO:tasks.workunit.client.1.smithi179.stdout:1/640: mknod d3/d5/d2f/d4f/d42/d53/d77/cdf 0 2024-04-02T04:47:55.792 INFO:tasks.workunit.client.1.smithi179.stdout:7/666: write d1/d9/d35/fdc [677448,119326] 0 2024-04-02T04:47:55.794 INFO:tasks.workunit.client.0.smithi070.stdout:8/578: dread d2/d3/d35/d3a/d4f/d59/f90 [0,4194304] 0 2024-04-02T04:47:55.795 INFO:tasks.workunit.client.1.smithi179.stdout:7/667: readlink d1/dd/d76/lb5 0 2024-04-02T04:47:55.795 INFO:tasks.workunit.client.1.smithi179.stdout:4/675: creat d1/d3/dc/fdf x:0 0 0 2024-04-02T04:47:55.796 INFO:tasks.workunit.client.0.smithi070.stdout:8/579: write d2/d3/d35/d3a/d4f/fab [768713,33087] 0 2024-04-02T04:47:55.796 INFO:tasks.workunit.client.1.smithi179.stdout:0/669: dwrite d1/da/df/fb5 [0,4194304] 0 2024-04-02T04:47:55.797 INFO:tasks.workunit.client.0.smithi070.stdout:4/545: unlink d1/d3/d85/d8e/f90 0 2024-04-02T04:47:55.797 INFO:tasks.workunit.client.0.smithi070.stdout:4/546: fdatasync d1/f93 0 2024-04-02T04:47:55.799 INFO:tasks.workunit.client.1.smithi179.stdout:6/613: truncate d3/d12/d23/d39/d80/f8f 3007277 0 2024-04-02T04:47:55.802 INFO:tasks.workunit.client.1.smithi179.stdout:9/623: symlink d0/d19/d27/d7a/d7e/lbc 0 2024-04-02T04:47:55.803 INFO:tasks.workunit.client.1.smithi179.stdout:3/649: write d8/d26/f33 [6472155,115869] 0 2024-04-02T04:47:55.804 INFO:tasks.workunit.client.1.smithi179.stdout:7/668: mknod d1/d9/d35/cec 0 2024-04-02T04:47:55.807 INFO:tasks.workunit.client.1.smithi179.stdout:6/614: dread d3/fa [0,4194304] 0 2024-04-02T04:47:55.808 INFO:tasks.workunit.client.1.smithi179.stdout:0/670: write d1/da/d28/d34/f7e [1724698,29676] 0 2024-04-02T04:47:55.810 INFO:tasks.workunit.client.1.smithi179.stdout:1/641: mknod d3/d5/d2f/d4f/d35/d49/db6/dd8/ce0 0 2024-04-02T04:47:55.812 INFO:tasks.workunit.client.1.smithi179.stdout:6/615: dread d3/d2d/d38/d64/fa8 [0,4194304] 0 2024-04-02T04:47:55.812 INFO:tasks.workunit.client.1.smithi179.stdout:1/642: write d3/d5/d2f/d4f/d35/fbf [266250,55798] 0 2024-04-02T04:47:55.813 INFO:tasks.workunit.client.1.smithi179.stdout:9/624: fsync d0/d1/d11/d29/d40/f63 0 2024-04-02T04:47:55.813 INFO:tasks.workunit.client.0.smithi070.stdout:4/547: mknod d1/d3/cb6 0 2024-04-02T04:47:55.815 INFO:tasks.workunit.client.1.smithi179.stdout:8/647: write d3/d32/d21/d50/d56/f6a [1817036,109883] 0 2024-04-02T04:47:55.817 INFO:tasks.workunit.client.0.smithi070.stdout:4/548: truncate d1/d3/f89 675416 0 2024-04-02T04:47:55.821 INFO:tasks.workunit.client.1.smithi179.stdout:0/671: symlink d1/da/d48/d5c/d83/d97/lc6 0 2024-04-02T04:47:55.822 INFO:tasks.workunit.client.1.smithi179.stdout:0/672: chown d1/db/f8e 69383386 1 2024-04-02T04:47:55.823 INFO:tasks.workunit.client.1.smithi179.stdout:0/673: read d1/da/d52/f5d [3524256,41631] 0 2024-04-02T04:47:55.826 INFO:tasks.workunit.client.1.smithi179.stdout:6/616: mkdir d3/d12/d23/d39/d4e/d6c/dcd 0 2024-04-02T04:47:55.830 INFO:tasks.workunit.client.1.smithi179.stdout:7/669: write d1/d9/d27/f49 [3332549,107246] 0 2024-04-02T04:47:55.842 INFO:tasks.workunit.client.1.smithi179.stdout:3/650: write d8/d26/d83/d89/d9c/fc2 [1462465,31161] 0 2024-04-02T04:47:55.844 INFO:tasks.workunit.client.1.smithi179.stdout:3/651: write d8/d26/d52/d85/f97 [410104,121273] 0 2024-04-02T04:47:55.844 INFO:tasks.workunit.client.1.smithi179.stdout:6/617: chown d3/d12/l78 0 1 2024-04-02T04:47:55.845 INFO:tasks.workunit.client.1.smithi179.stdout:6/618: readlink d3/d2d/l31 0 2024-04-02T04:47:55.845 INFO:tasks.workunit.client.1.smithi179.stdout:3/652: readlink d8/d26/d83/d89/l9d 0 2024-04-02T04:47:55.846 INFO:tasks.workunit.client.1.smithi179.stdout:3/653: chown d8/d26/d36/f49 9 1 2024-04-02T04:47:55.850 INFO:tasks.workunit.client.1.smithi179.stdout:7/670: fsync d1/d9/d27/f5f 0 2024-04-02T04:47:55.854 INFO:tasks.workunit.client.0.smithi070.stdout:1/588: sync 2024-04-02T04:47:55.855 INFO:tasks.workunit.client.0.smithi070.stdout:0/631: sync 2024-04-02T04:47:55.855 INFO:tasks.workunit.client.0.smithi070.stdout:1/589: chown d0/d7/f10 492 1 2024-04-02T04:47:55.856 INFO:tasks.workunit.client.0.smithi070.stdout:1/590: dread - d0/d4c/da2/faf zero size 2024-04-02T04:47:55.858 INFO:tasks.workunit.client.1.smithi179.stdout:0/674: dwrite d1/da/d52/fbb [4194304,4194304] 0 2024-04-02T04:47:55.860 INFO:tasks.workunit.client.0.smithi070.stdout:0/632: mknod d4/d3c/cba 0 2024-04-02T04:47:55.860 INFO:tasks.workunit.client.1.smithi179.stdout:7/671: dwrite d1/f4e [0,4194304] 0 2024-04-02T04:47:55.861 INFO:tasks.workunit.client.0.smithi070.stdout:8/580: link d2/c20 d2/d8/d23/d24/cb5 0 2024-04-02T04:47:55.861 INFO:tasks.workunit.client.1.smithi179.stdout:9/625: truncate d0/d1/d11/d29/f37 1101577 0 2024-04-02T04:47:55.863 INFO:tasks.workunit.client.1.smithi179.stdout:6/619: symlink d3/d2d/d38/d64/lce 0 2024-04-02T04:47:55.864 INFO:tasks.workunit.client.0.smithi070.stdout:1/591: rename d0/d7/d28/d83/lbb to d0/d54/lc0 0 2024-04-02T04:47:55.867 INFO:tasks.workunit.client.1.smithi179.stdout:3/654: dread d8/d26/f34 [0,4194304] 0 2024-04-02T04:47:55.868 INFO:tasks.workunit.client.1.smithi179.stdout:8/648: dwrite d3/d32/d21/f37 [0,4194304] 0 2024-04-02T04:47:55.869 INFO:tasks.workunit.client.1.smithi179.stdout:1/643: truncate d3/d5/d2f/d4f/d42/d53/f5e 402369 0 2024-04-02T04:47:55.871 INFO:tasks.workunit.client.0.smithi070.stdout:8/581: stat d2/d8/d29/d22/d5a/d94/ca3 0 2024-04-02T04:47:55.874 INFO:tasks.workunit.client.1.smithi179.stdout:1/644: read d3/d5/f13 [3962558,33293] 0 2024-04-02T04:47:55.874 INFO:tasks.workunit.client.0.smithi070.stdout:4/549: creat d1/d3/d9/fb7 x:0 0 0 2024-04-02T04:47:55.875 INFO:tasks.workunit.client.1.smithi179.stdout:0/675: fdatasync d1/f23 0 2024-04-02T04:47:55.875 INFO:tasks.workunit.client.0.smithi070.stdout:0/633: truncate d4/d5/f3f 1188460 0 2024-04-02T04:47:55.876 INFO:tasks.workunit.client.0.smithi070.stdout:1/592: dread d0/d7/f10 [0,4194304] 0 2024-04-02T04:47:55.877 INFO:tasks.workunit.client.0.smithi070.stdout:0/634: chown d4/d5/d21/cb4 129807144 1 2024-04-02T04:47:55.879 INFO:tasks.workunit.client.1.smithi179.stdout:7/672: read d1/dd/d23/f82 [29761,30388] 0 2024-04-02T04:47:55.879 INFO:tasks.workunit.client.0.smithi070.stdout:8/582: dwrite d2/d8/d23/d24/f5b [0,4194304] 0 2024-04-02T04:47:55.880 INFO:tasks.workunit.client.0.smithi070.stdout:4/550: symlink d1/d3/d9/d5e/lb8 0 2024-04-02T04:47:55.883 INFO:tasks.workunit.client.0.smithi070.stdout:0/635: symlink d4/d5/d58/d60/d83/lbb 0 2024-04-02T04:47:55.885 INFO:tasks.workunit.client.1.smithi179.stdout:8/649: mkdir d3/d32/d21/d50/d56/d94/d84/de4 0 2024-04-02T04:47:55.889 INFO:tasks.workunit.client.0.smithi070.stdout:4/551: dwrite d1/d3/d9/f9b [0,4194304] 0 2024-04-02T04:47:55.890 INFO:tasks.workunit.client.1.smithi179.stdout:0/676: unlink d1/da/d28/d37/f3e 0 2024-04-02T04:47:55.891 INFO:tasks.workunit.client.0.smithi070.stdout:4/552: dread - d1/d9c/d65/fb4 zero size 2024-04-02T04:47:55.891 INFO:tasks.workunit.client.1.smithi179.stdout:0/677: chown d1/da/d52 3 1 2024-04-02T04:47:55.899 INFO:tasks.workunit.client.1.smithi179.stdout:7/673: symlink d1/dd/d23/dab/led 0 2024-04-02T04:47:55.900 INFO:tasks.workunit.client.0.smithi070.stdout:4/553: unlink d1/d3/d78/d91/cb5 0 2024-04-02T04:47:55.901 INFO:tasks.workunit.client.0.smithi070.stdout:4/554: readlink d1/d3/l34 0 2024-04-02T04:47:55.901 INFO:tasks.workunit.client.0.smithi070.stdout:1/593: dread d0/d7/d28/d5c/d6e/f6f [0,4194304] 0 2024-04-02T04:47:55.903 INFO:tasks.workunit.client.1.smithi179.stdout:3/655: creat d8/d57/d63/dc3/fc8 x:0 0 0 2024-04-02T04:47:55.904 INFO:tasks.workunit.client.1.smithi179.stdout:7/674: dread d1/fe2 [0,4194304] 0 2024-04-02T04:47:55.904 INFO:tasks.workunit.client.0.smithi070.stdout:0/636: creat d4/d5/d13/d17/d6e/fbc x:0 0 0 2024-04-02T04:47:55.904 INFO:tasks.workunit.client.0.smithi070.stdout:5/728: sync 2024-04-02T04:47:55.904 INFO:tasks.workunit.client.0.smithi070.stdout:6/483: sync 2024-04-02T04:47:55.905 INFO:tasks.workunit.client.0.smithi070.stdout:5/729: chown d5/d9/c1f 6966 1 2024-04-02T04:47:55.909 INFO:tasks.workunit.client.1.smithi179.stdout:5/604: sync 2024-04-02T04:47:55.909 INFO:tasks.workunit.client.1.smithi179.stdout:2/501: sync 2024-04-02T04:47:55.910 INFO:tasks.workunit.client.1.smithi179.stdout:0/678: creat d1/da/d48/d7b/fc7 x:0 0 0 2024-04-02T04:47:55.912 INFO:tasks.workunit.client.1.smithi179.stdout:0/679: fdatasync d1/f2a 0 2024-04-02T04:47:55.913 INFO:tasks.workunit.client.0.smithi070.stdout:8/583: dread d2/f7f [0,4194304] 0 2024-04-02T04:47:55.914 INFO:tasks.workunit.client.1.smithi179.stdout:0/680: dread - d1/da/d48/d65/d7d/d80/f9e zero size 2024-04-02T04:47:55.914 INFO:tasks.workunit.client.1.smithi179.stdout:6/620: dwrite d3/d12/d23/d26/f76 [0,4194304] 0 2024-04-02T04:47:55.915 INFO:tasks.workunit.client.1.smithi179.stdout:1/645: write d3/d5/f25 [1914188,81588] 0 2024-04-02T04:47:55.916 INFO:tasks.workunit.client.1.smithi179.stdout:3/656: symlink d8/d57/d63/dc3/lc9 0 2024-04-02T04:47:55.916 INFO:tasks.workunit.client.1.smithi179.stdout:1/646: stat d3/d5/d2f/d5a/dcc 0 2024-04-02T04:47:55.917 INFO:tasks.workunit.client.0.smithi070.stdout:1/594: mkdir d0/d9/d1d/d22/d56/dc1 0 2024-04-02T04:47:55.920 INFO:tasks.workunit.client.1.smithi179.stdout:7/675: dread d1/d9/d27/d87/d8b/d75/db9/f92 [0,4194304] 0 2024-04-02T04:47:55.921 INFO:tasks.workunit.client.1.smithi179.stdout:2/502: unlink dc/d29/l84 0 2024-04-02T04:47:55.922 INFO:tasks.workunit.client.0.smithi070.stdout:6/484: stat d0/d43/d95/c96 0 2024-04-02T04:47:55.923 INFO:tasks.workunit.client.1.smithi179.stdout:9/626: write d0/d19/d27/d7a/d7e/fb8 [5629305,38537] 0 2024-04-02T04:47:55.925 INFO:tasks.workunit.client.0.smithi070.stdout:4/555: rename d1/c40 to d1/d98/cb9 0 2024-04-02T04:47:55.927 INFO:tasks.workunit.client.1.smithi179.stdout:5/605: creat d8/d1b/d23/d32/dbe/daf/fbf x:0 0 0 2024-04-02T04:47:55.927 INFO:tasks.workunit.client.0.smithi070.stdout:5/730: creat d5/d9/d10/db9/dd9/fe7 x:0 0 0 2024-04-02T04:47:55.931 INFO:tasks.workunit.client.0.smithi070.stdout:1/595: fsync d0/d7/d1a/f9a 0 2024-04-02T04:47:55.932 INFO:tasks.workunit.client.0.smithi070.stdout:1/596: readlink d0/d7/l53 0 2024-04-02T04:47:55.933 INFO:tasks.workunit.client.1.smithi179.stdout:0/681: creat d1/da/d67/d9f/fc8 x:0 0 0 2024-04-02T04:47:55.935 INFO:tasks.workunit.client.1.smithi179.stdout:1/647: symlink d3/d5/d96/le1 0 2024-04-02T04:47:55.937 INFO:tasks.workunit.client.1.smithi179.stdout:3/657: rename d8/d26/d36/c58 to d8/d26/d83/dbc/cca 0 2024-04-02T04:47:55.940 INFO:tasks.workunit.client.1.smithi179.stdout:2/503: dread dc/d29/d31/d57/f6b [0,4194304] 0 2024-04-02T04:47:55.943 INFO:tasks.workunit.client.1.smithi179.stdout:8/650: dwrite d3/d32/d21/d22/d6b/d91/f44 [0,4194304] 0 2024-04-02T04:47:55.944 INFO:tasks.workunit.client.1.smithi179.stdout:8/651: chown d3/d32/d21/f66 14651 1 2024-04-02T04:47:55.948 INFO:tasks.workunit.client.1.smithi179.stdout:6/621: mknod d3/d2d/ccf 0 2024-04-02T04:47:55.948 INFO:tasks.workunit.client.0.smithi070.stdout:4/556: write d1/d3/d9/f43 [891461,86140] 0 2024-04-02T04:47:55.949 INFO:tasks.workunit.client.0.smithi070.stdout:8/584: dwrite d2/d3/d52/f3c [0,4194304] 0 2024-04-02T04:47:55.951 INFO:tasks.workunit.client.0.smithi070.stdout:0/637: getdents d4/d5/d13/d2a/d3a/d45 0 2024-04-02T04:47:55.954 INFO:tasks.workunit.client.1.smithi179.stdout:4/676: sync 2024-04-02T04:47:55.954 INFO:tasks.workunit.client.0.smithi070.stdout:0/638: read d4/d5/d13/d43/f78 [1457411,116524] 0 2024-04-02T04:47:55.955 INFO:tasks.workunit.client.0.smithi070.stdout:4/557: dread d1/d3/f62 [0,4194304] 0 2024-04-02T04:47:55.956 INFO:tasks.workunit.client.0.smithi070.stdout:9/510: sync 2024-04-02T04:47:55.956 INFO:tasks.workunit.client.0.smithi070.stdout:7/598: sync 2024-04-02T04:47:55.956 INFO:tasks.workunit.client.0.smithi070.stdout:2/568: sync 2024-04-02T04:47:55.956 INFO:tasks.workunit.client.0.smithi070.stdout:3/590: sync 2024-04-02T04:47:55.956 INFO:tasks.workunit.client.0.smithi070.stdout:0/639: chown d4/d5/f5e 5 1 2024-04-02T04:47:55.959 INFO:tasks.workunit.client.0.smithi070.stdout:5/731: rename d5/d9/d23/d8b/lac to d5/d9/d10/d6a/d7d/le8 0 2024-04-02T04:47:55.963 INFO:tasks.workunit.client.1.smithi179.stdout:0/682: dwrite d1/da/d48/d65/d7d/f7a [8388608,4194304] 0 2024-04-02T04:47:55.963 INFO:tasks.workunit.client.0.smithi070.stdout:6/485: mkdir d0/d55/d79/d8d/d97 0 2024-04-02T04:47:55.967 INFO:tasks.workunit.client.0.smithi070.stdout:2/569: creat d2/d24/fb9 x:0 0 0 2024-04-02T04:47:55.972 INFO:tasks.workunit.client.0.smithi070.stdout:1/597: dread d0/d54/d64/fa9 [0,4194304] 0 2024-04-02T04:47:55.973 INFO:tasks.workunit.client.1.smithi179.stdout:2/504: creat dc/d29/d31/d57/fa0 x:0 0 0 2024-04-02T04:47:55.974 INFO:tasks.workunit.client.1.smithi179.stdout:2/505: stat dc/d29/d31 0 2024-04-02T04:47:55.974 INFO:tasks.workunit.client.0.smithi070.stdout:7/599: creat d7/d4e/d82/d57/fc2 x:0 0 0 2024-04-02T04:47:55.978 INFO:tasks.workunit.client.1.smithi179.stdout:5/606: creat d8/d1b/d23/d36/d53/d62/da9/fc0 x:0 0 0 2024-04-02T04:47:55.979 INFO:tasks.workunit.client.1.smithi179.stdout:5/607: chown d8/d1b/d23/d32/d4e 22749 1 2024-04-02T04:47:55.980 INFO:tasks.workunit.client.1.smithi179.stdout:5/608: write d8/d1b/f37 [839072,28147] 0 2024-04-02T04:47:55.981 INFO:tasks.workunit.client.1.smithi179.stdout:1/648: mknod d3/d5/d2f/d5a/dcc/ce2 0 2024-04-02T04:47:55.982 INFO:tasks.workunit.client.1.smithi179.stdout:5/609: chown d8/d1b/d23/d32/f7a 597 1 2024-04-02T04:47:55.983 INFO:tasks.workunit.client.1.smithi179.stdout:0/683: mknod d1/da/d52/d75/d9a/cc9 0 2024-04-02T04:47:55.986 INFO:tasks.workunit.client.1.smithi179.stdout:7/676: rename d1/fc8 to d1/dd/d23/fee 0 2024-04-02T04:47:55.988 INFO:tasks.workunit.client.0.smithi070.stdout:0/640: dread d4/d5/d13/d2a/f68 [0,4194304] 0 2024-04-02T04:47:55.991 INFO:tasks.workunit.client.1.smithi179.stdout:1/649: fsync d3/d85/f89 0 2024-04-02T04:47:55.991 INFO:tasks.workunit.client.0.smithi070.stdout:5/732: dwrite d5/d9/d10/d6a/d7d/fca [0,4194304] 0 2024-04-02T04:47:55.992 INFO:tasks.workunit.client.0.smithi070.stdout:5/733: chown d5/d9/d23/d8b 497331141 1 2024-04-02T04:47:55.993 INFO:tasks.workunit.client.1.smithi179.stdout:5/610: creat d8/d1b/d23/d32/d6e/fc1 x:0 0 0 2024-04-02T04:47:55.993 INFO:tasks.workunit.client.1.smithi179.stdout:1/650: read d3/d5/de/f3c [1467915,127922] 0 2024-04-02T04:47:55.994 INFO:tasks.workunit.client.1.smithi179.stdout:5/611: fdatasync d8/d1b/d23/d36/d53/d62/f75 0 2024-04-02T04:47:55.996 INFO:tasks.workunit.client.1.smithi179.stdout:3/658: rename d8/d57/c6e to d8/d26/d52/d85/d9a/ccb 0 2024-04-02T04:47:56.001 INFO:tasks.workunit.client.1.smithi179.stdout:0/684: dread d1/da/d28/d5e/f98 [0,4194304] 0 2024-04-02T04:47:56.002 INFO:tasks.workunit.client.1.smithi179.stdout:2/506: mknod dc/ca1 0 2024-04-02T04:47:56.003 INFO:tasks.workunit.client.1.smithi179.stdout:9/627: getdents d0/d1/d10 0 2024-04-02T04:47:56.004 INFO:tasks.workunit.client.0.smithi070.stdout:1/598: dread d0/d9/d1d/d22/f2d [0,4194304] 0 2024-04-02T04:47:56.008 INFO:tasks.workunit.client.1.smithi179.stdout:1/651: fsync d3/d5/f13 0 2024-04-02T04:47:56.009 INFO:tasks.workunit.client.1.smithi179.stdout:1/652: readlink d3/d5/d2f/d4f/d6a/lc2 0 2024-04-02T04:47:56.010 INFO:tasks.workunit.client.1.smithi179.stdout:3/659: readlink d8/l14 0 2024-04-02T04:47:56.012 INFO:tasks.workunit.client.1.smithi179.stdout:8/652: rename d3/d31/l46 to d3/d6c/d96/le5 0 2024-04-02T04:47:56.013 INFO:tasks.workunit.client.0.smithi070.stdout:0/641: truncate d4/d5/d13/d17/f62 460774 0 2024-04-02T04:47:56.014 INFO:tasks.workunit.client.0.smithi070.stdout:3/591: write d10/d17/f78 [1368896,21310] 0 2024-04-02T04:47:56.014 INFO:tasks.workunit.client.1.smithi179.stdout:7/677: creat d1/dd/d23/dab/db0/fef x:0 0 0 2024-04-02T04:47:56.016 INFO:tasks.workunit.client.0.smithi070.stdout:5/734: mkdir d5/d9/d23/d8b/de9 0 2024-04-02T04:47:56.016 INFO:tasks.workunit.client.0.smithi070.stdout:3/592: chown d10/d32/d8a/d8d/f9a 14234 1 2024-04-02T04:47:56.017 INFO:tasks.workunit.client.1.smithi179.stdout:3/660: dwrite d8/d26/d52/d85/f97 [0,4194304] 0 2024-04-02T04:47:56.017 INFO:tasks.workunit.client.0.smithi070.stdout:4/558: truncate d1/d3/d9/f1d 4092975 0 2024-04-02T04:47:56.019 INFO:tasks.workunit.client.1.smithi179.stdout:0/685: creat d1/da/d48/fca x:0 0 0 2024-04-02T04:47:56.019 INFO:tasks.workunit.client.0.smithi070.stdout:7/600: dwrite d7/d9/f78 [0,4194304] 0 2024-04-02T04:47:56.020 INFO:tasks.workunit.client.1.smithi179.stdout:0/686: chown d1/da/d48/d65/d7d/f7a 1369766 1 2024-04-02T04:47:56.020 INFO:tasks.workunit.client.0.smithi070.stdout:6/486: dread d0/d3/d19/d31/d61/f66 [0,4194304] 0 2024-04-02T04:47:56.021 INFO:tasks.workunit.client.0.smithi070.stdout:6/487: stat d0/d43/d80 0 2024-04-02T04:47:56.021 INFO:tasks.workunit.client.1.smithi179.stdout:2/507: fdatasync dc/d29/d4a/d4b/f51 0 2024-04-02T04:47:56.023 INFO:tasks.workunit.client.1.smithi179.stdout:0/687: truncate d1/da/d28/d81/fae 1211624 0 2024-04-02T04:47:56.023 INFO:tasks.workunit.client.1.smithi179.stdout:9/628: chown d0/d1/d10/l70 589 1 2024-04-02T04:47:56.025 INFO:tasks.workunit.client.0.smithi070.stdout:8/585: rename d2/d8/d29/c1c to d2/d8/d23/d24/cb6 0 2024-04-02T04:47:56.028 INFO:tasks.workunit.client.1.smithi179.stdout:1/653: mknod d3/d5/d2f/d4f/d35/d49/ce3 0 2024-04-02T04:47:56.030 INFO:tasks.workunit.client.0.smithi070.stdout:8/586: dread d2/d8/d29/db/f21 [0,4194304] 0 2024-04-02T04:47:56.030 INFO:tasks.workunit.client.0.smithi070.stdout:2/570: link d2/d10/da4/f9b d2/fba 0 2024-04-02T04:47:56.032 INFO:tasks.workunit.client.0.smithi070.stdout:2/571: truncate d2/d10/d5c/f8c 4542845 0 2024-04-02T04:47:56.034 INFO:tasks.workunit.client.0.smithi070.stdout:0/642: creat d4/d5/d13/d2a/d3a/d45/fbd x:0 0 0 2024-04-02T04:47:56.036 INFO:tasks.workunit.client.1.smithi179.stdout:4/677: rename d1/d3/df/d10/d2f/d7b/fa2 to d1/d3/d19/fe0 0 2024-04-02T04:47:56.036 INFO:tasks.workunit.client.1.smithi179.stdout:7/678: mknod d1/d9/d55/cf0 0 2024-04-02T04:47:56.037 INFO:tasks.workunit.client.1.smithi179.stdout:3/661: rmdir d8/d26/d83/d89/d9c 39 2024-04-02T04:47:56.040 INFO:tasks.workunit.client.1.smithi179.stdout:2/508: rmdir dc 39 2024-04-02T04:47:56.042 INFO:tasks.workunit.client.0.smithi070.stdout:7/601: unlink d7/dd/df/d34/d68/f9c 0 2024-04-02T04:47:56.042 INFO:tasks.workunit.client.1.smithi179.stdout:9/629: rmdir d0 39 2024-04-02T04:47:56.045 INFO:tasks.workunit.client.0.smithi070.stdout:9/511: link d4/d10/l9d d4/d10/d11/d7b/d19/la7 0 2024-04-02T04:47:56.048 INFO:tasks.workunit.client.1.smithi179.stdout:1/654: unlink d3/d5/d2f/d4f/d42/d53/d5b/f83 0 2024-04-02T04:47:56.050 INFO:tasks.workunit.client.0.smithi070.stdout:9/512: dread d4/d10/d12/d27/f5c [0,4194304] 0 2024-04-02T04:47:56.052 INFO:tasks.workunit.client.0.smithi070.stdout:8/587: mknod d2/d3/d52/d62/d7c/cb7 0 2024-04-02T04:47:56.054 INFO:tasks.workunit.client.0.smithi070.stdout:1/599: dwrite d0/d54/f63 [0,4194304] 0 2024-04-02T04:47:56.056 INFO:tasks.workunit.client.0.smithi070.stdout:0/643: rmdir d4/d3c 39 2024-04-02T04:47:56.057 INFO:tasks.workunit.client.0.smithi070.stdout:1/600: truncate d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/fb2 1006364 0 2024-04-02T04:47:56.059 INFO:tasks.workunit.client.0.smithi070.stdout:4/559: unlink d1/l44 0 2024-04-02T04:47:56.060 INFO:tasks.workunit.client.0.smithi070.stdout:3/593: write d10/f2c [884482,129789] 0 2024-04-02T04:47:56.061 INFO:tasks.workunit.client.0.smithi070.stdout:4/560: dread - d1/d3/d9/d5e/f87 zero size 2024-04-02T04:47:56.062 INFO:tasks.workunit.client.0.smithi070.stdout:3/594: read d10/d17/d51/d73/d8f/fab [2080893,115526] 0 2024-04-02T04:47:56.062 INFO:tasks.workunit.client.0.smithi070.stdout:5/735: creat d5/d9/d23/d8b/de9/fea x:0 0 0 2024-04-02T04:47:56.069 INFO:tasks.workunit.client.0.smithi070.stdout:6/488: mkdir d0/d20/d77/d98 0 2024-04-02T04:47:56.071 INFO:tasks.workunit.client.0.smithi070.stdout:9/513: mkdir d4/d10/d11/da8 0 2024-04-02T04:47:56.076 INFO:tasks.workunit.client.0.smithi070.stdout:0/644: stat d4/d3c/fa2 0 2024-04-02T04:47:56.078 INFO:tasks.workunit.client.0.smithi070.stdout:8/588: dread d2/d8/d23/f51 [0,4194304] 0 2024-04-02T04:47:56.086 INFO:tasks.workunit.client.0.smithi070.stdout:8/589: dwrite d2/d8/d23/d24/f37 [0,4194304] 0 2024-04-02T04:47:56.092 INFO:tasks.workunit.client.0.smithi070.stdout:8/590: read d2/d3/f61 [3274907,109040] 0 2024-04-02T04:47:56.094 INFO:tasks.workunit.client.0.smithi070.stdout:5/736: dwrite d5/d17/d2b/f8d [0,4194304] 0 2024-04-02T04:47:56.094 INFO:tasks.workunit.client.0.smithi070.stdout:9/514: creat d4/d10/d11/d7b/d19/d2f/d39/d57/fa9 x:0 0 0 2024-04-02T04:47:56.096 INFO:tasks.workunit.client.0.smithi070.stdout:9/515: fsync d4/d10/d11/f4e 0 2024-04-02T04:47:56.097 INFO:tasks.workunit.client.0.smithi070.stdout:9/516: chown d4/d10/d11/f14 442524 1 2024-04-02T04:47:56.097 INFO:tasks.workunit.client.1.smithi179.stdout:8/653: symlink d3/d1b/d2a/d83/le6 0 2024-04-02T04:47:56.098 INFO:tasks.workunit.client.1.smithi179.stdout:8/654: stat d3/d32/d21/d22 0 2024-04-02T04:47:56.099 INFO:tasks.workunit.client.0.smithi070.stdout:1/601: mknod d0/d7/d1a/d95/cc2 0 2024-04-02T04:47:56.101 INFO:tasks.workunit.client.1.smithi179.stdout:6/622: rename d3/d12/d6a/dc0 to d3/d12/d23/d39/d80/d93/dd0 0 2024-04-02T04:47:56.109 INFO:tasks.workunit.client.0.smithi070.stdout:4/561: link d1/d3/d9/f81 d1/d98/fba 0 2024-04-02T04:47:56.110 INFO:tasks.workunit.client.0.smithi070.stdout:4/562: chown d1/d9c/f73 20234953 1 2024-04-02T04:47:56.111 INFO:tasks.workunit.client.1.smithi179.stdout:9/630: write d0/d1/f7 [1547179,40771] 0 2024-04-02T04:47:56.112 INFO:tasks.workunit.client.0.smithi070.stdout:7/602: truncate d7/d36/fb9 1881751 0 2024-04-02T04:47:56.113 INFO:tasks.workunit.client.0.smithi070.stdout:7/603: stat d7/dd/l52 0 2024-04-02T04:47:56.115 INFO:tasks.workunit.client.0.smithi070.stdout:4/563: dread d1/d3/d78/d91/faa [0,4194304] 0 2024-04-02T04:47:56.116 INFO:tasks.workunit.client.0.smithi070.stdout:6/489: dread d0/d20/d5f/f6e [0,4194304] 0 2024-04-02T04:47:56.119 INFO:tasks.workunit.client.0.smithi070.stdout:5/737: creat d5/d9/d23/d33/feb x:0 0 0 2024-04-02T04:47:56.120 INFO:tasks.workunit.client.0.smithi070.stdout:5/738: write d5/fa [1369747,43807] 0 2024-04-02T04:47:56.121 INFO:tasks.workunit.client.0.smithi070.stdout:2/572: getdents d2/d12/d59 0 2024-04-02T04:47:56.122 INFO:tasks.workunit.client.0.smithi070.stdout:4/564: dwrite d1/d3/fb2 [0,4194304] 0 2024-04-02T04:47:56.125 INFO:tasks.workunit.client.1.smithi179.stdout:3/662: dread d8/d26/d83/d89/d9c/fb0 [0,4194304] 0 2024-04-02T04:47:56.127 INFO:tasks.workunit.client.0.smithi070.stdout:9/517: mknod d4/d10/d12/caa 0 2024-04-02T04:47:56.130 INFO:tasks.workunit.client.1.smithi179.stdout:0/688: truncate d1/da/d28/d37/f49 3813983 0 2024-04-02T04:47:56.131 INFO:tasks.workunit.client.1.smithi179.stdout:0/689: chown d1/da/d48/d7b/c91 11073614 1 2024-04-02T04:47:56.132 INFO:tasks.workunit.client.0.smithi070.stdout:0/645: creat d4/d5/d58/da1/fbe x:0 0 0 2024-04-02T04:47:56.135 INFO:tasks.workunit.client.1.smithi179.stdout:9/631: symlink d0/d1/d11/d3a/lbd 0 2024-04-02T04:47:56.137 INFO:tasks.workunit.client.0.smithi070.stdout:8/591: dwrite d2/d8/d29/d22/f46 [0,4194304] 0 2024-04-02T04:47:56.138 INFO:tasks.workunit.client.1.smithi179.stdout:5/612: rename d8/d1b/c29 to d8/d1b/d23/d30/d73/dba/cc2 0 2024-04-02T04:47:56.140 INFO:tasks.workunit.client.1.smithi179.stdout:5/613: chown d8/d1b/d23/f2f 2582 1 2024-04-02T04:47:56.143 INFO:tasks.workunit.client.0.smithi070.stdout:5/739: symlink d5/d9/d23/d30/da2/d75/lec 0 2024-04-02T04:47:56.146 INFO:tasks.workunit.client.0.smithi070.stdout:1/602: rename d0/d7/d28/c3a to d0/d7/d28/d83/cc3 0 2024-04-02T04:47:56.147 INFO:tasks.workunit.client.0.smithi070.stdout:1/603: write d0/d4c/fb7 [120157,104501] 0 2024-04-02T04:47:56.148 INFO:tasks.workunit.client.1.smithi179.stdout:4/678: truncate d1/f42 594622 0 2024-04-02T04:47:56.149 INFO:tasks.workunit.client.1.smithi179.stdout:2/509: creat dc/d29/d31/d66/fa2 x:0 0 0 2024-04-02T04:47:56.150 INFO:tasks.workunit.client.0.smithi070.stdout:3/595: getdents d10/d17/db2 0 2024-04-02T04:47:56.150 INFO:tasks.workunit.client.0.smithi070.stdout:3/596: chown d10 5 1 2024-04-02T04:47:56.151 INFO:tasks.workunit.client.1.smithi179.stdout:8/655: creat d3/d32/d21/d50/d56/d94/fe7 x:0 0 0 2024-04-02T04:47:56.152 INFO:tasks.workunit.client.0.smithi070.stdout:3/597: read - d10/d32/d8a/d8d/fc1 zero size 2024-04-02T04:47:56.155 INFO:tasks.workunit.client.1.smithi179.stdout:9/632: unlink d0/d19/d27/f68 0 2024-04-02T04:47:56.155 INFO:tasks.workunit.client.0.smithi070.stdout:1/604: dwrite d0/d7/f43 [0,4194304] 0 2024-04-02T04:47:56.156 INFO:tasks.workunit.client.1.smithi179.stdout:3/663: dread d8/d26/d36/f59 [0,4194304] 0 2024-04-02T04:47:56.159 INFO:tasks.workunit.client.1.smithi179.stdout:7/679: rename d1/d9/d35/cec to d1/d9/d55/dbf/d9f/cf1 0 2024-04-02T04:47:56.160 INFO:tasks.workunit.client.1.smithi179.stdout:7/680: stat d1/d9/d55/f9e 0 2024-04-02T04:47:56.162 INFO:tasks.workunit.client.1.smithi179.stdout:5/614: creat d8/d1b/d23/d36/d53/d62/da9/fc3 x:0 0 0 2024-04-02T04:47:56.164 INFO:tasks.workunit.client.1.smithi179.stdout:4/679: stat d1/d3/dc/c16 0 2024-04-02T04:47:56.165 INFO:tasks.workunit.client.0.smithi070.stdout:2/573: dwrite d2/fba [0,4194304] 0 2024-04-02T04:47:56.171 INFO:tasks.workunit.client.1.smithi179.stdout:9/633: fsync d0/d19/d27/d7a/f8e 0 2024-04-02T04:47:56.172 INFO:tasks.workunit.client.0.smithi070.stdout:3/598: mknod d10/d66/ccd 0 2024-04-02T04:47:56.173 INFO:tasks.workunit.client.1.smithi179.stdout:3/664: rmdir d8/d57/d8f 39 2024-04-02T04:47:56.175 INFO:tasks.workunit.client.1.smithi179.stdout:3/665: write d8/d6d/fb6 [662526,57366] 0 2024-04-02T04:47:56.176 INFO:tasks.workunit.client.1.smithi179.stdout:6/623: rename d3/d12/d23/d26/d57/ca4 to d3/d12/d23/d26/d9b/cd1 0 2024-04-02T04:47:56.177 INFO:tasks.workunit.client.1.smithi179.stdout:7/681: creat d1/d9/d7e/ff2 x:0 0 0 2024-04-02T04:47:56.183 INFO:tasks.workunit.client.1.smithi179.stdout:3/666: unlink d8/d26/cc0 0 2024-04-02T04:47:56.186 INFO:tasks.workunit.client.1.smithi179.stdout:9/634: symlink d0/d19/d27/d7a/lbe 0 2024-04-02T04:47:56.186 INFO:tasks.workunit.client.1.smithi179.stdout:3/667: mkdir d8/d26/d83/d89/d9c/dcc 0 2024-04-02T04:47:56.189 INFO:tasks.workunit.client.1.smithi179.stdout:8/656: getdents d3/d32/d21/d50/d56/dad/dbc 0 2024-04-02T04:47:56.189 INFO:tasks.workunit.client.1.smithi179.stdout:1/655: sync 2024-04-02T04:47:56.190 INFO:tasks.workunit.client.1.smithi179.stdout:2/510: rename dc/d29/d31/d83/f9d to dc/d29/fa3 0 2024-04-02T04:47:56.191 INFO:tasks.workunit.client.1.smithi179.stdout:7/682: write d1/d9/d27/d87/d8b/f6f [266610,90077] 0 2024-04-02T04:47:56.193 INFO:tasks.workunit.client.1.smithi179.stdout:1/656: creat d3/d5/d2f/d4f/fe4 x:0 0 0 2024-04-02T04:47:56.194 INFO:tasks.workunit.client.1.smithi179.stdout:5/615: rename d8/d1b/d23/d32/d46/fa6 to d8/d1b/d23/d32/d4b/fc4 0 2024-04-02T04:47:56.195 INFO:tasks.workunit.client.1.smithi179.stdout:3/668: mkdir d8/d26/dcd 0 2024-04-02T04:47:56.196 INFO:tasks.workunit.client.1.smithi179.stdout:9/635: mknod d0/d19/d27/d7a/d82/cbf 0 2024-04-02T04:47:56.198 INFO:tasks.workunit.client.1.smithi179.stdout:2/511: creat dc/d29/d4a/d1e/d43/fa4 x:0 0 0 2024-04-02T04:47:56.199 INFO:tasks.workunit.client.1.smithi179.stdout:4/680: rename d1/d3/df/d10/d2f/lab to d1/d3/df/d10/d2f/d60/le1 0 2024-04-02T04:47:56.200 INFO:tasks.workunit.client.1.smithi179.stdout:8/657: dwrite d3/d32/d21/d22/d6b/d91/fc0 [0,4194304] 0 2024-04-02T04:47:56.201 INFO:tasks.workunit.client.1.smithi179.stdout:6/624: dwrite d3/d12/d23/d39/d80/f8f [0,4194304] 0 2024-04-02T04:47:56.202 INFO:tasks.workunit.client.1.smithi179.stdout:5/616: symlink d8/d1b/d23/d30/d73/dba/lc5 0 2024-04-02T04:47:56.204 INFO:tasks.workunit.client.1.smithi179.stdout:1/657: mknod d3/d5/d2f/d4f/d6a/da0/dd4/ce5 0 2024-04-02T04:47:56.205 INFO:tasks.workunit.client.1.smithi179.stdout:2/512: symlink dc/d29/d31/d83/la5 0 2024-04-02T04:47:56.205 INFO:tasks.workunit.client.1.smithi179.stdout:7/683: write d1/dd/f1c [988912,74071] 0 2024-04-02T04:47:56.205 INFO:tasks.workunit.client.1.smithi179.stdout:9/636: rename d0/d1/d10/d6d/f8b to d0/d19/d1c/fc0 0 2024-04-02T04:47:56.209 INFO:tasks.workunit.client.1.smithi179.stdout:2/513: fdatasync dc/d29/d4a/f48 0 2024-04-02T04:47:56.211 INFO:tasks.workunit.client.1.smithi179.stdout:5/617: link d8/d1b/f9a d8/d1b/d23/db1/fc6 0 2024-04-02T04:47:56.214 INFO:tasks.workunit.client.1.smithi179.stdout:9/637: symlink d0/d19/d27/d7a/lc1 0 2024-04-02T04:47:56.214 INFO:tasks.workunit.client.1.smithi179.stdout:2/514: write dc/d29/d4a/f8c [3122127,50843] 0 2024-04-02T04:47:56.216 INFO:tasks.workunit.client.1.smithi179.stdout:4/681: dwrite d1/d3/dc/d27/d62/f54 [0,4194304] 0 2024-04-02T04:47:56.217 INFO:tasks.workunit.client.1.smithi179.stdout:8/658: dread d3/d32/d21/f66 [0,4194304] 0 2024-04-02T04:47:56.217 INFO:tasks.workunit.client.1.smithi179.stdout:6/625: dwrite d3/d12/d23/d39/d80/f8f [0,4194304] 0 2024-04-02T04:47:56.218 INFO:tasks.workunit.client.1.smithi179.stdout:5/618: creat d8/d1b/d23/d32/d4e/d51/fc7 x:0 0 0 2024-04-02T04:47:56.219 INFO:tasks.workunit.client.1.smithi179.stdout:8/659: chown d3/f59 3524261 1 2024-04-02T04:47:56.220 INFO:tasks.workunit.client.1.smithi179.stdout:2/515: truncate f2 2303169 0 2024-04-02T04:47:56.222 INFO:tasks.workunit.client.1.smithi179.stdout:4/682: creat d1/d3/dc/d27/d62/fe2 x:0 0 0 2024-04-02T04:47:56.222 INFO:tasks.workunit.client.1.smithi179.stdout:2/516: read dc/f5d [2617211,67403] 0 2024-04-02T04:47:56.222 INFO:tasks.workunit.client.1.smithi179.stdout:3/669: truncate d8/d57/d63/f84 1476890 0 2024-04-02T04:47:56.227 INFO:tasks.workunit.client.1.smithi179.stdout:8/660: chown d3/d32/d21/d50/d56/d94/d84/l92 15 1 2024-04-02T04:47:56.229 INFO:tasks.workunit.client.1.smithi179.stdout:2/517: unlink dc/d29/d31/d39/f4f 0 2024-04-02T04:47:56.230 INFO:tasks.workunit.client.1.smithi179.stdout:3/670: symlink d8/d26/d52/d7e/db8/lce 0 2024-04-02T04:47:56.231 INFO:tasks.workunit.client.1.smithi179.stdout:4/683: link d1/d93/cd5 d1/d3/dc/dc6/ce3 0 2024-04-02T04:47:56.233 INFO:tasks.workunit.client.1.smithi179.stdout:6/626: dread d3/d12/d23/d26/f76 [0,4194304] 0 2024-04-02T04:47:56.234 INFO:tasks.workunit.client.1.smithi179.stdout:4/684: stat d1/d15/d6e/l96 0 2024-04-02T04:47:56.235 INFO:tasks.workunit.client.1.smithi179.stdout:8/661: getdents d3/d1b/d2a 0 2024-04-02T04:47:56.236 INFO:tasks.workunit.client.1.smithi179.stdout:1/658: dread d3/d5/d2f/d4f/d35/d49/f7e [0,4194304] 0 2024-04-02T04:47:56.237 INFO:tasks.workunit.client.1.smithi179.stdout:2/518: link dc/d29/d4a/f24 dc/d79/fa6 0 2024-04-02T04:47:56.238 INFO:tasks.workunit.client.1.smithi179.stdout:8/662: fsync d3/f76 0 2024-04-02T04:47:56.239 INFO:tasks.workunit.client.1.smithi179.stdout:5/619: dwrite d8/d1b/d23/d32/d4e/d51/f66 [0,4194304] 0 2024-04-02T04:47:56.240 INFO:tasks.workunit.client.1.smithi179.stdout:2/519: readlink dc/d29/d4a/d1e/l59 0 2024-04-02T04:47:56.240 INFO:tasks.workunit.client.1.smithi179.stdout:1/659: chown d3/d5/d2f/d5a/lb2 1044 1 2024-04-02T04:47:56.242 INFO:tasks.workunit.client.1.smithi179.stdout:6/627: creat d3/d12/d23/d39/d4e/d66/fd2 x:0 0 0 2024-04-02T04:47:56.242 INFO:tasks.workunit.client.1.smithi179.stdout:9/638: dread d0/d19/d1c/f31 [0,4194304] 0 2024-04-02T04:47:56.243 INFO:tasks.workunit.client.1.smithi179.stdout:2/520: symlink dc/d29/d4a/d4b/la7 0 2024-04-02T04:47:56.245 INFO:tasks.workunit.client.1.smithi179.stdout:8/663: unlink d3/d32/d21/d50/d56/c58 0 2024-04-02T04:47:56.245 INFO:tasks.workunit.client.1.smithi179.stdout:9/639: read - d0/d1/d11/dab/fbb zero size 2024-04-02T04:47:56.246 INFO:tasks.workunit.client.1.smithi179.stdout:6/628: mkdir d3/d12/d23/d26/dd3 0 2024-04-02T04:47:56.246 INFO:tasks.workunit.client.1.smithi179.stdout:4/685: truncate d1/d3/f7 2509145 0 2024-04-02T04:47:56.247 INFO:tasks.workunit.client.1.smithi179.stdout:5/620: rename d8/d1b/d23/d32/d4e/d50/f6f to d8/d1b/d23/d36/d53/d62/da9/fc8 0 2024-04-02T04:47:56.248 INFO:tasks.workunit.client.1.smithi179.stdout:2/521: mknod dc/d29/d31/d57/ca8 0 2024-04-02T04:47:56.249 INFO:tasks.workunit.client.1.smithi179.stdout:4/686: creat d1/d93/fe4 x:0 0 0 2024-04-02T04:47:56.250 INFO:tasks.workunit.client.1.smithi179.stdout:9/640: fdatasync d0/d19/d27/d7a/d82/f98 0 2024-04-02T04:47:56.251 INFO:tasks.workunit.client.1.smithi179.stdout:1/660: dwrite d3/d5/d2f/d4f/d42/fbd [0,4194304] 0 2024-04-02T04:47:56.255 INFO:tasks.workunit.client.1.smithi179.stdout:6/629: rename d3/d12/d23/d39/d4e/d66/d7c to d3/d12/d23/d39/d80/d93/dd4 0 2024-04-02T04:47:56.258 INFO:tasks.workunit.client.1.smithi179.stdout:1/661: mknod d3/d5/d2f/d5a/dcc/ce6 0 2024-04-02T04:47:56.259 INFO:tasks.workunit.client.1.smithi179.stdout:4/687: mkdir d1/d3/dc/d27/dae/de5 0 2024-04-02T04:47:56.259 INFO:tasks.workunit.client.1.smithi179.stdout:9/641: truncate d0/d19/d1c/f52 447334 0 2024-04-02T04:47:56.260 INFO:tasks.workunit.client.1.smithi179.stdout:6/630: symlink d3/d2d/d43/ld5 0 2024-04-02T04:47:56.262 INFO:tasks.workunit.client.1.smithi179.stdout:8/664: dwrite d3/d32/d21/d22/d6b/d91/fc0 [0,4194304] 0 2024-04-02T04:47:56.264 INFO:tasks.workunit.client.1.smithi179.stdout:4/688: fsync d1/d3/dc/f1c 0 2024-04-02T04:47:56.267 INFO:tasks.workunit.client.1.smithi179.stdout:2/522: dwrite dc/d29/d4a/f72 [0,4194304] 0 2024-04-02T04:47:56.270 INFO:tasks.workunit.client.1.smithi179.stdout:5/621: dwrite d8/d1b/d23/d36/d53/d62/da9/fc0 [0,4194304] 0 2024-04-02T04:47:56.272 INFO:tasks.workunit.client.1.smithi179.stdout:2/523: chown dc/d29/c73 294148714 1 2024-04-02T04:47:56.272 INFO:tasks.workunit.client.1.smithi179.stdout:6/631: creat d3/d2d/d38/d64/fd6 x:0 0 0 2024-04-02T04:47:56.273 INFO:tasks.workunit.client.1.smithi179.stdout:9/642: mkdir d0/d19/d27/d7a/d87/daf/dc2 0 2024-04-02T04:47:56.275 INFO:tasks.workunit.client.1.smithi179.stdout:4/689: chown d1/d3/df/d10/c46 205262 1 2024-04-02T04:47:56.276 INFO:tasks.workunit.client.1.smithi179.stdout:6/632: fdatasync d3/d12/d23/d2f/f3c 0 2024-04-02T04:47:56.280 INFO:tasks.workunit.client.1.smithi179.stdout:1/662: getdents d3/d5/d2f/d4f/d42/d53 0 2024-04-02T04:47:56.280 INFO:tasks.workunit.client.1.smithi179.stdout:8/665: dread d3/d32/d21/d22/d6b/d91/f44 [0,4194304] 0 2024-04-02T04:47:56.282 INFO:tasks.workunit.client.1.smithi179.stdout:2/524: creat dc/fa9 x:0 0 0 2024-04-02T04:47:56.282 INFO:tasks.workunit.client.1.smithi179.stdout:9/643: creat d0/d19/d27/d7a/d87/daf/dc2/fc3 x:0 0 0 2024-04-02T04:47:56.283 INFO:tasks.workunit.client.1.smithi179.stdout:8/666: chown d3/d1b/d2a/fdb 309873 1 2024-04-02T04:47:56.285 INFO:tasks.workunit.client.1.smithi179.stdout:1/663: creat d3/d5/d2f/db7/fe7 x:0 0 0 2024-04-02T04:47:56.286 INFO:tasks.workunit.client.1.smithi179.stdout:5/622: dwrite d8/d1b/f37 [0,4194304] 0 2024-04-02T04:47:56.286 INFO:tasks.workunit.client.1.smithi179.stdout:9/644: mknod d0/d19/d27/d7a/d84/cc4 0 2024-04-02T04:47:56.288 INFO:tasks.workunit.client.1.smithi179.stdout:6/633: dwrite d3/f7f [0,4194304] 0 2024-04-02T04:47:56.289 INFO:tasks.workunit.client.1.smithi179.stdout:8/667: mknod d3/d32/d21/d22/d6b/db1/ce8 0 2024-04-02T04:47:56.290 INFO:tasks.workunit.client.1.smithi179.stdout:5/623: creat d8/fc9 x:0 0 0 2024-04-02T04:47:56.291 INFO:tasks.workunit.client.1.smithi179.stdout:1/664: rename d3/d5/de/d33/f7b to d3/d85/da2/dd1/fe8 0 2024-04-02T04:47:56.292 INFO:tasks.workunit.client.1.smithi179.stdout:9/645: mknod d0/d19/d27/d7a/d87/cc5 0 2024-04-02T04:47:56.293 INFO:tasks.workunit.client.1.smithi179.stdout:6/634: mkdir d3/d2d/d43/dd7 0 2024-04-02T04:47:56.293 INFO:tasks.workunit.client.1.smithi179.stdout:2/525: getdents dc/d29/d31/d57/d97 0 2024-04-02T04:47:56.295 INFO:tasks.workunit.client.1.smithi179.stdout:2/526: chown dc/d29/c70 1 1 2024-04-02T04:47:56.297 INFO:tasks.workunit.client.1.smithi179.stdout:5/624: readlink d8/d1b/d23/d36/d53/d89/l8b 0 2024-04-02T04:47:56.298 INFO:tasks.workunit.client.1.smithi179.stdout:1/665: mknod d3/d5/d2f/ce9 0 2024-04-02T04:47:56.299 INFO:tasks.workunit.client.1.smithi179.stdout:8/668: dread d3/d32/d21/d50/f5f [0,4194304] 0 2024-04-02T04:47:56.301 INFO:tasks.workunit.client.1.smithi179.stdout:9/646: link d0/d1/d11/d29/d40/c4b d0/d19/d27/d7a/d87/cc6 0 2024-04-02T04:47:56.302 INFO:tasks.workunit.client.1.smithi179.stdout:1/666: unlink d3/d5/de/l19 0 2024-04-02T04:47:56.306 INFO:tasks.workunit.client.1.smithi179.stdout:2/527: link dc/d29/d31/d39/f65 dc/d29/d4a/faa 0 2024-04-02T04:47:56.308 INFO:tasks.workunit.client.1.smithi179.stdout:9/647: dread d0/d19/d1c/f52 [0,4194304] 0 2024-04-02T04:47:56.310 INFO:tasks.workunit.client.1.smithi179.stdout:1/667: dwrite d3/d5/d2f/d4f/fe4 [0,4194304] 0 2024-04-02T04:47:56.310 INFO:tasks.workunit.client.1.smithi179.stdout:5/625: write d8/d1b/d23/f2b [2307303,130510] 0 2024-04-02T04:47:56.312 INFO:tasks.workunit.client.1.smithi179.stdout:8/669: truncate d3/d32/d21/d50/d56/d94/d84/fbd 154633 0 2024-04-02T04:47:56.312 INFO:tasks.workunit.client.1.smithi179.stdout:9/648: chown d0/d1/d11/d29/d80/l88 3 1 2024-04-02T04:47:56.315 INFO:tasks.workunit.client.1.smithi179.stdout:9/649: stat d0/d1/d11/d3a/l9c 0 2024-04-02T04:47:56.316 INFO:tasks.workunit.client.1.smithi179.stdout:1/668: write d3/d5/d2f/d4f/d35/d49/f8b [2434781,2634] 0 2024-04-02T04:47:56.318 INFO:tasks.workunit.client.1.smithi179.stdout:8/670: write d3/d32/d21/d45/d5a/fb5 [1174871,55171] 0 2024-04-02T04:47:56.320 INFO:tasks.workunit.client.1.smithi179.stdout:6/635: dwrite d3/d2c/f5b [0,4194304] 0 2024-04-02T04:47:56.320 INFO:tasks.workunit.client.1.smithi179.stdout:1/669: readlink d3/d5/d2f/d4f/d42/l71 0 2024-04-02T04:47:56.320 INFO:tasks.workunit.client.0.smithi070.stdout:0/646: dwrite d4/d5/d13/d2a/d29/f65 [0,4194304] 0 2024-04-02T04:47:56.321 INFO:tasks.workunit.client.0.smithi070.stdout:2/574: mkdir d2/d10/d53/dbb 0 2024-04-02T04:47:56.323 INFO:tasks.workunit.client.0.smithi070.stdout:6/490: getdents d0/d55/d5c 0 2024-04-02T04:47:56.324 INFO:tasks.workunit.client.0.smithi070.stdout:8/592: creat d2/d8/d29/fb8 x:0 0 0 2024-04-02T04:47:56.327 INFO:tasks.workunit.client.1.smithi179.stdout:6/636: dread f2 [0,4194304] 0 2024-04-02T04:47:56.329 INFO:tasks.workunit.client.1.smithi179.stdout:6/637: truncate d3/d12/d23/d26/d57/fa2 551145 0 2024-04-02T04:47:56.331 INFO:tasks.workunit.client.0.smithi070.stdout:0/647: creat d4/d5/d58/fbf x:0 0 0 2024-04-02T04:47:56.331 INFO:tasks.workunit.client.0.smithi070.stdout:0/648: fdatasync f0 0 2024-04-02T04:47:56.337 INFO:tasks.workunit.client.1.smithi179.stdout:1/670: creat d3/d85/da2/fea x:0 0 0 2024-04-02T04:47:56.339 INFO:tasks.workunit.client.1.smithi179.stdout:6/638: dwrite d3/d12/d23/d26/faf [0,4194304] 0 2024-04-02T04:47:56.340 INFO:tasks.workunit.client.1.smithi179.stdout:8/671: creat d3/d32/d21/d50/d56/d94/d84/fe9 x:0 0 0 2024-04-02T04:47:56.340 INFO:tasks.workunit.client.1.smithi179.stdout:8/672: read d3/d32/d21/d49/f52 [91887,84809] 0 2024-04-02T04:47:56.346 INFO:tasks.workunit.client.0.smithi070.stdout:5/740: getdents d5/d9/d10/d6a/d7d 0 2024-04-02T04:47:56.346 INFO:tasks.workunit.client.1.smithi179.stdout:4/690: dread d1/d3/dc/d27/d62/f76 [0,4194304] 0 2024-04-02T04:47:56.348 INFO:tasks.workunit.client.0.smithi070.stdout:7/604: rename d7/dd/df/d7a/d6c/d77/d54/c8a to d7/dd/cc3 0 2024-04-02T04:47:56.348 INFO:tasks.workunit.client.1.smithi179.stdout:5/626: write d8/d1b/d23/d30/f71 [12058,69906] 0 2024-04-02T04:47:56.349 INFO:tasks.workunit.client.1.smithi179.stdout:9/650: write d0/d19/d1c/f47 [142103,99132] 0 2024-04-02T04:47:56.351 INFO:tasks.workunit.client.1.smithi179.stdout:4/691: dread d1/d3/dc/f1c [0,4194304] 0 2024-04-02T04:47:56.351 INFO:tasks.workunit.client.0.smithi070.stdout:8/593: dwrite d2/d8/f6d [0,4194304] 0 2024-04-02T04:47:56.352 INFO:tasks.workunit.client.1.smithi179.stdout:5/627: truncate d8/d1b/d23/d32/d4e/d51/f9c 725477 0 2024-04-02T04:47:56.354 INFO:tasks.workunit.client.1.smithi179.stdout:1/671: unlink d3/fa 0 2024-04-02T04:47:56.355 INFO:tasks.workunit.client.0.smithi070.stdout:0/649: fsync d4/d5/d13/d2a/d29/f59 0 2024-04-02T04:47:56.360 INFO:tasks.workunit.client.1.smithi179.stdout:6/639: truncate d3/d12/d23/d26/d57/f8a 4714314 0 2024-04-02T04:47:56.361 INFO:tasks.workunit.client.1.smithi179.stdout:1/672: dread d3/d5/d2f/d4f/d35/d49/f7e [0,4194304] 0 2024-04-02T04:47:56.362 INFO:tasks.workunit.client.0.smithi070.stdout:7/605: unlink d7/d4e/ca0 0 2024-04-02T04:47:56.362 INFO:tasks.workunit.client.1.smithi179.stdout:6/640: dread - d3/d2d/da5/fbd zero size 2024-04-02T04:47:56.363 INFO:tasks.workunit.client.0.smithi070.stdout:0/650: dwrite d4/d5/d13/d2a/d3a/d80/f9c [0,4194304] 0 2024-04-02T04:47:56.365 INFO:tasks.workunit.client.0.smithi070.stdout:7/606: read - d7/dd/df/d79/fbf zero size 2024-04-02T04:47:56.365 INFO:tasks.workunit.client.0.smithi070.stdout:0/651: stat d4/d5/da 0 2024-04-02T04:47:56.365 INFO:tasks.workunit.client.0.smithi070.stdout:4/565: rename d1/d3/d78/da3 to d1/d3/d85/d8e/d99/db1/dbb 0 2024-04-02T04:47:56.367 INFO:tasks.workunit.client.0.smithi070.stdout:0/652: stat d4/d5/d13/d23/f4d 0 2024-04-02T04:47:56.368 INFO:tasks.workunit.client.1.smithi179.stdout:3/671: sync 2024-04-02T04:47:56.369 INFO:tasks.workunit.client.0.smithi070.stdout:5/741: dread d5/d9/d23/d30/da2/d75/f72 [0,4194304] 0 2024-04-02T04:47:56.369 INFO:tasks.workunit.client.1.smithi179.stdout:3/672: write d8/d26/d36/f5c [2507275,24190] 0 2024-04-02T04:47:56.376 INFO:tasks.workunit.client.0.smithi070.stdout:2/575: rmdir d2/d10/d53/dbb 0 2024-04-02T04:47:56.377 INFO:tasks.workunit.client.0.smithi070.stdout:2/576: write d2/d10/da4/fa0 [209154,93209] 0 2024-04-02T04:47:56.378 INFO:tasks.workunit.client.1.smithi179.stdout:6/641: creat d3/d12/d23/d39/d4e/d6c/fd8 x:0 0 0 2024-04-02T04:47:56.380 INFO:tasks.workunit.client.1.smithi179.stdout:7/684: dread d1/f32 [0,4194304] 0 2024-04-02T04:47:56.383 INFO:tasks.workunit.client.0.smithi070.stdout:3/599: rename d10/d32/f59 to d10/d17/db2/dbd/fce 0 2024-04-02T04:47:56.384 INFO:tasks.workunit.client.0.smithi070.stdout:3/600: write d10/d32/d3f/f6d [932320,11795] 0 2024-04-02T04:47:56.386 INFO:tasks.workunit.client.0.smithi070.stdout:7/607: rmdir d7/dd/df/d79/d46 39 2024-04-02T04:47:56.387 INFO:tasks.workunit.client.1.smithi179.stdout:8/673: dwrite d3/d1b/d2a/f3e [0,4194304] 0 2024-04-02T04:47:56.388 INFO:tasks.workunit.client.0.smithi070.stdout:8/594: mknod d2/d8/d29/d72/cb9 0 2024-04-02T04:47:56.389 INFO:tasks.workunit.client.0.smithi070.stdout:8/595: chown d2/d8/d23/d24/d79/l57 60313313 1 2024-04-02T04:47:56.389 INFO:tasks.workunit.client.1.smithi179.stdout:9/651: dread d0/d1/d11/f78 [0,4194304] 0 2024-04-02T04:47:56.392 INFO:tasks.workunit.client.0.smithi070.stdout:0/653: creat d4/d5/d13/d17/d98/fc0 x:0 0 0 2024-04-02T04:47:56.394 INFO:tasks.workunit.client.0.smithi070.stdout:5/742: rmdir d5/d14 39 2024-04-02T04:47:56.395 INFO:tasks.workunit.client.1.smithi179.stdout:1/673: write d3/d85/f89 [687679,86569] 0 2024-04-02T04:47:56.396 INFO:tasks.workunit.client.0.smithi070.stdout:5/743: read - d5/d9/d10/db9/dd9/fe7 zero size 2024-04-02T04:47:56.397 INFO:tasks.workunit.client.1.smithi179.stdout:7/685: rmdir d1/dd 39 2024-04-02T04:47:56.398 INFO:tasks.workunit.client.0.smithi070.stdout:2/577: symlink d2/d10/d21/d3d/lbc 0 2024-04-02T04:47:56.400 INFO:tasks.workunit.client.1.smithi179.stdout:3/673: dread d8/d26/d52/d85/f8b [0,4194304] 0 2024-04-02T04:47:56.400 INFO:tasks.workunit.client.0.smithi070.stdout:6/491: link d0/l12 d0/d43/l99 0 2024-04-02T04:47:56.400 INFO:tasks.workunit.client.1.smithi179.stdout:4/692: write d1/f3c [3358346,42418] 0 2024-04-02T04:47:56.401 INFO:tasks.workunit.client.1.smithi179.stdout:5/628: truncate d8/d1b/d23/d32/d4e/fab 861803 0 2024-04-02T04:47:56.403 INFO:tasks.workunit.client.0.smithi070.stdout:3/601: readlink d10/d17/l88 0 2024-04-02T04:47:56.404 INFO:tasks.workunit.client.1.smithi179.stdout:7/686: dwrite d1/f5d [0,4194304] 0 2024-04-02T04:47:56.405 INFO:tasks.workunit.client.0.smithi070.stdout:7/608: creat d7/dd/df/d7a/fc4 x:0 0 0 2024-04-02T04:47:56.407 INFO:tasks.workunit.client.1.smithi179.stdout:6/642: rename d3/d2d/d38/c41 to d3/d12/d23/d26/cd9 0 2024-04-02T04:47:56.409 INFO:tasks.workunit.client.1.smithi179.stdout:1/674: mknod d3/d5/d2f/d4f/d42/d53/d5b/ceb 0 2024-04-02T04:47:56.409 INFO:tasks.workunit.client.0.smithi070.stdout:8/596: read d2/f48 [1008299,47816] 0 2024-04-02T04:47:56.413 INFO:tasks.workunit.client.0.smithi070.stdout:0/654: dread - d4/d5/da/d2d/f89 zero size 2024-04-02T04:47:56.413 INFO:tasks.workunit.client.0.smithi070.stdout:7/609: dwrite d7/dd/df/d79/f48 [4194304,4194304] 0 2024-04-02T04:47:56.415 INFO:tasks.workunit.client.0.smithi070.stdout:7/610: chown d7/dd/df/d7a/d6c/d77/l73 1852 1 2024-04-02T04:47:56.416 INFO:tasks.workunit.client.1.smithi179.stdout:3/674: creat d8/d7a/fcf x:0 0 0 2024-04-02T04:47:56.418 INFO:tasks.workunit.client.1.smithi179.stdout:9/652: dread d0/f90 [0,4194304] 0 2024-04-02T04:47:56.418 INFO:tasks.workunit.client.1.smithi179.stdout:5/629: creat d8/d1b/d23/d36/fca x:0 0 0 2024-04-02T04:47:56.418 INFO:tasks.workunit.client.0.smithi070.stdout:6/492: write d0/d3/d19/d31/d61/f66 [722166,117664] 0 2024-04-02T04:47:56.419 INFO:tasks.workunit.client.1.smithi179.stdout:7/687: rmdir d1/d9/d55/dbf/d9f/da8 39 2024-04-02T04:47:56.419 INFO:tasks.workunit.client.1.smithi179.stdout:3/675: truncate d8/d26/d52/d85/dbb/fc6 990866 0 2024-04-02T04:47:56.420 INFO:tasks.workunit.client.0.smithi070.stdout:3/602: write d10/d17/d51/f64 [4251885,95856] 0 2024-04-02T04:47:56.423 INFO:tasks.workunit.client.1.smithi179.stdout:8/674: rename d3/l3a to d3/d32/d21/d51/d70/lea 0 2024-04-02T04:47:56.424 INFO:tasks.workunit.client.1.smithi179.stdout:8/675: write d3/d6c/d96/fa1 [4647758,64314] 0 2024-04-02T04:47:56.426 INFO:tasks.workunit.client.0.smithi070.stdout:4/566: link d1/d3/f62 d1/d3/d85/d8e/d99/fbc 0 2024-04-02T04:47:56.428 INFO:tasks.workunit.client.0.smithi070.stdout:5/744: symlink d5/led 0 2024-04-02T04:47:56.430 INFO:tasks.workunit.client.0.smithi070.stdout:0/655: mknod d4/d5/d58/d60/d83/da5/cc1 0 2024-04-02T04:47:56.431 INFO:tasks.workunit.client.0.smithi070.stdout:2/578: write d2/f32 [395689,24553] 0 2024-04-02T04:47:56.432 INFO:tasks.workunit.client.0.smithi070.stdout:7/611: mknod d7/dd/df/d7a/d6c/d77/d54/cc5 0 2024-04-02T04:47:56.435 INFO:tasks.workunit.client.1.smithi179.stdout:4/693: write d1/d3/f7 [1776979,41667] 0 2024-04-02T04:47:56.436 INFO:tasks.workunit.client.0.smithi070.stdout:3/603: truncate d10/d32/d3f/f4d 1842802 0 2024-04-02T04:47:56.436 INFO:tasks.workunit.client.1.smithi179.stdout:3/676: creat d8/d6d/fd0 x:0 0 0 2024-04-02T04:47:56.438 INFO:tasks.workunit.client.0.smithi070.stdout:8/597: unlink d2/d8/d23/d24/cb5 0 2024-04-02T04:47:56.439 INFO:tasks.workunit.client.0.smithi070.stdout:4/567: chown d1/d3/d9/c60 2588 1 2024-04-02T04:47:56.440 INFO:tasks.workunit.client.1.smithi179.stdout:1/675: dwrite d3/d5/d2f/d65/f92 [0,4194304] 0 2024-04-02T04:47:56.440 INFO:tasks.workunit.client.1.smithi179.stdout:6/643: truncate d3/f7f 3994832 0 2024-04-02T04:47:56.441 INFO:tasks.workunit.client.1.smithi179.stdout:6/644: stat d3/ff 0 2024-04-02T04:47:56.442 INFO:tasks.workunit.client.1.smithi179.stdout:8/676: creat d3/d32/d21/d45/feb x:0 0 0 2024-04-02T04:47:56.442 INFO:tasks.workunit.client.1.smithi179.stdout:1/676: fsync d3/d5/d2f/d4f/d35/fbf 0 2024-04-02T04:47:56.443 INFO:tasks.workunit.client.0.smithi070.stdout:2/579: symlink d2/d24/lbd 0 2024-04-02T04:47:56.444 INFO:tasks.workunit.client.1.smithi179.stdout:5/630: truncate d8/d1b/d23/d36/f41 305857 0 2024-04-02T04:47:56.445 INFO:tasks.workunit.client.1.smithi179.stdout:4/694: dwrite d1/d3/df/d10/d2f/d60/da6/fb6 [0,4194304] 0 2024-04-02T04:47:56.445 INFO:tasks.workunit.client.0.smithi070.stdout:7/612: creat d7/d36/fc6 x:0 0 0 2024-04-02T04:47:56.451 INFO:tasks.workunit.client.0.smithi070.stdout:2/580: dwrite d2/fba [0,4194304] 0 2024-04-02T04:47:56.451 INFO:tasks.workunit.client.1.smithi179.stdout:7/688: rename d1/dd/d23/c7d to d1/d9/d27/d87/cf3 0 2024-04-02T04:47:56.453 INFO:tasks.workunit.client.0.smithi070.stdout:2/581: write d2/d10/d5c/d7a/fab [65317,57427] 0 2024-04-02T04:47:56.454 INFO:tasks.workunit.client.0.smithi070.stdout:3/604: creat d10/d17/db2/dbd/fcf x:0 0 0 2024-04-02T04:47:56.462 INFO:tasks.workunit.client.0.smithi070.stdout:5/745: rename d5/d14/f68 to d5/d9/fee 0 2024-04-02T04:47:56.464 INFO:tasks.workunit.client.1.smithi179.stdout:0/690: dread d1/da/f3c [0,4194304] 0 2024-04-02T04:47:56.465 INFO:tasks.workunit.client.0.smithi070.stdout:0/656: dwrite d4/f9 [4194304,4194304] 0 2024-04-02T04:47:56.469 INFO:tasks.workunit.client.1.smithi179.stdout:0/691: dwrite d1/da/d52/f73 [0,4194304] 0 2024-04-02T04:47:56.470 INFO:tasks.workunit.client.0.smithi070.stdout:6/493: link d0/d20/f45 d0/d43/d95/f9a 0 2024-04-02T04:47:56.470 INFO:tasks.workunit.client.0.smithi070.stdout:2/582: creat d2/d10/d5c/fbe x:0 0 0 2024-04-02T04:47:56.472 INFO:tasks.workunit.client.0.smithi070.stdout:3/605: rmdir d10/d32/d3f 39 2024-04-02T04:47:56.474 INFO:tasks.workunit.client.1.smithi179.stdout:8/677: creat d3/d1b/d2a/d83/fec x:0 0 0 2024-04-02T04:47:56.474 INFO:tasks.workunit.client.1.smithi179.stdout:5/631: mkdir d8/d1b/d23/d36/dcb 0 2024-04-02T04:47:56.477 INFO:tasks.workunit.client.0.smithi070.stdout:6/494: dread d0/d3/d19/d31/d61/f66 [0,4194304] 0 2024-04-02T04:47:56.479 INFO:tasks.workunit.client.1.smithi179.stdout:4/695: mkdir d1/d93/de6 0 2024-04-02T04:47:56.482 INFO:tasks.workunit.client.1.smithi179.stdout:5/632: dwrite d8/d1b/d23/d32/dbe/daf/fbf [0,4194304] 0 2024-04-02T04:47:56.484 INFO:tasks.workunit.client.0.smithi070.stdout:4/568: mkdir d1/d3/d78/dad/dbd 0 2024-04-02T04:47:56.484 INFO:tasks.workunit.client.1.smithi179.stdout:5/633: stat d8/d1b/d23/d30/d73/f81 0 2024-04-02T04:47:56.484 INFO:tasks.workunit.client.1.smithi179.stdout:7/689: creat d1/dd/d76/ff4 x:0 0 0 2024-04-02T04:47:56.484 INFO:tasks.workunit.client.1.smithi179.stdout:9/653: link d0/d19/d27/d48/l5c d0/d19/d1c/d45/lc7 0 2024-04-02T04:47:56.486 INFO:tasks.workunit.client.1.smithi179.stdout:9/654: chown d0/d19/d27/d7a/d84 7 1 2024-04-02T04:47:56.487 INFO:tasks.workunit.client.0.smithi070.stdout:6/495: dwrite d0/d3/d19/f7f [0,4194304] 0 2024-04-02T04:47:56.487 INFO:tasks.workunit.client.1.smithi179.stdout:5/634: write d8/d1b/d23/d32/f7a [1383214,63115] 0 2024-04-02T04:47:56.488 INFO:tasks.workunit.client.0.smithi070.stdout:6/496: stat d0/d3/d19/d31/d37/f5d 0 2024-04-02T04:47:56.488 INFO:tasks.workunit.client.0.smithi070.stdout:4/569: write d1/d9c/f79 [1242007,38855] 0 2024-04-02T04:47:56.490 INFO:tasks.workunit.client.0.smithi070.stdout:4/570: fdatasync d1/d9c/d65/d7a/f9f 0 2024-04-02T04:47:56.493 INFO:tasks.workunit.client.1.smithi179.stdout:0/692: creat d1/fcb x:0 0 0 2024-04-02T04:47:56.494 INFO:tasks.workunit.client.1.smithi179.stdout:2/528: dread f6 [0,4194304] 0 2024-04-02T04:47:56.505 INFO:tasks.workunit.client.0.smithi070.stdout:8/598: creat d2/d8/d29/fba x:0 0 0 2024-04-02T04:47:56.505 INFO:tasks.workunit.client.0.smithi070.stdout:8/599: chown d2/f1f 243639 1 2024-04-02T04:47:56.510 INFO:tasks.workunit.client.1.smithi179.stdout:3/677: rename d8/f1c to d8/d26/d52/d85/d9a/fd1 0 2024-04-02T04:47:56.512 INFO:tasks.workunit.client.0.smithi070.stdout:5/746: link d5/d17/d27/fc5 d5/d17/da7/fef 0 2024-04-02T04:47:56.517 INFO:tasks.workunit.client.0.smithi070.stdout:6/497: creat d0/d3/d19/d31/d46/f9b x:0 0 0 2024-04-02T04:47:56.519 INFO:tasks.workunit.client.0.smithi070.stdout:4/571: creat d1/d9c/d65/d7a/fbe x:0 0 0 2024-04-02T04:47:56.521 INFO:tasks.workunit.client.1.smithi179.stdout:5/635: write d8/d1b/d23/d36/d53/d89/fa2 [256860,114103] 0 2024-04-02T04:47:56.521 INFO:tasks.workunit.client.0.smithi070.stdout:0/657: mknod d4/d5/d13/d2a/cc2 0 2024-04-02T04:47:56.524 INFO:tasks.workunit.client.1.smithi179.stdout:1/677: dread d3/d5/d2f/f47 [0,4194304] 0 2024-04-02T04:47:56.525 INFO:tasks.workunit.client.1.smithi179.stdout:9/655: dwrite d0/d1/d11/d29/f50 [0,4194304] 0 2024-04-02T04:47:56.525 INFO:tasks.workunit.client.1.smithi179.stdout:2/529: creat dc/d29/d31/d83/fab x:0 0 0 2024-04-02T04:47:56.527 INFO:tasks.workunit.client.0.smithi070.stdout:4/572: dwrite d1/f93 [4194304,4194304] 0 2024-04-02T04:47:56.527 INFO:tasks.workunit.client.1.smithi179.stdout:1/678: write d3/d5/d2f/d4f/d42/fb1 [546650,102839] 0 2024-04-02T04:47:56.527 INFO:tasks.workunit.client.1.smithi179.stdout:1/679: chown d3/d5/d2f/d5a/dcc/ce6 8032 1 2024-04-02T04:47:56.528 INFO:tasks.workunit.client.0.smithi070.stdout:4/573: write d1/fa [2513678,55977] 0 2024-04-02T04:47:56.529 INFO:tasks.workunit.client.1.smithi179.stdout:6/645: creat d3/d12/d23/d39/fda x:0 0 0 2024-04-02T04:47:56.530 INFO:tasks.workunit.client.0.smithi070.stdout:2/583: creat d2/d10/d5c/d7a/db0/fbf x:0 0 0 2024-04-02T04:47:56.532 INFO:tasks.workunit.client.0.smithi070.stdout:2/584: chown d2/d10/d53/l6d 16487655 1 2024-04-02T04:47:56.539 INFO:tasks.workunit.client.0.smithi070.stdout:8/600: write d2/d8/d23/d24/d79/fa0 [198043,4323] 0 2024-04-02T04:47:56.541 INFO:tasks.workunit.client.0.smithi070.stdout:5/747: mknod d5/d9/d23/d30/da2/d75/cf0 0 2024-04-02T04:47:56.541 INFO:tasks.workunit.client.0.smithi070.stdout:5/748: readlink d5/d17/d6c/l4d 0 2024-04-02T04:47:56.545 INFO:tasks.workunit.client.1.smithi179.stdout:5/636: unlink d8/d1b/f9a 0 2024-04-02T04:47:56.546 INFO:tasks.workunit.client.1.smithi179.stdout:5/637: truncate d8/d7e/fbc 471865 0 2024-04-02T04:47:56.547 INFO:tasks.workunit.client.1.smithi179.stdout:0/693: fsync d1/da/f93 0 2024-04-02T04:47:56.548 INFO:tasks.workunit.client.0.smithi070.stdout:4/574: symlink d1/d3/d85/d8e/d99/db1/lbf 0 2024-04-02T04:47:56.552 INFO:tasks.workunit.client.1.smithi179.stdout:2/530: creat dc/d29/d4a/d1e/d43/fac x:0 0 0 2024-04-02T04:47:56.555 INFO:tasks.workunit.client.0.smithi070.stdout:5/749: chown d5/d17/d27/d2e/f3b 1467 1 2024-04-02T04:47:56.558 INFO:tasks.workunit.client.1.smithi179.stdout:8/678: creat d3/d32/d21/fed x:0 0 0 2024-04-02T04:47:56.559 INFO:tasks.workunit.client.1.smithi179.stdout:3/678: mknod d8/d57/cd2 0 2024-04-02T04:47:56.561 INFO:tasks.workunit.client.1.smithi179.stdout:3/679: write d8/d26/d83/d89/d9c/fc2 [1417011,69504] 0 2024-04-02T04:47:56.561 INFO:tasks.workunit.client.0.smithi070.stdout:8/601: symlink d2/d3/lbb 0 2024-04-02T04:47:56.562 INFO:tasks.workunit.client.0.smithi070.stdout:8/602: write d2/d8/d29/fb8 [935750,39232] 0 2024-04-02T04:47:56.564 INFO:tasks.workunit.client.0.smithi070.stdout:8/603: chown d2/d8/d29/d22/d5a/f7e 592 1 2024-04-02T04:47:56.567 INFO:tasks.workunit.client.0.smithi070.stdout:0/658: write d4/d5/da/f63 [432523,41649] 0 2024-04-02T04:47:56.568 INFO:tasks.workunit.client.1.smithi179.stdout:7/690: dread d1/d9/d27/d87/d8b/d75/f9a [0,4194304] 0 2024-04-02T04:47:56.570 INFO:tasks.workunit.client.1.smithi179.stdout:7/691: write d1/dd/d23/f2f [5058608,82278] 0 2024-04-02T04:47:56.570 INFO:tasks.workunit.client.1.smithi179.stdout:9/656: fdatasync d0/f4c 0 2024-04-02T04:47:56.571 INFO:tasks.workunit.client.0.smithi070.stdout:3/606: dwrite d10/d17/f4a [0,4194304] 0 2024-04-02T04:47:56.575 INFO:tasks.workunit.client.0.smithi070.stdout:2/585: rename d2/cd to d2/cc0 0 2024-04-02T04:47:56.575 INFO:tasks.workunit.client.0.smithi070.stdout:4/575: write d1/d3/f80 [651889,99706] 0 2024-04-02T04:47:56.577 INFO:tasks.workunit.client.0.smithi070.stdout:4/576: write d1/d3/d9/f13 [8156618,43915] 0 2024-04-02T04:47:56.578 INFO:tasks.workunit.client.1.smithi179.stdout:9/657: dwrite d0/d19/d27/d7a/d84/f93 [0,4194304] 0 2024-04-02T04:47:56.579 INFO:tasks.workunit.client.1.smithi179.stdout:6/646: unlink d3/d12/d23/d26/cd9 0 2024-04-02T04:47:56.579 INFO:tasks.workunit.client.1.smithi179.stdout:9/658: dread - d0/d1/d10/d6d/d58/f75 zero size 2024-04-02T04:47:56.580 INFO:tasks.workunit.client.0.smithi070.stdout:0/659: symlink d4/d3c/lc3 0 2024-04-02T04:47:56.583 INFO:tasks.workunit.client.1.smithi179.stdout:4/696: getdents d1/d3 0 2024-04-02T04:47:56.584 INFO:tasks.workunit.client.0.smithi070.stdout:8/604: mknod d2/d8/d29/d22/d5a/d9e/cbc 0 2024-04-02T04:47:56.586 INFO:tasks.workunit.client.1.smithi179.stdout:3/680: write d8/d26/d52/d85/f8b [4637020,44661] 0 2024-04-02T04:47:56.586 INFO:tasks.workunit.client.0.smithi070.stdout:5/750: dwrite d5/d9/f5e [0,4194304] 0 2024-04-02T04:47:56.591 INFO:tasks.workunit.client.0.smithi070.stdout:4/577: symlink d1/d3/d9/d5e/lc0 0 2024-04-02T04:47:56.591 INFO:tasks.workunit.client.0.smithi070.stdout:4/578: chown d1/d9c/f73 53 1 2024-04-02T04:47:56.592 INFO:tasks.workunit.client.0.smithi070.stdout:3/607: creat d10/d32/d4b/dc5/fd0 x:0 0 0 2024-04-02T04:47:56.594 INFO:tasks.workunit.client.0.smithi070.stdout:0/660: rename d4/d5/d13/d2a/cc2 to d4/d5/d13/d2a/d29/cc4 0 2024-04-02T04:47:56.594 INFO:tasks.workunit.client.0.smithi070.stdout:5/751: dread d5/d17/d27/d2e/f91 [0,4194304] 0 2024-04-02T04:47:56.596 INFO:tasks.workunit.client.0.smithi070.stdout:5/752: write d5/d57/f5d [3845777,47925] 0 2024-04-02T04:47:56.598 INFO:tasks.workunit.client.0.smithi070.stdout:4/579: dwrite d1/d3/d9/d5e/f87 [0,4194304] 0 2024-04-02T04:47:56.599 INFO:tasks.workunit.client.0.smithi070.stdout:4/580: dread - d1/d3/d78/d91/fa4 zero size 2024-04-02T04:47:56.600 INFO:tasks.workunit.client.0.smithi070.stdout:8/605: creat d2/d3/d35/d3a/d4f/d59/fbd x:0 0 0 2024-04-02T04:47:56.601 INFO:tasks.workunit.client.1.smithi179.stdout:7/692: mknod d1/d9/d27/d87/d8b/d75/db9/dcf/cf5 0 2024-04-02T04:47:56.603 INFO:tasks.workunit.client.1.smithi179.stdout:5/638: write d8/d1b/d23/d42/d5a/f76 [1543666,99179] 0 2024-04-02T04:47:56.606 INFO:tasks.workunit.client.1.smithi179.stdout:1/680: creat d3/d5/de/fec x:0 0 0 2024-04-02T04:47:56.608 INFO:tasks.workunit.client.1.smithi179.stdout:9/659: fdatasync d0/d19/d27/d7a/f8e 0 2024-04-02T04:47:56.608 INFO:tasks.workunit.client.0.smithi070.stdout:0/661: creat d4/d5/d13/d23/fc5 x:0 0 0 2024-04-02T04:47:56.611 INFO:tasks.workunit.client.1.smithi179.stdout:0/694: dwrite d1/da/d28/d37/f49 [4194304,4194304] 0 2024-04-02T04:47:56.613 INFO:tasks.workunit.client.0.smithi070.stdout:5/753: dread - d5/d17/d27/f98 zero size 2024-04-02T04:47:56.613 INFO:tasks.workunit.client.1.smithi179.stdout:0/695: fdatasync d1/da/d48/d7b/fb6 0 2024-04-02T04:47:56.614 INFO:tasks.workunit.client.1.smithi179.stdout:1/681: dwrite d3/d5/de/f1a [0,4194304] 0 2024-04-02T04:47:56.614 INFO:tasks.workunit.client.0.smithi070.stdout:5/754: dread - d5/d14/dd8/ddb/fdc zero size 2024-04-02T04:47:56.615 INFO:tasks.workunit.client.0.smithi070.stdout:4/581: fdatasync d1/d3/d9/f75 0 2024-04-02T04:47:56.618 INFO:tasks.workunit.client.0.smithi070.stdout:3/608: mkdir d10/d32/dd1 0 2024-04-02T04:47:56.621 INFO:tasks.workunit.client.0.smithi070.stdout:0/662: creat d4/d5/d13/d2a/d3a/d45/fc6 x:0 0 0 2024-04-02T04:47:56.623 INFO:tasks.workunit.client.0.smithi070.stdout:2/586: getdents d2/d10/d53 0 2024-04-02T04:47:56.638 INFO:tasks.workunit.client.1.smithi179.stdout:5/639: read d8/d1b/d23/d36/d53/f78 [620457,1244] 0 2024-04-02T04:47:56.639 INFO:tasks.workunit.client.1.smithi179.stdout:2/531: rename dc/d29/d4a/d1e/d6c/f87 to dc/d29/d4a/fad 0 2024-04-02T04:47:56.644 INFO:tasks.workunit.client.0.smithi070.stdout:5/755: symlink d5/d9/d10/db9/dba/lf1 0 2024-04-02T04:47:56.644 INFO:tasks.workunit.client.1.smithi179.stdout:5/640: dwrite d8/d1b/d23/d32/d4e/f54 [0,4194304] 0 2024-04-02T04:47:56.646 INFO:tasks.workunit.client.0.smithi070.stdout:5/756: write d5/d17/d27/f4b [2709657,70064] 0 2024-04-02T04:47:56.647 INFO:tasks.workunit.client.0.smithi070.stdout:5/757: truncate d5/d17/da7/fef 5196323 0 2024-04-02T04:47:56.652 INFO:tasks.workunit.client.1.smithi179.stdout:9/660: rmdir d0/d19 39 2024-04-02T04:47:56.654 INFO:tasks.workunit.client.1.smithi179.stdout:6/647: symlink d3/d12/d23/d39/d4e/d6c/dcd/ldb 0 2024-04-02T04:47:56.655 INFO:tasks.workunit.client.0.smithi070.stdout:3/609: mkdir d10/d17/db2/dbf/dd2 0 2024-04-02T04:47:56.656 INFO:tasks.workunit.client.0.smithi070.stdout:3/610: fsync d10/d17/d51/d73/d8f/f9c 0 2024-04-02T04:47:56.656 INFO:tasks.workunit.client.1.smithi179.stdout:3/681: unlink d8/d26/d83/dbc/cca 0 2024-04-02T04:47:56.657 INFO:tasks.workunit.client.0.smithi070.stdout:0/663: fsync d4/d5/d13/d23/f36 0 2024-04-02T04:47:56.657 INFO:tasks.workunit.client.1.smithi179.stdout:0/696: fdatasync d1/da/df/f20 0 2024-04-02T04:47:56.658 INFO:tasks.workunit.client.0.smithi070.stdout:2/587: fsync d2/f9c 0 2024-04-02T04:47:56.659 INFO:tasks.workunit.client.1.smithi179.stdout:0/697: read d1/f23 [3688205,96347] 0 2024-04-02T04:47:56.660 INFO:tasks.workunit.client.1.smithi179.stdout:7/693: fdatasync d1/d9/d55/dbf/d9f/fd3 0 2024-04-02T04:47:56.662 INFO:tasks.workunit.client.0.smithi070.stdout:3/611: dwrite d10/d17/f48 [0,4194304] 0 2024-04-02T04:47:56.662 INFO:tasks.workunit.client.0.smithi070.stdout:5/758: fsync d5/d17/d2b/f67 0 2024-04-02T04:47:56.663 INFO:tasks.workunit.client.1.smithi179.stdout:1/682: read d3/d5/f13 [1292575,51191] 0 2024-04-02T04:47:56.665 INFO:tasks.workunit.client.1.smithi179.stdout:8/679: rename d3/d32/d21/d51/l75 to d3/d32/d21/d22/lee 0 2024-04-02T04:47:56.667 INFO:tasks.workunit.client.1.smithi179.stdout:2/532: mkdir dc/d29/d31/d83/dae 0 2024-04-02T04:47:56.667 INFO:tasks.workunit.client.0.smithi070.stdout:8/606: truncate d2/d8/f6d 1203287 0 2024-04-02T04:47:56.672 INFO:tasks.workunit.client.1.smithi179.stdout:1/683: dwrite d3/d5/d2f/d4f/d42/d55/fb8 [0,4194304] 0 2024-04-02T04:47:56.673 INFO:tasks.workunit.client.0.smithi070.stdout:0/664: fdatasync d4/d5/d58/d60/f92 0 2024-04-02T04:47:56.674 INFO:tasks.workunit.client.1.smithi179.stdout:1/684: stat d3/d85/da2/fea 0 2024-04-02T04:47:56.677 INFO:tasks.workunit.client.1.smithi179.stdout:0/698: dread d1/f62 [0,4194304] 0 2024-04-02T04:47:56.677 INFO:tasks.workunit.client.0.smithi070.stdout:2/588: mkdir d2/d10/d21/d71/dc1 0 2024-04-02T04:47:56.678 INFO:tasks.workunit.client.0.smithi070.stdout:2/589: chown d2/d10/d5c/f7f 171 1 2024-04-02T04:47:56.678 INFO:tasks.workunit.client.1.smithi179.stdout:6/648: stat d3/l7 0 2024-04-02T04:47:56.678 INFO:tasks.workunit.client.1.smithi179.stdout:1/685: dread d3/d5/d2f/d36/f67 [0,4194304] 0 2024-04-02T04:47:56.679 INFO:tasks.workunit.client.0.smithi070.stdout:2/590: read - d2/fb5 zero size 2024-04-02T04:47:56.683 INFO:tasks.workunit.client.0.smithi070.stdout:3/612: mknod d10/d17/d51/d73/cd3 0 2024-04-02T04:47:56.684 INFO:tasks.workunit.client.0.smithi070.stdout:5/759: fsync d5/d9/d23/d30/da2/d75/f72 0 2024-04-02T04:47:56.686 INFO:tasks.workunit.client.1.smithi179.stdout:7/694: unlink d1/dd/d16/l30 0 2024-04-02T04:47:56.689 INFO:tasks.workunit.client.1.smithi179.stdout:4/697: rename d1/f3c to d1/d15/d6e/d9c/fe7 0 2024-04-02T04:47:56.691 INFO:tasks.workunit.client.1.smithi179.stdout:5/641: dwrite d8/d1b/d23/d32/d4e/f61 [0,4194304] 0 2024-04-02T04:47:56.692 INFO:tasks.workunit.client.1.smithi179.stdout:8/680: creat d3/d32/d21/d50/d56/dad/fef x:0 0 0 2024-04-02T04:47:56.699 INFO:tasks.workunit.client.0.smithi070.stdout:4/582: truncate d1/f3e 676093 0 2024-04-02T04:47:56.705 INFO:tasks.workunit.client.0.smithi070.stdout:3/613: rename d10/d32/d8a/d8d/da1/lb8 to d10/d17/d51/d73/d8f/ld4 0 2024-04-02T04:47:56.707 INFO:tasks.workunit.client.1.smithi179.stdout:7/695: dread - d1/dd/d16/fbc zero size 2024-04-02T04:47:56.709 INFO:tasks.workunit.client.0.smithi070.stdout:8/607: dwrite d2/d8/d29/db/f13 [4194304,4194304] 0 2024-04-02T04:47:56.712 INFO:tasks.workunit.client.0.smithi070.stdout:0/665: truncate d4/d5/d13/d17/f62 1240392 0 2024-04-02T04:47:56.712 INFO:tasks.workunit.client.0.smithi070.stdout:3/614: dread d10/d32/f42 [0,4194304] 0 2024-04-02T04:47:56.713 INFO:tasks.workunit.client.1.smithi179.stdout:0/699: dwrite d1/da/d28/f3a [0,4194304] 0 2024-04-02T04:47:56.717 INFO:tasks.workunit.client.1.smithi179.stdout:6/649: dwrite d3/d2d/d43/f4b [0,4194304] 0 2024-04-02T04:47:56.717 INFO:tasks.workunit.client.1.smithi179.stdout:0/700: write d1/da/f4a [499431,118069] 0 2024-04-02T04:47:56.719 INFO:tasks.workunit.client.0.smithi070.stdout:5/760: mknod d5/d17/cf2 0 2024-04-02T04:47:56.720 INFO:tasks.workunit.client.1.smithi179.stdout:4/698: write d1/d15/d6e/d9c/fe7 [3385120,23107] 0 2024-04-02T04:47:56.720 INFO:tasks.workunit.client.0.smithi070.stdout:5/761: truncate d5/d9/fd4 445032 0 2024-04-02T04:47:56.723 INFO:tasks.workunit.client.1.smithi179.stdout:8/681: mkdir d3/d32/d21/d49/df0 0 2024-04-02T04:47:56.726 INFO:tasks.workunit.client.1.smithi179.stdout:2/533: unlink dc/d29/d4a/l21 0 2024-04-02T04:47:56.726 INFO:tasks.workunit.client.1.smithi179.stdout:2/534: fsync dc/f9f 0 2024-04-02T04:47:56.727 INFO:tasks.workunit.client.0.smithi070.stdout:0/666: unlink d4/d3c/la4 0 2024-04-02T04:47:56.729 INFO:tasks.workunit.client.1.smithi179.stdout:9/661: creat d0/d1/fc8 x:0 0 0 2024-04-02T04:47:56.730 INFO:tasks.workunit.client.0.smithi070.stdout:2/591: rename d2/d10/d21/f33 to d2/d10/d21/fc2 0 2024-04-02T04:47:56.732 INFO:tasks.workunit.client.1.smithi179.stdout:7/696: symlink d1/dd/d23/dab/db0/lf6 0 2024-04-02T04:47:56.733 INFO:tasks.workunit.client.0.smithi070.stdout:4/583: dread d1/d3/d78/d91/f96 [0,4194304] 0 2024-04-02T04:47:56.734 INFO:tasks.workunit.client.0.smithi070.stdout:3/615: dwrite d10/d17/db2/dbd/fce [0,4194304] 0 2024-04-02T04:47:56.735 INFO:tasks.workunit.client.0.smithi070.stdout:4/584: stat d1/d3/d78/dad/dbd 0 2024-04-02T04:47:56.736 INFO:tasks.workunit.client.0.smithi070.stdout:5/762: mknod d5/d9/d10/d6a/d7d/cf3 0 2024-04-02T04:47:56.739 INFO:tasks.workunit.client.1.smithi179.stdout:0/701: mknod d1/da/d28/d34/d39/d77/ccc 0 2024-04-02T04:47:56.742 INFO:tasks.workunit.client.1.smithi179.stdout:4/699: symlink d1/d3/dc/d26/le8 0 2024-04-02T04:47:56.742 INFO:tasks.workunit.client.1.smithi179.stdout:4/700: chown d1/d3/df/lb7 1833552 1 2024-04-02T04:47:56.744 INFO:tasks.workunit.client.0.smithi070.stdout:4/585: dwrite d1/d3/d9/f43 [0,4194304] 0 2024-04-02T04:47:56.744 INFO:tasks.workunit.client.0.smithi070.stdout:8/608: link d2/d8/d29/d22/d5a/d94/da2/c76 d2/d3/d35/cbe 0 2024-04-02T04:47:56.746 INFO:tasks.workunit.client.0.smithi070.stdout:5/763: dwrite d5/d17/d2b/f8d [0,4194304] 0 2024-04-02T04:47:56.748 INFO:tasks.workunit.client.0.smithi070.stdout:5/764: stat d5/d9/d10/d51/fbc 0 2024-04-02T04:47:56.753 INFO:tasks.workunit.client.1.smithi179.stdout:3/682: rename d8/d26/d52/d85 to d8/dd3 0 2024-04-02T04:47:56.754 INFO:tasks.workunit.client.0.smithi070.stdout:2/592: fdatasync d2/d10/d5c/f81 0 2024-04-02T04:47:56.758 INFO:tasks.workunit.client.1.smithi179.stdout:0/702: creat d1/da/df/fcd x:0 0 0 2024-04-02T04:47:56.759 INFO:tasks.workunit.client.1.smithi179.stdout:0/703: chown d1/da/l43 13 1 2024-04-02T04:47:56.760 INFO:tasks.workunit.client.0.smithi070.stdout:4/586: unlink d1/d3/d9/f75 0 2024-04-02T04:47:56.762 INFO:tasks.workunit.client.1.smithi179.stdout:5/642: creat d8/d1b/d23/d42/fcc x:0 0 0 2024-04-02T04:47:56.763 INFO:tasks.workunit.client.0.smithi070.stdout:8/609: creat d2/d8/d23/d24/d5f/fbf x:0 0 0 2024-04-02T04:47:56.765 INFO:tasks.workunit.client.0.smithi070.stdout:5/765: stat d5/l52 0 2024-04-02T04:47:56.770 INFO:tasks.workunit.client.1.smithi179.stdout:7/697: write d1/dd/d23/f25 [1622432,44224] 0 2024-04-02T04:47:56.770 INFO:tasks.workunit.client.0.smithi070.stdout:2/593: mkdir d2/d24/dc3 0 2024-04-02T04:47:56.775 INFO:tasks.workunit.client.0.smithi070.stdout:8/610: creat d2/d8/d29/d22/d5a/d94/fc0 x:0 0 0 2024-04-02T04:47:56.776 INFO:tasks.workunit.client.1.smithi179.stdout:8/682: truncate d3/d1b/fa2 2405831 0 2024-04-02T04:47:56.777 INFO:tasks.workunit.client.1.smithi179.stdout:6/650: mkdir d3/d12/d23/d39/d80/d93/ddc 0 2024-04-02T04:47:56.778 INFO:tasks.workunit.client.0.smithi070.stdout:5/766: chown d5/d9/d23/d30/da2/c47 482590 1 2024-04-02T04:47:56.779 INFO:tasks.workunit.client.0.smithi070.stdout:0/667: getdents d4/d5/d21/d6b 0 2024-04-02T04:47:56.780 INFO:tasks.workunit.client.0.smithi070.stdout:2/594: creat d2/d10/d21/d2b/fc4 x:0 0 0 2024-04-02T04:47:56.781 INFO:tasks.workunit.client.0.smithi070.stdout:2/595: truncate d2/d24/f96 278805 0 2024-04-02T04:47:56.782 INFO:tasks.workunit.client.0.smithi070.stdout:3/616: getdents d10/d66 0 2024-04-02T04:47:56.783 INFO:tasks.workunit.client.1.smithi179.stdout:3/683: dwrite d8/d26/d83/d89/fab [0,4194304] 0 2024-04-02T04:47:56.783 INFO:tasks.workunit.client.0.smithi070.stdout:3/617: stat d10/d17/d51/c87 0 2024-04-02T04:47:56.783 INFO:tasks.workunit.client.0.smithi070.stdout:3/618: write d10/f29 [3588923,30344] 0 2024-04-02T04:47:56.784 INFO:tasks.workunit.client.1.smithi179.stdout:3/684: write d8/d26/f4e [3275565,115763] 0 2024-04-02T04:47:56.784 INFO:tasks.workunit.client.1.smithi179.stdout:2/535: creat dc/d29/d31/faf x:0 0 0 2024-04-02T04:47:56.785 INFO:tasks.workunit.client.1.smithi179.stdout:2/536: dread - dc/d29/d31/d57/fa0 zero size 2024-04-02T04:47:56.785 INFO:tasks.workunit.client.0.smithi070.stdout:3/619: write fb [3871669,115755] 0 2024-04-02T04:47:56.785 INFO:tasks.workunit.client.0.smithi070.stdout:4/587: mknod d1/d3/d85/da8/cc1 0 2024-04-02T04:47:56.786 INFO:tasks.workunit.client.0.smithi070.stdout:4/588: stat d1/d3/d9/c1c 0 2024-04-02T04:47:56.788 INFO:tasks.workunit.client.0.smithi070.stdout:4/589: dread - d1/d9c/d65/fb4 zero size 2024-04-02T04:47:56.790 INFO:tasks.workunit.client.0.smithi070.stdout:4/590: write d1/d3/d9/f9b [1203232,60360] 0 2024-04-02T04:47:56.790 INFO:tasks.workunit.client.1.smithi179.stdout:6/651: mknod d3/d12/d23/d39/d4e/d6c/cdd 0 2024-04-02T04:47:56.793 INFO:tasks.workunit.client.1.smithi179.stdout:0/704: link d1/db/dad/fbd d1/da/d28/d37/fce 0 2024-04-02T04:47:56.795 INFO:tasks.workunit.client.1.smithi179.stdout:0/705: chown d1/da/d28/d34/f7e 215461 1 2024-04-02T04:47:56.798 INFO:tasks.workunit.client.0.smithi070.stdout:8/611: mknod d2/d8/d29/d22/cc1 0 2024-04-02T04:47:56.799 INFO:tasks.workunit.client.1.smithi179.stdout:4/701: link d1/d93/d9f/fb4 d1/d93/fe9 0 2024-04-02T04:47:56.799 INFO:tasks.workunit.client.0.smithi070.stdout:5/767: write d5/d17/d27/f28 [4453312,17978] 0 2024-04-02T04:47:56.800 INFO:tasks.workunit.client.0.smithi070.stdout:3/620: mkdir d10/d17/d51/d73/d9e/dd5 0 2024-04-02T04:47:56.806 INFO:tasks.workunit.client.0.smithi070.stdout:0/668: dwrite d4/d5/d13/f19 [0,4194304] 0 2024-04-02T04:47:56.806 INFO:tasks.workunit.client.0.smithi070.stdout:8/612: dwrite d2/d3/d52/d62/d7c/fa4 [0,4194304] 0 2024-04-02T04:47:56.810 INFO:tasks.workunit.client.1.smithi179.stdout:9/662: getdents d0/d1/d11/d29 0 2024-04-02T04:47:56.814 INFO:tasks.workunit.client.0.smithi070.stdout:3/621: rename d10/d32/d8a/d8d/da1 to d10/d17/db2/dbf/dd6 0 2024-04-02T04:47:56.815 INFO:tasks.workunit.client.1.smithi179.stdout:7/698: dwrite d1/dd/f20 [4194304,4194304] 0 2024-04-02T04:47:56.816 INFO:tasks.workunit.client.0.smithi070.stdout:3/622: truncate d10/d32/d8a/d8d/fc1 923939 0 2024-04-02T04:47:56.818 INFO:tasks.workunit.client.1.smithi179.stdout:5/643: dwrite d8/d1b/d23/d32/f7f [0,4194304] 0 2024-04-02T04:47:56.819 INFO:tasks.workunit.client.1.smithi179.stdout:4/702: creat d1/d93/d9f/fea x:0 0 0 2024-04-02T04:47:56.820 INFO:tasks.workunit.client.1.smithi179.stdout:3/685: symlink d8/d57/ld4 0 2024-04-02T04:47:56.823 INFO:tasks.workunit.client.0.smithi070.stdout:5/768: creat d5/d9/d10/db9/dba/de1/ff4 x:0 0 0 2024-04-02T04:47:56.824 INFO:tasks.workunit.client.1.smithi179.stdout:2/537: mkdir dc/d29/d31/d83/dae/db0 0 2024-04-02T04:47:56.828 INFO:tasks.workunit.client.0.smithi070.stdout:8/613: dread d2/d3/d35/d3a/d4f/d59/f8f [0,4194304] 0 2024-04-02T04:47:56.830 INFO:tasks.workunit.client.1.smithi179.stdout:0/706: mknod d1/da/d28/d34/d9c/ccf 0 2024-04-02T04:47:56.831 INFO:tasks.workunit.client.1.smithi179.stdout:0/707: write d1/f2a [6713047,64552] 0 2024-04-02T04:47:56.833 INFO:tasks.workunit.client.1.smithi179.stdout:0/708: dread - d1/db/dad/fbd zero size 2024-04-02T04:47:56.834 INFO:tasks.workunit.client.1.smithi179.stdout:7/699: symlink d1/d9/d27/d87/d8b/lf7 0 2024-04-02T04:47:56.837 INFO:tasks.workunit.client.0.smithi070.stdout:2/596: dread d2/d10/d21/d2b/f82 [0,4194304] 0 2024-04-02T04:47:56.839 INFO:tasks.workunit.client.1.smithi179.stdout:2/538: creat dc/d29/d4a/d1e/fb1 x:0 0 0 2024-04-02T04:47:56.841 INFO:tasks.workunit.client.1.smithi179.stdout:9/663: creat d0/d19/d27/d7a/d87/fc9 x:0 0 0 2024-04-02T04:47:56.841 INFO:tasks.workunit.client.0.smithi070.stdout:9/518: sync 2024-04-02T04:47:56.842 INFO:tasks.workunit.client.0.smithi070.stdout:1/605: sync 2024-04-02T04:47:56.842 INFO:tasks.workunit.client.0.smithi070.stdout:6/498: sync 2024-04-02T04:47:56.843 INFO:tasks.workunit.client.0.smithi070.stdout:7/613: sync 2024-04-02T04:47:56.843 INFO:tasks.workunit.client.0.smithi070.stdout:7/614: readlink d7/d36/l99 0 2024-04-02T04:47:56.844 INFO:tasks.workunit.client.0.smithi070.stdout:7/615: chown d7/dd/df/d7a/d6c/d77/d75/d80 13360741 1 2024-04-02T04:47:56.851 INFO:tasks.workunit.client.1.smithi179.stdout:4/703: rmdir d1/d93 39 2024-04-02T04:47:56.853 INFO:tasks.workunit.client.1.smithi179.stdout:3/686: symlink d8/d26/d83/d89/ld5 0 2024-04-02T04:47:56.854 INFO:tasks.workunit.client.1.smithi179.stdout:6/652: getdents d3/d2d/d43 0 2024-04-02T04:47:56.855 INFO:tasks.workunit.client.1.smithi179.stdout:6/653: write d3/d12/d23/d39/d4e/d66/fc1 [150835,77716] 0 2024-04-02T04:47:56.856 INFO:tasks.workunit.client.1.smithi179.stdout:2/539: mknod dc/d29/d31/d57/cb2 0 2024-04-02T04:47:56.857 INFO:tasks.workunit.client.1.smithi179.stdout:9/664: symlink d0/d19/d27/d7a/d87/daf/dc2/lca 0 2024-04-02T04:47:56.859 INFO:tasks.workunit.client.1.smithi179.stdout:0/709: rmdir d1 39 2024-04-02T04:47:56.860 INFO:tasks.workunit.client.1.smithi179.stdout:4/704: fdatasync d1/d3/dc/d27/d62/f69 0 2024-04-02T04:47:56.862 INFO:tasks.workunit.client.1.smithi179.stdout:6/654: write d3/d12/d44/fbb [206443,12162] 0 2024-04-02T04:47:56.863 INFO:tasks.workunit.client.1.smithi179.stdout:2/540: symlink dc/d79/lb3 0 2024-04-02T04:47:56.864 INFO:tasks.workunit.client.1.smithi179.stdout:2/541: chown dc/d29/d4a/d4b/la7 402799 1 2024-04-02T04:47:56.864 INFO:tasks.workunit.client.1.smithi179.stdout:2/542: stat dc/c2a 0 2024-04-02T04:47:56.870 INFO:tasks.workunit.client.1.smithi179.stdout:4/705: symlink d1/d3/dc/d27/dae/leb 0 2024-04-02T04:47:56.873 INFO:tasks.workunit.client.1.smithi179.stdout:2/543: dread - dc/d29/d4a/d1e/fb1 zero size 2024-04-02T04:47:56.875 INFO:tasks.workunit.client.1.smithi179.stdout:2/544: truncate dc/d29/d31/f41 4369539 0 2024-04-02T04:47:56.875 INFO:tasks.workunit.client.1.smithi179.stdout:9/665: mkdir d0/d19/d27/d7a/dcb 0 2024-04-02T04:47:56.877 INFO:tasks.workunit.client.1.smithi179.stdout:7/700: rename d1/dd/d16/d4d/c80 to d1/d9/d27/cf8 0 2024-04-02T04:47:56.882 INFO:tasks.workunit.client.1.smithi179.stdout:6/655: mkdir d3/d12/dde 0 2024-04-02T04:47:56.883 INFO:tasks.workunit.client.1.smithi179.stdout:2/545: chown dc/d29/d4a/d1e/d43 85461058 1 2024-04-02T04:47:56.883 INFO:tasks.workunit.client.1.smithi179.stdout:4/706: dread d1/d3/dc/d26/d8a/f4a [0,4194304] 0 2024-04-02T04:47:56.887 INFO:tasks.workunit.client.1.smithi179.stdout:3/687: rename d8/d26/d36/l70 to d8/dd3/dbb/ld6 0 2024-04-02T04:47:56.887 INFO:tasks.workunit.client.1.smithi179.stdout:3/688: chown d8/d57/ca5 1465 1 2024-04-02T04:47:56.890 INFO:tasks.workunit.client.1.smithi179.stdout:2/546: symlink dc/d29/d31/d83/dae/lb4 0 2024-04-02T04:47:56.893 INFO:tasks.workunit.client.0.smithi070.stdout:0/669: creat d4/d5/d13/fc7 x:0 0 0 2024-04-02T04:47:56.895 INFO:tasks.workunit.client.1.smithi179.stdout:7/701: rename d1/d9/d35/d6b/cc6 to d1/dd/d23/dab/cf9 0 2024-04-02T04:47:56.897 INFO:tasks.workunit.client.1.smithi179.stdout:3/689: creat d8/d26/d83/fd7 x:0 0 0 2024-04-02T04:47:56.898 INFO:tasks.workunit.client.1.smithi179.stdout:0/710: mknod d1/da/d48/d5c/d83/cd0 0 2024-04-02T04:47:56.899 INFO:tasks.workunit.client.0.smithi070.stdout:8/614: mkdir d2/d8/d23/d24/d5f/dc2 0 2024-04-02T04:47:56.900 INFO:tasks.workunit.client.1.smithi179.stdout:6/656: symlink d3/d2d/ldf 0 2024-04-02T04:47:56.901 INFO:tasks.workunit.client.1.smithi179.stdout:6/657: dread - d3/d12/fc4 zero size 2024-04-02T04:47:56.901 INFO:tasks.workunit.client.0.smithi070.stdout:2/597: creat d2/d12/d59/fc5 x:0 0 0 2024-04-02T04:47:56.902 INFO:tasks.workunit.client.1.smithi179.stdout:2/547: mknod dc/d29/d31/d83/cb5 0 2024-04-02T04:47:56.902 INFO:tasks.workunit.client.1.smithi179.stdout:2/548: chown dc/d29/d4a/d4b/f6d 284977 1 2024-04-02T04:47:56.904 INFO:tasks.workunit.client.1.smithi179.stdout:4/707: dread d1/d3/df/d10/f2d [0,4194304] 0 2024-04-02T04:47:56.904 INFO:tasks.workunit.client.0.smithi070.stdout:9/519: mknod d4/d10/d12/d27/d60/cab 0 2024-04-02T04:47:56.904 INFO:tasks.workunit.client.1.smithi179.stdout:4/708: write d1/d15/f56 [2026343,108923] 0 2024-04-02T04:47:56.905 INFO:tasks.workunit.client.1.smithi179.stdout:7/702: creat d1/dd/d76/ffa x:0 0 0 2024-04-02T04:47:56.909 INFO:tasks.workunit.client.1.smithi179.stdout:3/690: symlink d8/d6d/d94/ld8 0 2024-04-02T04:47:56.910 INFO:tasks.workunit.client.0.smithi070.stdout:9/520: dread d4/d10/f25 [0,4194304] 0 2024-04-02T04:47:56.911 INFO:tasks.workunit.client.1.smithi179.stdout:0/711: unlink d1/da/d28/d81/lb4 0 2024-04-02T04:47:56.911 INFO:tasks.workunit.client.0.smithi070.stdout:9/521: fdatasync d4/d10/d11/d7b/d19/f48 0 2024-04-02T04:47:56.912 INFO:tasks.workunit.client.0.smithi070.stdout:4/591: link d1/d9c/d65/l7e d1/d3/d85/d8e/d99/lc2 0 2024-04-02T04:47:56.913 INFO:tasks.workunit.client.1.smithi179.stdout:6/658: mkdir d3/d2c/de0 0 2024-04-02T04:47:56.915 INFO:tasks.workunit.client.0.smithi070.stdout:0/670: rename d4/d5/d13/d2a/d3a/d80/f9c to d4/d5/d13/d2a/d3a/d45/fc8 0 2024-04-02T04:47:56.916 INFO:tasks.workunit.client.1.smithi179.stdout:2/549: creat dc/d29/d31/d57/d97/fb6 x:0 0 0 2024-04-02T04:47:56.917 INFO:tasks.workunit.client.1.smithi179.stdout:9/666: dwrite d0/d1/d10/d6d/f2d [0,4194304] 0 2024-04-02T04:47:56.917 INFO:tasks.workunit.client.0.smithi070.stdout:5/769: mknod d5/d17/da7/cf5 0 2024-04-02T04:47:56.918 INFO:tasks.workunit.client.0.smithi070.stdout:0/671: dread d4/d5/d13/f19 [0,4194304] 0 2024-04-02T04:47:56.918 INFO:tasks.workunit.client.0.smithi070.stdout:5/770: chown d5/d9/d10/da9 399079117 1 2024-04-02T04:47:56.920 INFO:tasks.workunit.client.1.smithi179.stdout:4/709: rename d1/d3/dc/d27/d62/l81 to d1/d15/d6e/d9c/lec 0 2024-04-02T04:47:56.926 INFO:tasks.workunit.client.0.smithi070.stdout:2/598: chown d2/d10/d21/d2b/c43 23275 1 2024-04-02T04:47:56.926 INFO:tasks.workunit.client.0.smithi070.stdout:6/499: write d0/d3/d19/d31/d46/d6b/f72 [2596230,60396] 0 2024-04-02T04:47:56.928 INFO:tasks.workunit.client.0.smithi070.stdout:8/615: read d2/d8/f44 [1230635,118861] 0 2024-04-02T04:47:56.929 INFO:tasks.workunit.client.0.smithi070.stdout:1/606: unlink d0/d9/d1d/d22/d2c/d4f/ca6 0 2024-04-02T04:47:56.929 INFO:tasks.workunit.client.1.smithi179.stdout:9/667: fdatasync d0/d19/d1c/f31 0 2024-04-02T04:47:56.933 INFO:tasks.workunit.client.0.smithi070.stdout:7/616: write d7/dd/df/d79/d46/d65/f8c [3396027,41187] 0 2024-04-02T04:47:56.938 INFO:tasks.workunit.client.0.smithi070.stdout:3/623: truncate d10/d17/d4c/f62 2558372 0 2024-04-02T04:47:56.939 INFO:tasks.workunit.client.0.smithi070.stdout:4/592: readlink d1/l2f 0 2024-04-02T04:47:56.939 INFO:tasks.workunit.client.1.smithi179.stdout:2/550: write dc/d29/d4a/f15 [4213946,44919] 0 2024-04-02T04:47:56.941 INFO:tasks.workunit.client.1.smithi179.stdout:2/551: dread - dc/d29/d31/faf zero size 2024-04-02T04:47:56.943 INFO:tasks.workunit.client.1.smithi179.stdout:9/668: symlink d0/d19/d27/d7a/d87/daf/dc2/lcc 0 2024-04-02T04:47:56.943 INFO:tasks.workunit.client.1.smithi179.stdout:0/712: dwrite d1/da/df/f8a [0,4194304] 0 2024-04-02T04:47:56.944 INFO:tasks.workunit.client.1.smithi179.stdout:4/710: write d1/d15/f31 [377659,11484] 0 2024-04-02T04:47:56.946 INFO:tasks.workunit.client.1.smithi179.stdout:0/713: read - d1/da/df/d56/d84/fc5 zero size 2024-04-02T04:47:56.948 INFO:tasks.workunit.client.1.smithi179.stdout:9/669: truncate d0/d19/d1c/d64/fb7 907727 0 2024-04-02T04:47:56.948 INFO:tasks.workunit.client.1.smithi179.stdout:6/659: symlink d3/d12/d23/d39/d80/d93/ddc/le1 0 2024-04-02T04:47:56.949 INFO:tasks.workunit.client.1.smithi179.stdout:0/714: stat d1/da/d52/d75/dc0 0 2024-04-02T04:47:56.950 INFO:tasks.workunit.client.1.smithi179.stdout:0/715: fsync d1/da/d28/f41 0 2024-04-02T04:47:56.951 INFO:tasks.workunit.client.0.smithi070.stdout:9/522: dwrite d4/d10/d12/d27/d83/f8f [0,4194304] 0 2024-04-02T04:47:56.952 INFO:tasks.workunit.client.1.smithi179.stdout:0/716: read - d1/da/d48/d7b/fb6 zero size 2024-04-02T04:47:56.953 INFO:tasks.workunit.client.0.smithi070.stdout:6/500: mknod d0/d43/d95/c9c 0 2024-04-02T04:47:56.955 INFO:tasks.workunit.client.1.smithi179.stdout:4/711: dwrite d1/d3/dc/d26/d8a/d5d/f89 [0,4194304] 0 2024-04-02T04:47:56.956 INFO:tasks.workunit.client.1.smithi179.stdout:2/552: rename dc/d29/l95 to dc/d29/d31/d39/lb7 0 2024-04-02T04:47:56.957 INFO:tasks.workunit.client.1.smithi179.stdout:4/712: chown d1/d3/df/f8f 98006575 1 2024-04-02T04:47:56.957 INFO:tasks.workunit.client.0.smithi070.stdout:8/616: creat d2/d8/d29/d22/d5a/d94/da2/fc3 x:0 0 0 2024-04-02T04:47:56.959 INFO:tasks.workunit.client.1.smithi179.stdout:3/691: getdents d8/d26/d83/dbc 0 2024-04-02T04:47:56.961 INFO:tasks.workunit.client.1.smithi179.stdout:0/717: dwrite d1/da/d28/d34/f35 [0,4194304] 0 2024-04-02T04:47:56.963 INFO:tasks.workunit.client.1.smithi179.stdout:9/670: write d0/d1/d10/d6d/fe [4772277,103877] 0 2024-04-02T04:47:56.964 INFO:tasks.workunit.client.0.smithi070.stdout:7/617: unlink d7/d4e/d82/f59 0 2024-04-02T04:47:56.968 INFO:tasks.workunit.client.1.smithi179.stdout:6/660: rmdir d3/d12/d23/d39 39 2024-04-02T04:47:56.968 INFO:tasks.workunit.client.0.smithi070.stdout:3/624: fsync d10/d17/d4c/f84 0 2024-04-02T04:47:56.970 INFO:tasks.workunit.client.1.smithi179.stdout:2/553: mknod dc/d29/d31/d39/cb8 0 2024-04-02T04:47:56.972 INFO:tasks.workunit.client.1.smithi179.stdout:4/713: dread - d1/d3/df/d10/d2f/d60/d71/f85 zero size 2024-04-02T04:47:56.973 INFO:tasks.workunit.client.1.smithi179.stdout:4/714: chown d1/c2a 1349204 1 2024-04-02T04:47:56.975 INFO:tasks.workunit.client.1.smithi179.stdout:1/686: sync 2024-04-02T04:47:56.976 INFO:tasks.workunit.client.1.smithi179.stdout:8/683: sync 2024-04-02T04:47:56.976 INFO:tasks.workunit.client.1.smithi179.stdout:5/644: sync 2024-04-02T04:47:56.976 INFO:tasks.workunit.client.1.smithi179.stdout:3/692: rename d8/f22 to d8/dd3/fd9 0 2024-04-02T04:47:56.977 INFO:tasks.workunit.client.1.smithi179.stdout:1/687: fdatasync d3/d5/f21 0 2024-04-02T04:47:56.978 INFO:tasks.workunit.client.1.smithi179.stdout:0/718: creat d1/da/d48/d65/d7d/fd1 x:0 0 0 2024-04-02T04:47:56.981 INFO:tasks.workunit.client.0.smithi070.stdout:2/599: creat d2/d10/d19/fc6 x:0 0 0 2024-04-02T04:47:56.983 INFO:tasks.workunit.client.0.smithi070.stdout:9/523: rmdir d4/d10/d11 39 2024-04-02T04:47:56.985 INFO:tasks.workunit.client.0.smithi070.stdout:6/501: rmdir d0/d55/d8b/d94 39 2024-04-02T04:47:56.986 INFO:tasks.workunit.client.0.smithi070.stdout:6/502: write d0/d20/d5f/f62 [2854043,114989] 0 2024-04-02T04:47:56.988 INFO:tasks.workunit.client.0.smithi070.stdout:1/607: dread d0/d54/d64/d6d/f78 [0,4194304] 0 2024-04-02T04:47:56.990 INFO:tasks.workunit.client.1.smithi179.stdout:3/693: rmdir d8/d57/d63 39 2024-04-02T04:47:56.993 INFO:tasks.workunit.client.1.smithi179.stdout:5/645: symlink d8/d1b/d23/d30/d73/lcd 0 2024-04-02T04:47:56.995 INFO:tasks.workunit.client.0.smithi070.stdout:7/618: creat d7/dd/df/d34/d68/fc7 x:0 0 0 2024-04-02T04:47:56.999 INFO:tasks.workunit.client.1.smithi179.stdout:1/688: creat d3/d5/d2f/d4f/d42/d55/fed x:0 0 0 2024-04-02T04:47:57.000 INFO:tasks.workunit.client.0.smithi070.stdout:4/593: fdatasync d1/f26 0 2024-04-02T04:47:57.001 INFO:tasks.workunit.client.1.smithi179.stdout:8/684: dread d3/d32/d30/f7d [0,4194304] 0 2024-04-02T04:47:57.002 INFO:tasks.workunit.client.0.smithi070.stdout:5/771: rename d5/d9/d10/fc2 to d5/d9/d10/ff6 0 2024-04-02T04:47:57.003 INFO:tasks.workunit.client.1.smithi179.stdout:6/661: mknod d3/d12/d23/ce2 0 2024-04-02T04:47:57.003 INFO:tasks.workunit.client.0.smithi070.stdout:5/772: write d5/d9/d10/db9/dba/de1/ff4 [871014,13307] 0 2024-04-02T04:47:57.003 INFO:tasks.workunit.client.1.smithi179.stdout:6/662: write d3/d2d/d38/fa9 [3019285,44891] 0 2024-04-02T04:47:57.004 INFO:tasks.workunit.client.1.smithi179.stdout:5/646: dread d8/d1b/d23/d42/d5a/f6b [0,4194304] 0 2024-04-02T04:47:57.005 INFO:tasks.workunit.client.1.smithi179.stdout:6/663: chown d3/d12/d23/d2f 120980209 1 2024-04-02T04:47:57.005 INFO:tasks.workunit.client.0.smithi070.stdout:2/600: mkdir d2/d10/d5c/d7a/db0/dc7 0 2024-04-02T04:47:57.006 INFO:tasks.workunit.client.1.smithi179.stdout:4/715: mknod d1/d3/dc/dc6/ced 0 2024-04-02T04:47:57.008 INFO:tasks.workunit.client.1.smithi179.stdout:3/694: readlink d8/dd3/la3 0 2024-04-02T04:47:57.008 INFO:tasks.workunit.client.0.smithi070.stdout:9/524: write d4/d10/d11/d7b/d19/d53/f59 [5978523,74242] 0 2024-04-02T04:47:57.010 INFO:tasks.workunit.client.0.smithi070.stdout:8/617: write d2/d3/d52/f33 [443783,130728] 0 2024-04-02T04:47:57.012 INFO:tasks.workunit.client.0.smithi070.stdout:8/618: write d2/d8/d23/d24/d5f/fbf [181397,129732] 0 2024-04-02T04:47:57.013 INFO:tasks.workunit.client.1.smithi179.stdout:5/647: dwrite d8/d1b/d23/d32/d6e/fc1 [0,4194304] 0 2024-04-02T04:47:57.013 INFO:tasks.workunit.client.0.smithi070.stdout:7/619: chown d7/l14 304590913 1 2024-04-02T04:47:57.014 INFO:tasks.workunit.client.0.smithi070.stdout:8/619: truncate d2/d3/d52/f3c 4485071 0 2024-04-02T04:47:57.015 INFO:tasks.workunit.client.1.smithi179.stdout:8/685: creat d3/d31/ff1 x:0 0 0 2024-04-02T04:47:57.018 INFO:tasks.workunit.client.0.smithi070.stdout:3/625: rename d10/d32/d3f/c49 to d10/d17/db2/dbd/cd7 0 2024-04-02T04:47:57.019 INFO:tasks.workunit.client.1.smithi179.stdout:0/719: dread d1/f57 [0,4194304] 0 2024-04-02T04:47:57.020 INFO:tasks.workunit.client.0.smithi070.stdout:3/626: truncate d10/d17/f4a 5181224 0 2024-04-02T04:47:57.021 INFO:tasks.workunit.client.0.smithi070.stdout:3/627: write d10/d32/d8a/d8d/d91/f9f [66552,36934] 0 2024-04-02T04:47:57.022 INFO:tasks.workunit.client.1.smithi179.stdout:6/664: mknod d3/d12/d23/d26/d57/ce3 0 2024-04-02T04:47:57.022 INFO:tasks.workunit.client.1.smithi179.stdout:5/648: dwrite d8/d1b/d23/f69 [0,4194304] 0 2024-04-02T04:47:57.024 INFO:tasks.workunit.client.1.smithi179.stdout:5/649: dread - d8/d1b/d23/d32/d4e/d50/fb9 zero size 2024-04-02T04:47:57.024 INFO:tasks.workunit.client.1.smithi179.stdout:4/716: creat d1/d3/df/d10/d2f/d60/fee x:0 0 0 2024-04-02T04:47:57.025 INFO:tasks.workunit.client.1.smithi179.stdout:5/650: chown d8/f6c 1 1 2024-04-02T04:47:57.028 INFO:tasks.workunit.client.0.smithi070.stdout:0/672: link d4/d5/da/d2d/f89 d4/d5/da/fc9 0 2024-04-02T04:47:57.028 INFO:tasks.workunit.client.1.smithi179.stdout:3/695: mknod d8/d26/d83/d89/d9c/cda 0 2024-04-02T04:47:57.029 INFO:tasks.workunit.client.0.smithi070.stdout:2/601: symlink d2/d10/d53/lc8 0 2024-04-02T04:47:57.030 INFO:tasks.workunit.client.1.smithi179.stdout:9/671: getdents d0/d1/d10/d6d/d58 0 2024-04-02T04:47:57.030 INFO:tasks.workunit.client.1.smithi179.stdout:7/703: sync 2024-04-02T04:47:57.031 INFO:tasks.workunit.client.1.smithi179.stdout:7/704: chown d1/d9/d35/d6b/c93 2872 1 2024-04-02T04:47:57.032 INFO:tasks.workunit.client.1.smithi179.stdout:8/686: rmdir d3/d1b/d2a/d83 39 2024-04-02T04:47:57.033 INFO:tasks.workunit.client.0.smithi070.stdout:6/503: unlink d0/d3/d19/c71 0 2024-04-02T04:47:57.037 INFO:tasks.workunit.client.0.smithi070.stdout:5/773: write d5/d17/f53 [1887029,4005] 0 2024-04-02T04:47:57.041 INFO:tasks.workunit.client.0.smithi070.stdout:7/620: dread d7/dd/df/d7a/f5a [0,4194304] 0 2024-04-02T04:47:57.041 INFO:tasks.workunit.client.1.smithi179.stdout:0/720: creat d1/da/d28/d34/d39/d77/fd2 x:0 0 0 2024-04-02T04:47:57.042 INFO:tasks.workunit.client.1.smithi179.stdout:4/717: creat d1/d3/df/d10/d2f/d60/fef x:0 0 0 2024-04-02T04:47:57.043 INFO:tasks.workunit.client.0.smithi070.stdout:4/594: mkdir d1/dc3 0 2024-04-02T04:47:57.044 INFO:tasks.workunit.client.1.smithi179.stdout:1/689: creat d3/d5/de/fee x:0 0 0 2024-04-02T04:47:57.046 INFO:tasks.workunit.client.1.smithi179.stdout:1/690: write d3/d5/d2f/d4f/d42/d53/d77/f97 [190914,13148] 0 2024-04-02T04:47:57.046 INFO:tasks.workunit.client.0.smithi070.stdout:9/525: dwrite d4/d10/d11/f58 [0,4194304] 0 2024-04-02T04:47:57.047 INFO:tasks.workunit.client.1.smithi179.stdout:3/696: rename d8/dd3/la3 to d8/d26/d83/d89/d9c/dcc/ldb 0 2024-04-02T04:47:57.047 INFO:tasks.workunit.client.1.smithi179.stdout:1/691: fsync d3/f28 0 2024-04-02T04:47:57.048 INFO:tasks.workunit.client.1.smithi179.stdout:9/672: creat d0/d19/d27/d7a/d87/daf/fcd x:0 0 0 2024-04-02T04:47:57.049 INFO:tasks.workunit.client.1.smithi179.stdout:1/692: fdatasync d3/d85/f89 0 2024-04-02T04:47:57.051 INFO:tasks.workunit.client.0.smithi070.stdout:2/602: creat d2/d10/d21/d2b/fc9 x:0 0 0 2024-04-02T04:47:57.052 INFO:tasks.workunit.client.1.smithi179.stdout:6/665: dwrite d3/d12/d23/d2f/f56 [0,4194304] 0 2024-04-02T04:47:57.052 INFO:tasks.workunit.client.1.smithi179.stdout:2/554: getdents dc/d29/d4a/d1e/d6c 0 2024-04-02T04:47:57.053 INFO:tasks.workunit.client.0.smithi070.stdout:2/603: chown d2/d10/d5c/f81 199646 1 2024-04-02T04:47:57.054 INFO:tasks.workunit.client.0.smithi070.stdout:6/504: mknod d0/d55/d5c/c9d 0 2024-04-02T04:47:57.056 INFO:tasks.workunit.client.1.smithi179.stdout:3/697: dwrite d8/d6d/d94/fc5 [0,4194304] 0 2024-04-02T04:47:57.058 INFO:tasks.workunit.client.1.smithi179.stdout:3/698: write d8/d26/d36/f5c [469030,31592] 0 2024-04-02T04:47:57.060 INFO:tasks.workunit.client.0.smithi070.stdout:3/628: creat d10/d17/d51/d73/d9e/dd5/fd8 x:0 0 0 2024-04-02T04:47:57.061 INFO:tasks.workunit.client.0.smithi070.stdout:3/629: readlink d10/d32/l37 0 2024-04-02T04:47:57.061 INFO:tasks.workunit.client.0.smithi070.stdout:3/630: rename d10 to d10/d32/dd1/dd9 22 2024-04-02T04:47:57.062 INFO:tasks.workunit.client.0.smithi070.stdout:9/526: mknod d4/d10/d12/d56/cac 0 2024-04-02T04:47:57.065 INFO:tasks.workunit.client.0.smithi070.stdout:2/604: dread - d2/d10/d5c/f7f zero size 2024-04-02T04:47:57.068 INFO:tasks.workunit.client.0.smithi070.stdout:4/595: mkdir d1/dc3/dc4 0 2024-04-02T04:47:57.069 INFO:tasks.workunit.client.0.smithi070.stdout:3/631: creat d10/d17/d4c/fda x:0 0 0 2024-04-02T04:47:57.146 INFO:tasks.workunit.client.0.smithi070.stdout:8/620: getdents d2/d8/d29/d22/d5a/d94 0 2024-04-02T04:47:57.150 INFO:tasks.workunit.client.0.smithi070.stdout:4/596: creat d1/d3/d85/d8e/fc5 x:0 0 0 2024-04-02T04:47:57.152 INFO:tasks.workunit.client.0.smithi070.stdout:3/632: mkdir d10/d17/db2/dbd/ddb 0 2024-04-02T04:47:57.157 INFO:tasks.workunit.client.0.smithi070.stdout:5/774: dwrite d5/d9/d10/f11 [0,4194304] 0 2024-04-02T04:47:57.158 INFO:tasks.workunit.client.0.smithi070.stdout:2/605: mknod d2/d10/d21/d71/dc1/cca 0 2024-04-02T04:47:57.159 INFO:tasks.workunit.client.0.smithi070.stdout:5/775: chown d5/d9/c1f 7360 1 2024-04-02T04:47:57.161 INFO:tasks.workunit.client.0.smithi070.stdout:4/597: creat d1/d3/d85/d8e/fc6 x:0 0 0 2024-04-02T04:47:57.167 INFO:tasks.workunit.client.0.smithi070.stdout:8/621: symlink d2/d8/lc4 0 2024-04-02T04:47:57.169 INFO:tasks.workunit.client.0.smithi070.stdout:0/673: dwrite d4/d5/da/d2d/f66 [0,4194304] 0 2024-04-02T04:47:57.170 INFO:tasks.workunit.client.0.smithi070.stdout:0/674: chown d4/d3c/l5c 372714 1 2024-04-02T04:47:57.171 INFO:tasks.workunit.client.0.smithi070.stdout:5/776: mkdir d5/d9/d23/df7 0 2024-04-02T04:47:57.174 INFO:tasks.workunit.client.0.smithi070.stdout:2/606: mknod d2/d10/d21/d3d/ccb 0 2024-04-02T04:47:57.181 INFO:tasks.workunit.client.0.smithi070.stdout:8/622: creat d2/d8/d29/db/fc5 x:0 0 0 2024-04-02T04:47:57.188 INFO:tasks.workunit.client.0.smithi070.stdout:2/607: fdatasync d2/d10/d21/f89 0 2024-04-02T04:47:57.195 INFO:tasks.workunit.client.0.smithi070.stdout:2/608: dwrite d2/d12/d59/f8e [0,4194304] 0 2024-04-02T04:47:57.195 INFO:tasks.workunit.client.0.smithi070.stdout:2/609: readlink d2/d10/d21/d2b/l73 0 2024-04-02T04:47:57.196 INFO:tasks.workunit.client.0.smithi070.stdout:0/675: mknod d4/d5/d13/cca 0 2024-04-02T04:47:57.197 INFO:tasks.workunit.client.0.smithi070.stdout:2/610: chown d2/d10/d21/d71/dc1/cca 13 1 2024-04-02T04:47:57.203 INFO:tasks.workunit.client.1.smithi179.stdout:9/673: mknod d0/d19/d27/d7a/d7e/cce 0 2024-04-02T04:47:57.204 INFO:tasks.workunit.client.0.smithi070.stdout:8/623: creat d2/d8/d29/d22/d5a/d77/fc6 x:0 0 0 2024-04-02T04:47:57.207 INFO:tasks.workunit.client.0.smithi070.stdout:0/676: readlink d4/d5/d13/d2a/d29/l4f 0 2024-04-02T04:47:57.221 INFO:tasks.workunit.client.1.smithi179.stdout:3/699: unlink d8/d57/d63/f6b 0 2024-04-02T04:47:57.221 INFO:tasks.workunit.client.1.smithi179.stdout:5/651: getdents d8/d1b/d23/d36/d53 0 2024-04-02T04:47:57.223 INFO:tasks.workunit.client.1.smithi179.stdout:1/693: link d3/l2a d3/d5/d2f/d36/lef 0 2024-04-02T04:47:57.223 INFO:tasks.workunit.client.1.smithi179.stdout:3/700: read d8/fac [402344,580] 0 2024-04-02T04:47:57.244 INFO:tasks.workunit.client.0.smithi070.stdout:2/611: dread d2/d10/d21/d3d/f63 [0,4194304] 0 2024-04-02T04:47:57.246 INFO:tasks.workunit.client.1.smithi179.stdout:5/652: dread d8/d1b/d23/d36/d53/d62/da9/fc8 [0,4194304] 0 2024-04-02T04:47:57.251 INFO:tasks.workunit.client.1.smithi179.stdout:5/653: mknod d8/d1b/d23/d32/dbe/daf/cce 0 2024-04-02T04:47:57.254 INFO:tasks.workunit.client.1.smithi179.stdout:5/654: read d8/d1b/d23/d36/d53/f72 [492670,21074] 0 2024-04-02T04:47:57.255 INFO:tasks.workunit.client.1.smithi179.stdout:5/655: fdatasync d8/d1b/d23/d36/f41 0 2024-04-02T04:47:57.255 INFO:tasks.workunit.client.1.smithi179.stdout:1/694: link d3/d5/d96/ldd d3/d85/lf0 0 2024-04-02T04:47:57.256 INFO:tasks.workunit.client.1.smithi179.stdout:5/656: chown d8/d1b/d23/d36/l87 6 1 2024-04-02T04:47:57.259 INFO:tasks.workunit.client.1.smithi179.stdout:5/657: truncate d8/d1b/d23/d30/f5c 108449 0 2024-04-02T04:47:57.262 INFO:tasks.workunit.client.1.smithi179.stdout:5/658: creat d8/d1b/d23/d32/dbe/fcf x:0 0 0 2024-04-02T04:47:57.271 INFO:tasks.workunit.client.1.smithi179.stdout:2/555: creat dc/d29/d31/fb9 x:0 0 0 2024-04-02T04:47:57.272 INFO:tasks.workunit.client.0.smithi070.stdout:5/777: unlink d5/d17/d27/fc5 0 2024-04-02T04:47:57.274 INFO:tasks.workunit.client.1.smithi179.stdout:2/556: unlink dc/d29/d4a/d1e/d43/f5f 0 2024-04-02T04:47:57.277 INFO:tasks.workunit.client.0.smithi070.stdout:5/778: creat d5/d9/d10/ff8 x:0 0 0 2024-04-02T04:47:57.278 INFO:tasks.workunit.client.0.smithi070.stdout:5/779: write d5/d9/d10/f11 [2837013,59565] 0 2024-04-02T04:47:57.279 INFO:tasks.workunit.client.1.smithi179.stdout:2/557: dread dc/d29/fa3 [0,4194304] 0 2024-04-02T04:47:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: pgmap v17: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 92 MiB/s rd, 300 MiB/s wr, 576 op/s 2024-04-02T04:47:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: Adjusting osd_memory_target on smithi179 to 2240M 2024-04-02T04:47:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:57 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:57.280 INFO:tasks.workunit.client.1.smithi179.stdout:2/558: readlink dc/d29/d4a/d4b/l60 0 2024-04-02T04:47:57.280 INFO:tasks.workunit.client.1.smithi179.stdout:2/559: readlink dc/d29/d31/l8e 0 2024-04-02T04:47:57.281 INFO:tasks.workunit.client.0.smithi070.stdout:5/780: creat d5/d17/d2b/d7b/ff9 x:0 0 0 2024-04-02T04:47:57.282 INFO:tasks.workunit.client.1.smithi179.stdout:2/560: mknod dc/d29/d31/d83/cba 0 2024-04-02T04:47:57.284 INFO:tasks.workunit.client.1.smithi179.stdout:2/561: mknod dc/d29/d31/d57/d97/cbb 0 2024-04-02T04:47:57.285 INFO:tasks.workunit.client.0.smithi070.stdout:0/677: creat d4/d5/d58/d60/fcb x:0 0 0 2024-04-02T04:47:57.287 INFO:tasks.workunit.client.0.smithi070.stdout:0/678: mkdir d4/d5/d13/d17/d98/dcc 0 2024-04-02T04:47:57.289 INFO:tasks.workunit.client.1.smithi179.stdout:2/562: getdents dc/d29/d31/d83/dae 0 2024-04-02T04:47:57.290 INFO:tasks.workunit.client.0.smithi070.stdout:0/679: creat d4/d5/d58/da1/fcd x:0 0 0 2024-04-02T04:47:57.291 INFO:tasks.workunit.client.1.smithi179.stdout:3/701: creat d8/d26/d83/d89/fdc x:0 0 0 2024-04-02T04:47:57.294 INFO:tasks.workunit.client.1.smithi179.stdout:3/702: truncate d8/d26/d36/f59 1655578 0 2024-04-02T04:47:57.295 INFO:tasks.workunit.client.1.smithi179.stdout:3/703: dread - d8/d26/d36/f49 zero size 2024-04-02T04:47:57.298 INFO:tasks.workunit.client.0.smithi070.stdout:9/527: write d4/d13/f98 [2661555,49225] 0 2024-04-02T04:47:57.308 INFO:tasks.workunit.client.0.smithi070.stdout:3/633: write f1 [418630,99711] 0 2024-04-02T04:47:57.314 INFO:tasks.workunit.client.1.smithi179.stdout:7/705: rename d1/dd/d76/lb5 to d1/dd/lfb 0 2024-04-02T04:47:57.315 INFO:tasks.workunit.client.1.smithi179.stdout:8/687: truncate d3/f4 869379 0 2024-04-02T04:47:57.316 INFO:tasks.workunit.client.1.smithi179.stdout:8/688: read d3/f18 [2615734,3496] 0 2024-04-02T04:47:57.316 INFO:tasks.workunit.client.1.smithi179.stdout:0/721: rename d1/da/df/f33 to d1/da/d48/d65/d7d/fd3 0 2024-04-02T04:47:57.317 INFO:tasks.workunit.client.0.smithi070.stdout:2/612: rmdir d2 39 2024-04-02T04:47:57.320 INFO:tasks.workunit.client.1.smithi179.stdout:4/718: rename d1/d3/df/d10/d2f/d36/f58 to d1/d3/df/d10/d3a/ff0 0 2024-04-02T04:47:57.320 INFO:tasks.workunit.client.1.smithi179.stdout:5/659: dread d8/d1b/f3b [0,4194304] 0 2024-04-02T04:47:57.320 INFO:tasks.workunit.client.1.smithi179.stdout:6/666: rename d3 to d3/d12/d23/d26/dd3/de4 22 2024-04-02T04:47:57.321 INFO:tasks.workunit.client.1.smithi179.stdout:8/689: rmdir d3/d31/d8f 39 2024-04-02T04:47:57.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: pgmap v17: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 92 MiB/s rd, 300 MiB/s wr, 576 op/s 2024-04-02T04:47:57.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:57.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:57.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:47:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: Adjusting osd_memory_target on smithi179 to 2240M 2024-04-02T04:47:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:57 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:47:57.322 INFO:tasks.workunit.client.1.smithi179.stdout:0/722: rmdir d1/da/d28/d6d 39 2024-04-02T04:47:57.323 INFO:tasks.workunit.client.1.smithi179.stdout:0/723: write d1/da/f1b [2514202,91543] 0 2024-04-02T04:47:57.327 INFO:tasks.workunit.client.0.smithi070.stdout:2/613: link d2/d10/d21/d2b/c43 d2/d12/d59/ccc 0 2024-04-02T04:47:57.328 INFO:tasks.workunit.client.0.smithi070.stdout:8/624: write d2/f1f [2296728,67055] 0 2024-04-02T04:47:57.331 INFO:tasks.workunit.client.1.smithi179.stdout:6/667: creat d3/d12/d23/d39/d80/d93/ddc/fe5 x:0 0 0 2024-04-02T04:47:57.332 INFO:tasks.workunit.client.0.smithi070.stdout:2/614: rename d2/d10/da4/l7c to d2/d10/d21/d3d/d42/lcd 0 2024-04-02T04:47:57.333 INFO:tasks.workunit.client.0.smithi070.stdout:2/615: chown d2/d10/d5c/d7a/fab 154 1 2024-04-02T04:47:57.333 INFO:tasks.workunit.client.1.smithi179.stdout:9/674: rename d0/d19/l7c to d0/d1/lcf 0 2024-04-02T04:47:57.336 INFO:tasks.workunit.client.1.smithi179.stdout:6/668: dwrite d3/d12/d23/d26/d57/fa2 [0,4194304] 0 2024-04-02T04:47:57.339 INFO:tasks.workunit.client.0.smithi070.stdout:2/616: dwrite d2/d12/d3e/faa [0,4194304] 0 2024-04-02T04:47:57.343 INFO:tasks.workunit.client.1.smithi179.stdout:6/669: creat d3/d2c/d85/fe6 x:0 0 0 2024-04-02T04:47:57.344 INFO:tasks.workunit.client.0.smithi070.stdout:2/617: unlink d2/d12/d59/c99 0 2024-04-02T04:47:57.346 INFO:tasks.workunit.client.0.smithi070.stdout:2/618: fdatasync d2/d10/d19/f77 0 2024-04-02T04:47:57.350 INFO:tasks.workunit.client.0.smithi070.stdout:2/619: mkdir d2/d10/d21/d2b/dce 0 2024-04-02T04:47:57.354 INFO:tasks.workunit.client.1.smithi179.stdout:2/563: rmdir dc/d29 39 2024-04-02T04:47:57.357 INFO:tasks.workunit.client.1.smithi179.stdout:8/690: rmdir d3/d32/d21/d51 39 2024-04-02T04:47:57.359 INFO:tasks.workunit.client.1.smithi179.stdout:8/691: read d3/d32/d21/f25 [8258663,123124] 0 2024-04-02T04:47:57.360 INFO:tasks.workunit.client.1.smithi179.stdout:8/692: symlink d3/d6c/lf2 0 2024-04-02T04:47:57.362 INFO:tasks.workunit.client.0.smithi070.stdout:3/634: rmdir d10/d17/d51/d73 39 2024-04-02T04:47:57.363 INFO:tasks.workunit.client.1.smithi179.stdout:8/693: creat d3/d32/d21/d50/d65/ff3 x:0 0 0 2024-04-02T04:47:57.364 INFO:tasks.workunit.client.0.smithi070.stdout:3/635: creat d10/d32/d4b/fdc x:0 0 0 2024-04-02T04:47:57.368 INFO:tasks.workunit.client.1.smithi179.stdout:8/694: creat d3/d1b/d2a/d41/ff4 x:0 0 0 2024-04-02T04:47:57.369 INFO:tasks.workunit.client.1.smithi179.stdout:8/695: symlink d3/d32/d21/d50/dcf/lf5 0 2024-04-02T04:47:57.371 INFO:tasks.workunit.client.0.smithi070.stdout:0/680: fsync d4/d5/d58/da1/fcd 0 2024-04-02T04:47:57.374 INFO:tasks.workunit.client.0.smithi070.stdout:0/681: mknod d4/d5/cce 0 2024-04-02T04:47:57.374 INFO:tasks.workunit.client.0.smithi070.stdout:0/682: chown f0 104012 1 2024-04-02T04:47:57.375 INFO:tasks.workunit.client.0.smithi070.stdout:0/683: write d4/d5/d13/d2a/d3a/d80/fa9 [173973,78123] 0 2024-04-02T04:47:57.376 INFO:tasks.workunit.client.0.smithi070.stdout:9/528: truncate d4/d10/d11/d7b/d19/f24 3179062 0 2024-04-02T04:47:57.378 INFO:tasks.workunit.client.1.smithi179.stdout:1/695: rename d3/d5/d2f/d4f/d42/d55/c8f to d3/d5/d2f/d4f/cf1 0 2024-04-02T04:47:57.379 INFO:tasks.workunit.client.0.smithi070.stdout:9/529: dread - d4/d10/d11/d7b/d19/d82/f8a zero size 2024-04-02T04:47:57.380 INFO:tasks.workunit.client.1.smithi179.stdout:3/704: unlink d8/f98 0 2024-04-02T04:47:57.380 INFO:tasks.workunit.client.0.smithi070.stdout:0/684: symlink d4/d5/d13/d2a/d3a/d80/da8/lcf 0 2024-04-02T04:47:57.382 INFO:tasks.workunit.client.0.smithi070.stdout:3/636: unlink d10/d17/c8c 0 2024-04-02T04:47:57.383 INFO:tasks.workunit.client.1.smithi179.stdout:7/706: rename d1/d9/c22 to d1/dd/d16/d4d/cfc 0 2024-04-02T04:47:57.384 INFO:tasks.workunit.client.0.smithi070.stdout:9/530: unlink d4/d10/d11/d7b/f1d 0 2024-04-02T04:47:57.385 INFO:tasks.workunit.client.1.smithi179.stdout:3/705: creat d8/d57/d63/dc3/fdd x:0 0 0 2024-04-02T04:47:57.390 INFO:tasks.workunit.client.0.smithi070.stdout:3/637: unlink d10/d32/fcb 0 2024-04-02T04:47:57.392 INFO:tasks.workunit.client.1.smithi179.stdout:5/660: rename d8/cd to d8/d1b/d23/d36/dcb/cd0 0 2024-04-02T04:47:57.397 INFO:tasks.workunit.client.0.smithi070.stdout:9/531: rename d4/d10/d11/d7b/d19/d53/l65 to d4/d10/d12/d27/d60/lad 0 2024-04-02T04:47:57.397 INFO:tasks.workunit.client.1.smithi179.stdout:0/724: write d1/da/d28/d37/f95 [1388963,21015] 0 2024-04-02T04:47:57.398 INFO:tasks.workunit.client.1.smithi179.stdout:0/725: write d1/db/f2c [5133243,16470] 0 2024-04-02T04:47:57.400 INFO:tasks.workunit.client.0.smithi070.stdout:3/638: dread d10/d17/d4c/fa3 [0,4194304] 0 2024-04-02T04:47:57.403 INFO:tasks.workunit.client.1.smithi179.stdout:4/719: rename d1/d3/df/d10/d2f/d60/fda to d1/d3/dc/dc1/ff1 0 2024-04-02T04:47:57.404 INFO:tasks.workunit.client.0.smithi070.stdout:9/532: symlink d4/d10/d12/d55/d66/lae 0 2024-04-02T04:47:57.405 INFO:tasks.workunit.client.0.smithi070.stdout:8/625: truncate d2/d8/d23/d24/f37 808323 0 2024-04-02T04:47:57.406 INFO:tasks.workunit.client.1.smithi179.stdout:5/661: symlink d8/d1b/d23/d30/ld1 0 2024-04-02T04:47:57.406 INFO:tasks.workunit.client.0.smithi070.stdout:0/685: link d4/d5/d13/d2a/d29/c79 d4/d5/da/dab/cd0 0 2024-04-02T04:47:57.409 INFO:tasks.workunit.client.0.smithi070.stdout:3/639: unlink d10/d32/d3f/fca 0 2024-04-02T04:47:57.410 INFO:tasks.workunit.client.0.smithi070.stdout:0/686: dread d4/d5/da/d2d/f66 [0,4194304] 0 2024-04-02T04:47:57.410 INFO:tasks.workunit.client.1.smithi179.stdout:9/675: truncate d0/d19/d1c/f47 234605 0 2024-04-02T04:47:57.412 INFO:tasks.workunit.client.0.smithi070.stdout:9/533: rmdir d4/d10/d12/d56/d79 39 2024-04-02T04:47:57.416 INFO:tasks.workunit.client.1.smithi179.stdout:0/726: creat d1/da/d52/fd4 x:0 0 0 2024-04-02T04:47:57.417 INFO:tasks.workunit.client.0.smithi070.stdout:2/620: write d2/d10/d5c/f7f [225539,64886] 0 2024-04-02T04:47:57.418 INFO:tasks.workunit.client.1.smithi179.stdout:6/670: dwrite d3/d12/d23/d26/f4f [0,4194304] 0 2024-04-02T04:47:57.418 INFO:tasks.workunit.client.0.smithi070.stdout:9/534: dwrite d4/d10/d12/d27/d60/f88 [0,4194304] 0 2024-04-02T04:47:57.420 INFO:tasks.workunit.client.1.smithi179.stdout:2/564: rename dc/d29/d31/d39 to dc/d29/d31/d83/dae/db0/dbc 0 2024-04-02T04:47:57.420 INFO:tasks.workunit.client.1.smithi179.stdout:6/671: truncate d3/d2d/da5/fbd 501242 0 2024-04-02T04:47:57.421 INFO:tasks.workunit.client.1.smithi179.stdout:2/565: chown dc/d29/d4a/d4b/f6d 1264 1 2024-04-02T04:47:57.423 INFO:tasks.workunit.client.1.smithi179.stdout:4/720: creat d1/d3/df/d10/d2f/d60/d71/ff2 x:0 0 0 2024-04-02T04:47:57.425 INFO:tasks.workunit.client.0.smithi070.stdout:3/640: creat d10/d17/db2/dbd/fdd x:0 0 0 2024-04-02T04:47:57.426 INFO:tasks.workunit.client.0.smithi070.stdout:3/641: dread - d10/d32/d4b/dc5/fd0 zero size 2024-04-02T04:47:57.430 INFO:tasks.workunit.client.0.smithi070.stdout:8/626: mkdir d2/d8/d29/dc7 0 2024-04-02T04:47:57.431 INFO:tasks.workunit.client.0.smithi070.stdout:8/627: chown d2/d8/d29/d22/d5a/d94/fc0 1786470641 1 2024-04-02T04:47:57.433 INFO:tasks.workunit.client.0.smithi070.stdout:8/628: truncate d2/d3/f53 1176765 0 2024-04-02T04:47:57.437 INFO:tasks.workunit.client.1.smithi179.stdout:1/696: rename d3/d5/d2f/d4f/d35/d49 to d3/d5/d2f/d4f/d42/df2 0 2024-04-02T04:47:57.438 INFO:tasks.workunit.client.0.smithi070.stdout:8/629: dread d2/d8/d29/db/f13 [4194304,4194304] 0 2024-04-02T04:47:57.439 INFO:tasks.workunit.client.0.smithi070.stdout:8/630: read - d2/d8/d29/d22/d5a/d77/fc6 zero size 2024-04-02T04:47:57.439 INFO:tasks.workunit.client.1.smithi179.stdout:0/727: dread d1/da/d48/d5c/d83/f8b [4194304,4194304] 0 2024-04-02T04:47:57.440 INFO:tasks.workunit.client.1.smithi179.stdout:0/728: fdatasync d1/da/d48/d65/d7d/d80/f9e 0 2024-04-02T04:47:57.441 INFO:tasks.workunit.client.1.smithi179.stdout:6/672: creat d3/d9a/fe7 x:0 0 0 2024-04-02T04:47:57.441 INFO:tasks.workunit.client.1.smithi179.stdout:0/729: chown d1/da/d28/d34/d9c/ccf 55512255 1 2024-04-02T04:47:57.441 INFO:tasks.workunit.client.1.smithi179.stdout:0/730: fsync d1/db/f8e 0 2024-04-02T04:47:57.442 INFO:tasks.workunit.client.0.smithi070.stdout:2/621: creat d2/d10/da4/fcf x:0 0 0 2024-04-02T04:47:57.447 INFO:tasks.workunit.client.1.smithi179.stdout:6/673: dwrite d3/d12/d23/d26/faf [0,4194304] 0 2024-04-02T04:47:57.448 INFO:tasks.workunit.client.1.smithi179.stdout:6/674: readlink d3/d2d/d43/ld5 0 2024-04-02T04:47:57.449 INFO:tasks.workunit.client.0.smithi070.stdout:3/642: fsync d10/d32/d3f/f43 0 2024-04-02T04:47:57.449 INFO:tasks.workunit.client.1.smithi179.stdout:6/675: fdatasync d3/d9a/fe7 0 2024-04-02T04:47:57.453 INFO:tasks.workunit.client.1.smithi179.stdout:6/676: chown d3/d2c/f5b 1301 1 2024-04-02T04:47:57.454 INFO:tasks.workunit.client.1.smithi179.stdout:6/677: fdatasync d3/d12/d6a/f6e 0 2024-04-02T04:47:57.454 INFO:tasks.workunit.client.1.smithi179.stdout:7/707: dwrite d1/fe2 [0,4194304] 0 2024-04-02T04:47:57.457 INFO:tasks.workunit.client.0.smithi070.stdout:2/622: unlink d2/d10/d21/d71/fa5 0 2024-04-02T04:47:57.459 INFO:tasks.workunit.client.1.smithi179.stdout:1/697: mkdir d3/d5/d2f/d4f/d42/d55/df3 0 2024-04-02T04:47:57.461 INFO:tasks.workunit.client.0.smithi070.stdout:9/535: creat d4/d13/d89/faf x:0 0 0 2024-04-02T04:47:57.463 INFO:tasks.workunit.client.1.smithi179.stdout:3/706: truncate d8/d26/d36/f5e 2029141 0 2024-04-02T04:47:57.468 INFO:tasks.workunit.client.0.smithi070.stdout:0/687: rename d4/d5/da/dab to d4/d5/da/dd1 0 2024-04-02T04:47:57.469 INFO:tasks.workunit.client.1.smithi179.stdout:0/731: mknod d1/da/d28/d37/cd5 0 2024-04-02T04:47:57.481 INFO:tasks.workunit.client.0.smithi070.stdout:2/623: read - d2/d10/d19/f7b zero size 2024-04-02T04:47:57.484 INFO:tasks.workunit.client.1.smithi179.stdout:9/676: truncate d0/d1/d11/d29/f94 7267537 0 2024-04-02T04:47:57.485 INFO:tasks.workunit.client.1.smithi179.stdout:3/707: creat d8/d26/d52/d7e/db8/fde x:0 0 0 2024-04-02T04:47:57.487 INFO:tasks.workunit.client.1.smithi179.stdout:1/698: dwrite d3/d5/de/f3c [0,4194304] 0 2024-04-02T04:47:57.490 INFO:tasks.workunit.client.0.smithi070.stdout:8/631: dwrite d2/d8/d29/d22/f7a [0,4194304] 0 2024-04-02T04:47:57.496 INFO:tasks.workunit.client.0.smithi070.stdout:0/688: unlink d4/d5/ld 0 2024-04-02T04:47:57.504 INFO:tasks.workunit.client.1.smithi179.stdout:1/699: creat d3/d5/d2f/db7/ff4 x:0 0 0 2024-04-02T04:47:57.504 INFO:tasks.workunit.client.1.smithi179.stdout:1/700: chown d3/d5/c17 79207 1 2024-04-02T04:47:57.507 INFO:tasks.workunit.client.0.smithi070.stdout:9/536: rename d4/d10/d11/d7b/d19/d2f/d39/l91 to d4/d10/d12/d55/lb0 0 2024-04-02T04:47:57.507 INFO:tasks.workunit.client.0.smithi070.stdout:9/537: readlink d4/d10/d11/d7b/l6a 0 2024-04-02T04:47:57.507 INFO:tasks.workunit.client.1.smithi179.stdout:2/566: link dc/d29/c38 dc/d29/d31/d83/cbd 0 2024-04-02T04:47:57.508 INFO:tasks.workunit.client.1.smithi179.stdout:4/721: getdents d1/d15/d7c/d98 0 2024-04-02T04:47:57.511 INFO:tasks.workunit.client.1.smithi179.stdout:1/701: dwrite d3/d5/de/fee [0,4194304] 0 2024-04-02T04:47:57.515 INFO:tasks.workunit.client.1.smithi179.stdout:0/732: link d1/da/d28/d5e/fbe d1/da/d28/fd6 0 2024-04-02T04:47:57.517 INFO:tasks.workunit.client.0.smithi070.stdout:8/632: mkdir d2/d8/d29/d22/dc8 0 2024-04-02T04:47:57.519 INFO:tasks.workunit.client.1.smithi179.stdout:7/708: dwrite d1/f32 [0,4194304] 0 2024-04-02T04:47:57.521 INFO:tasks.workunit.client.1.smithi179.stdout:2/567: readlink dc/d29/d4a/d1e/l59 0 2024-04-02T04:47:57.522 INFO:tasks.workunit.client.0.smithi070.stdout:0/689: unlink d4/d5/c48 0 2024-04-02T04:47:57.523 INFO:tasks.workunit.client.0.smithi070.stdout:2/624: write d2/d12/d59/fb2 [755865,47247] 0 2024-04-02T04:47:57.524 INFO:tasks.workunit.client.0.smithi070.stdout:3/643: symlink d10/d17/d51/d73/lde 0 2024-04-02T04:47:57.524 INFO:tasks.workunit.client.0.smithi070.stdout:2/625: stat d2/d12/d59/fc5 0 2024-04-02T04:47:57.525 INFO:tasks.workunit.client.0.smithi070.stdout:2/626: chown d2/f1c 4417146 1 2024-04-02T04:47:57.525 INFO:tasks.workunit.client.0.smithi070.stdout:3/644: dread - d10/d17/db2/dbd/fdd zero size 2024-04-02T04:47:57.526 INFO:tasks.workunit.client.0.smithi070.stdout:2/627: chown d2/d24/lbd 414889899 1 2024-04-02T04:47:57.528 INFO:tasks.workunit.client.1.smithi179.stdout:6/678: getdents d3/d12/d23/d26 0 2024-04-02T04:47:57.533 INFO:tasks.workunit.client.0.smithi070.stdout:0/690: mkdir d4/d5/d13/d17/d98/dd2 0 2024-04-02T04:47:57.535 INFO:tasks.workunit.client.1.smithi179.stdout:1/702: mknod d3/d5/d2f/d4f/cf5 0 2024-04-02T04:47:57.535 INFO:tasks.workunit.client.1.smithi179.stdout:6/679: dwrite d3/d2c/f5b [0,4194304] 0 2024-04-02T04:47:57.536 INFO:tasks.workunit.client.0.smithi070.stdout:9/538: dread d4/d10/d11/d7b/d19/d53/f8b [0,4194304] 0 2024-04-02T04:47:57.537 INFO:tasks.workunit.client.1.smithi179.stdout:6/680: fdatasync d3/d12/d23/d26/d57/fa2 0 2024-04-02T04:47:57.537 INFO:tasks.workunit.client.0.smithi070.stdout:9/539: write d4/f9f [300725,79525] 0 2024-04-02T04:47:57.538 INFO:tasks.workunit.client.1.smithi179.stdout:6/681: chown d3/d42/l7b 46 1 2024-04-02T04:47:57.538 INFO:tasks.workunit.client.0.smithi070.stdout:2/628: mknod d2/d10/d5c/d7a/cd0 0 2024-04-02T04:47:57.545 INFO:tasks.workunit.client.0.smithi070.stdout:8/633: symlink d2/lc9 0 2024-04-02T04:47:57.546 INFO:tasks.workunit.client.1.smithi179.stdout:4/722: write d1/d3/d19/fa3 [571810,67575] 0 2024-04-02T04:47:57.550 INFO:tasks.workunit.client.1.smithi179.stdout:2/568: dwrite dc/d29/d31/d57/f62 [4194304,4194304] 0 2024-04-02T04:47:57.551 INFO:tasks.workunit.client.0.smithi070.stdout:2/629: truncate d2/d10/d5c/f83 3249827 0 2024-04-02T04:47:57.551 INFO:tasks.workunit.client.0.smithi070.stdout:8/634: dwrite d2/f9b [0,4194304] 0 2024-04-02T04:47:57.554 INFO:tasks.workunit.client.1.smithi179.stdout:9/677: getdents d0/d1/d11/dab 0 2024-04-02T04:47:57.556 INFO:tasks.workunit.client.1.smithi179.stdout:0/733: dwrite d1/da/d48/d65/d7d/f6c [0,4194304] 0 2024-04-02T04:47:57.559 INFO:tasks.workunit.client.0.smithi070.stdout:3/645: dwrite d10/d17/d4c/f84 [0,4194304] 0 2024-04-02T04:47:57.560 INFO:tasks.workunit.client.1.smithi179.stdout:1/703: symlink d3/d5/d2f/d4f/d42/d55/lf6 0 2024-04-02T04:47:57.562 INFO:tasks.workunit.client.0.smithi070.stdout:8/635: mknod d2/d8/d23/d24/d5f/cca 0 2024-04-02T04:47:57.563 INFO:tasks.workunit.client.0.smithi070.stdout:8/636: chown d2/cac 39691727 1 2024-04-02T04:47:57.571 INFO:tasks.workunit.client.1.smithi179.stdout:6/682: mkdir d3/d12/d6a/de8 0 2024-04-02T04:47:57.572 INFO:tasks.workunit.client.0.smithi070.stdout:9/540: dread d4/d10/d11/d7b/f2b [0,4194304] 0 2024-04-02T04:47:57.572 INFO:tasks.workunit.client.1.smithi179.stdout:6/683: write d3/d12/d23/d39/d4e/d66/fd2 [405661,41033] 0 2024-04-02T04:47:57.573 INFO:tasks.workunit.client.0.smithi070.stdout:3/646: rename fd to d10/d32/dd1/fdf 0 2024-04-02T04:47:57.574 INFO:tasks.workunit.client.0.smithi070.stdout:3/647: stat d10/d32/d8a/d8d/d91/f94 0 2024-04-02T04:47:57.576 INFO:tasks.workunit.client.0.smithi070.stdout:8/637: symlink d2/d3/d35/lcb 0 2024-04-02T04:47:57.578 INFO:tasks.workunit.client.1.smithi179.stdout:7/709: symlink d1/d9/d55/dbf/d9f/da8/dda/lfd 0 2024-04-02T04:47:57.578 INFO:tasks.workunit.client.0.smithi070.stdout:8/638: write d2/d8/d29/d22/d5a/d77/fc6 [944662,82330] 0 2024-04-02T04:47:57.580 INFO:tasks.workunit.client.1.smithi179.stdout:4/723: creat d1/d3/dc/d26/ff3 x:0 0 0 2024-04-02T04:47:57.581 INFO:tasks.workunit.client.0.smithi070.stdout:0/691: getdents d4/d5/da/dd1 0 2024-04-02T04:47:57.584 INFO:tasks.workunit.client.0.smithi070.stdout:9/541: mknod d4/d10/d12/d27/d60/d9e/cb1 0 2024-04-02T04:47:57.585 INFO:tasks.workunit.client.1.smithi179.stdout:0/734: creat d1/db/fd7 x:0 0 0 2024-04-02T04:47:57.588 INFO:tasks.workunit.client.0.smithi070.stdout:2/630: link d2/d10/d21/l29 d2/d24/dc3/ld1 0 2024-04-02T04:47:57.589 INFO:tasks.workunit.client.1.smithi179.stdout:7/710: truncate d1/d9/d35/f3b 2644477 0 2024-04-02T04:47:57.590 INFO:tasks.workunit.client.1.smithi179.stdout:4/724: creat d1/d3/dc/dc1/ff4 x:0 0 0 2024-04-02T04:47:57.590 INFO:tasks.workunit.client.0.smithi070.stdout:0/692: creat d4/d5/d13/d2a/d8d/fd3 x:0 0 0 2024-04-02T04:47:57.591 INFO:tasks.workunit.client.0.smithi070.stdout:0/693: fdatasync d4/d3c/f6a 0 2024-04-02T04:47:57.593 INFO:tasks.workunit.client.0.smithi070.stdout:9/542: rename d4/d34/f4a to d4/d10/d12/d27/d68/fb2 0 2024-04-02T04:47:57.594 INFO:tasks.workunit.client.1.smithi179.stdout:0/735: truncate d1/f1e 1154164 0 2024-04-02T04:47:57.595 INFO:tasks.workunit.client.0.smithi070.stdout:3/648: dread d10/d32/d3f/f38 [0,4194304] 0 2024-04-02T04:47:57.598 INFO:tasks.workunit.client.0.smithi070.stdout:8/639: getdents d2/d8/d29/d22/dc8 0 2024-04-02T04:47:57.604 INFO:tasks.workunit.client.0.smithi070.stdout:0/694: write d4/d5/d13/d2a/d3a/d45/fc8 [1480980,23036] 0 2024-04-02T04:47:57.606 INFO:tasks.workunit.client.0.smithi070.stdout:9/543: creat d4/d10/d12/fb3 x:0 0 0 2024-04-02T04:47:57.610 INFO:tasks.workunit.client.0.smithi070.stdout:3/649: unlink d10/d66/f9d 0 2024-04-02T04:47:57.611 INFO:tasks.workunit.client.0.smithi070.stdout:3/650: readlink d10/l6b 0 2024-04-02T04:47:57.612 INFO:tasks.workunit.client.0.smithi070.stdout:8/640: mkdir d2/d3/d35/d3a/d4f/d59/dcc 0 2024-04-02T04:47:57.612 INFO:tasks.workunit.client.0.smithi070.stdout:9/544: dwrite d4/d10/d12/d27/f97 [0,4194304] 0 2024-04-02T04:47:57.613 INFO:tasks.workunit.client.0.smithi070.stdout:8/641: readlink d2/d3/l28 0 2024-04-02T04:47:57.614 INFO:tasks.workunit.client.0.smithi070.stdout:9/545: write d4/d10/f71 [182188,111931] 0 2024-04-02T04:47:57.619 INFO:tasks.workunit.client.0.smithi070.stdout:3/651: mkdir d10/d32/d4b/dc5/de0 0 2024-04-02T04:47:57.622 INFO:tasks.workunit.client.0.smithi070.stdout:8/642: rename d2/d8/d23/d24/d66/la7 to d2/d3/d52/d62/lcd 0 2024-04-02T04:47:57.626 INFO:tasks.workunit.client.0.smithi070.stdout:3/652: dread d10/d32/d4b/f7d [0,4194304] 0 2024-04-02T04:47:57.628 INFO:tasks.workunit.client.0.smithi070.stdout:8/643: dread d2/d3/d52/f55 [0,4194304] 0 2024-04-02T04:47:57.629 INFO:tasks.workunit.client.0.smithi070.stdout:9/546: getdents d4/d10/d11/d7b/d19/d2f/d39/d57 0 2024-04-02T04:47:57.631 INFO:tasks.workunit.client.0.smithi070.stdout:0/695: write d4/d5/d13/d23/f36 [4339811,73450] 0 2024-04-02T04:47:57.632 INFO:tasks.workunit.client.0.smithi070.stdout:3/653: dread d10/d32/d3f/f6d [0,4194304] 0 2024-04-02T04:47:57.633 INFO:tasks.workunit.client.0.smithi070.stdout:8/644: write d2/d8/d29/d22/d5a/d94/da2/fc3 [1043755,22246] 0 2024-04-02T04:47:57.634 INFO:tasks.workunit.client.0.smithi070.stdout:2/631: dwrite d2/d12/d3e/fac [0,4194304] 0 2024-04-02T04:47:57.636 INFO:tasks.workunit.client.0.smithi070.stdout:9/547: mknod d4/d10/d11/d7b/d19/d53/d9c/cb4 0 2024-04-02T04:47:57.636 INFO:tasks.workunit.client.0.smithi070.stdout:8/645: rmdir d2 39 2024-04-02T04:47:57.638 INFO:tasks.workunit.client.0.smithi070.stdout:3/654: mkdir d10/d32/d4b/de1 0 2024-04-02T04:47:57.638 INFO:tasks.workunit.client.0.smithi070.stdout:9/548: readlink d4/d10/d12/d55/la1 0 2024-04-02T04:47:57.640 INFO:tasks.workunit.client.0.smithi070.stdout:2/632: rename d2/d10/da4 to d2/d24/dc3/dd2 0 2024-04-02T04:47:57.641 INFO:tasks.workunit.client.0.smithi070.stdout:2/633: stat d2/d12/d59 0 2024-04-02T04:47:57.643 INFO:tasks.workunit.client.0.smithi070.stdout:2/634: chown d2/d10/d21/d71/f9f 497 1 2024-04-02T04:47:57.643 INFO:tasks.workunit.client.0.smithi070.stdout:9/549: dread d4/d10/d11/d7b/f2b [0,4194304] 0 2024-04-02T04:47:57.644 INFO:tasks.workunit.client.0.smithi070.stdout:3/655: mkdir d10/d17/db2/de2 0 2024-04-02T04:47:57.646 INFO:tasks.workunit.client.0.smithi070.stdout:0/696: getdents d4/d5/da 0 2024-04-02T04:47:57.648 INFO:tasks.workunit.client.0.smithi070.stdout:8/646: creat d2/d8/d29/d22/d5a/d9e/fce x:0 0 0 2024-04-02T04:47:57.649 INFO:tasks.workunit.client.0.smithi070.stdout:8/647: stat d2/d3/d52/faf 0 2024-04-02T04:47:57.649 INFO:tasks.workunit.client.0.smithi070.stdout:2/635: dwrite d2/f32 [0,4194304] 0 2024-04-02T04:47:57.651 INFO:tasks.workunit.client.0.smithi070.stdout:2/636: chown d2/d12/d59/f8e 112 1 2024-04-02T04:47:57.651 INFO:tasks.workunit.client.1.smithi179.stdout:1/704: rename d3/d5/de/l62 to d3/d5/d2f/db7/lf7 0 2024-04-02T04:47:57.652 INFO:tasks.workunit.client.0.smithi070.stdout:0/697: creat d4/d5/d13/d23/fd4 x:0 0 0 2024-04-02T04:47:57.652 INFO:tasks.workunit.client.1.smithi179.stdout:4/725: creat d1/d15/d7c/d98/ff5 x:0 0 0 2024-04-02T04:47:57.653 INFO:tasks.workunit.client.1.smithi179.stdout:9/678: write d0/d19/d1c/fc0 [179966,82229] 0 2024-04-02T04:47:57.657 INFO:tasks.workunit.client.1.smithi179.stdout:9/679: dread - d0/d19/d27/fa7 zero size 2024-04-02T04:47:57.658 INFO:tasks.workunit.client.0.smithi070.stdout:9/550: write d4/d10/d11/f42 [2217984,12277] 0 2024-04-02T04:47:57.660 INFO:tasks.workunit.client.1.smithi179.stdout:6/684: dwrite d3/d12/d23/d39/d4e/d6c/f50 [4194304,4194304] 0 2024-04-02T04:47:57.660 INFO:tasks.workunit.client.0.smithi070.stdout:9/551: dread - d4/d10/d11/d7b/d19/d82/f8a zero size 2024-04-02T04:47:57.661 INFO:tasks.workunit.client.1.smithi179.stdout:1/705: mkdir d3/d5/d2f/d65/df8 0 2024-04-02T04:47:57.665 INFO:tasks.workunit.client.1.smithi179.stdout:0/736: truncate d1/da/d52/f73 1498444 0 2024-04-02T04:47:57.665 INFO:tasks.workunit.client.1.smithi179.stdout:0/737: readlink d1/l2e 0 2024-04-02T04:47:57.666 INFO:tasks.workunit.client.1.smithi179.stdout:0/738: chown d1/da/d48/d65/d7d/f7a 13 1 2024-04-02T04:47:57.667 INFO:tasks.workunit.client.1.smithi179.stdout:4/726: unlink d1/d3/df/d10/d2f/d60/d71/f74 0 2024-04-02T04:47:57.668 INFO:tasks.workunit.client.0.smithi070.stdout:8/648: dwrite d2/d3/f49 [0,4194304] 0 2024-04-02T04:47:57.670 INFO:tasks.workunit.client.0.smithi070.stdout:9/552: unlink d4/d10/d11/d7b/c9b 0 2024-04-02T04:47:57.672 INFO:tasks.workunit.client.1.smithi179.stdout:1/706: unlink d3/d5/d2f/d4f/d42/d53/d77/cdf 0 2024-04-02T04:47:57.673 INFO:tasks.workunit.client.1.smithi179.stdout:1/707: chown d3/d5/d2f/d5a/dcc 24426206 1 2024-04-02T04:47:57.673 INFO:tasks.workunit.client.0.smithi070.stdout:3/656: creat d10/d17/db2/fe3 x:0 0 0 2024-04-02T04:47:57.673 INFO:tasks.workunit.client.1.smithi179.stdout:0/739: fsync d1/da/df/f12 0 2024-04-02T04:47:57.676 INFO:tasks.workunit.client.0.smithi070.stdout:2/637: getdents d2/d10/d21 0 2024-04-02T04:47:57.677 INFO:tasks.workunit.client.1.smithi179.stdout:7/711: getdents d1/dd/d23 0 2024-04-02T04:47:57.678 INFO:tasks.workunit.client.0.smithi070.stdout:9/553: unlink d4/d10/d11/d7b/d19/d53/f5a 0 2024-04-02T04:47:57.680 INFO:tasks.workunit.client.1.smithi179.stdout:6/685: mknod d3/d12/d23/d39/d4e/d66/ce9 0 2024-04-02T04:47:57.682 INFO:tasks.workunit.client.0.smithi070.stdout:2/638: readlink d2/l58 0 2024-04-02T04:47:57.684 INFO:tasks.workunit.client.0.smithi070.stdout:9/554: dwrite d4/fc [0,4194304] 0 2024-04-02T04:47:57.685 INFO:tasks.workunit.client.1.smithi179.stdout:0/740: truncate d1/da/f3c 3842806 0 2024-04-02T04:47:57.690 INFO:tasks.workunit.client.0.smithi070.stdout:8/649: rename d2/d3/d35/d3a/d4f/d59/l6f to d2/d8/d23/d24/lcf 0 2024-04-02T04:47:57.691 INFO:tasks.workunit.client.1.smithi179.stdout:6/686: rename d3/d2c/d85/c8d to d3/d12/d23/d26/d9b/cea 0 2024-04-02T04:47:57.691 INFO:tasks.workunit.client.0.smithi070.stdout:0/698: dwrite d4/d5/d13/d17/f62 [0,4194304] 0 2024-04-02T04:47:57.693 INFO:tasks.workunit.client.0.smithi070.stdout:2/639: unlink d2/d10/d5c/d7a/f92 0 2024-04-02T04:47:57.696 INFO:tasks.workunit.client.0.smithi070.stdout:9/555: truncate d4/d10/d12/d56/f5b 10480 0 2024-04-02T04:47:57.699 INFO:tasks.workunit.client.1.smithi179.stdout:4/727: write d1/d3/df/d10/f11 [1225682,23721] 0 2024-04-02T04:47:57.702 INFO:tasks.workunit.client.1.smithi179.stdout:9/680: truncate d0/d1/f36 2993021 0 2024-04-02T04:47:57.704 INFO:tasks.workunit.client.0.smithi070.stdout:3/657: write d10/f4f [3724996,34094] 0 2024-04-02T04:47:57.704 INFO:tasks.workunit.client.1.smithi179.stdout:0/741: truncate d1/da/d28/f41 2347719 0 2024-04-02T04:47:57.706 INFO:tasks.workunit.client.1.smithi179.stdout:4/728: mknod d1/d3/df/d10/d2f/d60/d71/dce/cf6 0 2024-04-02T04:47:57.709 INFO:tasks.workunit.client.1.smithi179.stdout:9/681: dwrite d0/d1/d11/d29/f50 [0,4194304] 0 2024-04-02T04:47:57.710 INFO:tasks.workunit.client.1.smithi179.stdout:0/742: rmdir d1/da/d28/d37 39 2024-04-02T04:47:57.712 INFO:tasks.workunit.client.1.smithi179.stdout:0/743: readlink d1/da/l19 0 2024-04-02T04:47:57.713 INFO:tasks.workunit.client.0.smithi070.stdout:3/658: dread d10/d66/fb9 [0,4194304] 0 2024-04-02T04:47:57.714 INFO:tasks.workunit.client.0.smithi070.stdout:3/659: chown d10/d32/d3f/f35 184 1 2024-04-02T04:47:57.716 INFO:tasks.workunit.client.1.smithi179.stdout:1/708: dwrite d3/d5/d2f/d4f/d42/d55/fd6 [0,4194304] 0 2024-04-02T04:47:57.717 INFO:tasks.workunit.client.0.smithi070.stdout:3/660: write d10/d17/fcc [496188,115667] 0 2024-04-02T04:47:57.720 INFO:tasks.workunit.client.1.smithi179.stdout:7/712: dwrite d1/d9/d27/d87/d8b/d75/db9/d8d/fc5 [0,4194304] 0 2024-04-02T04:47:57.720 INFO:tasks.workunit.client.1.smithi179.stdout:4/729: rename d1/d3/df/d10/d2f/d60/l77 to d1/d3/df/d10/lf7 0 2024-04-02T04:47:57.724 INFO:tasks.workunit.client.0.smithi070.stdout:2/640: creat d2/fd3 x:0 0 0 2024-04-02T04:47:57.726 INFO:tasks.workunit.client.0.smithi070.stdout:9/556: link d4/d10/d12/d27/f5c d4/d10/d11/da8/fb5 0 2024-04-02T04:47:57.728 INFO:tasks.workunit.client.0.smithi070.stdout:9/557: truncate d4/d10/d12/f67 680714 0 2024-04-02T04:47:57.729 INFO:tasks.workunit.client.0.smithi070.stdout:0/699: dread d4/d5/d13/d2a/f20 [0,4194304] 0 2024-04-02T04:47:57.730 INFO:tasks.workunit.client.0.smithi070.stdout:0/700: write d4/d5/d58/da1/fcd [338263,76366] 0 2024-04-02T04:47:57.731 INFO:tasks.workunit.client.1.smithi179.stdout:0/744: mkdir d1/da/d28/d81/d9d/dd8 0 2024-04-02T04:47:57.732 INFO:tasks.workunit.client.0.smithi070.stdout:1/608: sync 2024-04-02T04:47:57.732 INFO:tasks.workunit.client.0.smithi070.stdout:8/650: write d2/d3/d52/d62/f95 [1191053,8114] 0 2024-04-02T04:47:57.740 INFO:tasks.workunit.client.0.smithi070.stdout:0/701: dwrite d4/d5/d13/d2a/d3a/d80/f9f [0,4194304] 0 2024-04-02T04:47:57.741 INFO:tasks.workunit.client.1.smithi179.stdout:7/713: rmdir d1/dd/d76 39 2024-04-02T04:47:57.743 INFO:tasks.workunit.client.1.smithi179.stdout:4/730: dwrite d1/d93/fbd [0,4194304] 0 2024-04-02T04:47:57.747 INFO:tasks.workunit.client.0.smithi070.stdout:1/609: dread d0/d7/d28/f7a [0,4194304] 0 2024-04-02T04:47:57.753 INFO:tasks.workunit.client.0.smithi070.stdout:8/651: creat d2/d8/d29/d72/fd0 x:0 0 0 2024-04-02T04:47:57.755 INFO:tasks.workunit.client.0.smithi070.stdout:2/641: unlink d2/c64 0 2024-04-02T04:47:57.759 INFO:tasks.workunit.client.0.smithi070.stdout:0/702: mknod d4/d5/d13/d17/cd5 0 2024-04-02T04:47:57.760 INFO:tasks.workunit.client.0.smithi070.stdout:3/661: symlink d10/d17/d51/d73/d9e/le4 0 2024-04-02T04:47:57.761 INFO:tasks.workunit.client.1.smithi179.stdout:9/682: creat d0/d1/d10/d6d/fd0 x:0 0 0 2024-04-02T04:47:57.763 INFO:tasks.workunit.client.0.smithi070.stdout:1/610: chown d0/d7/l3b 236102681 1 2024-04-02T04:47:57.763 INFO:tasks.workunit.client.0.smithi070.stdout:3/662: write d10/d17/f5d [463580,33011] 0 2024-04-02T04:47:57.763 INFO:tasks.workunit.client.0.smithi070.stdout:2/642: dwrite d2/d10/d5c/d7a/fb7 [0,4194304] 0 2024-04-02T04:47:57.764 INFO:tasks.workunit.client.1.smithi179.stdout:9/683: write d0/d1/d10/d6d/d58/f96 [1087681,98929] 0 2024-04-02T04:47:57.764 INFO:tasks.workunit.client.1.smithi179.stdout:0/745: mknod d1/da/dc3/cd9 0 2024-04-02T04:47:57.766 INFO:tasks.workunit.client.0.smithi070.stdout:3/663: truncate d10/d32/d3f/f6d 1130910 0 2024-04-02T04:47:57.766 INFO:tasks.workunit.client.0.smithi070.stdout:8/652: mknod d2/d8/d23/d24/d5f/cd1 0 2024-04-02T04:47:57.768 INFO:tasks.workunit.client.1.smithi179.stdout:8/696: sync 2024-04-02T04:47:57.768 INFO:tasks.workunit.client.1.smithi179.stdout:5/662: sync 2024-04-02T04:47:57.770 INFO:tasks.workunit.client.1.smithi179.stdout:8/697: fsync d3/d6c/f89 0 2024-04-02T04:47:57.772 INFO:tasks.workunit.client.1.smithi179.stdout:9/684: dwrite d0/d19/d27/d7a/d87/daf/dc2/fc3 [0,4194304] 0 2024-04-02T04:47:57.775 INFO:tasks.workunit.client.1.smithi179.stdout:4/731: write d1/d3/df/f8f [210296,110292] 0 2024-04-02T04:47:57.776 INFO:tasks.workunit.client.0.smithi070.stdout:2/643: dread d2/d10/d53/f55 [0,4194304] 0 2024-04-02T04:47:57.777 INFO:tasks.workunit.client.1.smithi179.stdout:4/732: chown d1/d3/dc/d26/d8a/fbe 1 1 2024-04-02T04:47:57.777 INFO:tasks.workunit.client.0.smithi070.stdout:1/611: mkdir d0/d9/d1d/dc4 0 2024-04-02T04:47:57.781 INFO:tasks.workunit.client.0.smithi070.stdout:8/653: symlink d2/d8/d23/d24/d79/ld2 0 2024-04-02T04:47:57.782 INFO:tasks.workunit.client.1.smithi179.stdout:1/709: getdents d3/d5/d2f/d4f/d42/df2 0 2024-04-02T04:47:57.783 INFO:tasks.workunit.client.0.smithi070.stdout:8/654: fsync d2/d8/d23/d24/d5f/fb0 0 2024-04-02T04:47:57.783 INFO:tasks.workunit.client.0.smithi070.stdout:9/558: dwrite d4/d10/d12/d27/d83/f8e [0,4194304] 0 2024-04-02T04:47:57.785 INFO:tasks.workunit.client.1.smithi179.stdout:8/698: read d3/d1b/d2a/f8b [376349,58148] 0 2024-04-02T04:47:57.788 INFO:tasks.workunit.client.0.smithi070.stdout:2/644: rename d2/d10/d21/d2b to d2/d10/d21/d71/dd4 0 2024-04-02T04:47:57.789 INFO:tasks.workunit.client.1.smithi179.stdout:7/714: creat d1/d9/ffe x:0 0 0 2024-04-02T04:47:57.790 INFO:tasks.workunit.client.1.smithi179.stdout:7/715: chown d1/d9/d27/d87/d8b/d75/fcb 4 1 2024-04-02T04:47:57.793 INFO:tasks.workunit.client.0.smithi070.stdout:9/559: creat d4/d10/d11/d7b/d19/d2f/d39/fb6 x:0 0 0 2024-04-02T04:47:57.794 INFO:tasks.workunit.client.1.smithi179.stdout:1/710: symlink d3/d5/d2f/d4f/d6a/da0/lf9 0 2024-04-02T04:47:57.794 INFO:tasks.workunit.client.0.smithi070.stdout:0/703: write d4/d5/d13/d2a/d29/f59 [3171162,95159] 0 2024-04-02T04:47:57.795 INFO:tasks.workunit.client.1.smithi179.stdout:9/685: dread d0/d19/d1c/f6b [0,4194304] 0 2024-04-02T04:47:57.801 INFO:tasks.workunit.client.1.smithi179.stdout:0/746: creat d1/da/d48/d65/fda x:0 0 0 2024-04-02T04:47:57.802 INFO:tasks.workunit.client.1.smithi179.stdout:0/747: stat d1/f62 0 2024-04-02T04:47:57.803 INFO:tasks.workunit.client.1.smithi179.stdout:9/686: dwrite d0/d1/fc8 [0,4194304] 0 2024-04-02T04:47:57.803 INFO:tasks.workunit.client.1.smithi179.stdout:1/711: truncate d3/d5/d2f/d4f/d42/d55/fa1 228203 0 2024-04-02T04:47:57.805 INFO:tasks.workunit.client.1.smithi179.stdout:0/748: dread - d1/db/dad/fbd zero size 2024-04-02T04:47:57.805 INFO:tasks.workunit.client.0.smithi070.stdout:1/612: link d0/d9/d1d/d22/d56/d79/d89/c97 d0/d9/cc5 0 2024-04-02T04:47:57.807 INFO:tasks.workunit.client.0.smithi070.stdout:8/655: truncate d2/d3/f49 34827 0 2024-04-02T04:47:57.808 INFO:tasks.workunit.client.1.smithi179.stdout:5/663: dwrite d8/d1b/d23/f2f [0,4194304] 0 2024-04-02T04:47:57.809 INFO:tasks.workunit.client.0.smithi070.stdout:0/704: link d4/d5/d13/d23/c42 d4/d5/d58/cd6 0 2024-04-02T04:47:57.811 INFO:tasks.workunit.client.0.smithi070.stdout:1/613: rename d0/d54/c86 to d0/d7/d28/d5c/d6e/cc6 0 2024-04-02T04:47:57.813 INFO:tasks.workunit.client.1.smithi179.stdout:8/699: creat d3/d32/ff6 x:0 0 0 2024-04-02T04:47:57.814 INFO:tasks.workunit.client.1.smithi179.stdout:8/700: chown d3/d1b/d2a 19528345 1 2024-04-02T04:47:57.815 INFO:tasks.workunit.client.0.smithi070.stdout:3/664: dread d10/d32/f52 [0,4194304] 0 2024-04-02T04:47:57.815 INFO:tasks.workunit.client.0.smithi070.stdout:0/705: dread d4/d5/d13/d2a/d29/f5d [0,4194304] 0 2024-04-02T04:47:57.815 INFO:tasks.workunit.client.1.smithi179.stdout:7/716: dwrite d1/d9/d55/f61 [0,4194304] 0 2024-04-02T04:47:57.815 INFO:tasks.workunit.client.1.smithi179.stdout:1/712: dwrite d3/d5/d2f/d4f/f37 [0,4194304] 0 2024-04-02T04:47:57.816 INFO:tasks.workunit.client.1.smithi179.stdout:7/717: chown d1/d9/d55/dbf/d9f/da8 9 1 2024-04-02T04:47:57.816 INFO:tasks.workunit.client.0.smithi070.stdout:3/665: dread - d10/d17/d51/d73/d9e/dd5/fd8 zero size 2024-04-02T04:47:57.817 INFO:tasks.workunit.client.1.smithi179.stdout:1/713: write d3/d5/d2f/d4f/d42/d53/d72/d86/f90 [4077600,52881] 0 2024-04-02T04:47:57.817 INFO:tasks.workunit.client.0.smithi070.stdout:3/666: readlink d10/d32/d4b/l50 0 2024-04-02T04:47:57.818 INFO:tasks.workunit.client.0.smithi070.stdout:8/656: mknod d2/d3/d35/d3a/d4f/cd3 0 2024-04-02T04:47:57.820 INFO:tasks.workunit.client.1.smithi179.stdout:9/687: rmdir d0/d1/d11/d29/d86 39 2024-04-02T04:47:57.820 INFO:tasks.workunit.client.1.smithi179.stdout:8/701: read d3/d31/f7f [2030386,101640] 0 2024-04-02T04:47:57.828 INFO:tasks.workunit.client.1.smithi179.stdout:8/702: dwrite d3/f76 [4194304,4194304] 0 2024-04-02T04:47:57.828 INFO:tasks.workunit.client.0.smithi070.stdout:2/645: symlink d2/d10/d21/d3d/d42/ld5 0 2024-04-02T04:47:57.828 INFO:tasks.workunit.client.0.smithi070.stdout:1/614: symlink d0/d9/d1d/d22/d2c/d4f/d77/lc7 0 2024-04-02T04:47:57.829 INFO:tasks.workunit.client.1.smithi179.stdout:5/664: symlink d8/d1b/d23/d32/d4e/ld2 0 2024-04-02T04:47:57.832 INFO:tasks.workunit.client.0.smithi070.stdout:0/706: creat d4/d5/d13/d2a/d3a/fd7 x:0 0 0 2024-04-02T04:47:57.833 INFO:tasks.workunit.client.1.smithi179.stdout:7/718: creat d1/d9/d35/d6b/fff x:0 0 0 2024-04-02T04:47:57.833 INFO:tasks.workunit.client.0.smithi070.stdout:3/667: readlink d10/d17/l2d 0 2024-04-02T04:47:57.834 INFO:tasks.workunit.client.0.smithi070.stdout:3/668: readlink d10/d32/l90 0 2024-04-02T04:47:57.835 INFO:tasks.workunit.client.1.smithi179.stdout:1/714: creat d3/d5/d2f/d4f/dab/ffa x:0 0 0 2024-04-02T04:47:57.836 INFO:tasks.workunit.client.1.smithi179.stdout:7/719: truncate d1/dd/d23/fc2 906854 0 2024-04-02T04:47:57.837 INFO:tasks.workunit.client.1.smithi179.stdout:9/688: creat d0/d19/d1c/fd1 x:0 0 0 2024-04-02T04:47:57.838 INFO:tasks.workunit.client.1.smithi179.stdout:0/749: symlink d1/da/df/d56/ldb 0 2024-04-02T04:47:57.838 INFO:tasks.workunit.client.1.smithi179.stdout:7/720: fsync d1/d9/d27/d87/d8b/d75/db9/d8d/fc5 0 2024-04-02T04:47:57.839 INFO:tasks.workunit.client.1.smithi179.stdout:8/703: symlink d3/d31/dc3/lf7 0 2024-04-02T04:47:57.840 INFO:tasks.workunit.client.1.smithi179.stdout:1/715: symlink d3/d5/d2f/db7/lfb 0 2024-04-02T04:47:57.841 INFO:tasks.workunit.client.1.smithi179.stdout:1/716: stat d3/d5/d2f/d4f/d42/d53/c6f 0 2024-04-02T04:47:57.843 INFO:tasks.workunit.client.1.smithi179.stdout:9/689: rmdir d0/d19/d27/d7a/d84 39 2024-04-02T04:47:57.847 INFO:tasks.workunit.client.1.smithi179.stdout:7/721: mkdir d1/d9/d55/dbf/d9f/da8/d100 0 2024-04-02T04:47:57.848 INFO:tasks.workunit.client.0.smithi070.stdout:1/615: dread d0/d9/d1d/f42 [0,4194304] 0 2024-04-02T04:47:57.848 INFO:tasks.workunit.client.1.smithi179.stdout:7/722: chown d1/d9/ca 48379 1 2024-04-02T04:47:57.849 INFO:tasks.workunit.client.1.smithi179.stdout:8/704: mkdir d3/d32/d21/d22/d6b/db1/df8 0 2024-04-02T04:47:57.851 INFO:tasks.workunit.client.1.smithi179.stdout:3/708: sync 2024-04-02T04:47:57.851 INFO:tasks.workunit.client.1.smithi179.stdout:2/569: sync 2024-04-02T04:47:57.852 INFO:tasks.workunit.client.1.smithi179.stdout:2/570: truncate dc/f9f 113643 0 2024-04-02T04:47:57.852 INFO:tasks.workunit.client.1.smithi179.stdout:0/750: symlink d1/da/ldc 0 2024-04-02T04:47:57.855 INFO:tasks.workunit.client.1.smithi179.stdout:1/717: mkdir d3/d5/d2f/d65/df8/dfc 0 2024-04-02T04:47:57.856 INFO:tasks.workunit.client.1.smithi179.stdout:2/571: dread dc/d29/d4a/f3f [0,4194304] 0 2024-04-02T04:47:57.857 INFO:tasks.workunit.client.1.smithi179.stdout:2/572: write dc/d29/d31/d57/d97/fb6 [21973,124321] 0 2024-04-02T04:47:57.860 INFO:tasks.workunit.client.1.smithi179.stdout:0/751: mkdir d1/da/d28/d34/ddd 0 2024-04-02T04:47:57.862 INFO:tasks.workunit.client.1.smithi179.stdout:8/705: mkdir d3/d32/d21/d49/df0/df9 0 2024-04-02T04:47:57.863 INFO:tasks.workunit.client.0.smithi070.stdout:2/646: write d2/d12/d59/f5e [4780708,67645] 0 2024-04-02T04:47:57.865 INFO:tasks.workunit.client.1.smithi179.stdout:9/690: link d0/d1/d11/l54 d0/d1/d10/ld2 0 2024-04-02T04:47:57.866 INFO:tasks.workunit.client.1.smithi179.stdout:1/718: symlink d3/d5/d2f/d4f/d6a/da0/lfd 0 2024-04-02T04:47:57.867 INFO:tasks.workunit.client.0.smithi070.stdout:3/669: dread - d10/d17/db2/dbd/fdd zero size 2024-04-02T04:47:57.868 INFO:tasks.workunit.client.1.smithi179.stdout:3/709: mkdir d8/d26/d52/d7e/ddf 0 2024-04-02T04:47:57.868 INFO:tasks.workunit.client.1.smithi179.stdout:3/710: stat d8/d6d/d94/caa 0 2024-04-02T04:47:57.870 INFO:tasks.workunit.client.0.smithi070.stdout:1/616: symlink d0/d9/lc8 0 2024-04-02T04:47:57.873 INFO:tasks.workunit.client.1.smithi179.stdout:0/752: rmdir d1/da/df 39 2024-04-02T04:47:57.874 INFO:tasks.workunit.client.1.smithi179.stdout:0/753: dread - d1/da/d48/d7b/fb6 zero size 2024-04-02T04:47:57.874 INFO:tasks.workunit.client.1.smithi179.stdout:3/711: dwrite d8/dd3/dbb/fc6 [0,4194304] 0 2024-04-02T04:47:57.876 INFO:tasks.workunit.client.1.smithi179.stdout:3/712: write d8/d26/d83/d89/d9c/fc2 [928122,89271] 0 2024-04-02T04:47:57.877 INFO:tasks.workunit.client.1.smithi179.stdout:9/691: mknod d0/d1/d10/d6d/d58/cd3 0 2024-04-02T04:47:57.878 INFO:tasks.workunit.client.1.smithi179.stdout:3/713: stat d8/d26/d83/d89/d9c/lb5 0 2024-04-02T04:47:57.884 INFO:tasks.workunit.client.1.smithi179.stdout:7/723: dwrite d1/dd/d23/f24 [4194304,4194304] 0 2024-04-02T04:47:57.885 INFO:tasks.workunit.client.1.smithi179.stdout:8/706: symlink d3/d32/d21/d51/lfa 0 2024-04-02T04:47:57.885 INFO:tasks.workunit.client.1.smithi179.stdout:7/724: chown d1/d9/dd9 323762261 1 2024-04-02T04:47:57.888 INFO:tasks.workunit.client.0.smithi070.stdout:3/670: creat d10/d32/d8a/fe5 x:0 0 0 2024-04-02T04:47:57.890 INFO:tasks.workunit.client.0.smithi070.stdout:8/657: truncate d2/d8/d23/d24/d66/f6a 3381083 0 2024-04-02T04:47:57.892 INFO:tasks.workunit.client.1.smithi179.stdout:9/692: mkdir d0/d19/d27/d7a/d7e/dd4 0 2024-04-02T04:47:57.896 INFO:tasks.workunit.client.0.smithi070.stdout:8/658: rename d2/d8/d23/d24/l45 to d2/d8/d29/db/ld4 0 2024-04-02T04:47:57.900 INFO:tasks.workunit.client.1.smithi179.stdout:2/573: dwrite dc/d29/d31/d57/f6b [0,4194304] 0 2024-04-02T04:47:57.900 INFO:tasks.workunit.client.1.smithi179.stdout:9/693: dwrite d0/d1/d11/dab/fbb [0,4194304] 0 2024-04-02T04:47:57.902 INFO:tasks.workunit.client.0.smithi070.stdout:0/707: truncate d4/d5/d13/d17/f62 2073918 0 2024-04-02T04:47:57.904 INFO:tasks.workunit.client.0.smithi070.stdout:2/647: dwrite d2/d12/fa9 [0,4194304] 0 2024-04-02T04:47:57.904 INFO:tasks.workunit.client.1.smithi179.stdout:1/719: write d3/f1f [647987,83427] 0 2024-04-02T04:47:57.906 INFO:tasks.workunit.client.0.smithi070.stdout:0/708: read f0 [1013761,15950] 0 2024-04-02T04:47:57.913 INFO:tasks.workunit.client.1.smithi179.stdout:0/754: mknod d1/da/d28/d6d/cde 0 2024-04-02T04:47:57.913 INFO:tasks.workunit.client.1.smithi179.stdout:8/707: dwrite d3/d32/d21/d22/d6b/d91/f44 [0,4194304] 0 2024-04-02T04:47:57.916 INFO:tasks.workunit.client.0.smithi070.stdout:1/617: dwrite d0/d9/d1d/d22/d56/fbf [0,4194304] 0 2024-04-02T04:47:57.916 INFO:tasks.workunit.client.1.smithi179.stdout:0/755: dwrite d1/da/d52/fd4 [0,4194304] 0 2024-04-02T04:47:57.923 INFO:tasks.workunit.client.0.smithi070.stdout:3/671: write d10/d17/f2a [4373202,38573] 0 2024-04-02T04:47:57.929 INFO:tasks.workunit.client.0.smithi070.stdout:2/648: dread d2/d10/d21/f89 [0,4194304] 0 2024-04-02T04:47:57.931 INFO:tasks.workunit.client.1.smithi179.stdout:2/574: dread dc/d29/d4a/f24 [0,4194304] 0 2024-04-02T04:47:57.931 INFO:tasks.workunit.client.1.smithi179.stdout:1/720: mkdir d3/d5/d2f/d4f/d42/d53/dfe 0 2024-04-02T04:47:57.933 INFO:tasks.workunit.client.0.smithi070.stdout:8/659: dwrite d2/d8/d23/d24/d66/f70 [0,4194304] 0 2024-04-02T04:47:57.936 INFO:tasks.workunit.client.0.smithi070.stdout:1/618: mknod d0/d9/d1d/d22/d2c/cc9 0 2024-04-02T04:47:57.937 INFO:tasks.workunit.client.1.smithi179.stdout:9/694: read d0/d1/d11/d29/d40/f49 [3554740,102539] 0 2024-04-02T04:47:57.941 INFO:tasks.workunit.client.1.smithi179.stdout:3/714: rename d8/c21 to d8/ce0 0 2024-04-02T04:47:57.942 INFO:tasks.workunit.client.0.smithi070.stdout:1/619: mknod d0/d54/d64/cca 0 2024-04-02T04:47:57.943 INFO:tasks.workunit.client.0.smithi070.stdout:0/709: link d4/d5/d13/d2a/d29/cc4 d4/d5/d58/d60/d83/da5/cd8 0 2024-04-02T04:47:57.944 INFO:tasks.workunit.client.0.smithi070.stdout:2/649: creat d2/fd6 x:0 0 0 2024-04-02T04:47:57.945 INFO:tasks.workunit.client.0.smithi070.stdout:2/650: chown d2/d10/d5c/d7a/db0/fbf 0 1 2024-04-02T04:47:57.946 INFO:tasks.workunit.client.0.smithi070.stdout:1/620: mknod d0/d7/d28/d5c/d6e/ccb 0 2024-04-02T04:47:57.948 INFO:tasks.workunit.client.0.smithi070.stdout:0/710: creat d4/d5/d58/d60/d83/fd9 x:0 0 0 2024-04-02T04:47:57.949 INFO:tasks.workunit.client.1.smithi179.stdout:8/708: link d3/d32/d21/ce1 d3/d32/d21/d51/d70/cfb 0 2024-04-02T04:47:57.951 INFO:tasks.workunit.client.0.smithi070.stdout:0/711: mknod d4/d5/d13/d17/d98/cda 0 2024-04-02T04:47:57.952 INFO:tasks.workunit.client.0.smithi070.stdout:0/712: chown d4/d5/d13/d43/f78 65 1 2024-04-02T04:47:57.952 INFO:tasks.workunit.client.1.smithi179.stdout:3/715: mkdir d8/d26/d52/d7e/de1 0 2024-04-02T04:47:57.952 INFO:tasks.workunit.client.0.smithi070.stdout:0/713: dread - d4/d5/d58/d60/fcb zero size 2024-04-02T04:47:57.953 INFO:tasks.workunit.client.1.smithi179.stdout:8/709: dread - d3/d1b/d2a/d83/fec zero size 2024-04-02T04:47:57.954 INFO:tasks.workunit.client.1.smithi179.stdout:8/710: dread - d3/d32/d21/d50/d56/d94/d84/fa9 zero size 2024-04-02T04:47:57.954 INFO:tasks.workunit.client.1.smithi179.stdout:3/716: creat d8/d26/d83/d89/d9c/dcc/fe2 x:0 0 0 2024-04-02T04:47:57.955 INFO:tasks.workunit.client.1.smithi179.stdout:3/717: chown d8/dd3/dbb/fbe 53495432 1 2024-04-02T04:47:57.955 INFO:tasks.workunit.client.1.smithi179.stdout:3/718: readlink d8/d57/ld4 0 2024-04-02T04:47:57.956 INFO:tasks.workunit.client.1.smithi179.stdout:8/711: truncate d3/d32/d21/d50/d56/fcd 1988536 0 2024-04-02T04:47:57.958 INFO:tasks.workunit.client.1.smithi179.stdout:8/712: write d3/d32/d21/d49/fda [443316,73316] 0 2024-04-02T04:47:57.958 INFO:tasks.workunit.client.0.smithi070.stdout:0/714: dwrite d4/d5/d13/d2a/d3a/d45/fc8 [0,4194304] 0 2024-04-02T04:47:57.958 INFO:tasks.workunit.client.1.smithi179.stdout:3/719: rename d8/d57/fa8 to d8/d57/d63/fe3 0 2024-04-02T04:47:57.961 INFO:tasks.workunit.client.1.smithi179.stdout:8/713: dread d3/d32/d21/d50/d56/d94/d84/f8e [0,4194304] 0 2024-04-02T04:47:57.965 INFO:tasks.workunit.client.0.smithi070.stdout:0/715: dwrite d4/d5/d13/d23/fd4 [0,4194304] 0 2024-04-02T04:47:57.966 INFO:tasks.workunit.client.1.smithi179.stdout:6/687: sync 2024-04-02T04:47:57.966 INFO:tasks.workunit.client.1.smithi179.stdout:4/733: sync 2024-04-02T04:47:57.968 INFO:tasks.workunit.client.1.smithi179.stdout:4/734: chown d1/d3/df/d10/fa1 128865148 1 2024-04-02T04:47:57.971 INFO:tasks.workunit.client.1.smithi179.stdout:6/688: creat d3/d12/d6a/de8/feb x:0 0 0 2024-04-02T04:47:57.973 INFO:tasks.workunit.client.1.smithi179.stdout:6/689: chown d3/d12/d23/d39/d80/f8f 4213 1 2024-04-02T04:47:57.978 INFO:tasks.workunit.client.0.smithi070.stdout:1/621: dread d0/d54/f63 [0,4194304] 0 2024-04-02T04:47:57.979 INFO:tasks.workunit.client.0.smithi070.stdout:1/622: readlink d0/d7/d1a/l8b 0 2024-04-02T04:47:57.979 INFO:tasks.workunit.client.1.smithi179.stdout:4/735: dwrite d1/d3/dc/d26/d8a/f82 [4194304,4194304] 0 2024-04-02T04:47:57.980 INFO:tasks.workunit.client.1.smithi179.stdout:1/721: write d3/d5/d2f/d4f/d42/d53/d72/d86/fa6 [1223062,48129] 0 2024-04-02T04:47:57.981 INFO:tasks.workunit.client.0.smithi070.stdout:1/623: read d0/d9/d1d/d22/f2d [4158099,125187] 0 2024-04-02T04:47:57.982 INFO:tasks.workunit.client.1.smithi179.stdout:6/690: mkdir d3/d12/d23/d39/d80/d93/ddc/dec 0 2024-04-02T04:47:57.982 INFO:tasks.workunit.client.1.smithi179.stdout:1/722: stat d3/d5/d2f/d36/f67 0 2024-04-02T04:47:57.982 INFO:tasks.workunit.client.1.smithi179.stdout:0/756: truncate d1/da/d48/d65/d7d/fd3 426207 0 2024-04-02T04:47:57.984 INFO:tasks.workunit.client.1.smithi179.stdout:4/736: mkdir d1/d3/df/d10/d2f/d60/da6/df8 0 2024-04-02T04:47:57.985 INFO:tasks.workunit.client.0.smithi070.stdout:1/624: mkdir d0/db4/dcc 0 2024-04-02T04:47:57.987 INFO:tasks.workunit.client.1.smithi179.stdout:6/691: mknod d3/d2d/da5/ced 0 2024-04-02T04:47:57.989 INFO:tasks.workunit.client.1.smithi179.stdout:9/695: dwrite d0/d1/f4 [4194304,4194304] 0 2024-04-02T04:47:57.990 INFO:tasks.workunit.client.1.smithi179.stdout:1/723: fsync d3/d5/d2f/d4f/d35/fbf 0 2024-04-02T04:47:57.991 INFO:tasks.workunit.client.1.smithi179.stdout:0/757: mknod d1/da/d48/d65/d7d/cdf 0 2024-04-02T04:47:57.991 INFO:tasks.workunit.client.1.smithi179.stdout:0/758: stat d1/db/dad/fbd 0 2024-04-02T04:47:57.994 INFO:tasks.workunit.client.1.smithi179.stdout:2/575: truncate dc/d29/d4a/d1e/f7b 3626195 0 2024-04-02T04:47:57.995 INFO:tasks.workunit.client.0.smithi070.stdout:2/651: dwrite d2/d10/d21/f89 [0,4194304] 0 2024-04-02T04:47:57.995 INFO:tasks.workunit.client.0.smithi070.stdout:1/625: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/lcd 0 2024-04-02T04:47:57.996 INFO:tasks.workunit.client.1.smithi179.stdout:2/576: write dc/d29/f34 [3967095,78900] 0 2024-04-02T04:47:58.001 INFO:tasks.workunit.client.1.smithi179.stdout:9/696: readlink d0/d1/d11/l54 0 2024-04-02T04:47:58.003 INFO:tasks.workunit.client.1.smithi179.stdout:3/720: truncate d8/dd3/dbb/fc6 3957475 0 2024-04-02T04:47:58.003 INFO:tasks.workunit.client.1.smithi179.stdout:9/697: dread - d0/d19/d27/d7a/d87/daf/fb4 zero size 2024-04-02T04:47:58.003 INFO:tasks.workunit.client.0.smithi070.stdout:2/652: mkdir d2/d10/d19/dd7 0 2024-04-02T04:47:58.005 INFO:tasks.workunit.client.0.smithi070.stdout:2/653: read - d2/d10/d21/d3d/d42/fad zero size 2024-04-02T04:47:58.009 INFO:tasks.workunit.client.0.smithi070.stdout:2/654: rename d2/d24/dc3 to d2/d10/d5c/d7a/db0/dd8 0 2024-04-02T04:47:58.011 INFO:tasks.workunit.client.1.smithi179.stdout:1/724: mkdir d3/d5/dff 0 2024-04-02T04:47:58.012 INFO:tasks.workunit.client.1.smithi179.stdout:9/698: mknod d0/d1/d11/dab/cd5 0 2024-04-02T04:47:58.013 INFO:tasks.workunit.client.0.smithi070.stdout:2/655: mknod d2/d10/d5c/d7a/db0/cd9 0 2024-04-02T04:47:58.015 INFO:tasks.workunit.client.1.smithi179.stdout:3/721: symlink d8/d26/d83/d89/d9c/le4 0 2024-04-02T04:47:58.017 INFO:tasks.workunit.client.0.smithi070.stdout:2/656: truncate d2/d10/d21/d71/dd4/fc9 82328 0 2024-04-02T04:47:58.018 INFO:tasks.workunit.client.1.smithi179.stdout:1/725: symlink d3/d5/d2f/db7/l100 0 2024-04-02T04:47:58.021 INFO:tasks.workunit.client.1.smithi179.stdout:9/699: chown d0/d19/d27/dad/fae 6169769 1 2024-04-02T04:47:58.021 INFO:tasks.workunit.client.1.smithi179.stdout:1/726: dread - d3/d5/d2f/d4f/d42/d53/d72/fbc zero size 2024-04-02T04:47:58.023 INFO:tasks.workunit.client.1.smithi179.stdout:0/759: dread d1/da/d48/d5c/fb9 [0,4194304] 0 2024-04-02T04:47:58.024 INFO:tasks.workunit.client.1.smithi179.stdout:3/722: dwrite d8/d26/d52/d7e/db8/fde [0,4194304] 0 2024-04-02T04:47:58.026 INFO:tasks.workunit.client.0.smithi070.stdout:1/626: dread d0/d54/f58 [0,4194304] 0 2024-04-02T04:47:58.027 INFO:tasks.workunit.client.1.smithi179.stdout:6/692: getdents d3/d12/d23 0 2024-04-02T04:47:58.027 INFO:tasks.workunit.client.0.smithi070.stdout:1/627: write d0/d7/f6c [2303242,18572] 0 2024-04-02T04:47:58.028 INFO:tasks.workunit.client.1.smithi179.stdout:3/723: readlink d8/dd3/l93 0 2024-04-02T04:47:58.034 INFO:tasks.workunit.client.1.smithi179.stdout:1/727: dwrite d3/d5/de/f16 [0,4194304] 0 2024-04-02T04:47:58.034 INFO:tasks.workunit.client.1.smithi179.stdout:9/700: unlink d0/d19/d27/d7a/d87/daf/dc2/fc3 0 2024-04-02T04:47:58.035 INFO:tasks.workunit.client.0.smithi070.stdout:1/628: mknod d0/d9/d1d/d22/d56/cce 0 2024-04-02T04:47:58.036 INFO:tasks.workunit.client.1.smithi179.stdout:9/701: chown d0/d19/d27/d48/f8c 351861 1 2024-04-02T04:47:58.037 INFO:tasks.workunit.client.1.smithi179.stdout:3/724: creat d8/d57/d63/dc3/fe5 x:0 0 0 2024-04-02T04:47:58.045 INFO:tasks.workunit.client.1.smithi179.stdout:4/737: truncate d1/d3/dc/dc1/ff1 2358882 0 2024-04-02T04:47:58.046 INFO:tasks.workunit.client.1.smithi179.stdout:1/728: chown d3/d5/de/f57 85155278 1 2024-04-02T04:47:58.046 INFO:tasks.workunit.client.1.smithi179.stdout:3/725: unlink d8/d26/f4e 0 2024-04-02T04:47:58.047 INFO:tasks.workunit.client.0.smithi070.stdout:2/657: rmdir d2 39 2024-04-02T04:47:58.050 INFO:tasks.workunit.client.1.smithi179.stdout:2/577: truncate dc/d29/d4a/f72 705845 0 2024-04-02T04:47:58.054 INFO:tasks.workunit.client.0.smithi070.stdout:2/658: getdents d2/d10/d5c/d7a/db0/dc7 0 2024-04-02T04:47:58.058 INFO:tasks.workunit.client.1.smithi179.stdout:0/760: dread d1/da/d67/fa8 [0,4194304] 0 2024-04-02T04:47:58.063 INFO:tasks.workunit.client.1.smithi179.stdout:6/693: write d3/d12/f21 [525072,89857] 0 2024-04-02T04:47:58.064 INFO:tasks.workunit.client.1.smithi179.stdout:5/665: sync 2024-04-02T04:47:58.064 INFO:tasks.workunit.client.1.smithi179.stdout:7/725: sync 2024-04-02T04:47:58.064 INFO:tasks.workunit.client.1.smithi179.stdout:1/729: dread d3/d5/d2f/fb0 [0,4194304] 0 2024-04-02T04:47:58.064 INFO:tasks.workunit.client.1.smithi179.stdout:6/694: fsync d3/d12/d23/d2f/f56 0 2024-04-02T04:47:58.065 INFO:tasks.workunit.client.1.smithi179.stdout:7/726: chown d1/d9/d27/d87/d8b/lf7 0 1 2024-04-02T04:47:58.069 INFO:tasks.workunit.client.1.smithi179.stdout:6/695: dwrite d3/d12/fc4 [0,4194304] 0 2024-04-02T04:47:58.084 INFO:tasks.workunit.client.1.smithi179.stdout:8/714: sync 2024-04-02T04:47:58.084 INFO:tasks.workunit.client.0.smithi070.stdout:5/781: sync 2024-04-02T04:47:58.084 INFO:tasks.workunit.client.0.smithi070.stdout:4/598: sync 2024-04-02T04:47:58.084 INFO:tasks.workunit.client.0.smithi070.stdout:6/505: sync 2024-04-02T04:47:58.084 INFO:tasks.workunit.client.0.smithi070.stdout:7/621: sync 2024-04-02T04:47:58.086 INFO:tasks.workunit.client.1.smithi179.stdout:9/702: dread d0/d19/d1c/fc0 [0,4194304] 0 2024-04-02T04:47:58.086 INFO:tasks.workunit.client.0.smithi070.stdout:1/629: write d0/d9/d1d/f5e [1260935,93334] 0 2024-04-02T04:47:58.087 INFO:tasks.workunit.client.1.smithi179.stdout:4/738: write d1/d15/d7c/fc2 [65778,16983] 0 2024-04-02T04:47:58.088 INFO:tasks.workunit.client.0.smithi070.stdout:2/659: link d2/d10/d21/d71/f76 d2/d10/d5c/d7a/fda 0 2024-04-02T04:47:58.088 INFO:tasks.workunit.client.1.smithi179.stdout:9/703: chown d0/d1/d10/d6d/d58/cd3 453723404 1 2024-04-02T04:47:58.089 INFO:tasks.workunit.client.0.smithi070.stdout:1/630: write d0/d4c/da2/fb5 [399136,124257] 0 2024-04-02T04:47:58.090 INFO:tasks.workunit.client.0.smithi070.stdout:6/506: stat d0/d3/d19/l2b 0 2024-04-02T04:47:58.178 INFO:tasks.workunit.client.0.smithi070.stdout:5/782: symlink d5/d9/d23/d8b/de9/lfa 0 2024-04-02T04:47:58.179 INFO:tasks.workunit.client.0.smithi070.stdout:5/783: truncate d5/d9/fb2 1210736 0 2024-04-02T04:47:58.180 INFO:tasks.workunit.client.0.smithi070.stdout:2/660: mknod d2/d10/d21/d71/dd4/cdb 0 2024-04-02T04:47:58.182 INFO:tasks.workunit.client.0.smithi070.stdout:4/599: symlink d1/d9c/d65/lc7 0 2024-04-02T04:47:58.185 INFO:tasks.workunit.client.1.smithi179.stdout:0/761: write d1/da/df/f20 [3966859,99967] 0 2024-04-02T04:47:58.185 INFO:tasks.workunit.client.0.smithi070.stdout:7/622: truncate d7/dd/df/d7a/d6c/f74 3127621 0 2024-04-02T04:47:58.189 INFO:tasks.workunit.client.1.smithi179.stdout:5/666: creat d8/fd3 x:0 0 0 2024-04-02T04:47:58.192 INFO:tasks.workunit.client.1.smithi179.stdout:7/727: mknod d1/d9/d55/dbf/c101 0 2024-04-02T04:47:58.193 INFO:tasks.workunit.client.1.smithi179.stdout:7/728: write d1/d9/d27/d87/d8b/f6f [473320,18036] 0 2024-04-02T04:47:58.193 INFO:tasks.workunit.client.0.smithi070.stdout:5/784: dread d5/d9/d23/d30/f79 [0,4194304] 0 2024-04-02T04:47:58.196 INFO:tasks.workunit.client.0.smithi070.stdout:7/623: chown d7/dd/caf 8226 1 2024-04-02T04:47:58.197 INFO:tasks.workunit.client.1.smithi179.stdout:6/696: rename d3/d12/d23/d26/d57/ce3 to d3/d42/dbc/cee 0 2024-04-02T04:47:58.199 INFO:tasks.workunit.client.0.smithi070.stdout:4/600: dwrite d1/f86 [0,4194304] 0 2024-04-02T04:47:58.200 INFO:tasks.workunit.client.0.smithi070.stdout:5/785: dwrite d5/d9/f5e [0,4194304] 0 2024-04-02T04:47:58.201 INFO:tasks.workunit.client.0.smithi070.stdout:1/631: link d0/d7/d28/d83/fbd d0/d54/fcf 0 2024-04-02T04:47:58.203 INFO:tasks.workunit.client.1.smithi179.stdout:8/715: dwrite d3/d32/d21/d50/f5f [0,4194304] 0 2024-04-02T04:47:58.207 INFO:tasks.workunit.client.1.smithi179.stdout:3/726: symlink d8/d26/dcd/le6 0 2024-04-02T04:47:58.210 INFO:tasks.workunit.client.1.smithi179.stdout:4/739: symlink d1/d3/df/d10/d2f/d60/d71/lf9 0 2024-04-02T04:47:58.211 INFO:tasks.workunit.client.1.smithi179.stdout:9/704: creat d0/d1/d11/d77/fd6 x:0 0 0 2024-04-02T04:47:58.212 INFO:tasks.workunit.client.1.smithi179.stdout:2/578: mkdir dc/d29/d4a/d1e/dbe 0 2024-04-02T04:47:58.213 INFO:tasks.workunit.client.1.smithi179.stdout:2/579: readlink dc/d29/l42 0 2024-04-02T04:47:58.213 INFO:tasks.workunit.client.1.smithi179.stdout:2/580: dread - dc/d29/d31/d83/fab zero size 2024-04-02T04:47:58.214 INFO:tasks.workunit.client.1.smithi179.stdout:2/581: fdatasync dc/d29/d31/d57/f6b 0 2024-04-02T04:47:58.218 INFO:tasks.workunit.client.0.smithi070.stdout:6/507: rename d0/d55/d5c/f82 to d0/d55/d8b/f9e 0 2024-04-02T04:47:58.220 INFO:tasks.workunit.client.0.smithi070.stdout:4/601: fsync d1/d3/f4c 0 2024-04-02T04:47:58.223 INFO:tasks.workunit.client.1.smithi179.stdout:7/729: chown d1/d9/d27/d87/fe0 108 1 2024-04-02T04:47:58.223 INFO:tasks.workunit.client.0.smithi070.stdout:1/632: creat d0/d4c/da2/fd0 x:0 0 0 2024-04-02T04:47:58.226 INFO:tasks.workunit.client.1.smithi179.stdout:5/667: rename d8/d1b/d23/d36/d53/d89 to d8/d1b/d23/d32/dbe/daf/dd4 0 2024-04-02T04:47:58.229 INFO:tasks.workunit.client.0.smithi070.stdout:1/633: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:58.230 INFO:tasks.workunit.client.0.smithi070.stdout:6/508: fdatasync d0/d3/d19/d31/f74 0 2024-04-02T04:47:58.234 INFO:tasks.workunit.client.1.smithi179.stdout:3/727: mkdir d8/d26/dcd/de7 0 2024-04-02T04:47:58.235 INFO:tasks.workunit.client.0.smithi070.stdout:4/602: fdatasync d1/d9c/d65/d7a/f9f 0 2024-04-02T04:47:58.240 INFO:tasks.workunit.client.0.smithi070.stdout:2/661: truncate d2/f9c 857960 0 2024-04-02T04:47:58.242 INFO:tasks.workunit.client.1.smithi179.stdout:2/582: mkdir dc/d29/d31/d57/d97/dbf 0 2024-04-02T04:47:58.247 INFO:tasks.workunit.client.0.smithi070.stdout:7/624: write d7/dd/df/f32 [192180,55784] 0 2024-04-02T04:47:58.248 INFO:tasks.workunit.client.0.smithi070.stdout:4/603: mkdir d1/d3/d85/d8e/dc8 0 2024-04-02T04:47:58.250 INFO:tasks.workunit.client.1.smithi179.stdout:7/730: creat d1/d9/d27/d87/d8b/d75/db9/dcf/f102 x:0 0 0 2024-04-02T04:47:58.250 INFO:tasks.workunit.client.0.smithi070.stdout:5/786: rmdir d5/d9/d23/df7 0 2024-04-02T04:47:58.253 INFO:tasks.workunit.client.0.smithi070.stdout:2/662: mkdir d2/d10/d21/d3d/ddc 0 2024-04-02T04:47:58.255 INFO:tasks.workunit.client.1.smithi179.stdout:6/697: rename d3/d12/d23/d26/faf to d3/d2d/d43/fef 0 2024-04-02T04:47:58.256 INFO:tasks.workunit.client.0.smithi070.stdout:1/634: mknod d0/d9/d1d/d22/d56/cd1 0 2024-04-02T04:47:58.260 INFO:tasks.workunit.client.0.smithi070.stdout:2/663: dwrite d2/d24/fb9 [0,4194304] 0 2024-04-02T04:47:58.261 INFO:tasks.workunit.client.0.smithi070.stdout:7/625: symlink d7/dd/df/d34/d68/lc8 0 2024-04-02T04:47:58.268 INFO:tasks.workunit.client.1.smithi179.stdout:4/740: dwrite d1/fc3 [0,4194304] 0 2024-04-02T04:47:58.270 INFO:tasks.workunit.client.0.smithi070.stdout:6/509: dwrite d0/f26 [0,4194304] 0 2024-04-02T04:47:58.271 INFO:tasks.workunit.client.0.smithi070.stdout:1/635: dwrite d0/d9/d1d/f5e [0,4194304] 0 2024-04-02T04:47:58.278 INFO:tasks.workunit.client.0.smithi070.stdout:7/626: dwrite d7/dd/f8d [0,4194304] 0 2024-04-02T04:47:58.279 INFO:tasks.workunit.client.0.smithi070.stdout:1/636: dread d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/fb2 [0,4194304] 0 2024-04-02T04:47:58.279 INFO:tasks.workunit.client.0.smithi070.stdout:2/664: creat d2/d10/d21/d71/fdd x:0 0 0 2024-04-02T04:47:58.279 INFO:tasks.workunit.client.1.smithi179.stdout:1/730: getdents d3/d5 0 2024-04-02T04:47:58.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:58 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:58.284 INFO:tasks.workunit.client.1.smithi179.stdout:6/698: fsync d3/d12/d23/d2f/f3a 0 2024-04-02T04:47:58.285 INFO:tasks.workunit.client.0.smithi070.stdout:5/787: unlink d5/d9/d23/dc3/le2 0 2024-04-02T04:47:58.288 INFO:tasks.workunit.client.1.smithi179.stdout:5/668: symlink d8/d1b/d23/d32/ld5 0 2024-04-02T04:47:58.288 INFO:tasks.workunit.client.1.smithi179.stdout:0/762: dwrite d1/f30 [0,4194304] 0 2024-04-02T04:47:58.293 INFO:tasks.workunit.client.1.smithi179.stdout:3/728: fdatasync d8/fa1 0 2024-04-02T04:47:58.293 INFO:tasks.workunit.client.1.smithi179.stdout:9/705: write d0/d19/d1c/f47 [785108,76775] 0 2024-04-02T04:47:58.293 INFO:tasks.workunit.client.0.smithi070.stdout:1/637: unlink d0/d4c/f51 0 2024-04-02T04:47:58.295 INFO:tasks.workunit.client.0.smithi070.stdout:1/638: write d0/d4c/da2/fd0 [485161,128785] 0 2024-04-02T04:47:58.296 INFO:tasks.workunit.client.1.smithi179.stdout:3/729: write d8/d26/d83/d89/d9c/fc2 [522499,92131] 0 2024-04-02T04:47:58.298 INFO:tasks.workunit.client.1.smithi179.stdout:3/730: chown d8/d26/d83/d89/d9c/ca4 83178669 1 2024-04-02T04:47:58.298 INFO:tasks.workunit.client.1.smithi179.stdout:8/716: dwrite d3/d1b/fa2 [0,4194304] 0 2024-04-02T04:47:58.299 INFO:tasks.workunit.client.0.smithi070.stdout:5/788: mknod d5/d9/d23/d8b/de9/cfb 0 2024-04-02T04:47:58.301 INFO:tasks.workunit.client.1.smithi179.stdout:6/699: readlink d3/l6 0 2024-04-02T04:47:58.302 INFO:tasks.workunit.client.1.smithi179.stdout:8/717: chown d3/d31/f7f 830779 1 2024-04-02T04:47:58.302 INFO:tasks.workunit.client.1.smithi179.stdout:6/700: chown d3/d12/dde 278 1 2024-04-02T04:47:58.303 INFO:tasks.workunit.client.0.smithi070.stdout:7/627: dwrite d7/dd/df/d7a/f64 [4194304,4194304] 0 2024-04-02T04:47:58.303 INFO:tasks.workunit.client.0.smithi070.stdout:6/510: dread d0/d3/d19/f3f [0,4194304] 0 2024-04-02T04:47:58.304 INFO:tasks.workunit.client.1.smithi179.stdout:8/718: chown d3/d32/d21/d51/f55 50366 1 2024-04-02T04:47:58.309 INFO:tasks.workunit.client.1.smithi179.stdout:9/706: creat d0/d19/d1c/d45/fd7 x:0 0 0 2024-04-02T04:47:58.311 INFO:tasks.workunit.client.0.smithi070.stdout:5/789: dwrite d5/d17/d2b/d7b/ff9 [0,4194304] 0 2024-04-02T04:47:58.312 INFO:tasks.workunit.client.0.smithi070.stdout:4/604: write d1/d9c/d65/f6a [2988394,85774] 0 2024-04-02T04:47:58.313 INFO:tasks.workunit.client.0.smithi070.stdout:5/790: fsync d5/d9/d10/ff8 0 2024-04-02T04:47:58.318 INFO:tasks.workunit.client.1.smithi179.stdout:2/583: rename dc/d29/d31/d83/dae/db0/dbc/c58 to dc/d29/d4a/cc0 0 2024-04-02T04:47:58.319 INFO:tasks.workunit.client.0.smithi070.stdout:6/511: dwrite d0/d3/d19/d31/f4e [4194304,4194304] 0 2024-04-02T04:47:58.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:58 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:58.322 INFO:tasks.workunit.client.0.smithi070.stdout:6/512: write d0/d3/d19/d31/d46/d6b/f70 [2706328,6520] 0 2024-04-02T04:47:58.325 INFO:tasks.workunit.client.0.smithi070.stdout:4/605: stat d1/c9a 0 2024-04-02T04:47:58.326 INFO:tasks.workunit.client.0.smithi070.stdout:2/665: rename d2/d10/d5c/d7a/db0/dd8/ld1 to d2/d10/lde 0 2024-04-02T04:47:58.326 INFO:tasks.workunit.client.0.smithi070.stdout:2/666: dread - d2/d10/d5c/d7a/db0/dd8/dd2/fcf zero size 2024-04-02T04:47:58.327 INFO:tasks.workunit.client.1.smithi179.stdout:8/719: creat d3/d32/d21/d45/ffc x:0 0 0 2024-04-02T04:47:58.330 INFO:tasks.workunit.client.1.smithi179.stdout:8/720: chown d3/d32/d21/f66 43098569 1 2024-04-02T04:47:58.330 INFO:tasks.workunit.client.0.smithi070.stdout:4/606: fdatasync d1/d3/d9/f22 0 2024-04-02T04:47:58.331 INFO:tasks.workunit.client.1.smithi179.stdout:2/584: mknod dc/d29/d31/d57/d97/cc1 0 2024-04-02T04:47:58.331 INFO:tasks.workunit.client.0.smithi070.stdout:4/607: chown d1/d3/d78/dad/dbd 5 1 2024-04-02T04:47:58.332 INFO:tasks.workunit.client.0.smithi070.stdout:5/791: dread d5/d17/d27/d2e/f91 [0,4194304] 0 2024-04-02T04:47:58.332 INFO:tasks.workunit.client.1.smithi179.stdout:3/731: fdatasync d8/d57/f5a 0 2024-04-02T04:47:58.333 INFO:tasks.workunit.client.1.smithi179.stdout:8/721: write d3/d1b/d2a/d41/ff4 [364289,54069] 0 2024-04-02T04:47:58.334 INFO:tasks.workunit.client.1.smithi179.stdout:1/731: dread d3/d5/f21 [0,4194304] 0 2024-04-02T04:47:58.334 INFO:tasks.workunit.client.0.smithi070.stdout:9/560: sync 2024-04-02T04:47:58.334 INFO:tasks.workunit.client.0.smithi070.stdout:2/667: creat d2/d10/d5c/d7a/db0/fdf x:0 0 0 2024-04-02T04:47:58.335 INFO:tasks.workunit.client.1.smithi179.stdout:3/732: write d8/d26/d83/d89/fab [2911824,116702] 0 2024-04-02T04:47:58.335 INFO:tasks.workunit.client.1.smithi179.stdout:8/722: fsync d3/d32/d21/d22/d6b/db1/fbe 0 2024-04-02T04:47:58.336 INFO:tasks.workunit.client.1.smithi179.stdout:6/701: symlink d3/d12/d23/d39/lf0 0 2024-04-02T04:47:58.336 INFO:tasks.workunit.client.1.smithi179.stdout:8/723: chown d3/d32/d21/l3f 1698 1 2024-04-02T04:47:58.340 INFO:tasks.workunit.client.0.smithi070.stdout:6/513: fdatasync d0/d55/d8b/d94/f53 0 2024-04-02T04:47:58.340 INFO:tasks.workunit.client.1.smithi179.stdout:9/707: dread d0/d1/d10/d6d/f20 [0,4194304] 0 2024-04-02T04:47:58.341 INFO:tasks.workunit.client.0.smithi070.stdout:4/608: dwrite d1/d9c/d65/d7a/f7d [0,4194304] 0 2024-04-02T04:47:58.346 INFO:tasks.workunit.client.0.smithi070.stdout:6/514: dread d0/d3/d19/f3f [0,4194304] 0 2024-04-02T04:47:58.346 INFO:tasks.workunit.client.0.smithi070.stdout:6/515: write d0/d3/d19/d31/d46/f9b [811247,2547] 0 2024-04-02T04:47:58.346 INFO:tasks.workunit.client.0.smithi070.stdout:5/792: unlink d5/d9/d10/d51/f59 0 2024-04-02T04:47:58.346 INFO:tasks.workunit.client.0.smithi070.stdout:6/516: chown d0/d3/d19/d31/d61/d7b 0 1 2024-04-02T04:47:58.346 INFO:tasks.workunit.client.1.smithi179.stdout:1/732: read - d3/d5/d2f/d4f/d42/fae zero size 2024-04-02T04:47:58.346 INFO:tasks.workunit.client.1.smithi179.stdout:0/763: dwrite d1/db/f7c [0,4194304] 0 2024-04-02T04:47:58.347 INFO:tasks.workunit.client.0.smithi070.stdout:5/793: chown d5/d9/d10/db9/dba/de1 11934 1 2024-04-02T04:47:58.348 INFO:tasks.workunit.client.0.smithi070.stdout:5/794: stat d5/d57/l9b 0 2024-04-02T04:47:58.348 INFO:tasks.workunit.client.0.smithi070.stdout:5/795: readlink d5/led 0 2024-04-02T04:47:58.349 INFO:tasks.workunit.client.1.smithi179.stdout:4/741: truncate d1/d3/df/f48 4181184 0 2024-04-02T04:47:58.349 INFO:tasks.workunit.client.1.smithi179.stdout:1/733: write d3/d5/de/f1a [1760107,98780] 0 2024-04-02T04:47:58.350 INFO:tasks.workunit.client.0.smithi070.stdout:5/796: dread - d5/d17/d50/fd7 zero size 2024-04-02T04:47:58.352 INFO:tasks.workunit.client.0.smithi070.stdout:1/639: dwrite d0/f1c [4194304,4194304] 0 2024-04-02T04:47:58.353 INFO:tasks.workunit.client.1.smithi179.stdout:1/734: chown d3/d5/de/d33/c74 578751 1 2024-04-02T04:47:58.360 INFO:tasks.workunit.client.1.smithi179.stdout:8/724: creat d3/d32/d30/ffd x:0 0 0 2024-04-02T04:47:58.360 INFO:tasks.workunit.client.1.smithi179.stdout:3/733: dread d8/d57/fb3 [0,4194304] 0 2024-04-02T04:47:58.361 INFO:tasks.workunit.client.0.smithi070.stdout:4/609: rename d1/f28 to d1/d3/d78/dad/dbd/fc9 0 2024-04-02T04:47:58.364 INFO:tasks.workunit.client.1.smithi179.stdout:5/669: rename d8/d1b/d23/d32/l94 to d8/d1b/d23/d42/ld6 0 2024-04-02T04:47:58.364 INFO:tasks.workunit.client.0.smithi070.stdout:7/628: dwrite d7/d9/f1b [0,4194304] 0 2024-04-02T04:47:58.364 INFO:tasks.workunit.client.0.smithi070.stdout:4/610: write d1/d3/d9/f64 [3747436,109056] 0 2024-04-02T04:47:58.366 INFO:tasks.workunit.client.0.smithi070.stdout:7/629: chown d7/dd/df/d7a/d6c/d77/l2b 11084662 1 2024-04-02T04:47:58.368 INFO:tasks.workunit.client.0.smithi070.stdout:2/668: symlink d2/d10/d21/le0 0 2024-04-02T04:47:58.369 INFO:tasks.workunit.client.0.smithi070.stdout:3/672: sync 2024-04-02T04:47:58.369 INFO:tasks.workunit.client.0.smithi070.stdout:0/716: sync 2024-04-02T04:47:58.369 INFO:tasks.workunit.client.0.smithi070.stdout:8/660: sync 2024-04-02T04:47:58.370 INFO:tasks.workunit.client.0.smithi070.stdout:5/797: readlink d5/l21 0 2024-04-02T04:47:58.371 INFO:tasks.workunit.client.0.smithi070.stdout:3/673: truncate d10/d17/db2/fe3 396809 0 2024-04-02T04:47:58.372 INFO:tasks.workunit.client.0.smithi070.stdout:1/640: readlink d0/d54/lc0 0 2024-04-02T04:47:58.372 INFO:tasks.workunit.client.0.smithi070.stdout:8/661: write d2/d8/d29/d22/d5a/d94/fc0 [495811,80040] 0 2024-04-02T04:47:58.373 INFO:tasks.workunit.client.1.smithi179.stdout:0/764: dread d1/da/d48/d65/d7d/d80/fa5 [0,4194304] 0 2024-04-02T04:47:58.373 INFO:tasks.workunit.client.1.smithi179.stdout:3/734: mkdir d8/dd3/d9a/de8 0 2024-04-02T04:47:58.376 INFO:tasks.workunit.client.0.smithi070.stdout:7/630: mknod d7/dd/df/d7a/d6c/cc9 0 2024-04-02T04:47:58.376 INFO:tasks.workunit.client.1.smithi179.stdout:7/731: sync 2024-04-02T04:47:58.376 INFO:tasks.workunit.client.1.smithi179.stdout:2/585: rename dc/d29/d4a/f24 to dc/d29/d31/d57/d97/fc2 0 2024-04-02T04:47:58.377 INFO:tasks.workunit.client.1.smithi179.stdout:5/670: mkdir d8/d1b/d23/d32/dbe/daf/dd7 0 2024-04-02T04:47:58.380 INFO:tasks.workunit.client.1.smithi179.stdout:0/765: dread d1/da/d28/d81/fae [0,4194304] 0 2024-04-02T04:47:58.380 INFO:tasks.workunit.client.0.smithi070.stdout:9/561: write d4/d10/d11/d7b/d19/d2f/d39/f44 [5018258,118221] 0 2024-04-02T04:47:58.381 INFO:tasks.workunit.client.1.smithi179.stdout:5/671: write d8/d1b/d23/d36/d53/d62/f75 [322187,20550] 0 2024-04-02T04:47:58.383 INFO:tasks.workunit.client.1.smithi179.stdout:8/725: mkdir d3/d32/d21/d22/d6b/d91/da8/dfe 0 2024-04-02T04:47:58.383 INFO:tasks.workunit.client.1.smithi179.stdout:0/766: chown d1/da/d28/d5e/f98 1 1 2024-04-02T04:47:58.385 INFO:tasks.workunit.client.0.smithi070.stdout:8/662: creat d2/d3/d35/fd5 x:0 0 0 2024-04-02T04:47:58.385 INFO:tasks.workunit.client.1.smithi179.stdout:6/702: write d3/d2d/f65 [8001651,61042] 0 2024-04-02T04:47:58.386 INFO:tasks.workunit.client.1.smithi179.stdout:7/732: dwrite d1/dd/d23/f2f [0,4194304] 0 2024-04-02T04:47:58.389 INFO:tasks.workunit.client.1.smithi179.stdout:6/703: fdatasync d3/d12/d23/d39/d4e/fbf 0 2024-04-02T04:47:58.390 INFO:tasks.workunit.client.0.smithi070.stdout:6/517: write d0/d55/d8b/d94/f65 [1400171,9356] 0 2024-04-02T04:47:58.395 INFO:tasks.workunit.client.0.smithi070.stdout:9/562: write d4/d10/d12/d27/d60/d9e/f77 [5032557,39526] 0 2024-04-02T04:47:58.399 INFO:tasks.workunit.client.1.smithi179.stdout:3/735: symlink d8/d26/dcd/de7/le9 0 2024-04-02T04:47:58.400 INFO:tasks.workunit.client.1.smithi179.stdout:9/708: truncate d0/d19/d1c/d64/fb7 285691 0 2024-04-02T04:47:58.400 INFO:tasks.workunit.client.1.smithi179.stdout:3/736: fsync d8/d26/d36/fa2 0 2024-04-02T04:47:58.401 INFO:tasks.workunit.client.0.smithi070.stdout:8/663: mknod d2/d8/d29/d72/cd6 0 2024-04-02T04:47:58.401 INFO:tasks.workunit.client.1.smithi179.stdout:3/737: write d8/d57/d63/dc3/fc8 [986310,14986] 0 2024-04-02T04:47:58.402 INFO:tasks.workunit.client.0.smithi070.stdout:8/664: write d2/d3/d52/d62/d7c/fa4 [2244276,103751] 0 2024-04-02T04:47:58.403 INFO:tasks.workunit.client.0.smithi070.stdout:0/717: write d4/d5/d13/d2a/d3a/f8e [54363,31714] 0 2024-04-02T04:47:58.403 INFO:tasks.workunit.client.1.smithi179.stdout:0/767: creat d1/da/d28/d34/d39/d77/fe0 x:0 0 0 2024-04-02T04:47:58.404 INFO:tasks.workunit.client.0.smithi070.stdout:0/718: chown d4/d5/d13/d17/ca7 2946860 1 2024-04-02T04:47:58.406 INFO:tasks.workunit.client.1.smithi179.stdout:5/672: dread d8/d1b/d23/d32/d6e/fc1 [0,4194304] 0 2024-04-02T04:47:58.406 INFO:tasks.workunit.client.1.smithi179.stdout:4/742: rename d1/d3/df/d10/d2f/d7b/l9e to d1/d3/df/d10/d2f/d60/d71/lfa 0 2024-04-02T04:47:58.407 INFO:tasks.workunit.client.0.smithi070.stdout:1/641: dread d0/d7/f20 [0,4194304] 0 2024-04-02T04:47:58.408 INFO:tasks.workunit.client.0.smithi070.stdout:6/518: fdatasync d0/d20/f27 0 2024-04-02T04:47:58.409 INFO:tasks.workunit.client.0.smithi070.stdout:2/669: link d2/d10/d53/f55 d2/d10/d5c/d7a/fe1 0 2024-04-02T04:47:58.410 INFO:tasks.workunit.client.0.smithi070.stdout:4/611: truncate d1/d3/d9/d5e/f87 2131979 0 2024-04-02T04:47:58.412 INFO:tasks.workunit.client.1.smithi179.stdout:6/704: mkdir d3/d12/d44/df1 0 2024-04-02T04:47:58.413 INFO:tasks.workunit.client.1.smithi179.stdout:6/705: truncate d3/d12/d23/d39/d4e/d66/fd2 1166491 0 2024-04-02T04:47:58.416 INFO:tasks.workunit.client.0.smithi070.stdout:3/674: link d10/d17/d51/c87 d10/d66/ce6 0 2024-04-02T04:47:58.419 INFO:tasks.workunit.client.0.smithi070.stdout:9/563: dread d4/d10/d11/d7b/f35 [0,4194304] 0 2024-04-02T04:47:58.424 INFO:tasks.workunit.client.0.smithi070.stdout:1/642: creat d0/d54/fd2 x:0 0 0 2024-04-02T04:47:58.426 INFO:tasks.workunit.client.1.smithi179.stdout:3/738: rmdir d8/d7a 39 2024-04-02T04:47:58.426 INFO:tasks.workunit.client.1.smithi179.stdout:3/739: fdatasync d8/d26/d83/fd7 0 2024-04-02T04:47:58.431 INFO:tasks.workunit.client.0.smithi070.stdout:1/643: dwrite d0/d9/d1d/d22/d2c/fb3 [0,4194304] 0 2024-04-02T04:47:58.432 INFO:tasks.workunit.client.0.smithi070.stdout:2/670: rename d2/d10/c45 to d2/d10/d5c/d7a/db0/dd8/dd2/ce2 0 2024-04-02T04:47:58.434 INFO:tasks.workunit.client.0.smithi070.stdout:4/612: readlink d1/d3/d9/l5a 0 2024-04-02T04:47:58.437 INFO:tasks.workunit.client.1.smithi179.stdout:4/743: symlink d1/d3/dc/d26/d8a/lfb 0 2024-04-02T04:47:58.438 INFO:tasks.workunit.client.1.smithi179.stdout:4/744: write d1/d3/fc5 [179126,58536] 0 2024-04-02T04:47:58.438 INFO:tasks.workunit.client.0.smithi070.stdout:8/665: creat d2/d8/d23/d24/d5f/dc2/fd7 x:0 0 0 2024-04-02T04:47:58.439 INFO:tasks.workunit.client.1.smithi179.stdout:4/745: dread - d1/d3/df/d10/d2f/d60/d71/ff2 zero size 2024-04-02T04:47:58.439 INFO:tasks.workunit.client.1.smithi179.stdout:7/733: symlink d1/d9/d55/dbf/d9f/l103 0 2024-04-02T04:47:58.441 INFO:tasks.workunit.client.0.smithi070.stdout:0/719: dwrite d4/d5/d13/f19 [0,4194304] 0 2024-04-02T04:47:58.441 INFO:tasks.workunit.client.1.smithi179.stdout:7/734: dread - d1/dd/d16/d4d/fe4 zero size 2024-04-02T04:47:58.442 INFO:tasks.workunit.client.1.smithi179.stdout:6/706: mkdir d3/df2 0 2024-04-02T04:47:58.443 INFO:tasks.workunit.client.1.smithi179.stdout:6/707: dread - d3/d12/d23/d26/d57/fc3 zero size 2024-04-02T04:47:58.444 INFO:tasks.workunit.client.0.smithi070.stdout:9/564: mkdir d4/d10/d11/d7b/d19/d82/db7 0 2024-04-02T04:47:58.444 INFO:tasks.workunit.client.1.smithi179.stdout:2/586: link cb dc/d29/d31/d83/dae/db0/dbc/cc3 0 2024-04-02T04:47:58.446 INFO:tasks.workunit.client.1.smithi179.stdout:2/587: chown dc/d29/d4a/fad 3 1 2024-04-02T04:47:58.446 INFO:tasks.workunit.client.1.smithi179.stdout:4/746: dwrite d1/d3/df/d10/d3a/faf [0,4194304] 0 2024-04-02T04:47:58.448 INFO:tasks.workunit.client.1.smithi179.stdout:3/740: creat d8/d6d/d94/fea x:0 0 0 2024-04-02T04:47:58.449 INFO:tasks.workunit.client.0.smithi070.stdout:1/644: fsync d0/d9/d1d/f16 0 2024-04-02T04:47:58.450 INFO:tasks.workunit.client.1.smithi179.stdout:0/768: fsync d1/f14 0 2024-04-02T04:47:58.452 INFO:tasks.workunit.client.0.smithi070.stdout:6/519: dwrite d0/d3/d19/f47 [8388608,4194304] 0 2024-04-02T04:47:58.455 INFO:tasks.workunit.client.0.smithi070.stdout:9/565: dwrite d4/d13/d89/faf [0,4194304] 0 2024-04-02T04:47:58.455 INFO:tasks.workunit.client.1.smithi179.stdout:1/735: rename d3/c4 to d3/d5/dff/c101 0 2024-04-02T04:47:58.457 INFO:tasks.workunit.client.1.smithi179.stdout:1/736: truncate d3/d5/d2f/d4f/d42/d55/fb8 4879807 0 2024-04-02T04:47:58.458 INFO:tasks.workunit.client.1.smithi179.stdout:1/737: fdatasync d3/d5/f6d 0 2024-04-02T04:47:58.458 INFO:tasks.workunit.client.0.smithi070.stdout:9/566: read - d4/d10/d12/fb3 zero size 2024-04-02T04:47:58.458 INFO:tasks.workunit.client.1.smithi179.stdout:1/738: readlink d3/d5/d2f/d65/l8e 0 2024-04-02T04:47:58.459 INFO:tasks.workunit.client.0.smithi070.stdout:3/675: rename d10/d17/d51/d73/d9e/le4 to d10/d32/d4b/dc5/de0/le7 0 2024-04-02T04:47:58.462 INFO:tasks.workunit.client.0.smithi070.stdout:1/645: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:47:58.463 INFO:tasks.workunit.client.1.smithi179.stdout:5/673: rmdir d8/d1b/d23/db1 39 2024-04-02T04:47:58.468 INFO:tasks.workunit.client.0.smithi070.stdout:0/720: mkdir d4/d5/d13/d2a/d3a/d80/da8/ddb 0 2024-04-02T04:47:58.469 INFO:tasks.workunit.client.0.smithi070.stdout:0/721: chown d4/d5/d13/d2a/d3a/c9e 9559647 1 2024-04-02T04:47:58.475 INFO:tasks.workunit.client.0.smithi070.stdout:2/671: write d2/d12/f38 [2779507,63162] 0 2024-04-02T04:47:58.478 INFO:tasks.workunit.client.0.smithi070.stdout:6/520: creat d0/d43/f9f x:0 0 0 2024-04-02T04:47:58.482 INFO:tasks.workunit.client.1.smithi179.stdout:8/726: rename d3/d32/d21/d49/f52 to d3/d32/d21/d45/d5a/fff 0 2024-04-02T04:47:58.483 INFO:tasks.workunit.client.0.smithi070.stdout:1/646: fsync d0/d54/fcf 0 2024-04-02T04:47:58.485 INFO:tasks.workunit.client.1.smithi179.stdout:0/769: dread d1/da/f2d [0,4194304] 0 2024-04-02T04:47:58.485 INFO:tasks.workunit.client.1.smithi179.stdout:5/674: creat d8/d7e/fd8 x:0 0 0 2024-04-02T04:47:58.486 INFO:tasks.workunit.client.1.smithi179.stdout:7/735: fdatasync d1/dd/d23/dab/fba 0 2024-04-02T04:47:58.488 INFO:tasks.workunit.client.1.smithi179.stdout:9/709: getdents d0/d1/d10/d6d/d58 0 2024-04-02T04:47:58.490 INFO:tasks.workunit.client.1.smithi179.stdout:9/710: write d0/d1/d10/d6d/d58/f75 [186559,75213] 0 2024-04-02T04:47:58.492 INFO:tasks.workunit.client.0.smithi070.stdout:9/567: rename d4/d10/d12/d27/d83/f8f to d4/d10/d11/d7b/d19/d2f/fb8 0 2024-04-02T04:47:58.494 INFO:tasks.workunit.client.0.smithi070.stdout:0/722: chown d4/d5/d21/d6b/f7b 12001002 1 2024-04-02T04:47:58.497 INFO:tasks.workunit.client.0.smithi070.stdout:2/672: link d2/d10/d5c/d7a/fab d2/d10/d5c/d7a/db0/fe3 0 2024-04-02T04:47:58.500 INFO:tasks.workunit.client.1.smithi179.stdout:4/747: write d1/d3/dc/d26/d8a/f55 [3241895,91785] 0 2024-04-02T04:47:58.500 INFO:tasks.workunit.client.0.smithi070.stdout:3/676: dread d10/d17/db2/dbd/fce [0,4194304] 0 2024-04-02T04:47:58.501 INFO:tasks.workunit.client.1.smithi179.stdout:4/748: write d1/d3/dc/d27/d62/fe2 [899686,96136] 0 2024-04-02T04:47:58.502 INFO:tasks.workunit.client.0.smithi070.stdout:8/666: truncate d2/d3/f9 1726978 0 2024-04-02T04:47:58.503 INFO:tasks.workunit.client.0.smithi070.stdout:8/667: readlink d2/d3/l28 0 2024-04-02T04:47:58.503 INFO:tasks.workunit.client.1.smithi179.stdout:5/675: mkdir d8/d1b/d23/d36/d53/dd9 0 2024-04-02T04:47:58.504 INFO:tasks.workunit.client.1.smithi179.stdout:5/676: dread - d8/d1b/d23/d32/d4b/fb0 zero size 2024-04-02T04:47:58.507 INFO:tasks.workunit.client.0.smithi070.stdout:9/568: mkdir d4/d34/db9 0 2024-04-02T04:47:58.508 INFO:tasks.workunit.client.0.smithi070.stdout:9/569: write d4/d10/d11/f4e [7393294,54907] 0 2024-04-02T04:47:58.509 INFO:tasks.workunit.client.0.smithi070.stdout:0/723: fsync d4/d5/d13/d17/d72/f90 0 2024-04-02T04:47:58.510 INFO:tasks.workunit.client.1.smithi179.stdout:6/708: write d3/d12/d23/d2f/f4a [635223,51810] 0 2024-04-02T04:47:58.512 INFO:tasks.workunit.client.0.smithi070.stdout:2/673: mkdir d2/d10/d5c/d7a/db0/de4 0 2024-04-02T04:47:58.515 INFO:tasks.workunit.client.1.smithi179.stdout:1/739: truncate d3/d5/d2f/d4f/d42/d55/fd6 2979102 0 2024-04-02T04:47:58.516 INFO:tasks.workunit.client.1.smithi179.stdout:8/727: creat d3/d32/d21/d22/d6b/d91/da8/dfe/f100 x:0 0 0 2024-04-02T04:47:58.516 INFO:tasks.workunit.client.0.smithi070.stdout:6/521: truncate d0/d3/f13 7412541 0 2024-04-02T04:47:58.518 INFO:tasks.workunit.client.1.smithi179.stdout:5/677: dread d8/d1b/d23/d32/f7a [0,4194304] 0 2024-04-02T04:47:58.521 INFO:tasks.workunit.client.0.smithi070.stdout:1/647: dwrite d0/d9/d1d/d22/d2c/d4f/f92 [0,4194304] 0 2024-04-02T04:47:58.524 INFO:tasks.workunit.client.0.smithi070.stdout:0/724: fdatasync d4/d5/d13/d43/f53 0 2024-04-02T04:47:58.525 INFO:tasks.workunit.client.1.smithi179.stdout:7/736: dread d1/dd/d76/f86 [0,4194304] 0 2024-04-02T04:47:58.526 INFO:tasks.workunit.client.0.smithi070.stdout:0/725: write d4/d5/d13/d2a/d8d/fb2 [2961748,85858] 0 2024-04-02T04:47:58.526 INFO:tasks.workunit.client.0.smithi070.stdout:2/674: creat d2/d12/d59/fe5 x:0 0 0 2024-04-02T04:47:58.527 INFO:tasks.workunit.client.1.smithi179.stdout:7/737: readlink d1/d9/d27/d87/d8b/d75/db9/d8d/la1 0 2024-04-02T04:47:58.529 INFO:tasks.workunit.client.0.smithi070.stdout:0/726: read d4/d5/da/d2d/f66 [2194778,62414] 0 2024-04-02T04:47:58.530 INFO:tasks.workunit.client.1.smithi179.stdout:2/588: rename dc/d29/d31/d66/fa2 to dc/fc4 0 2024-04-02T04:47:58.532 INFO:tasks.workunit.client.0.smithi070.stdout:3/677: mknod d10/d17/d51/d73/ce8 0 2024-04-02T04:47:58.534 INFO:tasks.workunit.client.0.smithi070.stdout:3/678: write d10/d17/db2/dbd/fdd [999679,130546] 0 2024-04-02T04:47:58.535 INFO:tasks.workunit.client.1.smithi179.stdout:1/740: read d3/d5/d2f/d4f/d42/d55/fa1 [49246,53829] 0 2024-04-02T04:47:58.536 INFO:tasks.workunit.client.0.smithi070.stdout:8/668: symlink d2/d8/d29/d22/dc8/ld8 0 2024-04-02T04:47:58.538 INFO:tasks.workunit.client.1.smithi179.stdout:8/728: rmdir d3/d32/d21/d22 39 2024-04-02T04:47:58.538 INFO:tasks.workunit.client.1.smithi179.stdout:2/589: dwrite dc/d29/d4a/f67 [0,4194304] 0 2024-04-02T04:47:58.538 INFO:tasks.workunit.client.0.smithi070.stdout:1/648: dread d0/d54/d64/d6d/f78 [0,4194304] 0 2024-04-02T04:47:58.538 INFO:tasks.workunit.client.0.smithi070.stdout:1/649: stat d0/l44 0 2024-04-02T04:47:58.545 INFO:tasks.workunit.client.1.smithi179.stdout:6/709: read - d3/d12/d23/d39/d80/d93/dd4/fb1 zero size 2024-04-02T04:47:58.546 INFO:tasks.workunit.client.1.smithi179.stdout:6/710: chown d3/d12/d44/c74 15942 1 2024-04-02T04:47:58.549 INFO:tasks.workunit.client.1.smithi179.stdout:6/711: write d3/d12/f62 [3026319,108796] 0 2024-04-02T04:47:58.550 INFO:tasks.workunit.client.0.smithi070.stdout:2/675: creat d2/d10/d5c/d7a/db0/dd8/dd2/fe6 x:0 0 0 2024-04-02T04:47:58.551 INFO:tasks.workunit.client.1.smithi179.stdout:9/711: creat d0/d1/d10/d6d/fd8 x:0 0 0 2024-04-02T04:47:58.552 INFO:tasks.workunit.client.0.smithi070.stdout:0/727: dread f0 [0,4194304] 0 2024-04-02T04:47:58.552 INFO:tasks.workunit.client.1.smithi179.stdout:1/741: dread d3/d5/f60 [4194304,4194304] 0 2024-04-02T04:47:58.553 INFO:tasks.workunit.client.0.smithi070.stdout:9/570: write d4/d10/d12/d27/f5c [65712,88901] 0 2024-04-02T04:47:58.556 INFO:tasks.workunit.client.1.smithi179.stdout:7/738: mkdir d1/d9/d55/dbf/d9f/da8/d104 0 2024-04-02T04:47:58.557 INFO:tasks.workunit.client.1.smithi179.stdout:7/739: dread - d1/dd/f8c zero size 2024-04-02T04:47:58.557 INFO:tasks.workunit.client.0.smithi070.stdout:6/522: getdents d0/d20/d5f 0 2024-04-02T04:47:58.558 INFO:tasks.workunit.client.0.smithi070.stdout:8/669: dwrite d2/d8/d23/d24/d5f/f93 [0,4194304] 0 2024-04-02T04:47:58.558 INFO:tasks.workunit.client.0.smithi070.stdout:6/523: write d0/d3/d19/f7f [4360934,10489] 0 2024-04-02T04:47:58.559 INFO:tasks.workunit.client.1.smithi179.stdout:3/741: rename d8/d7a/f8a to d8/dd3/dbb/feb 0 2024-04-02T04:47:58.560 INFO:tasks.workunit.client.0.smithi070.stdout:8/670: read d2/f7f [3484897,125663] 0 2024-04-02T04:47:58.561 INFO:tasks.workunit.client.0.smithi070.stdout:1/650: creat d0/d7/d28/d5c/d6e/fd3 x:0 0 0 2024-04-02T04:47:58.565 INFO:tasks.workunit.client.1.smithi179.stdout:2/590: creat dc/d29/d31/d66/fc5 x:0 0 0 2024-04-02T04:47:58.566 INFO:tasks.workunit.client.0.smithi070.stdout:2/676: truncate d2/d10/d21/d71/dd4/fb6 801809 0 2024-04-02T04:47:58.569 INFO:tasks.workunit.client.0.smithi070.stdout:1/651: dread d0/d7/d1a/f75 [0,4194304] 0 2024-04-02T04:47:58.569 INFO:tasks.workunit.client.0.smithi070.stdout:3/679: link d10/d17/d4c/f84 d10/d17/d51/d73/d9e/fe9 0 2024-04-02T04:47:58.570 INFO:tasks.workunit.client.1.smithi179.stdout:2/591: dread dc/f52 [0,4194304] 0 2024-04-02T04:47:58.571 INFO:tasks.workunit.client.0.smithi070.stdout:0/728: rename d4/d5/d13/d2a/d29/f65 to d4/d5/d13/d17/d98/fdc 0 2024-04-02T04:47:58.597 INFO:tasks.workunit.client.0.smithi070.stdout:8/671: symlink d2/d8/d29/d72/ld9 0 2024-04-02T04:47:58.598 INFO:tasks.workunit.client.0.smithi070.stdout:2/677: mkdir d2/d24/de7 0 2024-04-02T04:47:58.600 INFO:tasks.workunit.client.1.smithi179.stdout:7/740: dread d1/d9/d55/dbf/d9f/fd3 [0,4194304] 0 2024-04-02T04:47:58.600 INFO:tasks.workunit.client.1.smithi179.stdout:0/770: rename d1/da/d28/d5e/f98 to d1/da/d48/d5c/d83/fe1 0 2024-04-02T04:47:58.603 INFO:tasks.workunit.client.0.smithi070.stdout:8/672: dwrite d2/d3/d52/d62/f95 [0,4194304] 0 2024-04-02T04:47:58.606 INFO:tasks.workunit.client.1.smithi179.stdout:5/678: link d8/d1b/d23/d32/dbe/daf/cce d8/d1b/d23/d32/d46/cda 0 2024-04-02T04:47:58.609 INFO:tasks.workunit.client.0.smithi070.stdout:6/524: rename d0/l12 to d0/d55/la0 0 2024-04-02T04:47:58.610 INFO:tasks.workunit.client.0.smithi070.stdout:8/673: dwrite d2/d8/d29/db/fc5 [0,4194304] 0 2024-04-02T04:47:58.611 INFO:tasks.workunit.client.1.smithi179.stdout:2/592: mknod dc/d29/d31/d83/dae/cc6 0 2024-04-02T04:47:58.612 INFO:tasks.workunit.client.1.smithi179.stdout:2/593: stat dc/d29/d31/d83/dae/db0/dbc/c74 0 2024-04-02T04:47:58.612 INFO:tasks.workunit.client.0.smithi070.stdout:0/729: mknod d4/d5/d13/d17/cdd 0 2024-04-02T04:47:58.612 INFO:tasks.workunit.client.0.smithi070.stdout:6/525: fdatasync d0/d3/d19/d31/d46/d6b/f72 0 2024-04-02T04:47:58.612 INFO:tasks.workunit.client.0.smithi070.stdout:6/526: chown d0/d55/d8b 2 1 2024-04-02T04:47:58.614 INFO:tasks.workunit.client.0.smithi070.stdout:9/571: mknod d4/d10/d11/d7b/d19/d2f/d4d/d92/cba 0 2024-04-02T04:47:58.618 INFO:tasks.workunit.client.0.smithi070.stdout:2/678: dread d2/d12/fa9 [0,4194304] 0 2024-04-02T04:47:58.619 INFO:tasks.workunit.client.1.smithi179.stdout:0/771: write d1/db/d64/f99 [7726997,64527] 0 2024-04-02T04:47:58.619 INFO:tasks.workunit.client.0.smithi070.stdout:3/680: mknod d10/d32/d4b/de1/cea 0 2024-04-02T04:47:58.622 INFO:tasks.workunit.client.1.smithi179.stdout:5/679: creat d8/d1b/d23/d32/d4e/d51/fdb x:0 0 0 2024-04-02T04:47:58.623 INFO:tasks.workunit.client.1.smithi179.stdout:5/680: chown d8/d1b/d23/d32/d4e/d50/c86 342626 1 2024-04-02T04:47:58.623 INFO:tasks.workunit.client.0.smithi070.stdout:8/674: creat d2/d8/d29/d22/d5a/d77/fda x:0 0 0 2024-04-02T04:47:58.627 INFO:tasks.workunit.client.0.smithi070.stdout:6/527: mknod d0/d43/d80/ca1 0 2024-04-02T04:47:58.629 INFO:tasks.workunit.client.0.smithi070.stdout:0/730: rename d4/d5/d13/d17/cd5 to d4/d5/d58/da1/cde 0 2024-04-02T04:47:58.630 INFO:tasks.workunit.client.1.smithi179.stdout:7/741: dread d1/dd/d23/fee [0,4194304] 0 2024-04-02T04:47:58.630 INFO:tasks.workunit.client.1.smithi179.stdout:5/681: dwrite d8/d1b/d23/d32/d4e/f61 [4194304,4194304] 0 2024-04-02T04:47:58.631 INFO:tasks.workunit.client.1.smithi179.stdout:7/742: truncate d1/dd/f90 228561 0 2024-04-02T04:47:58.632 INFO:tasks.workunit.client.1.smithi179.stdout:0/772: readlink d1/da/d28/d37/l6f 0 2024-04-02T04:47:58.633 INFO:tasks.workunit.client.1.smithi179.stdout:7/743: stat d1/dd/l88 0 2024-04-02T04:47:58.633 INFO:tasks.workunit.client.0.smithi070.stdout:1/652: creat d0/d7/fd4 x:0 0 0 2024-04-02T04:47:58.634 INFO:tasks.workunit.client.1.smithi179.stdout:3/742: dwrite d8/f6f [0,4194304] 0 2024-04-02T04:47:58.635 INFO:tasks.workunit.client.1.smithi179.stdout:1/742: getdents d3/d5/d2f/d4f 0 2024-04-02T04:47:58.637 INFO:tasks.workunit.client.1.smithi179.stdout:2/594: unlink dc/d29/d4a/d1e/d43/fa4 0 2024-04-02T04:47:58.638 INFO:tasks.workunit.client.1.smithi179.stdout:3/743: truncate d8/d26/d83/d89/fdc 164557 0 2024-04-02T04:47:58.639 INFO:tasks.workunit.client.0.smithi070.stdout:9/572: dread d4/d13/f26 [0,4194304] 0 2024-04-02T04:47:58.640 INFO:tasks.workunit.client.1.smithi179.stdout:4/749: rename d1/d3/dc/d26/d8a/fbe to d1/d3/df/ffc 0 2024-04-02T04:47:58.641 INFO:tasks.workunit.client.1.smithi179.stdout:7/744: dwrite d1/d9/d27/d87/d8b/d75/db9/d8d/fc5 [0,4194304] 0 2024-04-02T04:47:58.642 INFO:tasks.workunit.client.1.smithi179.stdout:5/682: creat d8/d1b/d23/d32/d46/fdc x:0 0 0 2024-04-02T04:47:58.645 INFO:tasks.workunit.client.0.smithi070.stdout:0/731: rename d4/d5/d13/d23/fd4 to d4/d5/d13/d17/d6e/fdf 0 2024-04-02T04:47:58.646 INFO:tasks.workunit.client.0.smithi070.stdout:0/732: write d4/d5/d58/d60/fcb [2943,116615] 0 2024-04-02T04:47:58.647 INFO:tasks.workunit.client.0.smithi070.stdout:0/733: stat d4/d5/d58 0 2024-04-02T04:47:58.652 INFO:tasks.workunit.client.0.smithi070.stdout:8/675: mknod d2/d3/cdb 0 2024-04-02T04:47:58.653 INFO:tasks.workunit.client.0.smithi070.stdout:6/528: getdents d0/d20/d5f 0 2024-04-02T04:47:58.654 INFO:tasks.workunit.client.0.smithi070.stdout:8/676: chown d2/d3/d35/d3a/d4f/d59/f8f 22544 1 2024-04-02T04:47:58.654 INFO:tasks.workunit.client.1.smithi179.stdout:2/595: symlink dc/d29/d31/d83/dae/lc7 0 2024-04-02T04:47:58.655 INFO:tasks.workunit.client.0.smithi070.stdout:3/681: dwrite d10/d32/d3f/f38 [0,4194304] 0 2024-04-02T04:47:58.656 INFO:tasks.workunit.client.0.smithi070.stdout:3/682: chown d10/f2c 85561 1 2024-04-02T04:47:58.656 INFO:tasks.workunit.client.0.smithi070.stdout:9/573: creat d4/d10/d11/d7b/d19/d82/fbb x:0 0 0 2024-04-02T04:47:58.658 INFO:tasks.workunit.client.1.smithi179.stdout:6/712: rename d3/d2c/f5b to d3/d42/ff3 0 2024-04-02T04:47:58.659 INFO:tasks.workunit.client.0.smithi070.stdout:3/683: write d10/d32/d8a/fe5 [391538,77788] 0 2024-04-02T04:47:58.661 INFO:tasks.workunit.client.1.smithi179.stdout:4/750: chown d1/d3/dc/dc6/ce3 2056964 1 2024-04-02T04:47:58.663 INFO:tasks.workunit.client.0.smithi070.stdout:0/734: creat d4/d5/d21/d6b/fe0 x:0 0 0 2024-04-02T04:47:58.665 INFO:tasks.workunit.client.1.smithi179.stdout:5/683: fsync d8/d1b/d23/d32/d4b/fc4 0 2024-04-02T04:47:58.665 INFO:tasks.workunit.client.1.smithi179.stdout:3/744: dread d8/d57/d63/f84 [0,4194304] 0 2024-04-02T04:47:58.667 INFO:tasks.workunit.client.1.smithi179.stdout:0/773: creat d1/da/d28/d34/d39/da6/fe2 x:0 0 0 2024-04-02T04:47:58.668 INFO:tasks.workunit.client.1.smithi179.stdout:0/774: stat d1/da/f93 0 2024-04-02T04:47:58.669 INFO:tasks.workunit.client.1.smithi179.stdout:0/775: chown d1/db/dad 0 1 2024-04-02T04:47:58.670 INFO:tasks.workunit.client.0.smithi070.stdout:6/529: dread d0/d20/f34 [0,4194304] 0 2024-04-02T04:47:58.671 INFO:tasks.workunit.client.1.smithi179.stdout:5/684: dwrite d8/f3e [0,4194304] 0 2024-04-02T04:47:58.672 INFO:tasks.workunit.client.0.smithi070.stdout:9/574: rename d4/d10/d11/d7b/d19/c80 to d4/d10/d12/d55/d66/cbc 0 2024-04-02T04:47:58.675 INFO:tasks.workunit.client.0.smithi070.stdout:3/684: unlink d10/d32/d4b/f7d 0 2024-04-02T04:47:58.676 INFO:tasks.workunit.client.1.smithi179.stdout:2/596: dread dc/d29/d4a/d4b/f6d [0,4194304] 0 2024-04-02T04:47:58.678 INFO:tasks.workunit.client.0.smithi070.stdout:2/679: getdents d2/d10/d5c 0 2024-04-02T04:47:58.679 INFO:tasks.workunit.client.0.smithi070.stdout:2/680: write d2/fb5 [182413,88108] 0 2024-04-02T04:47:58.683 INFO:tasks.workunit.client.1.smithi179.stdout:3/745: mknod d8/d26/d83/dbc/cec 0 2024-04-02T04:47:58.684 INFO:tasks.workunit.client.0.smithi070.stdout:3/685: fdatasync d10/d66/fa9 0 2024-04-02T04:47:58.684 INFO:tasks.workunit.client.0.smithi070.stdout:3/686: chown d10/d32/d3f 2985180 1 2024-04-02T04:47:58.685 INFO:tasks.workunit.client.1.smithi179.stdout:0/776: mknod d1/db/ce3 0 2024-04-02T04:47:58.685 INFO:tasks.workunit.client.1.smithi179.stdout:1/743: dread d3/d5/d2f/d4f/d42/d53/d5b/f61 [0,4194304] 0 2024-04-02T04:47:58.687 INFO:tasks.workunit.client.1.smithi179.stdout:2/597: mknod dc/d29/d4a/d4b/cc8 0 2024-04-02T04:47:58.688 INFO:tasks.workunit.client.1.smithi179.stdout:8/729: sync 2024-04-02T04:47:58.690 INFO:tasks.workunit.client.0.smithi070.stdout:0/735: rename d4/d5/d13/d2a/f6c to d4/d5/da/d2d/fe1 0 2024-04-02T04:47:58.690 INFO:tasks.workunit.client.1.smithi179.stdout:6/713: symlink d3/d12/d23/d39/d4e/d66/lf4 0 2024-04-02T04:47:58.691 INFO:tasks.workunit.client.0.smithi070.stdout:9/575: truncate f3 2631101 0 2024-04-02T04:47:58.692 INFO:tasks.workunit.client.1.smithi179.stdout:7/745: rmdir d1/d9/d27/d87/d8b/d75/dd6 0 2024-04-02T04:47:58.693 INFO:tasks.workunit.client.0.smithi070.stdout:8/677: getdents d2/d3/d52/d62/d7c 0 2024-04-02T04:47:58.693 INFO:tasks.workunit.client.1.smithi179.stdout:4/751: fsync d1/d3/dc/dc1/fd2 0 2024-04-02T04:47:58.694 INFO:tasks.workunit.client.1.smithi179.stdout:4/752: fdatasync d1/d3/f7 0 2024-04-02T04:47:58.695 INFO:tasks.workunit.client.0.smithi070.stdout:8/678: readlink d2/d3/d35/d3a/d4f/d59/l8d 0 2024-04-02T04:47:58.695 INFO:tasks.workunit.client.1.smithi179.stdout:3/746: mkdir d8/d26/d83/dbc/ded 0 2024-04-02T04:47:58.695 INFO:tasks.workunit.client.0.smithi070.stdout:6/530: dread d0/d3/d19/d31/d61/f88 [4194304,4194304] 0 2024-04-02T04:47:58.695 INFO:tasks.workunit.client.0.smithi070.stdout:8/679: write d2/d3/d52/d62/f95 [2739186,23532] 0 2024-04-02T04:47:58.785 INFO:tasks.workunit.client.1.smithi179.stdout:9/712: sync 2024-04-02T04:47:58.786 INFO:tasks.workunit.client.1.smithi179.stdout:9/713: dread - d0/d19/d27/d7a/d87/fc9 zero size 2024-04-02T04:47:58.795 INFO:tasks.workunit.client.1.smithi179.stdout:0/777: creat d1/da/d48/d5c/d83/fe4 x:0 0 0 2024-04-02T04:47:58.795 INFO:tasks.workunit.client.1.smithi179.stdout:0/778: fsync d1/da/d28/f3a 0 2024-04-02T04:47:58.796 INFO:tasks.workunit.client.0.smithi070.stdout:0/736: truncate d4/d5/d13/f8a 3479472 0 2024-04-02T04:47:58.798 INFO:tasks.workunit.client.0.smithi070.stdout:9/576: creat d4/d13/fbd x:0 0 0 2024-04-02T04:47:58.801 INFO:tasks.workunit.client.1.smithi179.stdout:2/598: rename cb to dc/d29/d4a/d4b/cc9 0 2024-04-02T04:47:58.802 INFO:tasks.workunit.client.0.smithi070.stdout:3/687: mknod d10/d17/d51/ceb 0 2024-04-02T04:47:58.803 INFO:tasks.workunit.client.0.smithi070.stdout:2/681: mkdir d2/d10/d21/d3d/ddc/de8 0 2024-04-02T04:47:58.804 INFO:tasks.workunit.client.0.smithi070.stdout:2/682: read - d2/d12/d59/fe5 zero size 2024-04-02T04:47:58.807 INFO:tasks.workunit.client.0.smithi070.stdout:6/531: unlink d0/d3/d19/d31/d46/d6b/f7d 0 2024-04-02T04:47:58.810 INFO:tasks.workunit.client.0.smithi070.stdout:0/737: symlink d4/d5/d58/le2 0 2024-04-02T04:47:58.812 INFO:tasks.workunit.client.0.smithi070.stdout:9/577: mknod d4/d10/d11/cbe 0 2024-04-02T04:47:58.816 INFO:tasks.workunit.client.1.smithi179.stdout:5/685: sync 2024-04-02T04:47:58.818 INFO:tasks.workunit.client.1.smithi179.stdout:9/714: mknod d0/d19/d1c/d64/cd9 0 2024-04-02T04:47:58.819 INFO:tasks.workunit.client.1.smithi179.stdout:9/715: chown d0/d19/f56 2657 1 2024-04-02T04:47:58.819 INFO:tasks.workunit.client.1.smithi179.stdout:8/730: dread d3/d1b/d2a/f3e [0,4194304] 0 2024-04-02T04:47:58.820 INFO:tasks.workunit.client.0.smithi070.stdout:0/738: rmdir d4/d5/d58 39 2024-04-02T04:47:58.822 INFO:tasks.workunit.client.1.smithi179.stdout:0/779: creat d1/da/df/fe5 x:0 0 0 2024-04-02T04:47:58.824 INFO:tasks.workunit.client.0.smithi070.stdout:9/578: mknod d4/d10/d11/d7b/d19/d2f/d39/d57/cbf 0 2024-04-02T04:47:58.825 INFO:tasks.workunit.client.1.smithi179.stdout:9/716: dwrite d0/d1/d11/d77/fd6 [0,4194304] 0 2024-04-02T04:47:58.825 INFO:tasks.workunit.client.1.smithi179.stdout:0/780: read d1/da/d52/fbb [7844619,120632] 0 2024-04-02T04:47:58.826 INFO:tasks.workunit.client.1.smithi179.stdout:0/781: chown d1/da/dc3 37132858 1 2024-04-02T04:47:58.826 INFO:tasks.workunit.client.1.smithi179.stdout:2/599: symlink dc/d79/lca 0 2024-04-02T04:47:58.826 INFO:tasks.workunit.client.1.smithi179.stdout:0/782: fdatasync d1/da/f4a 0 2024-04-02T04:47:58.827 INFO:tasks.workunit.client.1.smithi179.stdout:2/600: readlink dc/d29/d31/d83/la5 0 2024-04-02T04:47:58.831 INFO:tasks.workunit.client.0.smithi070.stdout:2/683: creat d2/d10/d21/d71/dd4/dce/fe9 x:0 0 0 2024-04-02T04:47:58.832 INFO:tasks.workunit.client.1.smithi179.stdout:6/714: mknod d3/df2/cf5 0 2024-04-02T04:47:58.832 INFO:tasks.workunit.client.0.smithi070.stdout:2/684: chown d2/d12/l87 68 1 2024-04-02T04:47:58.833 INFO:tasks.workunit.client.0.smithi070.stdout:6/532: dread d0/d43/f51 [4194304,4194304] 0 2024-04-02T04:47:58.835 INFO:tasks.workunit.client.1.smithi179.stdout:0/783: dwrite d1/fcb [0,4194304] 0 2024-04-02T04:47:58.839 INFO:tasks.workunit.client.1.smithi179.stdout:3/747: dwrite d8/d26/d36/f49 [0,4194304] 0 2024-04-02T04:47:58.842 INFO:tasks.workunit.client.1.smithi179.stdout:7/746: truncate d1/d9/d7e/f8a 206439 0 2024-04-02T04:47:58.846 INFO:tasks.workunit.client.1.smithi179.stdout:9/717: rmdir d0/d1/d11/d3a 39 2024-04-02T04:47:58.849 INFO:tasks.workunit.client.1.smithi179.stdout:4/753: dwrite d1/d3/df/f48 [0,4194304] 0 2024-04-02T04:47:58.852 INFO:tasks.workunit.client.1.smithi179.stdout:6/715: fdatasync d3/d12/d44/fb3 0 2024-04-02T04:47:58.856 INFO:tasks.workunit.client.1.smithi179.stdout:0/784: write d1/fc1 [588947,45523] 0 2024-04-02T04:47:58.861 INFO:tasks.workunit.client.1.smithi179.stdout:1/744: link d3/d5/d2f/d5a/l5c d3/d5/d2f/d65/l102 0 2024-04-02T04:47:58.865 INFO:tasks.workunit.client.1.smithi179.stdout:2/601: mkdir dc/dcb 0 2024-04-02T04:47:58.866 INFO:tasks.workunit.client.1.smithi179.stdout:2/602: write dc/f89 [397680,108828] 0 2024-04-02T04:47:58.869 INFO:tasks.workunit.client.1.smithi179.stdout:3/748: dread d8/d26/d36/f4d [0,4194304] 0 2024-04-02T04:47:58.869 INFO:tasks.workunit.client.1.smithi179.stdout:6/716: mknod d3/d12/d6a/de8/cf6 0 2024-04-02T04:47:58.871 INFO:tasks.workunit.client.0.smithi070.stdout:2/685: rename d2/d10/d5c/d7a/db0/fbf to d2/d24/fea 0 2024-04-02T04:47:58.872 INFO:tasks.workunit.client.1.smithi179.stdout:0/785: rmdir d1/da/d28/d34/d39 39 2024-04-02T04:47:58.873 INFO:tasks.workunit.client.1.smithi179.stdout:0/786: write d1/da/f4a [1096527,61880] 0 2024-04-02T04:47:58.873 INFO:tasks.workunit.client.0.smithi070.stdout:3/688: getdents d10/d17/db2 0 2024-04-02T04:47:58.874 INFO:tasks.workunit.client.0.smithi070.stdout:2/686: dread d2/d24/f96 [0,4194304] 0 2024-04-02T04:47:58.874 INFO:tasks.workunit.client.0.smithi070.stdout:2/687: chown d2/d10/d5c/d7a/db0/fe3 124531919 1 2024-04-02T04:47:58.876 INFO:tasks.workunit.client.0.smithi070.stdout:0/739: rmdir d4/d5/d13/d17/d98/dcc 0 2024-04-02T04:47:58.878 INFO:tasks.workunit.client.1.smithi179.stdout:9/718: symlink d0/d1/d11/d29/lda 0 2024-04-02T04:47:58.878 INFO:tasks.workunit.client.1.smithi179.stdout:0/787: dread d1/da/f4a [0,4194304] 0 2024-04-02T04:47:58.879 INFO:tasks.workunit.client.0.smithi070.stdout:6/533: mkdir d0/da2 0 2024-04-02T04:47:58.879 INFO:tasks.workunit.client.1.smithi179.stdout:0/788: chown d1/da/d48/d5c/d83/d97 16 1 2024-04-02T04:47:58.881 INFO:tasks.workunit.client.1.smithi179.stdout:2/603: write dc/d29/d4a/d1e/d6c/f9e [142115,119384] 0 2024-04-02T04:47:58.882 INFO:tasks.workunit.client.1.smithi179.stdout:2/604: stat dc/d29/c73 0 2024-04-02T04:47:58.883 INFO:tasks.workunit.client.0.smithi070.stdout:3/689: creat d10/d17/d51/d73/d9e/fec x:0 0 0 2024-04-02T04:47:58.883 INFO:tasks.workunit.client.1.smithi179.stdout:6/717: fsync d3/d2d/d43/fef 0 2024-04-02T04:47:58.884 INFO:tasks.workunit.client.0.smithi070.stdout:2/688: symlink d2/d10/d5c/d7a/db0/dd8/dd2/d90/leb 0 2024-04-02T04:47:58.885 INFO:tasks.workunit.client.1.smithi179.stdout:5/686: getdents d8/d1b/d23/d32/dbe/daf/db7/dad 0 2024-04-02T04:47:58.885 INFO:tasks.workunit.client.0.smithi070.stdout:2/689: truncate d2/d12/d59/fb2 1611841 0 2024-04-02T04:47:58.886 INFO:tasks.workunit.client.0.smithi070.stdout:0/740: stat d4/d5/d13/d2a/d8d/f8f 0 2024-04-02T04:47:58.888 INFO:tasks.workunit.client.0.smithi070.stdout:9/579: getdents d4/d10/d11 0 2024-04-02T04:47:58.891 INFO:tasks.workunit.client.1.smithi179.stdout:8/731: write d3/d32/d21/d22/f2d [791698,126700] 0 2024-04-02T04:47:58.894 INFO:tasks.workunit.client.0.smithi070.stdout:2/690: rmdir d2/d10/d5c/d7a/db0/dd8 39 2024-04-02T04:47:58.900 INFO:tasks.workunit.client.0.smithi070.stdout:2/691: fsync d2/d10/d5c/f81 0 2024-04-02T04:47:58.900 INFO:tasks.workunit.client.0.smithi070.stdout:6/534: dread d0/d3/d19/d31/f36 [4194304,4194304] 0 2024-04-02T04:47:58.902 INFO:tasks.workunit.client.1.smithi179.stdout:1/745: write d3/d5/d2f/d65/f6b [1027031,38542] 0 2024-04-02T04:47:58.902 INFO:tasks.workunit.client.1.smithi179.stdout:3/749: truncate d8/d26/d36/f59 2007514 0 2024-04-02T04:47:58.905 INFO:tasks.workunit.client.1.smithi179.stdout:6/718: creat d3/d12/d23/d26/d57/ff7 x:0 0 0 2024-04-02T04:47:58.907 INFO:tasks.workunit.client.0.smithi070.stdout:9/580: dread d4/d10/d12/d27/d60/d9e/f77 [0,4194304] 0 2024-04-02T04:47:58.907 INFO:tasks.workunit.client.0.smithi070.stdout:2/692: creat d2/d12/d59/fec x:0 0 0 2024-04-02T04:47:58.907 INFO:tasks.workunit.client.1.smithi179.stdout:5/687: mkdir d8/d1b/d23/d32/d4e/ddd 0 2024-04-02T04:47:58.909 INFO:tasks.workunit.client.0.smithi070.stdout:6/535: dwrite d0/d55/d8b/d94/f6c [4194304,4194304] 0 2024-04-02T04:47:58.911 INFO:tasks.workunit.client.0.smithi070.stdout:9/581: chown d4/d10/d12/d55/d66/f7c 91131 1 2024-04-02T04:47:58.911 INFO:tasks.workunit.client.0.smithi070.stdout:3/690: dwrite d10/d66/fb9 [0,4194304] 0 2024-04-02T04:47:58.912 INFO:tasks.workunit.client.1.smithi179.stdout:9/719: symlink d0/d1/d11/d29/ldb 0 2024-04-02T04:47:58.913 INFO:tasks.workunit.client.1.smithi179.stdout:4/754: getdents d1/d3/dc/d27/dae 0 2024-04-02T04:47:58.914 INFO:tasks.workunit.client.0.smithi070.stdout:2/693: dread - d2/d10/d5c/d7a/db0/dd8/dd2/fcf zero size 2024-04-02T04:47:58.918 INFO:tasks.workunit.client.0.smithi070.stdout:9/582: dread d4/d10/d12/d27/d68/fb2 [0,4194304] 0 2024-04-02T04:47:58.918 INFO:tasks.workunit.client.1.smithi179.stdout:0/789: unlink d1/da/d28/d34/d39/da6/fe2 0 2024-04-02T04:47:58.921 INFO:tasks.workunit.client.1.smithi179.stdout:2/605: rename f6 to dc/d29/d4a/d1e/d8a/fcc 0 2024-04-02T04:47:58.923 INFO:tasks.workunit.client.0.smithi070.stdout:2/694: dwrite d2/d10/d5c/d7a/db0/dd8/dd2/fe6 [0,4194304] 0 2024-04-02T04:47:58.924 INFO:tasks.workunit.client.0.smithi070.stdout:2/695: dread - d2/d10/d21/d71/fdd zero size 2024-04-02T04:47:58.925 INFO:tasks.workunit.client.0.smithi070.stdout:6/536: symlink d0/d55/d8b/la3 0 2024-04-02T04:47:58.926 INFO:tasks.workunit.client.0.smithi070.stdout:2/696: chown d2/d10/d21/d3d/ccb 3695 1 2024-04-02T04:47:58.927 INFO:tasks.workunit.client.0.smithi070.stdout:2/697: write d2/d10/d21/d71/fdd [39852,126705] 0 2024-04-02T04:47:58.928 INFO:tasks.workunit.client.0.smithi070.stdout:9/583: fsync d4/d10/d12/d27/f69 0 2024-04-02T04:47:58.929 INFO:tasks.workunit.client.1.smithi179.stdout:5/688: dread d8/d1b/d23/d32/f99 [0,4194304] 0 2024-04-02T04:47:58.930 INFO:tasks.workunit.client.0.smithi070.stdout:9/584: write d4/d10/d12/d55/f7e [4070851,53853] 0 2024-04-02T04:47:58.931 INFO:tasks.workunit.client.0.smithi070.stdout:9/585: readlink d4/d13/l3e 0 2024-04-02T04:47:58.937 INFO:tasks.workunit.client.0.smithi070.stdout:7/631: sync 2024-04-02T04:47:58.937 INFO:tasks.workunit.client.0.smithi070.stdout:5/798: sync 2024-04-02T04:47:58.938 INFO:tasks.workunit.client.1.smithi179.stdout:8/732: dwrite d3/d31/f69 [0,4194304] 0 2024-04-02T04:47:58.941 INFO:tasks.workunit.client.1.smithi179.stdout:2/606: rmdir dc/d29/d4a/d1e 39 2024-04-02T04:47:58.941 INFO:tasks.workunit.client.0.smithi070.stdout:3/691: link d10/d32/f42 d10/d32/d4b/dc5/fed 0 2024-04-02T04:47:58.942 INFO:tasks.workunit.client.0.smithi070.stdout:6/537: mknod d0/da2/ca4 0 2024-04-02T04:47:58.943 INFO:tasks.workunit.client.1.smithi179.stdout:0/790: dread d1/da/df/f20 [0,4194304] 0 2024-04-02T04:47:58.944 INFO:tasks.workunit.client.0.smithi070.stdout:5/799: creat d5/d17/d6c/ffc x:0 0 0 2024-04-02T04:47:58.946 INFO:tasks.workunit.client.1.smithi179.stdout:1/746: dwrite d3/d5/d2f/d36/f94 [0,4194304] 0 2024-04-02T04:47:58.946 INFO:tasks.workunit.client.1.smithi179.stdout:0/791: stat d1/da/d28/d37/fce 0 2024-04-02T04:47:58.946 INFO:tasks.workunit.client.0.smithi070.stdout:9/586: symlink d4/d10/d11/d7b/d19/d2f/d8c/lc0 0 2024-04-02T04:47:58.947 INFO:tasks.workunit.client.1.smithi179.stdout:4/755: write d1/d3/dc/d27/d62/f5e [4300627,21353] 0 2024-04-02T04:47:58.947 INFO:tasks.workunit.client.1.smithi179.stdout:0/792: dread - d1/da/df/d56/d84/fc5 zero size 2024-04-02T04:47:58.949 INFO:tasks.workunit.client.0.smithi070.stdout:3/692: mkdir d10/d32/dd1/dee 0 2024-04-02T04:47:58.951 INFO:tasks.workunit.client.1.smithi179.stdout:0/793: chown d1/db/d64/f99 39727662 1 2024-04-02T04:47:58.953 INFO:tasks.workunit.client.1.smithi179.stdout:2/607: write dc/d29/fa3 [1383496,6705] 0 2024-04-02T04:47:58.954 INFO:tasks.workunit.client.1.smithi179.stdout:0/794: read d1/f2a [4942483,24439] 0 2024-04-02T04:47:58.955 INFO:tasks.workunit.client.1.smithi179.stdout:5/689: mkdir d8/d1b/d23/d42/dde 0 2024-04-02T04:47:58.956 INFO:tasks.workunit.client.1.smithi179.stdout:1/747: dwrite d3/d5/d2f/d4f/f4c [0,4194304] 0 2024-04-02T04:47:58.957 INFO:tasks.workunit.client.1.smithi179.stdout:0/795: fsync d1/da/df/fb5 0 2024-04-02T04:47:58.961 INFO:tasks.workunit.client.1.smithi179.stdout:9/720: rename d0/d19/d27/d7a/d87/cc6 to d0/d1/d11/d3a/cdc 0 2024-04-02T04:47:58.962 INFO:tasks.workunit.client.1.smithi179.stdout:0/796: write d1/da/d28/d34/d39/d77/fd2 [734625,22160] 0 2024-04-02T04:47:58.962 INFO:tasks.workunit.client.1.smithi179.stdout:7/747: sync 2024-04-02T04:47:58.966 INFO:tasks.workunit.client.1.smithi179.stdout:2/608: dwrite dc/f89 [0,4194304] 0 2024-04-02T04:47:58.966 INFO:tasks.workunit.client.1.smithi179.stdout:7/748: stat d1/dd/d23/f24 0 2024-04-02T04:47:58.971 INFO:tasks.workunit.client.1.smithi179.stdout:6/719: link d3/d2d/d38/d64/ca0 d3/d12/d23/d39/d4e/d66/cf8 0 2024-04-02T04:47:58.973 INFO:tasks.workunit.client.1.smithi179.stdout:0/797: mkdir d1/da/d67/de6 0 2024-04-02T04:47:58.973 INFO:tasks.workunit.client.1.smithi179.stdout:7/749: dwrite d1/d9/d7e/ff2 [0,4194304] 0 2024-04-02T04:47:58.974 INFO:tasks.workunit.client.1.smithi179.stdout:8/733: write d3/d31/f7f [4669321,79723] 0 2024-04-02T04:47:58.974 INFO:tasks.workunit.client.1.smithi179.stdout:5/690: symlink d8/d1b/ldf 0 2024-04-02T04:47:58.979 INFO:tasks.workunit.client.1.smithi179.stdout:2/609: creat dc/d29/d31/d57/fcd x:0 0 0 2024-04-02T04:47:58.980 INFO:tasks.workunit.client.1.smithi179.stdout:1/748: mknod d3/d5/d2f/d4f/d35/c103 0 2024-04-02T04:47:58.982 INFO:tasks.workunit.client.1.smithi179.stdout:1/749: write d3/d5/d2f/d4f/d42/df2/f8b [1983543,42674] 0 2024-04-02T04:47:58.982 INFO:tasks.workunit.client.1.smithi179.stdout:6/720: symlink d3/d12/d23/d39/d4e/d6c/lf9 0 2024-04-02T04:47:58.984 INFO:tasks.workunit.client.1.smithi179.stdout:4/756: dread d1/d3/df/d10/f11 [0,4194304] 0 2024-04-02T04:47:58.984 INFO:tasks.workunit.client.1.smithi179.stdout:7/750: creat d1/d9/d35/d6b/f105 x:0 0 0 2024-04-02T04:47:58.985 INFO:tasks.workunit.client.1.smithi179.stdout:5/691: creat d8/d1b/d23/d30/fe0 x:0 0 0 2024-04-02T04:47:58.990 INFO:tasks.workunit.client.1.smithi179.stdout:6/721: mknod d3/d12/d23/d39/d80/d93/ddc/cfa 0 2024-04-02T04:47:58.991 INFO:tasks.workunit.client.1.smithi179.stdout:1/750: creat d3/d5/f104 x:0 0 0 2024-04-02T04:47:58.992 INFO:tasks.workunit.client.1.smithi179.stdout:3/750: sync 2024-04-02T04:47:58.993 INFO:tasks.workunit.client.1.smithi179.stdout:1/751: truncate d3/f1f 969000 0 2024-04-02T04:47:58.994 INFO:tasks.workunit.client.1.smithi179.stdout:5/692: dwrite d8/d1b/d23/d32/dbe/fcf [0,4194304] 0 2024-04-02T04:47:58.995 INFO:tasks.workunit.client.1.smithi179.stdout:2/610: mkdir dc/d29/d4a/d1e/d8a/dce 0 2024-04-02T04:47:58.997 INFO:tasks.workunit.client.1.smithi179.stdout:4/757: creat d1/d3/dc0/ffd x:0 0 0 2024-04-02T04:47:58.998 INFO:tasks.workunit.client.1.smithi179.stdout:7/751: creat d1/d9/d55/dbf/d9f/da8/d104/f106 x:0 0 0 2024-04-02T04:47:58.999 INFO:tasks.workunit.client.1.smithi179.stdout:5/693: fdatasync d8/d1b/d23/f47 0 2024-04-02T04:47:59.000 INFO:tasks.workunit.client.1.smithi179.stdout:3/751: dread d8/d57/d63/f84 [0,4194304] 0 2024-04-02T04:47:59.000 INFO:tasks.workunit.client.1.smithi179.stdout:6/722: symlink d3/d2c/de0/lfb 0 2024-04-02T04:47:59.007 INFO:tasks.workunit.client.1.smithi179.stdout:6/723: chown d3/d12/d23/c3e 35907 1 2024-04-02T04:47:59.007 INFO:tasks.workunit.client.1.smithi179.stdout:5/694: unlink d8/d1b/d23/d32/d4e/f61 0 2024-04-02T04:47:59.007 INFO:tasks.workunit.client.1.smithi179.stdout:5/695: dread - d8/d1b/d23/d30/f34 zero size 2024-04-02T04:47:59.012 INFO:tasks.workunit.client.1.smithi179.stdout:5/696: getdents d8/d1b/d23 0 2024-04-02T04:47:59.014 INFO:tasks.workunit.client.1.smithi179.stdout:9/721: truncate d0/d1/d10/f6f 4079628 0 2024-04-02T04:47:59.018 INFO:tasks.workunit.client.1.smithi179.stdout:9/722: creat d0/d19/d27/d7a/d7e/fdd x:0 0 0 2024-04-02T04:47:59.018 INFO:tasks.workunit.client.1.smithi179.stdout:8/734: truncate d3/d32/d21/d49/fda 505765 0 2024-04-02T04:47:59.019 INFO:tasks.workunit.client.1.smithi179.stdout:5/697: dwrite d8/d1b/d23/d42/fcc [0,4194304] 0 2024-04-02T04:47:59.020 INFO:tasks.workunit.client.1.smithi179.stdout:0/798: truncate d1/da/d28/d37/f49 3773930 0 2024-04-02T04:47:59.020 INFO:tasks.workunit.client.1.smithi179.stdout:1/752: chown d3/d5/d2f/d4f/d6a/da0 306116913 1 2024-04-02T04:47:59.020 INFO:tasks.workunit.client.1.smithi179.stdout:8/735: stat d3/d6c/d96/fa1 0 2024-04-02T04:47:59.022 INFO:tasks.workunit.client.1.smithi179.stdout:0/799: dread - d1/da/d48/fca zero size 2024-04-02T04:47:59.024 INFO:tasks.workunit.client.1.smithi179.stdout:0/800: stat d1/da/d48/l82 0 2024-04-02T04:47:59.026 INFO:tasks.workunit.client.1.smithi179.stdout:2/611: write dc/d29/d4a/f3f [1066349,8948] 0 2024-04-02T04:47:59.026 INFO:tasks.workunit.client.1.smithi179.stdout:7/752: write d1/d9/d35/fb4 [693068,72355] 0 2024-04-02T04:47:59.034 INFO:tasks.workunit.client.1.smithi179.stdout:5/698: creat d8/d1b/fe1 x:0 0 0 2024-04-02T04:47:59.036 INFO:tasks.workunit.client.1.smithi179.stdout:1/753: creat d3/d5/d2f/d4f/d42/d53/d77/f105 x:0 0 0 2024-04-02T04:47:59.039 INFO:tasks.workunit.client.1.smithi179.stdout:0/801: truncate d1/f1e 737034 0 2024-04-02T04:47:59.039 INFO:tasks.workunit.client.1.smithi179.stdout:3/752: dwrite d8/d57/d63/faf [0,4194304] 0 2024-04-02T04:47:59.041 INFO:tasks.workunit.client.1.smithi179.stdout:6/724: dwrite d3/d2d/d43/f47 [0,4194304] 0 2024-04-02T04:47:59.043 INFO:tasks.workunit.client.1.smithi179.stdout:2/612: link dc/d29/d31/d57/d97/fb6 dc/dcb/fcf 0 2024-04-02T04:47:59.044 INFO:tasks.workunit.client.1.smithi179.stdout:8/736: write d3/d32/d21/d50/d65/f9b [484941,38901] 0 2024-04-02T04:47:59.045 INFO:tasks.workunit.client.1.smithi179.stdout:7/753: mkdir d1/d9/d55/dbf/d9f/da8/d100/d107 0 2024-04-02T04:47:59.052 INFO:tasks.workunit.client.1.smithi179.stdout:0/802: mknod d1/da/d28/d81/ce7 0 2024-04-02T04:47:59.053 INFO:tasks.workunit.client.1.smithi179.stdout:6/725: creat d3/d12/d23/ffc x:0 0 0 2024-04-02T04:47:59.055 INFO:tasks.workunit.client.1.smithi179.stdout:7/754: chown d1/d9/d55/dbf/c94 43968700 1 2024-04-02T04:47:59.056 INFO:tasks.workunit.client.1.smithi179.stdout:9/723: dwrite d0/d1/d11/d29/d40/f49 [0,4194304] 0 2024-04-02T04:47:59.057 INFO:tasks.workunit.client.1.smithi179.stdout:0/803: write d1/da/d28/d37/f95 [3496603,47653] 0 2024-04-02T04:47:59.058 INFO:tasks.workunit.client.1.smithi179.stdout:7/755: write d1/dd/d16/fcd [2902756,27104] 0 2024-04-02T04:47:59.058 INFO:tasks.workunit.client.1.smithi179.stdout:9/724: readlink d0/d1/d11/d29/d80/la3 0 2024-04-02T04:47:59.064 INFO:tasks.workunit.client.0.smithi070.stdout:5/800: rmdir d5/d17/d27/d2e 39 2024-04-02T04:47:59.067 INFO:tasks.workunit.client.1.smithi179.stdout:8/737: creat d3/d32/d21/d45/d5a/f101 x:0 0 0 2024-04-02T04:47:59.068 INFO:tasks.workunit.client.0.smithi070.stdout:2/698: dwrite d2/f9c [0,4194304] 0 2024-04-02T04:47:59.069 INFO:tasks.workunit.client.0.smithi070.stdout:7/632: getdents d7/d9 0 2024-04-02T04:47:59.069 INFO:tasks.workunit.client.0.smithi070.stdout:6/538: link d0/d55/d8b/l92 d0/d55/la5 0 2024-04-02T04:47:59.070 INFO:tasks.workunit.client.1.smithi179.stdout:6/726: creat d3/d2c/d85/ffd x:0 0 0 2024-04-02T04:47:59.074 INFO:tasks.workunit.client.0.smithi070.stdout:9/587: dwrite d4/d10/d11/d7b/d19/d82/f8a [0,4194304] 0 2024-04-02T04:47:59.075 INFO:tasks.workunit.client.0.smithi070.stdout:9/588: stat d4/d10/d11/da8/fb5 0 2024-04-02T04:47:59.076 INFO:tasks.workunit.client.0.smithi070.stdout:3/693: fsync d10/d32/f55 0 2024-04-02T04:47:59.077 INFO:tasks.workunit.client.1.smithi179.stdout:1/754: rename d3/d5/d2f/d4f/d35/l3e to d3/d5/d2f/d4f/d42/d53/d72/l106 0 2024-04-02T04:47:59.077 INFO:tasks.workunit.client.0.smithi070.stdout:9/589: write d4/d10/d11/f42 [3140170,60678] 0 2024-04-02T04:47:59.085 INFO:tasks.workunit.client.1.smithi179.stdout:5/699: getdents d8/d1b/d23/d32/d4e/d51 0 2024-04-02T04:47:59.086 INFO:tasks.workunit.client.0.smithi070.stdout:4/613: sync 2024-04-02T04:47:59.087 INFO:tasks.workunit.client.0.smithi070.stdout:1/653: sync 2024-04-02T04:47:59.087 INFO:tasks.workunit.client.1.smithi179.stdout:5/700: truncate d8/d1b/f98 499671 0 2024-04-02T04:47:59.091 INFO:tasks.workunit.client.0.smithi070.stdout:3/694: rmdir d10/d17/d51/d73 39 2024-04-02T04:47:59.092 INFO:tasks.workunit.client.1.smithi179.stdout:6/727: dread d3/d12/d23/d26/f4f [0,4194304] 0 2024-04-02T04:47:59.093 INFO:tasks.workunit.client.1.smithi179.stdout:8/738: rename d3/d32/d21/d45/ffc to d3/d32/d21/d49/df0/df9/f102 0 2024-04-02T04:47:59.094 INFO:tasks.workunit.client.1.smithi179.stdout:6/728: write d3/d12/d23/d2f/f3c [2109535,98573] 0 2024-04-02T04:47:59.094 INFO:tasks.workunit.client.0.smithi070.stdout:9/590: truncate d4/d34/f7a 1011515 0 2024-04-02T04:47:59.097 INFO:tasks.workunit.client.1.smithi179.stdout:7/756: creat d1/d9/d55/dbf/d9f/da8/dda/dea/f108 x:0 0 0 2024-04-02T04:47:59.097 INFO:tasks.workunit.client.1.smithi179.stdout:7/757: chown d1/d9/ffe 696378 1 2024-04-02T04:47:59.099 INFO:tasks.workunit.client.1.smithi179.stdout:7/758: read d1/d9/d55/f64 [2148854,13568] 0 2024-04-02T04:47:59.099 INFO:tasks.workunit.client.0.smithi070.stdout:2/699: dread d2/d10/d21/d71/f76 [0,4194304] 0 2024-04-02T04:47:59.100 INFO:tasks.workunit.client.0.smithi070.stdout:2/700: stat d2/d24/f96 0 2024-04-02T04:47:59.106 INFO:tasks.workunit.client.0.smithi070.stdout:3/695: fdatasync d10/d32/d3f/f43 0 2024-04-02T04:47:59.107 INFO:tasks.workunit.client.0.smithi070.stdout:9/591: unlink d4/d13/c49 0 2024-04-02T04:47:59.110 INFO:tasks.workunit.client.1.smithi179.stdout:0/804: dwrite d1/da/f4a [0,4194304] 0 2024-04-02T04:47:59.111 INFO:tasks.workunit.client.0.smithi070.stdout:5/801: creat d5/d17/ffd x:0 0 0 2024-04-02T04:47:59.111 INFO:tasks.workunit.client.1.smithi179.stdout:9/725: dwrite d0/f4c [0,4194304] 0 2024-04-02T04:47:59.113 INFO:tasks.workunit.client.0.smithi070.stdout:7/633: truncate d7/dd/df/d79/f48 5988629 0 2024-04-02T04:47:59.114 INFO:tasks.workunit.client.1.smithi179.stdout:0/805: truncate d1/da/d67/fa8 5132972 0 2024-04-02T04:47:59.114 INFO:tasks.workunit.client.1.smithi179.stdout:2/613: link dc/d29/l49 dc/ld0 0 2024-04-02T04:47:59.116 INFO:tasks.workunit.client.0.smithi070.stdout:1/654: truncate d0/d9/f3e 1194082 0 2024-04-02T04:47:59.116 INFO:tasks.workunit.client.0.smithi070.stdout:6/539: truncate d0/d3/d19/f7f 1856531 0 2024-04-02T04:47:59.118 INFO:tasks.workunit.client.0.smithi070.stdout:5/802: dwrite d5/d17/d50/fd7 [0,4194304] 0 2024-04-02T04:47:59.118 INFO:tasks.workunit.client.1.smithi179.stdout:5/701: write d8/d1b/d23/d32/d4e/fa0 [448928,117200] 0 2024-04-02T04:47:59.119 INFO:tasks.workunit.client.0.smithi070.stdout:6/540: chown d0/d55/d8b/d94/f53 5 1 2024-04-02T04:47:59.123 INFO:tasks.workunit.client.1.smithi179.stdout:1/755: dread d3/d5/de/f4e [0,4194304] 0 2024-04-02T04:47:59.124 INFO:tasks.workunit.client.0.smithi070.stdout:3/696: creat d10/d17/db2/dbf/fef x:0 0 0 2024-04-02T04:47:59.126 INFO:tasks.workunit.client.1.smithi179.stdout:8/739: dwrite d3/f27 [0,4194304] 0 2024-04-02T04:47:59.126 INFO:tasks.workunit.client.0.smithi070.stdout:9/592: rmdir d4/d10/d12/d55/d66 39 2024-04-02T04:47:59.129 INFO:tasks.workunit.client.0.smithi070.stdout:2/701: mkdir d2/d10/d5c/d7a/db0/dc7/ded 0 2024-04-02T04:47:59.129 INFO:tasks.workunit.client.1.smithi179.stdout:9/726: mknod d0/d19/d27/d7a/d87/daf/cde 0 2024-04-02T04:47:59.131 INFO:tasks.workunit.client.0.smithi070.stdout:7/634: mkdir d7/dd/df/dca 0 2024-04-02T04:47:59.133 INFO:tasks.workunit.client.1.smithi179.stdout:9/727: read d0/d1/d10/d6d/f2d [170482,126116] 0 2024-04-02T04:47:59.137 INFO:tasks.workunit.client.0.smithi070.stdout:4/614: rename d1/fa to d1/d9c/d65/fca 0 2024-04-02T04:47:59.138 INFO:tasks.workunit.client.1.smithi179.stdout:5/702: creat d8/d1b/d23/d36/d53/d62/da9/fe2 x:0 0 0 2024-04-02T04:47:59.139 INFO:tasks.workunit.client.1.smithi179.stdout:1/756: symlink d3/d5/d2f/db7/l107 0 2024-04-02T04:47:59.141 INFO:tasks.workunit.client.1.smithi179.stdout:2/614: dread dc/d29/d31/f77 [0,4194304] 0 2024-04-02T04:47:59.142 INFO:tasks.workunit.client.1.smithi179.stdout:9/728: dread d0/d1/d11/d29/d40/f49 [0,4194304] 0 2024-04-02T04:47:59.146 INFO:tasks.workunit.client.1.smithi179.stdout:4/758: sync 2024-04-02T04:47:59.146 INFO:tasks.workunit.client.1.smithi179.stdout:6/729: creat d3/d2d/d38/ffe x:0 0 0 2024-04-02T04:47:59.147 INFO:tasks.workunit.client.1.smithi179.stdout:6/730: write d3/d2d/da5/fbd [277919,107198] 0 2024-04-02T04:47:59.149 INFO:tasks.workunit.client.0.smithi070.stdout:9/593: creat d4/d34/fc1 x:0 0 0 2024-04-02T04:47:59.151 INFO:tasks.workunit.client.0.smithi070.stdout:9/594: read d4/d10/d12/f67 [565844,117746] 0 2024-04-02T04:47:59.151 INFO:tasks.workunit.client.1.smithi179.stdout:7/759: link d1/dd/d16/d4d/c89 d1/dd/d76/c109 0 2024-04-02T04:47:59.154 INFO:tasks.workunit.client.1.smithi179.stdout:4/759: dwrite d1/d3/df/d10/d2f/d60/fdd [0,4194304] 0 2024-04-02T04:47:59.155 INFO:tasks.workunit.client.0.smithi070.stdout:7/635: rename d7/dd/df/d34/c12 to d7/dd/df/d7a/d6c/d77/d54/ccb 0 2024-04-02T04:47:59.155 INFO:tasks.workunit.client.0.smithi070.stdout:7/636: chown d7/d4e/l5c 213752807 1 2024-04-02T04:47:59.156 INFO:tasks.workunit.client.1.smithi179.stdout:4/760: chown d1/d3/df/d10/d2f/c8b 123 1 2024-04-02T04:47:59.157 INFO:tasks.workunit.client.0.smithi070.stdout:5/803: symlink d5/d17/d2b/daf/lfe 0 2024-04-02T04:47:59.157 INFO:tasks.workunit.client.0.smithi070.stdout:1/655: write d0/d9/d1d/f4e [4039271,112013] 0 2024-04-02T04:47:59.160 INFO:tasks.workunit.client.1.smithi179.stdout:0/806: dwrite d1/da/d52/fbb [0,4194304] 0 2024-04-02T04:47:59.165 INFO:tasks.workunit.client.1.smithi179.stdout:8/740: creat d3/d1b/d2a/db0/f103 x:0 0 0 2024-04-02T04:47:59.166 INFO:tasks.workunit.client.1.smithi179.stdout:9/729: creat d0/d19/d27/d48/fdf x:0 0 0 2024-04-02T04:47:59.169 INFO:tasks.workunit.client.1.smithi179.stdout:8/741: dwrite d3/d1b/fa4 [4194304,4194304] 0 2024-04-02T04:47:59.171 INFO:tasks.workunit.client.0.smithi070.stdout:1/656: rename d0/d7/d28/d5c/d6e/fd3 to d0/d9/d1d/d22/d2c/d4f/fd5 0 2024-04-02T04:47:59.173 INFO:tasks.workunit.client.0.smithi070.stdout:2/702: dwrite d2/d10/d19/f1e [0,4194304] 0 2024-04-02T04:47:59.175 INFO:tasks.workunit.client.0.smithi070.stdout:2/703: chown d2/d10/d5c/f81 3430343 1 2024-04-02T04:47:59.176 INFO:tasks.workunit.client.0.smithi070.stdout:2/704: fsync d2/d10/d21/d71/dd4/fc4 0 2024-04-02T04:47:59.181 INFO:tasks.workunit.client.0.smithi070.stdout:4/615: dread d1/d3/d78/d91/faa [0,4194304] 0 2024-04-02T04:47:59.182 INFO:tasks.workunit.client.1.smithi179.stdout:5/703: dwrite d8/d1b/d23/d30/f34 [0,4194304] 0 2024-04-02T04:47:59.182 INFO:tasks.workunit.client.0.smithi070.stdout:4/616: fsync d1/d9c/d65/d7a/f9f 0 2024-04-02T04:47:59.183 INFO:tasks.workunit.client.1.smithi179.stdout:1/757: getdents d3/d5/d2f/d65/df8/dfc 0 2024-04-02T04:47:59.184 INFO:tasks.workunit.client.0.smithi070.stdout:4/617: chown d1/d3/d78/d91/ca2 4569 1 2024-04-02T04:47:59.185 INFO:tasks.workunit.client.0.smithi070.stdout:5/804: symlink d5/d9/d10/d6a/d7d/dc4/lff 0 2024-04-02T04:47:59.187 INFO:tasks.workunit.client.1.smithi179.stdout:4/761: symlink d1/d3/df/d10/d2f/d60/lfe 0 2024-04-02T04:47:59.188 INFO:tasks.workunit.client.1.smithi179.stdout:4/762: write d1/d93/fe4 [450431,27980] 0 2024-04-02T04:47:59.189 INFO:tasks.workunit.client.0.smithi070.stdout:6/541: getdents d0/d43/d80 0 2024-04-02T04:47:59.191 INFO:tasks.workunit.client.0.smithi070.stdout:4/618: dwrite d1/d3/d78/d91/faf [0,4194304] 0 2024-04-02T04:47:59.192 INFO:tasks.workunit.client.1.smithi179.stdout:0/807: fdatasync d1/da/d48/f54 0 2024-04-02T04:47:59.193 INFO:tasks.workunit.client.1.smithi179.stdout:0/808: write d1/fcb [2093707,64882] 0 2024-04-02T04:47:59.195 INFO:tasks.workunit.client.0.smithi070.stdout:1/657: creat d0/db4/fd6 x:0 0 0 2024-04-02T04:47:59.197 INFO:tasks.workunit.client.1.smithi179.stdout:6/731: mkdir d3/d12/d23/d39/dff 0 2024-04-02T04:47:59.199 INFO:tasks.workunit.client.0.smithi070.stdout:9/595: creat d4/d10/d11/d7b/d19/fc2 x:0 0 0 2024-04-02T04:47:59.200 INFO:tasks.workunit.client.1.smithi179.stdout:8/742: truncate d3/d1b/f23 3141355 0 2024-04-02T04:47:59.203 INFO:tasks.workunit.client.1.smithi179.stdout:7/760: rename d1/d9/d27/d87/lde to d1/d9/d35/d6b/l10a 0 2024-04-02T04:47:59.206 INFO:tasks.workunit.client.1.smithi179.stdout:2/615: write f2 [256426,33329] 0 2024-04-02T04:47:59.206 INFO:tasks.workunit.client.0.smithi070.stdout:9/596: dread d4/d10/f2c [0,4194304] 0 2024-04-02T04:47:59.206 INFO:tasks.workunit.client.0.smithi070.stdout:6/542: mkdir d0/d3/d19/d31/d61/d7b/da6 0 2024-04-02T04:47:59.207 INFO:tasks.workunit.client.0.smithi070.stdout:6/543: fsync d0/f26 0 2024-04-02T04:47:59.207 INFO:tasks.workunit.client.1.smithi179.stdout:5/704: symlink d8/d1b/d23/d32/d46/le3 0 2024-04-02T04:47:59.208 INFO:tasks.workunit.client.0.smithi070.stdout:4/619: truncate d1/d3/d9/f5f 1392551 0 2024-04-02T04:47:59.210 INFO:tasks.workunit.client.0.smithi070.stdout:1/658: symlink d0/d9/d1d/d22/d2c/d4f/d77/ld7 0 2024-04-02T04:47:59.210 INFO:tasks.workunit.client.1.smithi179.stdout:3/753: sync 2024-04-02T04:47:59.212 INFO:tasks.workunit.client.0.smithi070.stdout:5/805: mknod d5/d9/d10/c100 0 2024-04-02T04:47:59.212 INFO:tasks.workunit.client.1.smithi179.stdout:0/809: creat d1/da/d52/fe8 x:0 0 0 2024-04-02T04:47:59.215 INFO:tasks.workunit.client.0.smithi070.stdout:5/806: dread d5/d9/f5e [0,4194304] 0 2024-04-02T04:47:59.215 INFO:tasks.workunit.client.1.smithi179.stdout:5/705: dwrite d8/fd3 [0,4194304] 0 2024-04-02T04:47:59.216 INFO:tasks.workunit.client.0.smithi070.stdout:6/544: fsync d0/d43/f51 0 2024-04-02T04:47:59.216 INFO:tasks.workunit.client.0.smithi070.stdout:5/807: readlink d5/d17/d50/l9d 0 2024-04-02T04:47:59.216 INFO:tasks.workunit.client.1.smithi179.stdout:5/706: dread - d8/d7e/fd8 zero size 2024-04-02T04:47:59.217 INFO:tasks.workunit.client.1.smithi179.stdout:8/743: mkdir d3/d32/d21/d22/d6b/db1/d104 0 2024-04-02T04:47:59.220 INFO:tasks.workunit.client.0.smithi070.stdout:2/705: creat d2/d10/d21/fee x:0 0 0 2024-04-02T04:47:59.224 INFO:tasks.workunit.client.0.smithi070.stdout:1/659: creat d0/d9/d1d/fd8 x:0 0 0 2024-04-02T04:47:59.224 INFO:tasks.workunit.client.1.smithi179.stdout:2/616: read dc/d29/d4a/d1e/f7b [1850903,23965] 0 2024-04-02T04:47:59.224 INFO:tasks.workunit.client.1.smithi179.stdout:2/617: dread - dc/d29/d31/fb9 zero size 2024-04-02T04:47:59.228 INFO:tasks.workunit.client.1.smithi179.stdout:1/758: write d3/d5/d2f/d4f/d42/fae [982376,122223] 0 2024-04-02T04:47:59.229 INFO:tasks.workunit.client.1.smithi179.stdout:3/754: mkdir d8/d26/dcd/dee 0 2024-04-02T04:47:59.231 INFO:tasks.workunit.client.0.smithi070.stdout:9/597: rename d4/d10/f71 to d4/d13/fc3 0 2024-04-02T04:47:59.234 INFO:tasks.workunit.client.1.smithi179.stdout:9/730: creat d0/d19/d27/fe0 x:0 0 0 2024-04-02T04:47:59.236 INFO:tasks.workunit.client.1.smithi179.stdout:0/810: creat d1/da/d52/d75/d9a/fe9 x:0 0 0 2024-04-02T04:47:59.236 INFO:tasks.workunit.client.1.smithi179.stdout:6/732: write d3/d12/d44/fb3 [543308,15818] 0 2024-04-02T04:47:59.238 INFO:tasks.workunit.client.1.smithi179.stdout:5/707: mknod d8/d1b/d23/d30/d73/ce4 0 2024-04-02T04:47:59.240 INFO:tasks.workunit.client.0.smithi070.stdout:2/706: unlink d2/l58 0 2024-04-02T04:47:59.240 INFO:tasks.workunit.client.1.smithi179.stdout:8/744: fdatasync d3/d32/d21/d49/df0/df9/f102 0 2024-04-02T04:47:59.241 INFO:tasks.workunit.client.0.smithi070.stdout:2/707: truncate d2/fd6 854502 0 2024-04-02T04:47:59.241 INFO:tasks.workunit.client.0.smithi070.stdout:2/708: fsync d2/f9c 0 2024-04-02T04:47:59.242 INFO:tasks.workunit.client.0.smithi070.stdout:1/660: fsync d0/d9/d1d/f16 0 2024-04-02T04:47:59.245 INFO:tasks.workunit.client.0.smithi070.stdout:9/598: creat d4/d10/d11/d7b/d19/d82/fc4 x:0 0 0 2024-04-02T04:47:59.252 INFO:tasks.workunit.client.1.smithi179.stdout:6/733: dread d3/d9a/f9f [0,4194304] 0 2024-04-02T04:47:59.255 INFO:tasks.workunit.client.0.smithi070.stdout:5/808: dwrite d5/d9/d10/f19 [0,4194304] 0 2024-04-02T04:47:59.258 INFO:tasks.workunit.client.1.smithi179.stdout:8/745: mkdir d3/d32/d21/d45/d105 0 2024-04-02T04:47:59.259 INFO:tasks.workunit.client.1.smithi179.stdout:8/746: fdatasync d3/d32/d21/d22/d6b/db1/fcb 0 2024-04-02T04:47:59.259 INFO:tasks.workunit.client.1.smithi179.stdout:7/761: creat d1/dd/f10b x:0 0 0 2024-04-02T04:47:59.261 INFO:tasks.workunit.client.0.smithi070.stdout:9/599: mknod d4/d13/cc5 0 2024-04-02T04:47:59.262 INFO:tasks.workunit.client.0.smithi070.stdout:6/545: dwrite d0/f78 [0,4194304] 0 2024-04-02T04:47:59.263 INFO:tasks.workunit.client.0.smithi070.stdout:4/620: link d1/d98/cb9 d1/d98/ccb 0 2024-04-02T04:47:59.264 INFO:tasks.workunit.client.1.smithi179.stdout:3/755: dwrite d8/d26/d36/f4d [0,4194304] 0 2024-04-02T04:47:59.264 INFO:tasks.workunit.client.1.smithi179.stdout:3/756: chown d8/d26/d36/c43 321861 1 2024-04-02T04:47:59.265 INFO:tasks.workunit.client.1.smithi179.stdout:3/757: dread - d8/fa1 zero size 2024-04-02T04:47:59.266 INFO:tasks.workunit.client.1.smithi179.stdout:3/758: stat d8/d26/d36/c79 0 2024-04-02T04:47:59.267 INFO:tasks.workunit.client.0.smithi070.stdout:6/546: dread d0/d3/d19/f3f [0,4194304] 0 2024-04-02T04:47:59.269 INFO:tasks.workunit.client.1.smithi179.stdout:0/811: link d1/da/d67/d9f/fc8 d1/da/d67/de6/fea 0 2024-04-02T04:47:59.269 INFO:tasks.workunit.client.1.smithi179.stdout:0/812: stat d1/l2e 0 2024-04-02T04:47:59.269 INFO:tasks.workunit.client.0.smithi070.stdout:2/709: dwrite d2/d10/d21/d3d/f5d [0,4194304] 0 2024-04-02T04:47:59.272 INFO:tasks.workunit.client.0.smithi070.stdout:1/661: link d0/d54/f63 d0/d7/d28/d5c/d6e/fd9 0 2024-04-02T04:47:59.272 INFO:tasks.workunit.client.1.smithi179.stdout:5/708: mkdir d8/d1b/d23/d32/d4e/ddd/de5 0 2024-04-02T04:47:59.279 INFO:tasks.workunit.client.0.smithi070.stdout:4/621: dread d1/f26 [0,4194304] 0 2024-04-02T04:47:59.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:47:59 smithi070 ceph-mon[29212]: pgmap v18: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 78 MiB/s rd, 200 MiB/s wr, 415 op/s 2024-04-02T04:47:59.280 INFO:tasks.workunit.client.0.smithi070.stdout:6/547: chown d0/l4d 246 1 2024-04-02T04:47:59.281 INFO:tasks.workunit.client.0.smithi070.stdout:2/710: fdatasync d2/d10/d21/d71/fae 0 2024-04-02T04:47:59.283 INFO:tasks.workunit.client.0.smithi070.stdout:1/662: fsync d0/d9/d1d/d22/d2c/f76 0 2024-04-02T04:47:59.284 INFO:tasks.workunit.client.0.smithi070.stdout:9/600: symlink d4/d10/lc6 0 2024-04-02T04:47:59.286 INFO:tasks.workunit.client.0.smithi070.stdout:9/601: chown d4/d10/d11/d7b/d19/d2f/d39/ca3 445444 1 2024-04-02T04:47:59.286 INFO:tasks.workunit.client.0.smithi070.stdout:5/809: fdatasync d5/d17/da7/fef 0 2024-04-02T04:47:59.287 INFO:tasks.workunit.client.1.smithi179.stdout:8/747: mknod d3/d32/d21/d49/df0/df9/c106 0 2024-04-02T04:47:59.288 INFO:tasks.workunit.client.1.smithi179.stdout:1/759: creat d3/d5/d2f/d4f/d42/d53/f108 x:0 0 0 2024-04-02T04:47:59.289 INFO:tasks.workunit.client.1.smithi179.stdout:6/734: creat d3/d12/d23/d26/dd3/f100 x:0 0 0 2024-04-02T04:47:59.289 INFO:tasks.workunit.client.1.smithi179.stdout:1/760: chown d3/f95 320 1 2024-04-02T04:47:59.290 INFO:tasks.workunit.client.0.smithi070.stdout:8/680: sync 2024-04-02T04:47:59.290 INFO:tasks.workunit.client.1.smithi179.stdout:5/709: mkdir d8/d1b/d23/d32/dbe/daf/db7/de6 0 2024-04-02T04:47:59.291 INFO:tasks.workunit.client.0.smithi070.stdout:2/711: fsync d2/d10/d21/d3d/f63 0 2024-04-02T04:47:59.292 INFO:tasks.workunit.client.0.smithi070.stdout:2/712: chown d2/d10/d5c/d7a/db0/dc7 35310 1 2024-04-02T04:47:59.293 INFO:tasks.workunit.client.1.smithi179.stdout:9/731: dwrite d0/d1/d11/d29/f94 [0,4194304] 0 2024-04-02T04:47:59.294 INFO:tasks.workunit.client.1.smithi179.stdout:2/618: rename dc/d29/d4a/c4d to dc/d29/d4a/d1e/cd1 0 2024-04-02T04:47:59.296 INFO:tasks.workunit.client.1.smithi179.stdout:3/759: mkdir d8/d26/d52/d7e/def 0 2024-04-02T04:47:59.298 INFO:tasks.workunit.client.0.smithi070.stdout:1/663: unlink d0/d7/d1a/db9/lb8 0 2024-04-02T04:47:59.300 INFO:tasks.workunit.client.1.smithi179.stdout:8/748: mkdir d3/d32/d21/d50/d56/d94/d84/d107 0 2024-04-02T04:47:59.303 INFO:tasks.workunit.client.1.smithi179.stdout:2/619: dwrite dc/d29/d31/d57/fcd [0,4194304] 0 2024-04-02T04:47:59.304 INFO:tasks.workunit.client.1.smithi179.stdout:0/813: dread d1/da/f3c [0,4194304] 0 2024-04-02T04:47:59.306 INFO:tasks.workunit.client.1.smithi179.stdout:0/814: write d1/da/d52/fe8 [826675,62313] 0 2024-04-02T04:47:59.307 INFO:tasks.workunit.client.0.smithi070.stdout:5/810: unlink d5/d17/d6c/l4d 0 2024-04-02T04:47:59.307 INFO:tasks.workunit.client.1.smithi179.stdout:0/815: truncate d1/fc1 1460561 0 2024-04-02T04:47:59.308 INFO:tasks.workunit.client.1.smithi179.stdout:0/816: readlink d1/da/d52/l87 0 2024-04-02T04:47:59.311 INFO:tasks.workunit.client.1.smithi179.stdout:1/761: dwrite d3/d5/f1d [0,4194304] 0 2024-04-02T04:47:59.311 INFO:tasks.workunit.client.1.smithi179.stdout:5/710: unlink d8/d1b/d23/d32/d4b/f70 0 2024-04-02T04:47:59.312 INFO:tasks.workunit.client.0.smithi070.stdout:4/622: getdents d1/d3/d85/d8e/dc8 0 2024-04-02T04:47:59.312 INFO:tasks.workunit.client.1.smithi179.stdout:5/711: readlink d8/d1b/l21 0 2024-04-02T04:47:59.312 INFO:tasks.workunit.client.1.smithi179.stdout:6/735: rmdir d3/d12/d23 39 2024-04-02T04:47:59.313 INFO:tasks.workunit.client.0.smithi070.stdout:4/623: stat d1/d9c/d65/d7a/f7d 0 2024-04-02T04:47:59.315 INFO:tasks.workunit.client.0.smithi070.stdout:5/811: dwrite d5/d9/d23/fb4 [0,4194304] 0 2024-04-02T04:47:59.317 INFO:tasks.workunit.client.0.smithi070.stdout:8/681: mknod d2/d8/d29/d22/d5a/d9e/cdc 0 2024-04-02T04:47:59.318 INFO:tasks.workunit.client.0.smithi070.stdout:8/682: truncate d2/f9b 4456220 0 2024-04-02T04:47:59.321 INFO:tasks.workunit.client.0.smithi070.stdout:2/713: fsync d2/d10/d19/f7b 0 2024-04-02T04:47:59.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:47:59 smithi179 ceph-mon[36603]: pgmap v18: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 78 MiB/s rd, 200 MiB/s wr, 415 op/s 2024-04-02T04:47:59.323 INFO:tasks.workunit.client.0.smithi070.stdout:6/548: dread d0/d3/d19/d31/d46/f68 [0,4194304] 0 2024-04-02T04:47:59.324 INFO:tasks.workunit.client.0.smithi070.stdout:2/714: write d2/d10/d19/f1e [250504,75455] 0 2024-04-02T04:47:59.325 INFO:tasks.workunit.client.0.smithi070.stdout:9/602: write d4/d10/d12/d27/f69 [967541,59418] 0 2024-04-02T04:47:59.326 INFO:tasks.workunit.client.1.smithi179.stdout:3/760: dread d8/dd3/fd9 [0,4194304] 0 2024-04-02T04:47:59.326 INFO:tasks.workunit.client.1.smithi179.stdout:2/620: mkdir dc/d29/d4a/d1e/d6c/dd2 0 2024-04-02T04:47:59.327 INFO:tasks.workunit.client.0.smithi070.stdout:2/715: readlink d2/d10/d19/l75 0 2024-04-02T04:47:59.328 INFO:tasks.workunit.client.0.smithi070.stdout:8/683: dwrite d2/d3/d35/f78 [0,4194304] 0 2024-04-02T04:47:59.329 INFO:tasks.workunit.client.1.smithi179.stdout:0/817: symlink d1/db/d64/leb 0 2024-04-02T04:47:59.330 INFO:tasks.workunit.client.0.smithi070.stdout:8/684: read - d2/d8/d29/fba zero size 2024-04-02T04:47:59.331 INFO:tasks.workunit.client.1.smithi179.stdout:1/762: symlink d3/da8/l109 0 2024-04-02T04:47:59.335 INFO:tasks.workunit.client.1.smithi179.stdout:9/732: symlink d0/d19/d27/le1 0 2024-04-02T04:47:59.336 INFO:tasks.workunit.client.0.smithi070.stdout:6/549: symlink d0/d55/la7 0 2024-04-02T04:47:59.337 INFO:tasks.workunit.client.1.smithi179.stdout:4/763: sync 2024-04-02T04:47:59.338 INFO:tasks.workunit.client.0.smithi070.stdout:2/716: mknod d2/d10/d19/cef 0 2024-04-02T04:47:59.339 INFO:tasks.workunit.client.1.smithi179.stdout:0/818: stat d1/da/d48/d5c/l8d 0 2024-04-02T04:47:59.340 INFO:tasks.workunit.client.1.smithi179.stdout:3/761: read d8/dd3/dbb/fc6 [2487057,50103] 0 2024-04-02T04:47:59.341 INFO:tasks.workunit.client.1.smithi179.stdout:5/712: creat d8/d1b/d23/d42/dde/fe7 x:0 0 0 2024-04-02T04:47:59.342 INFO:tasks.workunit.client.0.smithi070.stdout:6/550: chown d0/d43/l99 1764052856 1 2024-04-02T04:47:59.344 INFO:tasks.workunit.client.1.smithi179.stdout:8/749: link d3/d32/d21/d22/d6b/d91/cdc d3/d32/d21/d45/d105/c108 0 2024-04-02T04:47:59.345 INFO:tasks.workunit.client.0.smithi070.stdout:2/717: mkdir d2/d10/d21/d3d/df0 0 2024-04-02T04:47:59.349 INFO:tasks.workunit.client.1.smithi179.stdout:4/764: rename d1/d15/d6e/d9c/fac to d1/d3/df/d10/d2f/d60/fff 0 2024-04-02T04:47:59.349 INFO:tasks.workunit.client.1.smithi179.stdout:3/762: dwrite d8/d26/d36/f5c [0,4194304] 0 2024-04-02T04:47:59.350 INFO:tasks.workunit.client.1.smithi179.stdout:5/713: dwrite d8/d1b/d23/d32/d6e/f95 [0,4194304] 0 2024-04-02T04:47:59.354 INFO:tasks.workunit.client.1.smithi179.stdout:0/819: mkdir d1/da/d52/d75/d9a/dec 0 2024-04-02T04:47:59.356 INFO:tasks.workunit.client.0.smithi070.stdout:2/718: fsync d2/d12/fa7 0 2024-04-02T04:47:59.357 INFO:tasks.workunit.client.0.smithi070.stdout:5/812: dread d5/d9/d10/d51/fbc [0,4194304] 0 2024-04-02T04:47:59.359 INFO:tasks.workunit.client.0.smithi070.stdout:8/685: creat d2/d8/d29/d22/fdd x:0 0 0 2024-04-02T04:47:59.360 INFO:tasks.workunit.client.1.smithi179.stdout:8/750: mkdir d3/d1b/d2a/d41/d109 0 2024-04-02T04:47:59.360 INFO:tasks.workunit.client.0.smithi070.stdout:6/551: symlink d0/la8 0 2024-04-02T04:47:59.363 INFO:tasks.workunit.client.1.smithi179.stdout:3/763: mknod d8/dd3/cf0 0 2024-04-02T04:47:59.365 INFO:tasks.workunit.client.1.smithi179.stdout:5/714: rmdir d8/d1b/d23/d36/d53/d62/db2 39 2024-04-02T04:47:59.365 INFO:tasks.workunit.client.0.smithi070.stdout:4/624: write d1/f3f [3889227,57560] 0 2024-04-02T04:47:59.366 INFO:tasks.workunit.client.1.smithi179.stdout:5/715: fsync d8/d1b/d23/fb4 0 2024-04-02T04:47:59.369 INFO:tasks.workunit.client.0.smithi070.stdout:8/686: mkdir d2/d8/d29/d22/d5a/d94/da2/dde 0 2024-04-02T04:47:59.370 INFO:tasks.workunit.client.1.smithi179.stdout:0/820: fdatasync d1/da/d48/d5c/fb9 0 2024-04-02T04:47:59.370 INFO:tasks.workunit.client.0.smithi070.stdout:9/603: write d4/d34/f7a [356761,42822] 0 2024-04-02T04:47:59.370 INFO:tasks.workunit.client.0.smithi070.stdout:8/687: dread - d2/d8/d29/d22/d5a/d9e/fce zero size 2024-04-02T04:47:59.370 INFO:tasks.workunit.client.0.smithi070.stdout:6/552: creat d0/d3/d19/d31/d46/fa9 x:0 0 0 2024-04-02T04:47:59.370 INFO:tasks.workunit.client.0.smithi070.stdout:8/688: write d2/d3/f53 [438524,81055] 0 2024-04-02T04:47:59.371 INFO:tasks.workunit.client.1.smithi179.stdout:8/751: mkdir d3/d32/d21/d45/d10a 0 2024-04-02T04:47:59.372 INFO:tasks.workunit.client.0.smithi070.stdout:8/689: read d2/f9b [3166302,68285] 0 2024-04-02T04:47:59.374 INFO:tasks.workunit.client.1.smithi179.stdout:3/764: dwrite d8/d26/d36/f8d [0,4194304] 0 2024-04-02T04:47:59.375 INFO:tasks.workunit.client.1.smithi179.stdout:9/733: dwrite d0/d1/f7 [0,4194304] 0 2024-04-02T04:47:59.375 INFO:tasks.workunit.client.1.smithi179.stdout:2/621: dread dc/d29/d4a/f56 [0,4194304] 0 2024-04-02T04:47:59.377 INFO:tasks.workunit.client.0.smithi070.stdout:9/604: dwrite d4/d10/d12/d27/f69 [0,4194304] 0 2024-04-02T04:47:59.377 INFO:tasks.workunit.client.1.smithi179.stdout:6/736: getdents d3/d12/d23/d39 0 2024-04-02T04:47:59.379 INFO:tasks.workunit.client.1.smithi179.stdout:0/821: mkdir d1/da/df/ded 0 2024-04-02T04:47:59.382 INFO:tasks.workunit.client.1.smithi179.stdout:9/734: dread d0/d19/d1c/fc0 [0,4194304] 0 2024-04-02T04:47:59.383 INFO:tasks.workunit.client.0.smithi070.stdout:2/719: link d2/d12/fa6 d2/d10/d5c/d7a/db0/de4/ff1 0 2024-04-02T04:47:59.383 INFO:tasks.workunit.client.0.smithi070.stdout:9/605: dwrite d4/d13/f98 [0,4194304] 0 2024-04-02T04:47:59.385 INFO:tasks.workunit.client.0.smithi070.stdout:9/606: fsync d4/d10/d11/d7b/fa2 0 2024-04-02T04:47:59.390 INFO:tasks.workunit.client.1.smithi179.stdout:3/765: write d8/d26/d83/fa0 [2616337,108942] 0 2024-04-02T04:47:59.391 INFO:tasks.workunit.client.1.smithi179.stdout:6/737: symlink d3/d12/d23/d26/l101 0 2024-04-02T04:47:59.392 INFO:tasks.workunit.client.1.smithi179.stdout:7/762: sync 2024-04-02T04:47:59.393 INFO:tasks.workunit.client.1.smithi179.stdout:4/765: write d1/d3/dc/d26/f29 [3323802,93616] 0 2024-04-02T04:47:59.395 INFO:tasks.workunit.client.0.smithi070.stdout:2/720: symlink d2/d10/d53/lf2 0 2024-04-02T04:47:59.395 INFO:tasks.workunit.client.1.smithi179.stdout:2/622: dwrite dc/f5d [0,4194304] 0 2024-04-02T04:47:59.395 INFO:tasks.workunit.client.0.smithi070.stdout:2/721: dread - d2/d10/d5c/fbe zero size 2024-04-02T04:47:59.396 INFO:tasks.workunit.client.1.smithi179.stdout:4/766: write d1/d15/d7c/f84 [1507826,56871] 0 2024-04-02T04:47:59.398 INFO:tasks.workunit.client.0.smithi070.stdout:8/690: link d2/d8/d29/fba d2/d3/d35/d3a/fdf 0 2024-04-02T04:47:59.399 INFO:tasks.workunit.client.0.smithi070.stdout:8/691: chown d2/d3/d35/d3a/d4f/d59/fbd 55236 1 2024-04-02T04:47:59.400 INFO:tasks.workunit.client.1.smithi179.stdout:5/716: link d8/d1b/d23/d30/d73/lcd d8/d1b/d23/d36/le8 0 2024-04-02T04:47:59.401 INFO:tasks.workunit.client.0.smithi070.stdout:9/607: rename d4/d13/fc3 to d4/d10/d12/d27/d60/d9e/fc7 0 2024-04-02T04:47:59.402 INFO:tasks.workunit.client.1.smithi179.stdout:2/623: truncate dc/d29/d31/d57/fa0 273387 0 2024-04-02T04:47:59.402 INFO:tasks.workunit.client.0.smithi070.stdout:5/813: write d5/d9/fc9 [1025815,55859] 0 2024-04-02T04:47:59.403 INFO:tasks.workunit.client.1.smithi179.stdout:2/624: chown dc/d29/d31/d57/d97/dbf 340851887 1 2024-04-02T04:47:59.404 INFO:tasks.workunit.client.0.smithi070.stdout:5/814: write d5/fa [845932,55581] 0 2024-04-02T04:47:59.405 INFO:tasks.workunit.client.0.smithi070.stdout:6/553: link d0/d3/d19/d31/d46/fa9 d0/d55/d8b/faa 0 2024-04-02T04:47:59.408 INFO:tasks.workunit.client.1.smithi179.stdout:6/738: creat d3/d42/f102 x:0 0 0 2024-04-02T04:47:59.408 INFO:tasks.workunit.client.1.smithi179.stdout:9/735: dread d0/d1/d10/d6d/d58/f9d [0,4194304] 0 2024-04-02T04:47:59.411 INFO:tasks.workunit.client.0.smithi070.stdout:8/692: symlink d2/d8/d29/d72/le0 0 2024-04-02T04:47:59.413 INFO:tasks.workunit.client.0.smithi070.stdout:4/625: dwrite d1/d9c/f45 [0,4194304] 0 2024-04-02T04:47:59.413 INFO:tasks.workunit.client.0.smithi070.stdout:8/693: chown d2/d8/d23/d24/d79/l57 108 1 2024-04-02T04:47:59.417 INFO:tasks.workunit.client.0.smithi070.stdout:9/608: dread d4/d10/d12/d27/f97 [0,4194304] 0 2024-04-02T04:47:59.420 INFO:tasks.workunit.client.0.smithi070.stdout:6/554: rmdir d0/d3/d19/d31/d46/d6b 39 2024-04-02T04:47:59.421 INFO:tasks.workunit.client.0.smithi070.stdout:6/555: chown d0/d3/d19 789613964 1 2024-04-02T04:47:59.421 INFO:tasks.workunit.client.1.smithi179.stdout:8/752: getdents d3/d6c 0 2024-04-02T04:47:59.422 INFO:tasks.workunit.client.0.smithi070.stdout:2/722: creat d2/d10/d5c/d7a/db0/dc7/ded/ff3 x:0 0 0 2024-04-02T04:47:59.422 INFO:tasks.workunit.client.0.smithi070.stdout:5/815: dwrite d5/d17/d27/d2e/f9f [0,4194304] 0 2024-04-02T04:47:59.424 INFO:tasks.workunit.client.0.smithi070.stdout:9/609: dread d4/d10/d12/d27/d60/d9e/f77 [0,4194304] 0 2024-04-02T04:47:59.425 INFO:tasks.workunit.client.0.smithi070.stdout:9/610: chown d4/d10/d11/c4c 813800115 1 2024-04-02T04:47:59.425 INFO:tasks.workunit.client.1.smithi179.stdout:6/739: rmdir d3/d12/d6a 39 2024-04-02T04:47:59.426 INFO:tasks.workunit.client.0.smithi070.stdout:2/723: dread d2/d10/d21/d71/dd4/fb6 [0,4194304] 0 2024-04-02T04:47:59.430 INFO:tasks.workunit.client.1.smithi179.stdout:9/736: rename d0/d19/d27/d7a/d87/daf/dc2/lcc to d0/d19/d27/d7a/d7e/dd4/le2 0 2024-04-02T04:47:59.431 INFO:tasks.workunit.client.0.smithi070.stdout:6/556: dread - d0/d3/d19/d31/d46/fa9 zero size 2024-04-02T04:47:59.431 INFO:tasks.workunit.client.1.smithi179.stdout:8/753: dwrite d3/d31/ff1 [0,4194304] 0 2024-04-02T04:47:59.432 INFO:tasks.workunit.client.0.smithi070.stdout:2/724: dwrite d2/d12/d59/f8e [0,4194304] 0 2024-04-02T04:47:59.433 INFO:tasks.workunit.client.1.smithi179.stdout:8/754: read - d3/d31/dc3/fc6 zero size 2024-04-02T04:47:59.434 INFO:tasks.workunit.client.0.smithi070.stdout:5/816: rename d5/d17/d2b/f6f to d5/d14/f101 0 2024-04-02T04:47:59.436 INFO:tasks.workunit.client.0.smithi070.stdout:5/817: truncate d5/d17/d27/fe4 319953 0 2024-04-02T04:47:59.437 INFO:tasks.workunit.client.0.smithi070.stdout:4/626: fsync d1/f12 0 2024-04-02T04:47:59.441 INFO:tasks.workunit.client.1.smithi179.stdout:7/763: creat d1/d9/f10c x:0 0 0 2024-04-02T04:47:59.446 INFO:tasks.workunit.client.1.smithi179.stdout:8/755: readlink d3/d31/l54 0 2024-04-02T04:47:59.447 INFO:tasks.workunit.client.1.smithi179.stdout:5/717: link d8/d1b/d23/d42/c58 d8/d1b/d23/d32/d4e/d50/ce9 0 2024-04-02T04:47:59.449 INFO:tasks.workunit.client.0.smithi070.stdout:4/627: dwrite d1/d3/d78/d91/fa4 [0,4194304] 0 2024-04-02T04:47:59.450 INFO:tasks.workunit.client.1.smithi179.stdout:6/740: creat d3/d12/d23/d39/d80/d93/dd0/f103 x:0 0 0 2024-04-02T04:47:59.451 INFO:tasks.workunit.client.0.smithi070.stdout:9/611: dwrite d4/d10/d12/d27/d68/fb2 [0,4194304] 0 2024-04-02T04:47:59.453 INFO:tasks.workunit.client.0.smithi070.stdout:4/628: write d1/d3/d85/f92 [1315212,112937] 0 2024-04-02T04:47:59.453 INFO:tasks.workunit.client.1.smithi179.stdout:9/737: dread d0/d19/d1c/f47 [0,4194304] 0 2024-04-02T04:47:59.458 INFO:tasks.workunit.client.0.smithi070.stdout:8/694: creat d2/d8/d29/d22/fe1 x:0 0 0 2024-04-02T04:47:59.459 INFO:tasks.workunit.client.1.smithi179.stdout:5/718: symlink d8/d1b/d23/d30/lea 0 2024-04-02T04:47:59.461 INFO:tasks.workunit.client.1.smithi179.stdout:1/763: sync 2024-04-02T04:47:59.461 INFO:tasks.workunit.client.0.smithi070.stdout:8/695: dread d2/d8/d29/db/fc5 [0,4194304] 0 2024-04-02T04:47:59.463 INFO:tasks.workunit.client.1.smithi179.stdout:6/741: fsync d3/fa 0 2024-04-02T04:47:59.464 INFO:tasks.workunit.client.1.smithi179.stdout:7/764: symlink d1/dd/l10d 0 2024-04-02T04:47:59.467 INFO:tasks.workunit.client.1.smithi179.stdout:9/738: mkdir d0/d1/d10/de3 0 2024-04-02T04:47:59.471 INFO:tasks.workunit.client.1.smithi179.stdout:0/822: truncate d1/da/d67/fa8 5089970 0 2024-04-02T04:47:59.474 INFO:tasks.workunit.client.1.smithi179.stdout:1/764: write d3/d5/d2f/d4f/d42/d53/d77/f97 [1104682,59986] 0 2024-04-02T04:47:59.475 INFO:tasks.workunit.client.1.smithi179.stdout:1/765: read d3/d5/d2f/d4f/d42/d55/fa1 [48686,95601] 0 2024-04-02T04:47:59.476 INFO:tasks.workunit.client.1.smithi179.stdout:1/766: read - d3/d5/d2f/d4f/dab/fde zero size 2024-04-02T04:47:59.476 INFO:tasks.workunit.client.0.smithi070.stdout:4/629: mknod d1/d3/d85/ccc 0 2024-04-02T04:47:59.477 INFO:tasks.workunit.client.1.smithi179.stdout:6/742: symlink d3/d12/d23/d39/d80/d93/dd4/l104 0 2024-04-02T04:47:59.479 INFO:tasks.workunit.client.0.smithi070.stdout:8/696: creat d2/d8/d23/d24/d5f/fe2 x:0 0 0 2024-04-02T04:47:59.479 INFO:tasks.workunit.client.0.smithi070.stdout:0/741: sync 2024-04-02T04:47:59.480 INFO:tasks.workunit.client.1.smithi179.stdout:6/743: write d3/d2d/da5/fbd [1218782,123914] 0 2024-04-02T04:47:59.482 INFO:tasks.workunit.client.1.smithi179.stdout:9/739: mkdir d0/d19/d27/d7a/d87/de4 0 2024-04-02T04:47:59.482 INFO:tasks.workunit.client.0.smithi070.stdout:9/612: mknod d4/d10/d12/d55/cc8 0 2024-04-02T04:47:59.484 INFO:tasks.workunit.client.1.smithi179.stdout:2/625: dwrite dc/d29/d4a/d4b/f51 [4194304,4194304] 0 2024-04-02T04:47:59.485 INFO:tasks.workunit.client.0.smithi070.stdout:2/725: creat d2/ff4 x:0 0 0 2024-04-02T04:47:59.485 INFO:tasks.workunit.client.1.smithi179.stdout:9/740: chown d0/d19/d27/d48/f8c 27801402 1 2024-04-02T04:47:59.485 INFO:tasks.workunit.client.1.smithi179.stdout:3/766: dwrite d8/d26/d36/f5e [0,4194304] 0 2024-04-02T04:47:59.487 INFO:tasks.workunit.client.0.smithi070.stdout:5/818: creat d5/d9/d23/d30/da2/f102 x:0 0 0 2024-04-02T04:47:59.487 INFO:tasks.workunit.client.1.smithi179.stdout:9/741: chown d0/d1/d11/d29/d40/f67 32663310 1 2024-04-02T04:47:59.487 INFO:tasks.workunit.client.0.smithi070.stdout:5/819: chown d5/d9/d10 95843087 1 2024-04-02T04:47:59.488 INFO:tasks.workunit.client.0.smithi070.stdout:4/630: unlink d1/d3/d9/d5e/l83 0 2024-04-02T04:47:59.489 INFO:tasks.workunit.client.0.smithi070.stdout:6/557: write d0/d43/f51 [6533893,20959] 0 2024-04-02T04:47:59.491 INFO:tasks.workunit.client.1.smithi179.stdout:7/765: getdents d1/d9/d27/d87/d8b/d75/db9/ddd 0 2024-04-02T04:47:59.492 INFO:tasks.workunit.client.1.smithi179.stdout:7/766: chown d1/d9/d27/d87/d8b/d75/db9/dcf 61740004 1 2024-04-02T04:47:59.493 INFO:tasks.workunit.client.1.smithi179.stdout:7/767: dread - d1/d9/d35/d6b/fff zero size 2024-04-02T04:47:59.494 INFO:tasks.workunit.client.0.smithi070.stdout:8/697: symlink d2/d8/d29/d22/d5a/le3 0 2024-04-02T04:47:59.496 INFO:tasks.workunit.client.1.smithi179.stdout:1/767: mkdir d3/d5/d2f/d4f/d42/d55/dc5/d10a 0 2024-04-02T04:47:59.500 INFO:tasks.workunit.client.0.smithi070.stdout:9/613: stat d4/d10/d12/d56/d79/f85 0 2024-04-02T04:47:59.506 INFO:tasks.workunit.client.0.smithi070.stdout:6/558: readlink d0/l35 0 2024-04-02T04:47:59.507 INFO:tasks.workunit.client.1.smithi179.stdout:5/719: write d8/d1b/d23/d32/d46/f7d [264823,79291] 0 2024-04-02T04:47:59.507 INFO:tasks.workunit.client.1.smithi179.stdout:8/756: link d3/d32/d21/l3f d3/d32/l10b 0 2024-04-02T04:47:59.509 INFO:tasks.workunit.client.1.smithi179.stdout:0/823: symlink d1/da/d48/d5c/lee 0 2024-04-02T04:47:59.509 INFO:tasks.workunit.client.1.smithi179.stdout:6/744: dread f2 [0,4194304] 0 2024-04-02T04:47:59.512 INFO:tasks.workunit.client.0.smithi070.stdout:0/742: rename d4/d5/d13/d23/c42 to d4/d5/d21/d6b/ce3 0 2024-04-02T04:47:59.514 INFO:tasks.workunit.client.0.smithi070.stdout:9/614: mkdir d4/d10/d12/d27/d60/dc9 0 2024-04-02T04:47:59.515 INFO:tasks.workunit.client.0.smithi070.stdout:9/615: truncate d4/d13/d89/faf 4957913 0 2024-04-02T04:47:59.516 INFO:tasks.workunit.client.1.smithi179.stdout:9/742: mknod d0/d1/d10/ce5 0 2024-04-02T04:47:59.516 INFO:tasks.workunit.client.1.smithi179.stdout:4/767: sync 2024-04-02T04:47:59.518 INFO:tasks.workunit.client.1.smithi179.stdout:7/768: creat d1/d9/d35/d6b/f10e x:0 0 0 2024-04-02T04:47:59.528 INFO:tasks.workunit.client.1.smithi179.stdout:0/824: mknod d1/da/d28/d34/d9c/cef 0 2024-04-02T04:47:59.529 INFO:tasks.workunit.client.0.smithi070.stdout:5/820: dwrite d5/d9/f5e [0,4194304] 0 2024-04-02T04:47:59.533 INFO:tasks.workunit.client.1.smithi179.stdout:9/743: readlink d0/d1/d11/d29/d86/l99 0 2024-04-02T04:47:59.533 INFO:tasks.workunit.client.0.smithi070.stdout:4/631: link d1/d3/d85/d8e/cb3 d1/d3/d85/d8e/d99/ccd 0 2024-04-02T04:47:59.535 INFO:tasks.workunit.client.0.smithi070.stdout:4/632: write d1/d9c/f79 [2067836,105905] 0 2024-04-02T04:47:59.535 INFO:tasks.workunit.client.1.smithi179.stdout:4/768: mkdir d1/d3/dc/d27/d62/d100 0 2024-04-02T04:47:59.536 INFO:tasks.workunit.client.0.smithi070.stdout:4/633: chown d1/d3/d9/f8b 69 1 2024-04-02T04:47:59.537 INFO:tasks.workunit.client.0.smithi070.stdout:8/698: creat d2/d8/d29/fe4 x:0 0 0 2024-04-02T04:47:59.538 INFO:tasks.workunit.client.0.smithi070.stdout:4/634: dread - d1/d3/d9/fb7 zero size 2024-04-02T04:47:59.540 INFO:tasks.workunit.client.0.smithi070.stdout:4/635: readlink d1/d3/d9/d5e/lb8 0 2024-04-02T04:47:59.540 INFO:tasks.workunit.client.0.smithi070.stdout:0/743: fdatasync d4/d5/d58/d60/f81 0 2024-04-02T04:47:59.541 INFO:tasks.workunit.client.0.smithi070.stdout:2/726: dwrite d2/d10/d19/f39 [0,4194304] 0 2024-04-02T04:47:59.542 INFO:tasks.workunit.client.1.smithi179.stdout:1/768: truncate d3/d5/d2f/d4f/d42/d53/f5e 354507 0 2024-04-02T04:47:59.542 INFO:tasks.workunit.client.0.smithi070.stdout:4/636: chown d1/d3/l97 46444575 1 2024-04-02T04:47:59.542 INFO:tasks.workunit.client.0.smithi070.stdout:6/559: write d0/d55/d8b/f9e [452848,71512] 0 2024-04-02T04:47:59.546 INFO:tasks.workunit.client.1.smithi179.stdout:1/769: dread d3/d5/d2f/d4f/f4c [0,4194304] 0 2024-04-02T04:47:59.546 INFO:tasks.workunit.client.0.smithi070.stdout:5/821: creat d5/d9/d10/db9/dd9/f103 x:0 0 0 2024-04-02T04:47:59.553 INFO:tasks.workunit.client.1.smithi179.stdout:6/745: mknod d3/d12/d23/d39/d4e/c105 0 2024-04-02T04:47:59.553 INFO:tasks.workunit.client.1.smithi179.stdout:6/746: fdatasync d3/d42/dbe/fca 0 2024-04-02T04:47:59.554 INFO:tasks.workunit.client.0.smithi070.stdout:6/560: dwrite d0/f78 [4194304,4194304] 0 2024-04-02T04:47:59.554 INFO:tasks.workunit.client.1.smithi179.stdout:8/757: truncate d3/f76 4949373 0 2024-04-02T04:47:59.554 INFO:tasks.workunit.client.0.smithi070.stdout:2/727: dwrite d2/d10/d21/d3d/d42/fad [0,4194304] 0 2024-04-02T04:47:59.555 INFO:tasks.workunit.client.1.smithi179.stdout:8/758: dread - d3/d32/d21/d50/d56/d94/fe7 zero size 2024-04-02T04:47:59.557 INFO:tasks.workunit.client.1.smithi179.stdout:0/825: write d1/da/df/f20 [2835540,57068] 0 2024-04-02T04:47:59.558 INFO:tasks.workunit.client.1.smithi179.stdout:0/826: chown d1/f36 1982430 1 2024-04-02T04:47:59.566 INFO:tasks.workunit.client.0.smithi070.stdout:2/728: dwrite d2/d10/d21/d3d/d42/fad [0,4194304] 0 2024-04-02T04:47:59.567 INFO:tasks.workunit.client.1.smithi179.stdout:9/744: read d0/d1/f36 [2476943,51990] 0 2024-04-02T04:47:59.568 INFO:tasks.workunit.client.1.smithi179.stdout:4/769: mkdir d1/d93/d101 0 2024-04-02T04:47:59.569 INFO:tasks.workunit.client.1.smithi179.stdout:4/770: chown d1/d3/df/f8f 5419 1 2024-04-02T04:47:59.570 INFO:tasks.workunit.client.1.smithi179.stdout:5/720: sync 2024-04-02T04:47:59.574 INFO:tasks.workunit.client.1.smithi179.stdout:7/769: fsync d1/f15 0 2024-04-02T04:47:59.574 INFO:tasks.workunit.client.0.smithi070.stdout:0/744: write d4/d5/f84 [500161,63331] 0 2024-04-02T04:47:59.575 INFO:tasks.workunit.client.1.smithi179.stdout:1/770: dread - d3/d85/da2/fb5 zero size 2024-04-02T04:47:59.576 INFO:tasks.workunit.client.0.smithi070.stdout:6/561: symlink d0/d20/lab 0 2024-04-02T04:47:59.577 INFO:tasks.workunit.client.1.smithi179.stdout:2/626: getdents dc/d29/d4a/d1e 0 2024-04-02T04:47:59.578 INFO:tasks.workunit.client.0.smithi070.stdout:5/822: write d5/f56 [1274232,127410] 0 2024-04-02T04:47:59.579 INFO:tasks.workunit.client.1.smithi179.stdout:6/747: rmdir d3/d12/d23/d39/d4e/d6c/dcd 39 2024-04-02T04:47:59.580 INFO:tasks.workunit.client.0.smithi070.stdout:5/823: write d5/d17/ffd [772923,49652] 0 2024-04-02T04:47:59.581 INFO:tasks.workunit.client.1.smithi179.stdout:8/759: mkdir d3/d31/dc3/d10c 0 2024-04-02T04:47:59.582 INFO:tasks.workunit.client.0.smithi070.stdout:6/562: dread d0/d55/d8b/d94/f53 [0,4194304] 0 2024-04-02T04:47:59.583 INFO:tasks.workunit.client.1.smithi179.stdout:3/767: getdents d8 0 2024-04-02T04:47:59.583 INFO:tasks.workunit.client.1.smithi179.stdout:3/768: stat d8/d6d/d94/f95 0 2024-04-02T04:47:59.583 INFO:tasks.workunit.client.0.smithi070.stdout:4/637: mkdir d1/dce 0 2024-04-02T04:47:59.584 INFO:tasks.workunit.client.0.smithi070.stdout:4/638: stat d1/d3/c59 0 2024-04-02T04:47:59.584 INFO:tasks.workunit.client.0.smithi070.stdout:5/824: dread d5/d9/d10/db9/dba/de1/ff4 [0,4194304] 0 2024-04-02T04:47:59.586 INFO:tasks.workunit.client.0.smithi070.stdout:2/729: creat d2/d24/ff5 x:0 0 0 2024-04-02T04:47:59.589 INFO:tasks.workunit.client.1.smithi179.stdout:3/769: dwrite d8/d26/d83/fa0 [0,4194304] 0 2024-04-02T04:47:59.589 INFO:tasks.workunit.client.0.smithi070.stdout:8/699: creat d2/d8/fe5 x:0 0 0 2024-04-02T04:47:59.590 INFO:tasks.workunit.client.0.smithi070.stdout:0/745: creat d4/d5/d58/d60/d83/fe4 x:0 0 0 2024-04-02T04:47:59.593 INFO:tasks.workunit.client.1.smithi179.stdout:7/770: truncate d1/dd/d23/fee 2224360 0 2024-04-02T04:47:59.594 INFO:tasks.workunit.client.1.smithi179.stdout:7/771: dread - d1/d9/d55/dbf/d9f/da8/d104/f106 zero size 2024-04-02T04:47:59.595 INFO:tasks.workunit.client.0.smithi070.stdout:5/825: rmdir d5/d9/d23/d30/da2/d75 39 2024-04-02T04:47:59.597 INFO:tasks.workunit.client.0.smithi070.stdout:2/730: symlink d2/d10/d21/d71/dd4/dce/lf6 0 2024-04-02T04:47:59.599 INFO:tasks.workunit.client.0.smithi070.stdout:8/700: truncate d2/d8/d23/d24/d5f/f4c 213110 0 2024-04-02T04:47:59.599 INFO:tasks.workunit.client.1.smithi179.stdout:1/771: truncate d3/d85/da2/fb5 412277 0 2024-04-02T04:47:59.599 INFO:tasks.workunit.client.1.smithi179.stdout:2/627: truncate dc/d29/d4a/f56 739715 0 2024-04-02T04:47:59.600 INFO:tasks.workunit.client.1.smithi179.stdout:9/745: dwrite d0/d1/d11/d29/d86/f9e [0,4194304] 0 2024-04-02T04:47:59.600 INFO:tasks.workunit.client.1.smithi179.stdout:2/628: stat dc/d29/d4a/d1e/cd1 0 2024-04-02T04:47:59.604 INFO:tasks.workunit.client.0.smithi070.stdout:0/746: mknod d4/d5/d58/d60/d83/ce5 0 2024-04-02T04:47:59.605 INFO:tasks.workunit.client.0.smithi070.stdout:0/747: fdatasync d4/d5/d21/d6b/fe0 0 2024-04-02T04:47:59.606 INFO:tasks.workunit.client.0.smithi070.stdout:8/701: dwrite d2/d8/d29/d22/fe1 [0,4194304] 0 2024-04-02T04:47:59.610 INFO:tasks.workunit.client.0.smithi070.stdout:4/639: rename d1/c84 to d1/d3/d85/d8e/ccf 0 2024-04-02T04:47:59.613 INFO:tasks.workunit.client.1.smithi179.stdout:0/827: rename d1/da/d48/d5c/f78 to d1/da/d48/d7b/ff0 0 2024-04-02T04:47:59.614 INFO:tasks.workunit.client.1.smithi179.stdout:0/828: write d1/da/d28/d34/f7e [1094986,115061] 0 2024-04-02T04:47:59.614 INFO:tasks.workunit.client.0.smithi070.stdout:5/826: mknod d5/d17/d6c/c104 0 2024-04-02T04:47:59.617 INFO:tasks.workunit.client.1.smithi179.stdout:4/771: mkdir d1/d3/dc/d27/dae/de5/d102 0 2024-04-02T04:47:59.618 INFO:tasks.workunit.client.0.smithi070.stdout:8/702: dwrite d2/d8/d29/d22/d5a/d94/fc0 [0,4194304] 0 2024-04-02T04:47:59.622 INFO:tasks.workunit.client.1.smithi179.stdout:3/770: readlink d8/d26/d36/l42 0 2024-04-02T04:47:59.627 INFO:tasks.workunit.client.0.smithi070.stdout:8/703: dread d2/d3/f53 [0,4194304] 0 2024-04-02T04:47:59.627 INFO:tasks.workunit.client.1.smithi179.stdout:5/721: truncate f2 5298293 0 2024-04-02T04:47:59.628 INFO:tasks.workunit.client.0.smithi070.stdout:6/563: dwrite d0/d43/d95/f9a [0,4194304] 0 2024-04-02T04:47:59.629 INFO:tasks.workunit.client.1.smithi179.stdout:5/722: truncate d8/d1b/d23/d32/d4e/d50/f80 4257112 0 2024-04-02T04:47:59.634 INFO:tasks.workunit.client.0.smithi070.stdout:6/564: read d0/d3/d19/f47 [5513535,117102] 0 2024-04-02T04:47:59.636 INFO:tasks.workunit.client.0.smithi070.stdout:4/640: dread d1/d3/d9/d5e/f77 [0,4194304] 0 2024-04-02T04:47:59.636 INFO:tasks.workunit.client.1.smithi179.stdout:7/772: symlink d1/dd/d76/l10f 0 2024-04-02T04:47:59.637 INFO:tasks.workunit.client.1.smithi179.stdout:5/723: dwrite d8/d1b/d23/d32/d4e/d51/fc7 [0,4194304] 0 2024-04-02T04:47:59.641 INFO:tasks.workunit.client.1.smithi179.stdout:9/746: symlink d0/d19/d27/d7a/d7e/dd4/le6 0 2024-04-02T04:47:59.643 INFO:tasks.workunit.client.1.smithi179.stdout:6/748: symlink d3/d12/d44/df1/l106 0 2024-04-02T04:47:59.644 INFO:tasks.workunit.client.0.smithi070.stdout:0/748: rename d4/d5/d21/f9b to d4/d5/d13/d2a/d3a/db3/fe6 0 2024-04-02T04:47:59.645 INFO:tasks.workunit.client.1.smithi179.stdout:1/772: dread d3/d5/d2f/d4f/f4c [0,4194304] 0 2024-04-02T04:47:59.646 INFO:tasks.workunit.client.0.smithi070.stdout:5/827: dread d5/d9/d10/d6a/d7d/fc8 [0,4194304] 0 2024-04-02T04:47:59.647 INFO:tasks.workunit.client.1.smithi179.stdout:8/760: creat d3/d1b/d2a/d41/d109/f10d x:0 0 0 2024-04-02T04:47:59.650 INFO:tasks.workunit.client.0.smithi070.stdout:4/641: unlink d1/d3/d78/d91/faf 0 2024-04-02T04:47:59.652 INFO:tasks.workunit.client.1.smithi179.stdout:4/772: creat d1/d15/d6e/f103 x:0 0 0 2024-04-02T04:47:59.655 INFO:tasks.workunit.client.0.smithi070.stdout:6/565: rename d0/l35 to d0/da2/lac 0 2024-04-02T04:47:59.655 INFO:tasks.workunit.client.0.smithi070.stdout:4/642: dwrite d1/f3f [0,4194304] 0 2024-04-02T04:47:59.656 INFO:tasks.workunit.client.0.smithi070.stdout:6/566: read d0/d43/f51 [5612820,1181] 0 2024-04-02T04:47:59.657 INFO:tasks.workunit.client.0.smithi070.stdout:0/749: mkdir d4/d5/da/d2d/de7 0 2024-04-02T04:47:59.659 INFO:tasks.workunit.client.1.smithi179.stdout:7/773: symlink d1/d9/d27/d87/d8b/l110 0 2024-04-02T04:47:59.662 INFO:tasks.workunit.client.1.smithi179.stdout:6/749: unlink d3/d12/d23/d39/c81 0 2024-04-02T04:47:59.662 INFO:tasks.workunit.client.0.smithi070.stdout:2/731: getdents d2/d10/d5c/d7a/db0/de4 0 2024-04-02T04:47:59.663 INFO:tasks.workunit.client.0.smithi070.stdout:2/732: chown d2/d10/d5c/d7a/fab 1747413 1 2024-04-02T04:47:59.665 INFO:tasks.workunit.client.0.smithi070.stdout:2/733: write d2/d10/d21/d3d/d42/fad [3025411,62004] 0 2024-04-02T04:47:59.667 INFO:tasks.workunit.client.1.smithi179.stdout:7/774: dwrite d1/d9/d55/fd7 [0,4194304] 0 2024-04-02T04:47:59.672 INFO:tasks.workunit.client.0.smithi070.stdout:6/567: creat d0/d55/d5c/fad x:0 0 0 2024-04-02T04:47:59.675 INFO:tasks.workunit.client.0.smithi070.stdout:0/750: symlink d4/d5/d13/d2a/d3a/le8 0 2024-04-02T04:47:59.677 INFO:tasks.workunit.client.1.smithi179.stdout:0/829: mknod d1/da/d48/d65/cf1 0 2024-04-02T04:47:59.680 INFO:tasks.workunit.client.1.smithi179.stdout:3/771: mknod d8/d57/d8f/cf1 0 2024-04-02T04:47:59.680 INFO:tasks.workunit.client.1.smithi179.stdout:3/772: stat d8/d26/d52/fb4 0 2024-04-02T04:47:59.682 INFO:tasks.workunit.client.0.smithi070.stdout:4/643: dread d1/d3/d9/f9b [0,4194304] 0 2024-04-02T04:47:59.682 INFO:tasks.workunit.client.0.smithi070.stdout:2/734: chown d2/d12/d59/ccc 48 1 2024-04-02T04:47:59.683 INFO:tasks.workunit.client.0.smithi070.stdout:3/697: sync 2024-04-02T04:47:59.683 INFO:tasks.workunit.client.0.smithi070.stdout:7/637: sync 2024-04-02T04:47:59.684 INFO:tasks.workunit.client.0.smithi070.stdout:3/698: chown d10/d32/d4b/c5a 233499 1 2024-04-02T04:47:59.685 INFO:tasks.workunit.client.0.smithi070.stdout:3/699: dread - d10/d17/d4c/fda zero size 2024-04-02T04:47:59.688 INFO:tasks.workunit.client.0.smithi070.stdout:5/828: rename d5/d9/d10/d6a/d7d/dc4 to d5/d105 0 2024-04-02T04:47:59.688 INFO:tasks.workunit.client.0.smithi070.stdout:5/829: readlink d5/l21 0 2024-04-02T04:47:59.689 INFO:tasks.workunit.client.1.smithi179.stdout:1/773: mknod d3/d5/d2f/d4f/d42/d55/dc5/d10a/c10b 0 2024-04-02T04:47:59.690 INFO:tasks.workunit.client.0.smithi070.stdout:7/638: dwrite d7/dd/f8d [4194304,4194304] 0 2024-04-02T04:47:59.697 INFO:tasks.workunit.client.0.smithi070.stdout:5/830: dwrite d5/d17/d50/f5a [0,4194304] 0 2024-04-02T04:47:59.697 INFO:tasks.workunit.client.1.smithi179.stdout:1/774: dread d3/d5/d2f/d4f/d42/df2/f8b [0,4194304] 0 2024-04-02T04:47:59.698 INFO:tasks.workunit.client.0.smithi070.stdout:5/831: dread - d5/d9/d23/d30/da2/fdd zero size 2024-04-02T04:47:59.699 INFO:tasks.workunit.client.0.smithi070.stdout:5/832: dread - d5/d9/d23/dc3/fd0 zero size 2024-04-02T04:47:59.704 INFO:tasks.workunit.client.1.smithi179.stdout:7/775: getdents d1/d9/d27/d87/d8b/d75/db9/ddd 0 2024-04-02T04:47:59.705 INFO:tasks.workunit.client.1.smithi179.stdout:7/776: chown d1/f15 56 1 2024-04-02T04:47:59.706 INFO:tasks.workunit.client.1.smithi179.stdout:9/747: truncate d0/d1/d11/d29/d86/f9e 912287 0 2024-04-02T04:47:59.707 INFO:tasks.workunit.client.0.smithi070.stdout:2/735: truncate d2/d10/d53/f55 266816 0 2024-04-02T04:47:59.707 INFO:tasks.workunit.client.0.smithi070.stdout:6/568: write d0/d55/d8b/d94/f53 [742474,38521] 0 2024-04-02T04:47:59.709 INFO:tasks.workunit.client.1.smithi179.stdout:2/629: rename dc/ld0 to dc/d29/d31/ld3 0 2024-04-02T04:47:59.712 INFO:tasks.workunit.client.1.smithi179.stdout:9/748: dwrite d0/d1/d11/d77/f74 [0,4194304] 0 2024-04-02T04:47:59.718 INFO:tasks.workunit.client.0.smithi070.stdout:6/569: creat d0/d43/fae x:0 0 0 2024-04-02T04:47:59.722 INFO:tasks.workunit.client.0.smithi070.stdout:5/833: dwrite d5/d9/d10/d6a/d7d/fc8 [0,4194304] 0 2024-04-02T04:47:59.725 INFO:tasks.workunit.client.1.smithi179.stdout:8/761: getdents d3/d32/d21/d51 0 2024-04-02T04:47:59.725 INFO:tasks.workunit.client.0.smithi070.stdout:0/751: rename d4/d5/d13/d2a/d3a/c3e to d4/d5/d13/ce9 0 2024-04-02T04:47:59.725 INFO:tasks.workunit.client.0.smithi070.stdout:4/644: dwrite d1/f12 [4194304,4194304] 0 2024-04-02T04:47:59.727 INFO:tasks.workunit.client.0.smithi070.stdout:3/700: dwrite d10/d32/d3f/f65 [0,4194304] 0 2024-04-02T04:47:59.738 INFO:tasks.workunit.client.1.smithi179.stdout:5/724: rename d8/d1b/d23/d32/d46/l8c to d8/d1b/d23/d36/leb 0 2024-04-02T04:47:59.738 INFO:tasks.workunit.client.1.smithi179.stdout:0/830: write d1/da/d28/d37/f49 [2185375,38609] 0 2024-04-02T04:47:59.740 INFO:tasks.workunit.client.1.smithi179.stdout:0/831: chown d1/da/l19 126523723 1 2024-04-02T04:47:59.740 INFO:tasks.workunit.client.1.smithi179.stdout:5/725: dread - d8/d1b/d23/d32/d46/fdc zero size 2024-04-02T04:47:59.741 INFO:tasks.workunit.client.0.smithi070.stdout:5/834: rmdir d5/d9/d10/db9 39 2024-04-02T04:47:59.741 INFO:tasks.workunit.client.0.smithi070.stdout:3/701: fsync d10/d32/d8a/d8d/d91/f94 0 2024-04-02T04:47:59.742 INFO:tasks.workunit.client.0.smithi070.stdout:4/645: mknod d1/d3/d78/cd0 0 2024-04-02T04:47:59.745 INFO:tasks.workunit.client.0.smithi070.stdout:2/736: getdents d2/d10/d5c/d7a/db0/dd8/dd2/d90 0 2024-04-02T04:47:59.749 INFO:tasks.workunit.client.1.smithi179.stdout:4/773: rename d1/d3/dc0/ffd to d1/d3/dc/d27/f104 0 2024-04-02T04:47:59.749 INFO:tasks.workunit.client.1.smithi179.stdout:0/832: dwrite d1/da/d28/d34/d39/d77/f9b [0,4194304] 0 2024-04-02T04:47:59.750 INFO:tasks.workunit.client.0.smithi070.stdout:3/702: mknod d10/d32/d3f/dc0/cf0 0 2024-04-02T04:47:59.753 INFO:tasks.workunit.client.0.smithi070.stdout:5/835: rename d5/d9/d23/f48 to d5/d9/d23/f106 0 2024-04-02T04:47:59.754 INFO:tasks.workunit.client.0.smithi070.stdout:5/836: dread - d5/d17/d27/f98 zero size 2024-04-02T04:47:59.755 INFO:tasks.workunit.client.1.smithi179.stdout:3/773: getdents d8/d57 0 2024-04-02T04:47:59.758 INFO:tasks.workunit.client.1.smithi179.stdout:5/726: mkdir d8/d1b/d23/d32/dbe/daf/dec 0 2024-04-02T04:47:59.759 INFO:tasks.workunit.client.1.smithi179.stdout:5/727: fdatasync d8/d1b/d23/d32/d4e/fb8 0 2024-04-02T04:47:59.760 INFO:tasks.workunit.client.1.smithi179.stdout:2/630: dread dc/d29/d4a/d1e/d43/f47 [0,4194304] 0 2024-04-02T04:47:59.762 INFO:tasks.workunit.client.1.smithi179.stdout:7/777: creat d1/dd/f111 x:0 0 0 2024-04-02T04:47:59.763 INFO:tasks.workunit.client.1.smithi179.stdout:7/778: write d1/d9/d27/f49 [688069,25537] 0 2024-04-02T04:47:59.766 INFO:tasks.workunit.client.0.smithi070.stdout:6/570: dread d0/d3/d19/d31/d61/f66 [0,4194304] 0 2024-04-02T04:47:59.768 INFO:tasks.workunit.client.0.smithi070.stdout:6/571: chown d0/d55/d8b/d94/f53 9757 1 2024-04-02T04:47:59.768 INFO:tasks.workunit.client.1.smithi179.stdout:8/762: truncate d3/d32/d21/d49/fb9 311864 0 2024-04-02T04:47:59.769 INFO:tasks.workunit.client.0.smithi070.stdout:6/572: readlink d0/d55/la5 0 2024-04-02T04:47:59.771 INFO:tasks.workunit.client.1.smithi179.stdout:4/774: dread d1/d3/dc/d26/f29 [0,4194304] 0 2024-04-02T04:47:59.771 INFO:tasks.workunit.client.0.smithi070.stdout:0/752: dwrite d4/d5/d13/d43/f6f [0,4194304] 0 2024-04-02T04:47:59.771 INFO:tasks.workunit.client.0.smithi070.stdout:4/646: fdatasync d1/d3/d78/dad/dbd/fc9 0 2024-04-02T04:47:59.775 INFO:tasks.workunit.client.1.smithi179.stdout:9/749: dwrite d0/d19/d1c/d64/fb7 [0,4194304] 0 2024-04-02T04:47:59.778 INFO:tasks.workunit.client.0.smithi070.stdout:4/647: read d1/d3/d85/f92 [174995,113326] 0 2024-04-02T04:47:59.779 INFO:tasks.workunit.client.1.smithi179.stdout:3/774: symlink d8/d26/d83/dbc/lf2 0 2024-04-02T04:47:59.779 INFO:tasks.workunit.client.0.smithi070.stdout:5/837: mknod d5/d9/d23/d30/da2/d75/ddf/c107 0 2024-04-02T04:47:59.780 INFO:tasks.workunit.client.1.smithi179.stdout:0/833: dread d1/da/d48/d65/d7d/d80/fa5 [0,4194304] 0 2024-04-02T04:47:59.781 INFO:tasks.workunit.client.1.smithi179.stdout:4/775: dwrite d1/d3/d19/fa3 [0,4194304] 0 2024-04-02T04:47:59.785 INFO:tasks.workunit.client.1.smithi179.stdout:7/779: mkdir d1/d9/d27/d87/d8b/d112 0 2024-04-02T04:47:59.789 INFO:tasks.workunit.client.0.smithi070.stdout:5/838: dwrite d5/d9/d23/d8b/de9/fea [0,4194304] 0 2024-04-02T04:47:59.789 INFO:tasks.workunit.client.0.smithi070.stdout:2/737: rmdir d2/d24/de7 0 2024-04-02T04:47:59.796 INFO:tasks.workunit.client.0.smithi070.stdout:3/703: rmdir d10/d32/dd1/dee 0 2024-04-02T04:47:59.797 INFO:tasks.workunit.client.1.smithi179.stdout:1/775: rename d3/d5/c4d to d3/d5/d2f/d4f/d42/d55/c10c 0 2024-04-02T04:47:59.798 INFO:tasks.workunit.client.1.smithi179.stdout:9/750: symlink d0/d1/d10/d6d/d58/le7 0 2024-04-02T04:47:59.800 INFO:tasks.workunit.client.0.smithi070.stdout:2/738: dwrite d2/fb5 [0,4194304] 0 2024-04-02T04:47:59.802 INFO:tasks.workunit.client.1.smithi179.stdout:3/775: creat d8/d6d/ff3 x:0 0 0 2024-04-02T04:47:59.804 INFO:tasks.workunit.client.1.smithi179.stdout:3/776: dread - d8/d26/d83/d89/d9c/dcc/fe2 zero size 2024-04-02T04:47:59.808 INFO:tasks.workunit.client.0.smithi070.stdout:4/648: creat d1/dce/fd1 x:0 0 0 2024-04-02T04:47:59.810 INFO:tasks.workunit.client.1.smithi179.stdout:6/750: rename d3/d12/d23/d39/d4e/d6c/f50 to d3/d2d/d43/dd7/f107 0 2024-04-02T04:47:59.812 INFO:tasks.workunit.client.1.smithi179.stdout:8/763: dread d3/d32/d21/d50/d56/f9c [0,4194304] 0 2024-04-02T04:47:59.812 INFO:tasks.workunit.client.0.smithi070.stdout:6/573: creat d0/d3/d19/d31/faf x:0 0 0 2024-04-02T04:47:59.813 INFO:tasks.workunit.client.1.smithi179.stdout:8/764: read - d3/d32/d21/d22/d6b/d91/da8/dfe/f100 zero size 2024-04-02T04:47:59.814 INFO:tasks.workunit.client.1.smithi179.stdout:5/728: creat d8/d1b/d23/db1/fed x:0 0 0 2024-04-02T04:47:59.815 INFO:tasks.workunit.client.1.smithi179.stdout:5/729: write d8/d1b/d23/d42/fcc [3500942,120133] 0 2024-04-02T04:47:59.815 INFO:tasks.workunit.client.0.smithi070.stdout:0/753: creat d4/d5/d13/d2a/fea x:0 0 0 2024-04-02T04:47:59.817 INFO:tasks.workunit.client.1.smithi179.stdout:2/631: creat dc/fd4 x:0 0 0 2024-04-02T04:47:59.819 INFO:tasks.workunit.client.0.smithi070.stdout:2/739: truncate d2/d10/d5c/d7a/fda 1368849 0 2024-04-02T04:47:59.821 INFO:tasks.workunit.client.1.smithi179.stdout:4/776: creat d1/d93/d101/f105 x:0 0 0 2024-04-02T04:47:59.823 INFO:tasks.workunit.client.1.smithi179.stdout:3/777: creat d8/d57/ff4 x:0 0 0 2024-04-02T04:47:59.823 INFO:tasks.workunit.client.1.smithi179.stdout:9/751: dread d0/d1/d11/d29/d40/f5d [0,4194304] 0 2024-04-02T04:47:59.824 INFO:tasks.workunit.client.0.smithi070.stdout:5/839: dread d5/d17/d2b/d7b/ff9 [0,4194304] 0 2024-04-02T04:47:59.824 INFO:tasks.workunit.client.1.smithi179.stdout:3/778: fdatasync d8/d26/d36/f49 0 2024-04-02T04:47:59.824 INFO:tasks.workunit.client.1.smithi179.stdout:9/752: chown d0/d19/d27/fe0 30 1 2024-04-02T04:47:59.825 INFO:tasks.workunit.client.1.smithi179.stdout:9/753: chown d0/d1/f7 166 1 2024-04-02T04:47:59.828 INFO:tasks.workunit.client.0.smithi070.stdout:6/574: write d0/d3/d19/d31/d46/f93 [828504,23645] 0 2024-04-02T04:47:59.830 INFO:tasks.workunit.client.1.smithi179.stdout:0/834: rename d1/db/l31 to d1/da/d52/d75/dc0/lf2 0 2024-04-02T04:47:59.831 INFO:tasks.workunit.client.0.smithi070.stdout:0/754: creat d4/d5/d58/d60/d83/feb x:0 0 0 2024-04-02T04:47:59.831 INFO:tasks.workunit.client.1.smithi179.stdout:8/765: dread d3/d1b/fa2 [0,4194304] 0 2024-04-02T04:47:59.832 INFO:tasks.workunit.client.1.smithi179.stdout:3/779: dwrite d8/d26/d52/d7e/db8/fde [0,4194304] 0 2024-04-02T04:47:59.832 INFO:tasks.workunit.client.1.smithi179.stdout:0/835: write d1/da/df/f8a [4506156,114856] 0 2024-04-02T04:47:59.832 INFO:tasks.workunit.client.1.smithi179.stdout:6/751: mkdir d3/d42/d108 0 2024-04-02T04:47:59.838 INFO:tasks.workunit.client.0.smithi070.stdout:2/740: readlink d2/d10/d21/l46 0 2024-04-02T04:47:59.839 INFO:tasks.workunit.client.0.smithi070.stdout:3/704: link d10/d32/d3f/f38 d10/ff1 0 2024-04-02T04:47:59.841 INFO:tasks.workunit.client.0.smithi070.stdout:2/741: dread - d2/d10/d21/d71/fae zero size 2024-04-02T04:47:59.842 INFO:tasks.workunit.client.0.smithi070.stdout:6/575: dwrite d0/d3/d19/d31/d46/f9b [0,4194304] 0 2024-04-02T04:47:59.843 INFO:tasks.workunit.client.0.smithi070.stdout:2/742: dread - d2/d24/ff5 zero size 2024-04-02T04:47:59.843 INFO:tasks.workunit.client.0.smithi070.stdout:3/705: dread d10/d17/db2/fe3 [0,4194304] 0 2024-04-02T04:47:59.844 INFO:tasks.workunit.client.0.smithi070.stdout:5/840: rename d5/d14/c6b to d5/d9/d23/dc3/c108 0 2024-04-02T04:47:59.845 INFO:tasks.workunit.client.0.smithi070.stdout:3/706: fsync d10/f29 0 2024-04-02T04:47:59.845 INFO:tasks.workunit.client.1.smithi179.stdout:5/730: rmdir d8/d1b/d23/d32/dbe/daf 39 2024-04-02T04:47:59.849 INFO:tasks.workunit.client.1.smithi179.stdout:2/632: mknod dc/d29/d4a/d1e/d8a/cd5 0 2024-04-02T04:47:59.849 INFO:tasks.workunit.client.0.smithi070.stdout:2/743: dread d2/d12/d59/f8e [0,4194304] 0 2024-04-02T04:47:59.864 INFO:tasks.workunit.client.0.smithi070.stdout:1/664: sync 2024-04-02T04:47:59.864 INFO:tasks.workunit.client.0.smithi070.stdout:9/616: sync 2024-04-02T04:47:59.866 INFO:tasks.workunit.client.1.smithi179.stdout:7/780: rename d1/d9/d35/fdc to d1/dd/d23/f113 0 2024-04-02T04:47:59.869 INFO:tasks.workunit.client.0.smithi070.stdout:9/617: dwrite d4/d13/f98 [0,4194304] 0 2024-04-02T04:47:59.871 INFO:tasks.workunit.client.0.smithi070.stdout:9/618: chown d4/d34/f7a 85830 1 2024-04-02T04:47:59.873 INFO:tasks.workunit.client.1.smithi179.stdout:8/766: mknod d3/d32/d21/d50/d56/c10e 0 2024-04-02T04:47:59.876 INFO:tasks.workunit.client.1.smithi179.stdout:0/836: mkdir d1/da/df/df3 0 2024-04-02T04:47:59.878 INFO:tasks.workunit.client.1.smithi179.stdout:6/752: creat d3/d2d/d38/d64/f109 x:0 0 0 2024-04-02T04:47:59.882 INFO:tasks.workunit.client.1.smithi179.stdout:2/633: creat dc/d29/d31/d83/dae/fd6 x:0 0 0 2024-04-02T04:47:59.883 INFO:tasks.workunit.client.1.smithi179.stdout:4/777: mkdir d1/d3/df/d106 0 2024-04-02T04:47:59.884 INFO:tasks.workunit.client.1.smithi179.stdout:1/776: fsync d3/d5/d96/fa7 0 2024-04-02T04:47:59.888 INFO:tasks.workunit.client.1.smithi179.stdout:7/781: unlink d1/d9/d27/d87/d8b/fd4 0 2024-04-02T04:47:59.888 INFO:tasks.workunit.client.1.smithi179.stdout:2/634: dwrite dc/d29/d4a/d1e/d43/f7f [0,4194304] 0 2024-04-02T04:47:59.889 INFO:tasks.workunit.client.1.smithi179.stdout:7/782: fsync d1/dd/f90 0 2024-04-02T04:47:59.889 INFO:tasks.workunit.client.1.smithi179.stdout:7/783: chown d1/dd/d16/f48 1921 1 2024-04-02T04:47:59.891 INFO:tasks.workunit.client.1.smithi179.stdout:8/767: creat d3/d32/d21/d45/f10f x:0 0 0 2024-04-02T04:47:59.894 INFO:tasks.workunit.client.1.smithi179.stdout:0/837: unlink d1/da/d28/d81/fae 0 2024-04-02T04:47:59.900 INFO:tasks.workunit.client.1.smithi179.stdout:5/731: dread d8/f1a [0,4194304] 0 2024-04-02T04:47:59.901 INFO:tasks.workunit.client.1.smithi179.stdout:4/778: symlink d1/d3/df/d10/l107 0 2024-04-02T04:47:59.902 INFO:tasks.workunit.client.1.smithi179.stdout:4/779: write d1/fc3 [2781701,3187] 0 2024-04-02T04:47:59.905 INFO:tasks.workunit.client.1.smithi179.stdout:7/784: dread d1/dd/d23/f24 [4194304,4194304] 0 2024-04-02T04:47:59.905 INFO:tasks.workunit.client.1.smithi179.stdout:3/780: symlink d8/d57/d63/lf5 0 2024-04-02T04:47:59.906 INFO:tasks.workunit.client.1.smithi179.stdout:3/781: stat d8/dd3/dbb/fc6 0 2024-04-02T04:47:59.906 INFO:tasks.workunit.client.1.smithi179.stdout:7/785: write d1/d9/fd0 [87712,110106] 0 2024-04-02T04:47:59.910 INFO:tasks.workunit.client.1.smithi179.stdout:8/768: mkdir d3/d32/d21/d22/d6b/d91/d110 0 2024-04-02T04:47:59.911 INFO:tasks.workunit.client.1.smithi179.stdout:8/769: chown d3/d1b/d2a/d41 110897 1 2024-04-02T04:47:59.930 INFO:tasks.workunit.client.0.smithi070.stdout:5/841: unlink d5/d9/d23/d30/da2/c47 0 2024-04-02T04:47:59.931 INFO:tasks.workunit.client.0.smithi070.stdout:4/649: creat d1/d3/fd2 x:0 0 0 2024-04-02T04:47:59.933 INFO:tasks.workunit.client.0.smithi070.stdout:3/707: rename d10/d32/f55 to d10/d17/db2/dbd/ddb/ff2 0 2024-04-02T04:47:59.934 INFO:tasks.workunit.client.0.smithi070.stdout:3/708: write fb [4968054,105488] 0 2024-04-02T04:47:59.936 INFO:tasks.workunit.client.0.smithi070.stdout:2/744: unlink d2/d10/d21/d3d/f5d 0 2024-04-02T04:47:59.936 INFO:tasks.workunit.client.0.smithi070.stdout:3/709: write d10/d32/d3f/f1f [103177,53372] 0 2024-04-02T04:47:59.939 INFO:tasks.workunit.client.0.smithi070.stdout:1/665: truncate d0/d4c/f96 345341 0 2024-04-02T04:47:59.942 INFO:tasks.workunit.client.0.smithi070.stdout:2/745: dwrite d2/fba [0,4194304] 0 2024-04-02T04:47:59.944 INFO:tasks.workunit.client.0.smithi070.stdout:5/842: write d5/d17/d2b/d7b/ff9 [3235911,30292] 0 2024-04-02T04:47:59.945 INFO:tasks.workunit.client.1.smithi179.stdout:0/838: symlink d1/da/d28/d34/d9c/lf4 0 2024-04-02T04:47:59.945 INFO:tasks.workunit.client.1.smithi179.stdout:7/786: dwrite d1/d9/d27/d87/d8b/fa2 [0,4194304] 0 2024-04-02T04:47:59.946 INFO:tasks.workunit.client.1.smithi179.stdout:8/770: fdatasync d3/d32/d21/f25 0 2024-04-02T04:47:59.946 INFO:tasks.workunit.client.1.smithi179.stdout:5/732: mkdir d8/d1b/d23/d30/d73/dee 0 2024-04-02T04:47:59.947 INFO:tasks.workunit.client.1.smithi179.stdout:1/777: mknod d3/d5/d2f/d4f/dab/c10d 0 2024-04-02T04:47:59.948 INFO:tasks.workunit.client.1.smithi179.stdout:9/754: link d0/d1/d10/d6d/c2a d0/d19/d27/d7a/d87/daf/dc2/ce8 0 2024-04-02T04:47:59.948 INFO:tasks.workunit.client.0.smithi070.stdout:4/650: creat d1/d3/d85/fd3 x:0 0 0 2024-04-02T04:47:59.949 INFO:tasks.workunit.client.1.smithi179.stdout:8/771: stat d3/d32/d21/d22/d6b/db1 0 2024-04-02T04:47:59.950 INFO:tasks.workunit.client.1.smithi179.stdout:3/782: chown d8/d57/ld4 183815 1 2024-04-02T04:47:59.951 INFO:tasks.workunit.client.1.smithi179.stdout:1/778: stat d3/d5/d2f/d4f/d42/df2/l9a 0 2024-04-02T04:47:59.955 INFO:tasks.workunit.client.1.smithi179.stdout:5/733: symlink d8/d1b/d23/d30/lef 0 2024-04-02T04:47:59.955 INFO:tasks.workunit.client.1.smithi179.stdout:3/783: chown d8/d26/d52/cae 119446 1 2024-04-02T04:47:59.957 INFO:tasks.workunit.client.1.smithi179.stdout:5/734: chown d8/d1b/l83 86129 1 2024-04-02T04:47:59.957 INFO:tasks.workunit.client.1.smithi179.stdout:0/839: dwrite d1/db/f7c [0,4194304] 0 2024-04-02T04:47:59.957 INFO:tasks.workunit.client.1.smithi179.stdout:8/772: write d3/d32/d21/d50/d56/d94/d99/fb6 [824290,126628] 0 2024-04-02T04:47:59.959 INFO:tasks.workunit.client.0.smithi070.stdout:3/710: rmdir d10/d32/d3f 39 2024-04-02T04:47:59.960 INFO:tasks.workunit.client.1.smithi179.stdout:0/840: read - d1/da/d48/d65/d7d/d80/fb0 zero size 2024-04-02T04:47:59.961 INFO:tasks.workunit.client.0.smithi070.stdout:0/755: creat d4/d5/d21/fec x:0 0 0 2024-04-02T04:47:59.962 INFO:tasks.workunit.client.0.smithi070.stdout:0/756: stat d4/d5/d13/d17 0 2024-04-02T04:47:59.962 INFO:tasks.workunit.client.1.smithi179.stdout:0/841: chown d1/da/d52/fbb 197 1 2024-04-02T04:47:59.964 INFO:tasks.workunit.client.0.smithi070.stdout:1/666: rmdir d0/d7/d1a/d95 39 2024-04-02T04:47:59.965 INFO:tasks.workunit.client.0.smithi070.stdout:1/667: read - d0/d54/fd2 zero size 2024-04-02T04:47:59.965 INFO:tasks.workunit.client.1.smithi179.stdout:6/753: getdents d3/d42/dbe 0 2024-04-02T04:47:59.965 INFO:tasks.workunit.client.0.smithi070.stdout:6/576: write d0/d3/d19/d31/f36 [2191206,114877] 0 2024-04-02T04:47:59.968 INFO:tasks.workunit.client.0.smithi070.stdout:2/746: symlink d2/d10/d5c/d7a/db0/dc7/lf7 0 2024-04-02T04:47:59.971 INFO:tasks.workunit.client.0.smithi070.stdout:0/757: dwrite d4/d5/d58/d60/d83/fe4 [0,4194304] 0 2024-04-02T04:47:59.973 INFO:tasks.workunit.client.0.smithi070.stdout:0/758: stat d4/d5/d58/da1/fcd 0 2024-04-02T04:47:59.973 INFO:tasks.workunit.client.1.smithi179.stdout:8/773: fdatasync d3/d32/d21/d49/f9f 0 2024-04-02T04:47:59.974 INFO:tasks.workunit.client.0.smithi070.stdout:5/843: creat d5/d9/d23/d30/da2/d75/ddf/f109 x:0 0 0 2024-04-02T04:47:59.974 INFO:tasks.workunit.client.0.smithi070.stdout:0/759: stat d4/d5/da/fc9 0 2024-04-02T04:47:59.981 INFO:tasks.workunit.client.1.smithi179.stdout:2/635: getdents dc/dcb 0 2024-04-02T04:47:59.983 INFO:tasks.workunit.client.1.smithi179.stdout:1/779: creat d3/dbe/f10e x:0 0 0 2024-04-02T04:47:59.988 INFO:tasks.workunit.client.1.smithi179.stdout:0/842: truncate d1/f2a 233491 0 2024-04-02T04:47:59.990 INFO:tasks.workunit.client.1.smithi179.stdout:6/754: mkdir d3/d12/d44/d10a 0 2024-04-02T04:47:59.996 INFO:tasks.workunit.client.1.smithi179.stdout:8/774: symlink d3/d32/d21/d22/da5/l111 0 2024-04-02T04:48:00.004 INFO:tasks.workunit.client.1.smithi179.stdout:6/755: write d3/d12/d23/d26/f4f [2685885,122413] 0 2024-04-02T04:48:00.004 INFO:tasks.workunit.client.1.smithi179.stdout:7/787: dwrite d1/dd/d76/f86 [0,4194304] 0 2024-04-02T04:48:00.004 INFO:tasks.workunit.client.1.smithi179.stdout:8/775: dwrite d3/d32/d21/d50/d56/d94/d84/fe9 [0,4194304] 0 2024-04-02T04:48:00.005 INFO:tasks.workunit.client.1.smithi179.stdout:8/776: dread - d3/d1b/fdf zero size 2024-04-02T04:48:00.007 INFO:tasks.workunit.client.1.smithi179.stdout:1/780: creat d3/d5/d2f/d4f/d42/d53/d72/d86/f10f x:0 0 0 2024-04-02T04:48:00.007 INFO:tasks.workunit.client.1.smithi179.stdout:5/735: link d8/d1b/d23/d32/d4e/f59 d8/d1b/d23/d36/d53/d62/da9/ff0 0 2024-04-02T04:48:00.013 INFO:tasks.workunit.client.1.smithi179.stdout:0/843: creat d1/da/d28/ff5 x:0 0 0 2024-04-02T04:48:00.013 INFO:tasks.workunit.client.1.smithi179.stdout:2/636: getdents dc/d29/d31/d66 0 2024-04-02T04:48:00.015 INFO:tasks.workunit.client.1.smithi179.stdout:5/736: unlink d8/f3e 0 2024-04-02T04:48:00.015 INFO:tasks.workunit.client.1.smithi179.stdout:2/637: read - dc/fd4 zero size 2024-04-02T04:48:00.018 INFO:tasks.workunit.client.1.smithi179.stdout:8/777: mkdir d3/d32/d21/d50/d112 0 2024-04-02T04:48:00.018 INFO:tasks.workunit.client.1.smithi179.stdout:1/781: dwrite d3/d5/d2f/d4f/d42/fbd [0,4194304] 0 2024-04-02T04:48:00.019 INFO:tasks.workunit.client.0.smithi070.stdout:8/704: sync 2024-04-02T04:48:00.021 INFO:tasks.workunit.client.0.smithi070.stdout:4/651: creat d1/d3/d85/d8e/d99/fd4 x:0 0 0 2024-04-02T04:48:00.022 INFO:tasks.workunit.client.1.smithi179.stdout:0/844: rmdir d1/da/d67/de6 39 2024-04-02T04:48:00.023 INFO:tasks.workunit.client.1.smithi179.stdout:9/755: truncate d0/d1/d11/d29/f94 6300906 0 2024-04-02T04:48:00.023 INFO:tasks.workunit.client.0.smithi070.stdout:2/747: creat d2/d10/d21/d71/dc1/ff8 x:0 0 0 2024-04-02T04:48:00.024 INFO:tasks.workunit.client.1.smithi179.stdout:6/756: link d3/fa d3/f10b 0 2024-04-02T04:48:00.025 INFO:tasks.workunit.client.0.smithi070.stdout:8/705: truncate d2/d8/d23/d24/d5f/fe2 545558 0 2024-04-02T04:48:00.026 INFO:tasks.workunit.client.0.smithi070.stdout:2/748: write d2/ff4 [403055,11237] 0 2024-04-02T04:48:00.028 INFO:tasks.workunit.client.1.smithi179.stdout:2/638: fdatasync dc/d29/d31/d57/f71 0 2024-04-02T04:48:00.028 INFO:tasks.workunit.client.0.smithi070.stdout:4/652: dread d1/d3/f80 [0,4194304] 0 2024-04-02T04:48:00.029 INFO:tasks.workunit.client.1.smithi179.stdout:1/782: fdatasync d3/d5/d2f/d4f/f4c 0 2024-04-02T04:48:00.034 INFO:tasks.workunit.client.1.smithi179.stdout:6/757: dwrite d3/d9a/fe7 [0,4194304] 0 2024-04-02T04:48:00.034 INFO:tasks.workunit.client.0.smithi070.stdout:3/711: dwrite d10/d17/db2/fe3 [0,4194304] 0 2024-04-02T04:48:00.035 INFO:tasks.workunit.client.0.smithi070.stdout:8/706: chown d2/d8/d29/d22/c2f 14894371 1 2024-04-02T04:48:00.037 INFO:tasks.workunit.client.1.smithi179.stdout:3/784: truncate d8/d26/f34 4114829 0 2024-04-02T04:48:00.038 INFO:tasks.workunit.client.0.smithi070.stdout:2/749: creat d2/d12/d59/ff9 x:0 0 0 2024-04-02T04:48:00.041 INFO:tasks.workunit.client.1.smithi179.stdout:4/780: sync 2024-04-02T04:48:00.041 INFO:tasks.workunit.client.0.smithi070.stdout:2/750: truncate d2/d10/d5c/d7a/db0/dc7/ded/ff3 856060 0 2024-04-02T04:48:00.041 INFO:tasks.workunit.client.0.smithi070.stdout:2/751: readlink d2/d24/l49 0 2024-04-02T04:48:00.042 INFO:tasks.workunit.client.1.smithi179.stdout:7/788: write d1/f83 [1662818,62096] 0 2024-04-02T04:48:00.043 INFO:tasks.workunit.client.0.smithi070.stdout:4/653: mknod d1/dc3/cd5 0 2024-04-02T04:48:00.048 INFO:tasks.workunit.client.1.smithi179.stdout:1/783: chown d3/d5/ld5 126772917 1 2024-04-02T04:48:00.050 INFO:tasks.workunit.client.1.smithi179.stdout:8/778: mknod d3/d32/d21/d22/d6b/db1/d104/c113 0 2024-04-02T04:48:00.054 INFO:tasks.workunit.client.0.smithi070.stdout:5/844: write d5/d17/d2b/d7b/f92 [4632238,103970] 0 2024-04-02T04:48:00.055 INFO:tasks.workunit.client.0.smithi070.stdout:8/707: rmdir d2/d3/d35/d8e 39 2024-04-02T04:48:00.057 INFO:tasks.workunit.client.1.smithi179.stdout:5/737: rename d8/d1b/d23/d32/d6e to d8/d1b/d23/d32/dbe/daf/db7/dad/df1 0 2024-04-02T04:48:00.057 INFO:tasks.workunit.client.0.smithi070.stdout:5/845: dread d5/d17/d27/fe4 [0,4194304] 0 2024-04-02T04:48:00.057 INFO:tasks.workunit.client.1.smithi179.stdout:9/756: getdents d0/d9f 0 2024-04-02T04:48:00.057 INFO:tasks.workunit.client.1.smithi179.stdout:8/779: fdatasync d3/d32/d21/d22/f2d 0 2024-04-02T04:48:00.058 INFO:tasks.workunit.client.1.smithi179.stdout:9/757: fdatasync d0/d1/f7b 0 2024-04-02T04:48:00.058 INFO:tasks.workunit.client.1.smithi179.stdout:0/845: write d1/f14 [3289781,13978] 0 2024-04-02T04:48:00.058 INFO:tasks.workunit.client.1.smithi179.stdout:8/780: stat d3/d32/d21/d45/c7b 0 2024-04-02T04:48:00.059 INFO:tasks.workunit.client.0.smithi070.stdout:2/752: rmdir d2/d10/d5c/d7a/db0/dc7 39 2024-04-02T04:48:00.060 INFO:tasks.workunit.client.1.smithi179.stdout:9/758: fsync d0/d1/d10/d6d/d58/f75 0 2024-04-02T04:48:00.060 INFO:tasks.workunit.client.1.smithi179.stdout:8/781: readlink d3/d32/d21/d22/d6b/d91/laf 0 2024-04-02T04:48:00.061 INFO:tasks.workunit.client.1.smithi179.stdout:8/782: write d3/d32/d30/ffd [181690,30213] 0 2024-04-02T04:48:00.062 INFO:tasks.workunit.client.0.smithi070.stdout:0/760: dwrite d4/d5/da/d2d/f66 [0,4194304] 0 2024-04-02T04:48:00.062 INFO:tasks.workunit.client.1.smithi179.stdout:3/785: symlink d8/d7a/lf6 0 2024-04-02T04:48:00.063 INFO:tasks.workunit.client.1.smithi179.stdout:8/783: chown d3/d32/d21/d51/fd3 199142 1 2024-04-02T04:48:00.064 INFO:tasks.workunit.client.0.smithi070.stdout:4/654: creat d1/d98/fd6 x:0 0 0 2024-04-02T04:48:00.065 INFO:tasks.workunit.client.0.smithi070.stdout:6/577: link d0/d55/d5c/c9d d0/d55/d8b/cb0 0 2024-04-02T04:48:00.066 INFO:tasks.workunit.client.0.smithi070.stdout:6/578: fdatasync d0/d55/d5c/fad 0 2024-04-02T04:48:00.066 INFO:tasks.workunit.client.1.smithi179.stdout:7/789: rmdir d1/dd/d23/dab 39 2024-04-02T04:48:00.066 INFO:tasks.workunit.client.0.smithi070.stdout:5/846: dread d5/d9/d10/d6a/d7d/fc8 [0,4194304] 0 2024-04-02T04:48:00.072 INFO:tasks.workunit.client.0.smithi070.stdout:8/708: mkdir d2/d8/d29/d22/d5a/d94/de6 0 2024-04-02T04:48:00.073 INFO:tasks.workunit.client.0.smithi070.stdout:8/709: stat d2/d3/d35/c3e 0 2024-04-02T04:48:00.080 INFO:tasks.workunit.client.0.smithi070.stdout:0/761: fdatasync d4/d5/d13/d2a/d8d/f8f 0 2024-04-02T04:48:00.087 INFO:tasks.workunit.client.0.smithi070.stdout:5/847: mknod d5/d17/d2b/daf/c10a 0 2024-04-02T04:48:00.089 INFO:tasks.workunit.client.0.smithi070.stdout:2/753: dread d2/d10/d5c/f8c [0,4194304] 0 2024-04-02T04:48:00.090 INFO:tasks.workunit.client.0.smithi070.stdout:3/712: rename d10/d17/d51/cc2 to d10/d17/d51/cf3 0 2024-04-02T04:48:00.091 INFO:tasks.workunit.client.0.smithi070.stdout:3/713: write d10/d66/fb9 [2696564,98051] 0 2024-04-02T04:48:00.092 INFO:tasks.workunit.client.0.smithi070.stdout:8/710: chown d2/d3/d35/cbe 298367 1 2024-04-02T04:48:00.097 INFO:tasks.workunit.client.0.smithi070.stdout:4/655: creat d1/d3/d85/d8e/dc8/fd7 x:0 0 0 2024-04-02T04:48:00.101 INFO:tasks.workunit.client.0.smithi070.stdout:5/848: chown d5/f22 663 1 2024-04-02T04:48:00.103 INFO:tasks.workunit.client.0.smithi070.stdout:2/754: creat d2/d10/d5c/d7a/db0/dd8/dd2/ffa x:0 0 0 2024-04-02T04:48:00.105 INFO:tasks.workunit.client.0.smithi070.stdout:5/849: dread d5/d9/f5e [0,4194304] 0 2024-04-02T04:48:00.105 INFO:tasks.workunit.client.0.smithi070.stdout:3/714: creat d10/d17/db2/dbd/ddb/ff4 x:0 0 0 2024-04-02T04:48:00.115 INFO:tasks.workunit.client.0.smithi070.stdout:4/656: unlink d1/f32 0 2024-04-02T04:48:00.116 INFO:tasks.workunit.client.0.smithi070.stdout:4/657: dread - d1/d3/d85/d8e/d99/fd4 zero size 2024-04-02T04:48:00.117 INFO:tasks.workunit.client.0.smithi070.stdout:6/579: creat d0/d3/d19/d31/fb1 x:0 0 0 2024-04-02T04:48:00.117 INFO:tasks.workunit.client.0.smithi070.stdout:6/580: chown d0/d20/f27 16 1 2024-04-02T04:48:00.119 INFO:tasks.workunit.client.0.smithi070.stdout:5/850: dread d5/d17/f53 [0,4194304] 0 2024-04-02T04:48:00.121 INFO:tasks.workunit.client.0.smithi070.stdout:2/755: creat d2/d10/d53/ffb x:0 0 0 2024-04-02T04:48:00.125 INFO:tasks.workunit.client.0.smithi070.stdout:5/851: dwrite d5/d17/d2b/d7b/ff9 [0,4194304] 0 2024-04-02T04:48:00.127 INFO:tasks.workunit.client.0.smithi070.stdout:8/711: fsync d2/d8/d23/d24/f37 0 2024-04-02T04:48:00.134 INFO:tasks.workunit.client.0.smithi070.stdout:0/762: write d4/d5/da/d2d/fe1 [3489945,21767] 0 2024-04-02T04:48:00.141 INFO:tasks.workunit.client.0.smithi070.stdout:0/763: dwrite d4/f9 [8388608,4194304] 0 2024-04-02T04:48:00.142 INFO:tasks.workunit.client.0.smithi070.stdout:3/715: creat d10/d32/d3f/ff5 x:0 0 0 2024-04-02T04:48:00.144 INFO:tasks.workunit.client.0.smithi070.stdout:2/756: dread d2/d12/d59/f8e [0,4194304] 0 2024-04-02T04:48:00.146 INFO:tasks.workunit.client.0.smithi070.stdout:7/639: sync 2024-04-02T04:48:00.146 INFO:tasks.workunit.client.0.smithi070.stdout:9/619: sync 2024-04-02T04:48:00.147 INFO:tasks.workunit.client.0.smithi070.stdout:5/852: dread - d5/d17/d6c/f6d zero size 2024-04-02T04:48:00.151 INFO:tasks.workunit.client.1.smithi179.stdout:9/759: creat d0/d1/d11/dab/fe9 x:0 0 0 2024-04-02T04:48:00.152 INFO:tasks.workunit.client.0.smithi070.stdout:8/712: mknod d2/d8/d23/d24/d79/ce7 0 2024-04-02T04:48:00.152 INFO:tasks.workunit.client.1.smithi179.stdout:3/786: rmdir d8/d26/d83/dbc 39 2024-04-02T04:48:00.153 INFO:tasks.workunit.client.1.smithi179.stdout:3/787: chown d8/d26/l67 1508433039 1 2024-04-02T04:48:00.154 INFO:tasks.workunit.client.0.smithi070.stdout:5/853: dwrite d5/f1d [0,4194304] 0 2024-04-02T04:48:00.156 INFO:tasks.workunit.client.0.smithi070.stdout:4/658: symlink d1/d9c/d65/ld8 0 2024-04-02T04:48:00.157 INFO:tasks.workunit.client.0.smithi070.stdout:6/581: rename d0/d3/d19/d31/d46/d6b/c91 to d0/da2/cb2 0 2024-04-02T04:48:00.158 INFO:tasks.workunit.client.0.smithi070.stdout:6/582: stat d0/d20/d5f/f6e 0 2024-04-02T04:48:00.161 INFO:tasks.workunit.client.1.smithi179.stdout:7/790: unlink d1/c60 0 2024-04-02T04:48:00.162 INFO:tasks.workunit.client.0.smithi070.stdout:3/716: creat d10/d66/ff6 x:0 0 0 2024-04-02T04:48:00.163 INFO:tasks.workunit.client.1.smithi179.stdout:1/784: symlink d3/d5/d2f/d65/df8/dfc/l110 0 2024-04-02T04:48:00.166 INFO:tasks.workunit.client.0.smithi070.stdout:0/764: rmdir d4/d5/d13/d2a/d3a/d45 39 2024-04-02T04:48:00.166 INFO:tasks.workunit.client.1.smithi179.stdout:7/791: dwrite d1/fe2 [0,4194304] 0 2024-04-02T04:48:00.168 INFO:tasks.workunit.client.0.smithi070.stdout:2/757: symlink d2/d12/lfc 0 2024-04-02T04:48:00.168 INFO:tasks.workunit.client.1.smithi179.stdout:0/846: write d1/da/d67/d9f/fc8 [446058,11988] 0 2024-04-02T04:48:00.169 INFO:tasks.workunit.client.0.smithi070.stdout:2/758: chown d2/d10/d19/cef 129601636 1 2024-04-02T04:48:00.170 INFO:tasks.workunit.client.0.smithi070.stdout:0/765: dwrite f0 [0,4194304] 0 2024-04-02T04:48:00.172 INFO:tasks.workunit.client.1.smithi179.stdout:2/639: rename dc/d29/d31/d83/dae/db0/dbc/lb7 to dc/d79/ld7 0 2024-04-02T04:48:00.173 INFO:tasks.workunit.client.0.smithi070.stdout:7/640: mknod d7/d36/ccc 0 2024-04-02T04:48:00.177 INFO:tasks.workunit.client.1.smithi179.stdout:6/758: link d3/d12/d23/d39/d4e/d6c/cdd d3/d42/c10c 0 2024-04-02T04:48:00.179 INFO:tasks.workunit.client.1.smithi179.stdout:6/759: fdatasync d3/d12/d44/fbb 0 2024-04-02T04:48:00.179 INFO:tasks.workunit.client.0.smithi070.stdout:8/713: mkdir d2/d3/d35/d3a/d4f/de8 0 2024-04-02T04:48:00.181 INFO:tasks.workunit.client.0.smithi070.stdout:5/854: symlink d5/d9/d10/d6a/l10b 0 2024-04-02T04:48:00.181 INFO:tasks.workunit.client.0.smithi070.stdout:5/855: chown d5/d9/f5e 1476 1 2024-04-02T04:48:00.182 INFO:tasks.workunit.client.0.smithi070.stdout:4/659: creat d1/d3/d85/d8e/fd9 x:0 0 0 2024-04-02T04:48:00.184 INFO:tasks.workunit.client.0.smithi070.stdout:6/583: rmdir d0/da2 39 2024-04-02T04:48:00.184 INFO:tasks.workunit.client.0.smithi070.stdout:4/660: write d1/d3/fb2 [1241292,111579] 0 2024-04-02T04:48:00.185 INFO:tasks.workunit.client.1.smithi179.stdout:3/788: read d8/d26/d83/d89/d9c/fb0 [1272763,22627] 0 2024-04-02T04:48:00.186 INFO:tasks.workunit.client.1.smithi179.stdout:4/781: creat d1/f108 x:0 0 0 2024-04-02T04:48:00.186 INFO:tasks.workunit.client.1.smithi179.stdout:9/760: dwrite d0/d1/d11/d29/d40/f5d [0,4194304] 0 2024-04-02T04:48:00.186 INFO:tasks.workunit.client.1.smithi179.stdout:3/789: chown d8/d7a 6 1 2024-04-02T04:48:00.187 INFO:tasks.workunit.client.0.smithi070.stdout:1/668: sync 2024-04-02T04:48:00.190 INFO:tasks.workunit.client.0.smithi070.stdout:1/669: read d0/d9/fc [456938,98662] 0 2024-04-02T04:48:00.191 INFO:tasks.workunit.client.0.smithi070.stdout:2/759: rmdir d2/d10/d21/d71/dd4/dce 39 2024-04-02T04:48:00.191 INFO:tasks.workunit.client.0.smithi070.stdout:1/670: stat d0/d7/d28/d83 0 2024-04-02T04:48:00.192 INFO:tasks.workunit.client.1.smithi179.stdout:9/761: dwrite d0/d19/d27/fe0 [0,4194304] 0 2024-04-02T04:48:00.193 INFO:tasks.workunit.client.1.smithi179.stdout:9/762: readlink d0/d19/d1c/l43 0 2024-04-02T04:48:00.194 INFO:tasks.workunit.client.0.smithi070.stdout:1/671: readlink d0/d9/d1d/lb 0 2024-04-02T04:48:00.194 INFO:tasks.workunit.client.0.smithi070.stdout:2/760: fdatasync d2/d10/d5c/d7a/db0/dd8/dd2/ffa 0 2024-04-02T04:48:00.196 INFO:tasks.workunit.client.1.smithi179.stdout:7/792: creat d1/d9/d27/d87/d8b/d75/db9/dcf/f114 x:0 0 0 2024-04-02T04:48:00.198 INFO:tasks.workunit.client.1.smithi179.stdout:7/793: write d1/dd/d16/fcd [2095988,3584] 0 2024-04-02T04:48:00.198 INFO:tasks.workunit.client.1.smithi179.stdout:7/794: write d1/dd/d16/d4d/fe4 [834576,94546] 0 2024-04-02T04:48:00.199 INFO:tasks.workunit.client.1.smithi179.stdout:7/795: dread - d1/d9/d27/d87/d8b/d75/db9/dcf/f114 zero size 2024-04-02T04:48:00.199 INFO:tasks.workunit.client.1.smithi179.stdout:7/796: readlink d1/dd/d76/l10f 0 2024-04-02T04:48:00.205 INFO:tasks.workunit.client.1.smithi179.stdout:2/640: rename dc/d29/d31/d57/fcd to dc/d29/d4a/d1e/dbe/fd8 0 2024-04-02T04:48:00.207 INFO:tasks.workunit.client.1.smithi179.stdout:7/797: dwrite d1/d9/d35/fb4 [0,4194304] 0 2024-04-02T04:48:00.211 INFO:tasks.workunit.client.0.smithi070.stdout:9/620: write f3 [1114906,58101] 0 2024-04-02T04:48:00.213 INFO:tasks.workunit.client.1.smithi179.stdout:6/760: rmdir d3/d12/d23/d39 39 2024-04-02T04:48:00.217 INFO:tasks.workunit.client.1.smithi179.stdout:4/782: write d1/d3/df/d10/f11 [1734188,117096] 0 2024-04-02T04:48:00.219 INFO:tasks.workunit.client.1.smithi179.stdout:8/784: link d3/d31/c5b d3/d32/d21/d22/d6b/d91/c114 0 2024-04-02T04:48:00.220 INFO:tasks.workunit.client.0.smithi070.stdout:2/761: fsync d2/d12/fa7 0 2024-04-02T04:48:00.222 INFO:tasks.workunit.client.0.smithi070.stdout:6/584: write d0/d3/d19/d31/f74 [257209,58098] 0 2024-04-02T04:48:00.223 INFO:tasks.workunit.client.0.smithi070.stdout:7/641: dwrite d7/d9/f27 [0,4194304] 0 2024-04-02T04:48:00.227 INFO:tasks.workunit.client.0.smithi070.stdout:9/621: dwrite d4/d10/d11/d7b/d19/d82/f8a [0,4194304] 0 2024-04-02T04:48:00.229 INFO:tasks.workunit.client.0.smithi070.stdout:9/622: chown d4/d10/d11/d7b/d19/d2f/d39/d57/cbf 480202 1 2024-04-02T04:48:00.230 INFO:tasks.workunit.client.0.smithi070.stdout:9/623: truncate d4/d34/fc1 127907 0 2024-04-02T04:48:00.230 INFO:tasks.workunit.client.0.smithi070.stdout:4/661: rename d1/f33 to d1/d3/d85/d8e/fda 0 2024-04-02T04:48:00.238 INFO:tasks.workunit.client.1.smithi179.stdout:9/763: rename d0/d19/d1c/d45 to d0/d19/d27/d7a/d87/dea 0 2024-04-02T04:48:00.239 INFO:tasks.workunit.client.1.smithi179.stdout:9/764: write d0/d19/d27/d48/fdf [633684,32821] 0 2024-04-02T04:48:00.240 INFO:tasks.workunit.client.0.smithi070.stdout:4/662: dwrite d1/d3/f39 [4194304,4194304] 0 2024-04-02T04:48:00.240 INFO:tasks.workunit.client.1.smithi179.stdout:9/765: dread - d0/d19/d27/fa7 zero size 2024-04-02T04:48:00.244 INFO:tasks.workunit.client.1.smithi179.stdout:2/641: mknod dc/d29/d4a/d1e/d8a/cd9 0 2024-04-02T04:48:00.248 INFO:tasks.workunit.client.1.smithi179.stdout:5/738: symlink d8/d1b/d23/d36/d53/d62/db2/lf2 0 2024-04-02T04:48:00.248 INFO:tasks.workunit.client.0.smithi070.stdout:8/714: link d2/d3/cdb d2/d8/d29/d22/dc8/ce9 0 2024-04-02T04:48:00.251 INFO:tasks.workunit.client.0.smithi070.stdout:6/585: mkdir d0/d43/db3 0 2024-04-02T04:48:00.251 INFO:tasks.workunit.client.0.smithi070.stdout:6/586: stat d0/d3/d19/d31/c8c 0 2024-04-02T04:48:00.254 INFO:tasks.workunit.client.0.smithi070.stdout:2/762: symlink d2/d10/d21/d3d/lfd 0 2024-04-02T04:48:00.260 INFO:tasks.workunit.client.1.smithi179.stdout:4/783: rmdir d1/d3/df/d10/d2f/d60/da6 39 2024-04-02T04:48:00.260 INFO:tasks.workunit.client.0.smithi070.stdout:2/763: dread d2/d10/d21/d3d/d42/fad [0,4194304] 0 2024-04-02T04:48:00.263 INFO:tasks.workunit.client.1.smithi179.stdout:1/785: link d3/d5/d2f/db7/l100 d3/d85/da2/dd1/l111 0 2024-04-02T04:48:00.263 INFO:tasks.workunit.client.1.smithi179.stdout:0/847: link d1/da/d48/d5c/d83/d97/lc6 d1/da/d28/d5e/lf6 0 2024-04-02T04:48:00.265 INFO:tasks.workunit.client.1.smithi179.stdout:6/761: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:48:00.265 INFO:tasks.workunit.client.0.smithi070.stdout:3/717: creat d10/d17/d51/d73/ff7 x:0 0 0 2024-04-02T04:48:00.266 INFO:tasks.workunit.client.1.smithi179.stdout:2/642: symlink dc/d29/d4a/d1e/d6c/lda 0 2024-04-02T04:48:00.269 INFO:tasks.workunit.client.0.smithi070.stdout:7/642: dwrite d7/dd/df/d79/d46/d65/d7b/fab [0,4194304] 0 2024-04-02T04:48:00.269 INFO:tasks.workunit.client.1.smithi179.stdout:5/739: creat d8/d1b/d23/d32/dbe/daf/ff3 x:0 0 0 2024-04-02T04:48:00.270 INFO:tasks.workunit.client.0.smithi070.stdout:4/663: mkdir d1/d3/d85/ddb 0 2024-04-02T04:48:00.270 INFO:tasks.workunit.client.1.smithi179.stdout:7/798: symlink d1/d9/l115 0 2024-04-02T04:48:00.270 INFO:tasks.workunit.client.1.smithi179.stdout:0/848: dread d1/da/f93 [0,4194304] 0 2024-04-02T04:48:00.271 INFO:tasks.workunit.client.0.smithi070.stdout:7/643: chown d7/dd/df/d79/c3d 6048988 1 2024-04-02T04:48:00.271 INFO:tasks.workunit.client.1.smithi179.stdout:0/849: write d1/da/df/f12 [5050844,85195] 0 2024-04-02T04:48:00.272 INFO:tasks.workunit.client.1.smithi179.stdout:0/850: truncate d1/db/f8e 4293439 0 2024-04-02T04:48:00.274 INFO:tasks.workunit.client.1.smithi179.stdout:0/851: write d1/da/d52/fbb [7710529,4289] 0 2024-04-02T04:48:00.274 INFO:tasks.workunit.client.1.smithi179.stdout:0/852: fsync d1/f36 0 2024-04-02T04:48:00.275 INFO:tasks.workunit.client.1.smithi179.stdout:3/790: creat d8/d26/d83/dbc/ded/ff7 x:0 0 0 2024-04-02T04:48:00.276 INFO:tasks.workunit.client.0.smithi070.stdout:3/718: dwrite d10/f4f [0,4194304] 0 2024-04-02T04:48:00.277 INFO:tasks.workunit.client.0.smithi070.stdout:4/664: truncate d1/d3/d85/d8e/dc8/fd7 1022359 0 2024-04-02T04:48:00.278 INFO:tasks.workunit.client.1.smithi179.stdout:8/785: creat d3/d32/d21/d50/d112/f115 x:0 0 0 2024-04-02T04:48:00.280 INFO:tasks.workunit.client.0.smithi070.stdout:6/587: mknod d0/d3/d19/d31/d61/cb4 0 2024-04-02T04:48:00.284 INFO:tasks.workunit.client.0.smithi070.stdout:2/764: rmdir d2 39 2024-04-02T04:48:00.285 INFO:tasks.workunit.client.0.smithi070.stdout:6/588: write d0/f78 [4871021,9835] 0 2024-04-02T04:48:00.287 INFO:tasks.workunit.client.1.smithi179.stdout:6/762: symlink d3/d12/d23/d26/dd3/l10d 0 2024-04-02T04:48:00.287 INFO:tasks.workunit.client.0.smithi070.stdout:4/665: dwrite d1/d3/d85/d8e/fd9 [0,4194304] 0 2024-04-02T04:48:00.289 INFO:tasks.workunit.client.1.smithi179.stdout:2/643: symlink dc/d29/d31/d57/ldb 0 2024-04-02T04:48:00.289 INFO:tasks.workunit.client.0.smithi070.stdout:4/666: dread - d1/d3/d85/fd3 zero size 2024-04-02T04:48:00.293 INFO:tasks.workunit.client.0.smithi070.stdout:1/672: link d0/d7/d1a/d66/f88 d0/d7/d1a/fda 0 2024-04-02T04:48:00.295 INFO:tasks.workunit.client.1.smithi179.stdout:2/644: dwrite f2 [0,4194304] 0 2024-04-02T04:48:00.296 INFO:tasks.workunit.client.0.smithi070.stdout:8/715: unlink d2/d3/d35/d8e/la9 0 2024-04-02T04:48:00.297 INFO:tasks.workunit.client.0.smithi070.stdout:8/716: stat d2/d8/d29/c3f 0 2024-04-02T04:48:00.297 INFO:tasks.workunit.client.0.smithi070.stdout:3/719: rmdir d10/d32/d4b/de1 39 2024-04-02T04:48:00.298 INFO:tasks.workunit.client.0.smithi070.stdout:3/720: write d10/d17/f5d [1842449,57893] 0 2024-04-02T04:48:00.300 INFO:tasks.workunit.client.1.smithi179.stdout:4/784: symlink d1/d3/df/d10/d2f/l109 0 2024-04-02T04:48:00.301 INFO:tasks.workunit.client.1.smithi179.stdout:8/786: creat d3/d1b/d2a/d41/d109/f116 x:0 0 0 2024-04-02T04:48:00.309 INFO:tasks.workunit.client.0.smithi070.stdout:2/765: chown d2/d10/d21/d71/dd4/l97 36083 1 2024-04-02T04:48:00.312 INFO:tasks.workunit.client.1.smithi179.stdout:1/786: dread d3/d5/de/f3c [0,4194304] 0 2024-04-02T04:48:00.312 INFO:tasks.workunit.client.0.smithi070.stdout:0/766: rename d4/d5/d13/d2a/d3a/d45/l9a to d4/d5/d58/d60/led 0 2024-04-02T04:48:00.312 INFO:tasks.workunit.client.1.smithi179.stdout:5/740: symlink d8/d1b/d23/d32/dbe/daf/dec/lf4 0 2024-04-02T04:48:00.313 INFO:tasks.workunit.client.1.smithi179.stdout:3/791: dread d8/d26/f3f [0,4194304] 0 2024-04-02T04:48:00.316 INFO:tasks.workunit.client.1.smithi179.stdout:2/645: mkdir dc/d29/d31/d83/dae/ddc 0 2024-04-02T04:48:00.317 INFO:tasks.workunit.client.0.smithi070.stdout:4/667: symlink d1/d3/d85/d8e/d99/ldc 0 2024-04-02T04:48:00.319 INFO:tasks.workunit.client.1.smithi179.stdout:4/785: symlink d1/d3/df/d10/d2f/d60/d71/dce/l10a 0 2024-04-02T04:48:00.319 INFO:tasks.workunit.client.0.smithi070.stdout:0/767: dwrite d4/d5/d13/d2a/d3a/d45/fbd [0,4194304] 0 2024-04-02T04:48:00.320 INFO:tasks.workunit.client.0.smithi070.stdout:7/644: creat d7/dd/db1/fcd x:0 0 0 2024-04-02T04:48:00.326 INFO:tasks.workunit.client.0.smithi070.stdout:3/721: creat d10/d17/db2/ff8 x:0 0 0 2024-04-02T04:48:00.328 INFO:tasks.workunit.client.0.smithi070.stdout:2/766: truncate d2/d10/d21/d71/dd4/fb6 1554211 0 2024-04-02T04:48:00.329 INFO:tasks.workunit.client.1.smithi179.stdout:5/741: creat d8/d1b/d23/ff5 x:0 0 0 2024-04-02T04:48:00.332 INFO:tasks.workunit.client.0.smithi070.stdout:1/673: rename d0/d9/d1d/d22/d2c/d4f/f50 to d0/d9/d1d/d22/d56/d79/d89/fdb 0 2024-04-02T04:48:00.332 INFO:tasks.workunit.client.0.smithi070.stdout:1/674: chown d0/db4/d9f 336077522 1 2024-04-02T04:48:00.332 INFO:tasks.workunit.client.1.smithi179.stdout:6/763: dwrite d3/d2d/fb6 [0,4194304] 0 2024-04-02T04:48:00.332 INFO:tasks.workunit.client.1.smithi179.stdout:3/792: rmdir d8/d57/d63 39 2024-04-02T04:48:00.333 INFO:tasks.workunit.client.1.smithi179.stdout:3/793: chown d8/c17 5229935 1 2024-04-02T04:48:00.338 INFO:tasks.workunit.client.1.smithi179.stdout:7/799: rename d1/f32 to d1/d9/d27/d87/f116 0 2024-04-02T04:48:00.341 INFO:tasks.workunit.client.0.smithi070.stdout:6/589: truncate d0/f26 2204848 0 2024-04-02T04:48:00.342 INFO:tasks.workunit.client.1.smithi179.stdout:2/646: mkdir dc/d29/d31/d57/d97/ddd 0 2024-04-02T04:48:00.344 INFO:tasks.workunit.client.0.smithi070.stdout:9/624: getdents d4/d10/d12/d56 0 2024-04-02T04:48:00.345 INFO:tasks.workunit.client.1.smithi179.stdout:4/786: mkdir d1/d3/dc/d27/d10b 0 2024-04-02T04:48:00.347 INFO:tasks.workunit.client.1.smithi179.stdout:8/787: mkdir d3/d32/d21/d50/d56/d94/d84/d107/d117 0 2024-04-02T04:48:00.348 INFO:tasks.workunit.client.1.smithi179.stdout:8/788: fsync d3/d32/d21/fed 0 2024-04-02T04:48:00.349 INFO:tasks.workunit.client.1.smithi179.stdout:1/787: mkdir d3/d85/d112 0 2024-04-02T04:48:00.354 INFO:tasks.workunit.client.0.smithi070.stdout:7/645: mkdir d7/d4e/d82/d57/dce 0 2024-04-02T04:48:00.354 INFO:tasks.workunit.client.1.smithi179.stdout:9/766: sync 2024-04-02T04:48:00.355 INFO:tasks.workunit.client.0.smithi070.stdout:7/646: fdatasync d7/dd/df/d79/fbf 0 2024-04-02T04:48:00.355 INFO:tasks.workunit.client.1.smithi179.stdout:9/767: fsync d0/d1/d10/d6d/d58/f96 0 2024-04-02T04:48:00.359 INFO:tasks.workunit.client.0.smithi070.stdout:3/722: chown d10/d17/d51/d73/cbe 41366020 1 2024-04-02T04:48:00.362 INFO:tasks.workunit.client.0.smithi070.stdout:2/767: mknod d2/d12/cfe 0 2024-04-02T04:48:00.363 INFO:tasks.workunit.client.0.smithi070.stdout:2/768: write d2/d12/d59/fe5 [542413,28001] 0 2024-04-02T04:48:00.365 INFO:tasks.workunit.client.1.smithi179.stdout:0/853: rename d1/db/dad to d1/da/d52/d75/dc0/df7 0 2024-04-02T04:48:00.366 INFO:tasks.workunit.client.0.smithi070.stdout:1/675: write d0/d7/f10 [3582046,60064] 0 2024-04-02T04:48:00.369 INFO:tasks.workunit.client.0.smithi070.stdout:2/769: dwrite d2/f32 [0,4194304] 0 2024-04-02T04:48:00.372 INFO:tasks.workunit.client.1.smithi179.stdout:2/647: mknod dc/d29/d4a/d1e/d8a/cde 0 2024-04-02T04:48:00.374 INFO:tasks.workunit.client.0.smithi070.stdout:6/590: readlink d0/d3/d19/l48 0 2024-04-02T04:48:00.374 INFO:tasks.workunit.client.0.smithi070.stdout:6/591: chown d0/d3/l8 126 1 2024-04-02T04:48:00.380 INFO:tasks.workunit.client.1.smithi179.stdout:7/800: write d1/dd/d16/f48 [4627739,20163] 0 2024-04-02T04:48:00.380 INFO:tasks.workunit.client.0.smithi070.stdout:4/668: mknod d1/d3/d85/ddb/cdd 0 2024-04-02T04:48:00.381 INFO:tasks.workunit.client.0.smithi070.stdout:5/856: sync 2024-04-02T04:48:00.381 INFO:tasks.workunit.client.0.smithi070.stdout:6/592: dwrite d0/d3/d19/d31/d46/f93 [0,4194304] 0 2024-04-02T04:48:00.381 INFO:tasks.workunit.client.1.smithi179.stdout:1/788: symlink d3/d85/da2/l113 0 2024-04-02T04:48:00.382 INFO:tasks.workunit.client.0.smithi070.stdout:4/669: write d1/d3/d85/d8e/fac [356756,63259] 0 2024-04-02T04:48:00.384 INFO:tasks.workunit.client.0.smithi070.stdout:0/768: symlink d4/d5/d13/d17/d98/dd2/lee 0 2024-04-02T04:48:00.387 INFO:tasks.workunit.client.0.smithi070.stdout:8/717: creat d2/fea x:0 0 0 2024-04-02T04:48:00.389 INFO:tasks.workunit.client.1.smithi179.stdout:9/768: truncate d0/d19/d27/d7a/d84/fb6 275990 0 2024-04-02T04:48:00.392 INFO:tasks.workunit.client.1.smithi179.stdout:6/764: rename d3/d42/dbc to d3/d12/d44/d10a/d10e 0 2024-04-02T04:48:00.393 INFO:tasks.workunit.client.1.smithi179.stdout:0/854: mkdir d1/da/d52/d75/d9a/df8 0 2024-04-02T04:48:00.395 INFO:tasks.workunit.client.0.smithi070.stdout:1/676: unlink d0/d9/d1d/d22/d2c/d4f/f61 0 2024-04-02T04:48:00.396 INFO:tasks.workunit.client.1.smithi179.stdout:9/769: dwrite d0/d1/d10/d6d/fd0 [0,4194304] 0 2024-04-02T04:48:00.396 INFO:tasks.workunit.client.1.smithi179.stdout:9/770: chown d0/c23 65149 1 2024-04-02T04:48:00.398 INFO:tasks.workunit.client.0.smithi070.stdout:2/770: fdatasync d2/d10/d19/f7b 0 2024-04-02T04:48:00.401 INFO:tasks.workunit.client.1.smithi179.stdout:8/789: creat d3/d32/d21/d22/d6b/d91/d110/f118 x:0 0 0 2024-04-02T04:48:00.401 INFO:tasks.workunit.client.1.smithi179.stdout:8/790: readlink d3/d31/ld9 0 2024-04-02T04:48:00.404 INFO:tasks.workunit.client.0.smithi070.stdout:9/625: creat d4/d10/d11/d7b/d19/d2f/d39/da5/fca x:0 0 0 2024-04-02T04:48:00.407 INFO:tasks.workunit.client.1.smithi179.stdout:8/791: dwrite d3/d1b/d2a/d41/d109/f116 [0,4194304] 0 2024-04-02T04:48:00.410 INFO:tasks.workunit.client.1.smithi179.stdout:1/789: mknod d3/d5/d2f/d4f/d6a/da0/c114 0 2024-04-02T04:48:00.411 INFO:tasks.workunit.client.0.smithi070.stdout:9/626: dwrite d4/d13/d89/faf [0,4194304] 0 2024-04-02T04:48:00.411 INFO:tasks.workunit.client.0.smithi070.stdout:4/670: mknod d1/d3/d85/d8e/cde 0 2024-04-02T04:48:00.420 INFO:tasks.workunit.client.1.smithi179.stdout:3/794: rmdir d8/d26/dcd/dee 0 2024-04-02T04:48:00.422 INFO:tasks.workunit.client.0.smithi070.stdout:0/769: unlink d4/d5/da/f6d 0 2024-04-02T04:48:00.423 INFO:tasks.workunit.client.0.smithi070.stdout:7/647: symlink d7/d4e/d82/d57/dce/lcf 0 2024-04-02T04:48:00.425 INFO:tasks.workunit.client.0.smithi070.stdout:8/718: creat d2/d8/d23/d24/d5f/feb x:0 0 0 2024-04-02T04:48:00.426 INFO:tasks.workunit.client.1.smithi179.stdout:6/765: creat d3/d12/d44/f10f x:0 0 0 2024-04-02T04:48:00.426 INFO:tasks.workunit.client.1.smithi179.stdout:7/801: dwrite d1/d9/d7e/fb8 [0,4194304] 0 2024-04-02T04:48:00.429 INFO:tasks.workunit.client.0.smithi070.stdout:1/677: truncate d0/d7/d1a/f9a 538648 0 2024-04-02T04:48:00.430 INFO:tasks.workunit.client.1.smithi179.stdout:4/787: link d1/d3/dc/dc1/fd2 d1/d3/dc/d27/dae/f10c 0 2024-04-02T04:48:00.431 INFO:tasks.workunit.client.1.smithi179.stdout:4/788: dread - d1/d93/d9f/fb4 zero size 2024-04-02T04:48:00.434 INFO:tasks.workunit.client.0.smithi070.stdout:8/719: dwrite d2/d3/d52/f33 [0,4194304] 0 2024-04-02T04:48:00.441 INFO:tasks.workunit.client.0.smithi070.stdout:5/857: symlink d5/d9/d10/db9/dba/l10c 0 2024-04-02T04:48:00.443 INFO:tasks.workunit.client.0.smithi070.stdout:5/858: readlink d5/d9/d10/db9/dba/l10c 0 2024-04-02T04:48:00.447 INFO:tasks.workunit.client.1.smithi179.stdout:2/648: rename dc/d29/d4a/c2e to dc/d29/d31/d83/d8d/cdf 0 2024-04-02T04:48:00.447 INFO:tasks.workunit.client.0.smithi070.stdout:4/671: creat d1/d3/d85/d8e/d99/db1/fdf x:0 0 0 2024-04-02T04:48:00.448 INFO:tasks.workunit.client.1.smithi179.stdout:2/649: truncate dc/d29/d31/faf 786071 0 2024-04-02T04:48:00.448 INFO:tasks.workunit.client.1.smithi179.stdout:6/766: creat d3/d9a/f110 x:0 0 0 2024-04-02T04:48:00.451 INFO:tasks.workunit.client.1.smithi179.stdout:7/802: creat d1/d9/d55/dbf/d9f/da8/d104/f117 x:0 0 0 2024-04-02T04:48:00.451 INFO:tasks.workunit.client.1.smithi179.stdout:5/742: dwrite d8/d1b/d23/d32/dbe/daf/db7/dad/df1/f8a [0,4194304] 0 2024-04-02T04:48:00.455 INFO:tasks.workunit.client.0.smithi070.stdout:0/770: rmdir d4/d5/d13/d43 39 2024-04-02T04:48:00.457 INFO:tasks.workunit.client.0.smithi070.stdout:6/593: truncate d0/d3/d19/d31/d46/d6b/f70 3505845 0 2024-04-02T04:48:00.460 INFO:tasks.workunit.client.1.smithi179.stdout:0/855: write d1/da/d48/d5c/d83/fb1 [1470284,126920] 0 2024-04-02T04:48:00.461 INFO:tasks.workunit.client.1.smithi179.stdout:0/856: write d1/da/d48/fca [277461,119320] 0 2024-04-02T04:48:00.461 INFO:tasks.workunit.client.1.smithi179.stdout:4/789: write d1/d3/df/d10/d2f/d7b/f86 [145417,22614] 0 2024-04-02T04:48:00.462 INFO:tasks.workunit.client.1.smithi179.stdout:0/857: stat d1/f62 0 2024-04-02T04:48:00.463 INFO:tasks.workunit.client.1.smithi179.stdout:4/790: write d1/d3/dc/d26/d8a/fcd [478387,96864] 0 2024-04-02T04:48:00.465 INFO:tasks.workunit.client.0.smithi070.stdout:6/594: dwrite d0/d3/d19/d31/d46/d6b/f72 [0,4194304] 0 2024-04-02T04:48:00.465 INFO:tasks.workunit.client.1.smithi179.stdout:9/771: rename d0/d19/d27/d7a/d84/f93 to d0/d19/d27/d7a/dcb/feb 0 2024-04-02T04:48:00.467 INFO:tasks.workunit.client.0.smithi070.stdout:7/648: write d7/d9/f21 [610365,88835] 0 2024-04-02T04:48:00.473 INFO:tasks.workunit.client.0.smithi070.stdout:5/859: creat d5/d14/f10d x:0 0 0 2024-04-02T04:48:00.474 INFO:tasks.workunit.client.0.smithi070.stdout:5/860: chown d5/d9/d10/d51/faa 8033 1 2024-04-02T04:48:00.478 INFO:tasks.workunit.client.0.smithi070.stdout:0/771: symlink d4/d5/d13/d2a/d29/lef 0 2024-04-02T04:48:00.480 INFO:tasks.workunit.client.0.smithi070.stdout:3/723: getdents d10/d17/db2/dbf/dd2 0 2024-04-02T04:48:00.480 INFO:tasks.workunit.client.1.smithi179.stdout:4/791: symlink d1/d3/df/d10/d2f/d36/l10d 0 2024-04-02T04:48:00.483 INFO:tasks.workunit.client.1.smithi179.stdout:8/792: rename d3/d32/d21/d22/d6b/d91/laf to d3/d32/d21/d22/d6b/l119 0 2024-04-02T04:48:00.485 INFO:tasks.workunit.client.0.smithi070.stdout:7/649: symlink d7/dd/df/d7a/d6c/d77/d54/ld0 0 2024-04-02T04:48:00.486 INFO:tasks.workunit.client.0.smithi070.stdout:0/772: dwrite d4/d5/d13/d2a/d3a/d80/fa9 [0,4194304] 0 2024-04-02T04:48:00.490 INFO:tasks.workunit.client.1.smithi179.stdout:7/803: symlink d1/d9/l118 0 2024-04-02T04:48:00.490 INFO:tasks.workunit.client.0.smithi070.stdout:0/773: chown d4/d5/d58/d60/d83/da5/lb1 0 1 2024-04-02T04:48:00.492 INFO:tasks.workunit.client.0.smithi070.stdout:2/771: link d2/d10/d21/d71/dc1/ff8 d2/d10/d21/d71/dd4/dce/fff 0 2024-04-02T04:48:00.493 INFO:tasks.workunit.client.1.smithi179.stdout:5/743: mkdir d8/d1b/d23/d42/d5a/df6 0 2024-04-02T04:48:00.495 INFO:tasks.workunit.client.1.smithi179.stdout:0/858: mknod d1/da/cf9 0 2024-04-02T04:48:00.496 INFO:tasks.workunit.client.0.smithi070.stdout:5/861: creat d5/d9/d10/d6a/f10e x:0 0 0 2024-04-02T04:48:00.498 INFO:tasks.workunit.client.1.smithi179.stdout:4/792: dread d1/d3/df/d10/d2f/fa7 [0,4194304] 0 2024-04-02T04:48:00.498 INFO:tasks.workunit.client.1.smithi179.stdout:1/790: rename d3/d5/d2f/d4f/d42/d55/dc5/ccb to d3/d85/d112/c115 0 2024-04-02T04:48:00.500 INFO:tasks.workunit.client.1.smithi179.stdout:8/793: mkdir d3/d32/d21/d22/d6b/d91/da8/d11a 0 2024-04-02T04:48:00.500 INFO:tasks.workunit.client.1.smithi179.stdout:8/794: readlink d3/d31/dc3/lf7 0 2024-04-02T04:48:00.502 INFO:tasks.workunit.client.1.smithi179.stdout:9/772: symlink d0/d19/d27/lec 0 2024-04-02T04:48:00.503 INFO:tasks.workunit.client.1.smithi179.stdout:2/650: write dc/d79/fa6 [5058037,98790] 0 2024-04-02T04:48:00.510 INFO:tasks.workunit.client.0.smithi070.stdout:8/720: dwrite d2/d3/f49 [0,4194304] 0 2024-04-02T04:48:00.513 INFO:tasks.workunit.client.1.smithi179.stdout:1/791: dread d3/d5/d2f/d4f/f37 [0,4194304] 0 2024-04-02T04:48:00.515 INFO:tasks.workunit.client.1.smithi179.stdout:1/792: fdatasync d3/d5/d2f/d4f/d42/d53/d72/d86/fa6 0 2024-04-02T04:48:00.515 INFO:tasks.workunit.client.1.smithi179.stdout:1/793: stat d3/d5/d2f/d36/cc3 0 2024-04-02T04:48:00.517 INFO:tasks.workunit.client.1.smithi179.stdout:3/795: rename d8/d26/d83/l96 to d8/d57/d63/dc3/lf8 0 2024-04-02T04:48:00.517 INFO:tasks.workunit.client.0.smithi070.stdout:6/595: truncate d0/d55/d8b/d94/f65 290463 0 2024-04-02T04:48:00.517 INFO:tasks.workunit.client.0.smithi070.stdout:6/596: stat d0/d3/d19/d31/d37/l3b 0 2024-04-02T04:48:00.518 INFO:tasks.workunit.client.1.smithi179.stdout:3/796: fsync d8/d26/d36/f8d 0 2024-04-02T04:48:00.518 INFO:tasks.workunit.client.1.smithi179.stdout:1/794: write d3/d5/d2f/d4f/d42/d53/d77/f97 [916557,74454] 0 2024-04-02T04:48:00.519 INFO:tasks.workunit.client.1.smithi179.stdout:3/797: stat d8/d57/d63/dc3/fdd 0 2024-04-02T04:48:00.519 INFO:tasks.workunit.client.0.smithi070.stdout:9/627: rename d4/d10/d12/d55/d66/lae to d4/d10/d11/d7b/d19/lcb 0 2024-04-02T04:48:00.521 INFO:tasks.workunit.client.0.smithi070.stdout:5/862: truncate d5/d17/d2b/f67 648154 0 2024-04-02T04:48:00.522 INFO:tasks.workunit.client.1.smithi179.stdout:8/795: unlink d3/d32/d21/d50/d56/dad/fef 0 2024-04-02T04:48:00.523 INFO:tasks.workunit.client.1.smithi179.stdout:3/798: dread d8/d6d/fb6 [0,4194304] 0 2024-04-02T04:48:00.523 INFO:tasks.workunit.client.0.smithi070.stdout:1/678: getdents d0/d7/d28/d83 0 2024-04-02T04:48:00.524 INFO:tasks.workunit.client.1.smithi179.stdout:3/799: fdatasync d8/d6d/d94/fc5 0 2024-04-02T04:48:00.524 INFO:tasks.workunit.client.0.smithi070.stdout:3/724: getdents d10/d17/db2/dbf/dd2 0 2024-04-02T04:48:00.524 INFO:tasks.workunit.client.1.smithi179.stdout:3/800: stat d8/l35 0 2024-04-02T04:48:00.525 INFO:tasks.workunit.client.1.smithi179.stdout:3/801: fsync d8/d26/d83/dbc/ded/ff7 0 2024-04-02T04:48:00.528 INFO:tasks.workunit.client.0.smithi070.stdout:7/650: truncate d7/dd/df/d34/fba 980455 0 2024-04-02T04:48:00.528 INFO:tasks.workunit.client.0.smithi070.stdout:8/721: truncate d2/d8/d23/d24/d66/f99 852832 0 2024-04-02T04:48:00.536 INFO:tasks.workunit.client.1.smithi179.stdout:9/773: rename d0/d19/d1c/d64 to d0/d1/d11/dab/ded 0 2024-04-02T04:48:00.536 INFO:tasks.workunit.client.1.smithi179.stdout:9/774: chown d0/d1/d10/d6d/f25 40718 1 2024-04-02T04:48:00.537 INFO:tasks.workunit.client.0.smithi070.stdout:2/772: mkdir d2/d10/d21/d71/dd4/d100 0 2024-04-02T04:48:00.539 INFO:tasks.workunit.client.1.smithi179.stdout:4/793: creat d1/d93/de6/f10e x:0 0 0 2024-04-02T04:48:00.541 INFO:tasks.workunit.client.0.smithi070.stdout:2/773: dread d2/d10/d19/f39 [0,4194304] 0 2024-04-02T04:48:00.543 INFO:tasks.workunit.client.0.smithi070.stdout:9/628: mknod d4/d10/d12/d55/d66/ccc 0 2024-04-02T04:48:00.545 INFO:tasks.workunit.client.0.smithi070.stdout:5/863: rmdir d5/d9/d23/d33 39 2024-04-02T04:48:00.546 INFO:tasks.workunit.client.1.smithi179.stdout:5/744: dwrite d8/d1b/d23/d32/d4e/f59 [0,4194304] 0 2024-04-02T04:48:00.546 INFO:tasks.workunit.client.0.smithi070.stdout:5/864: write d5/d17/d27/f28 [1320550,68042] 0 2024-04-02T04:48:00.550 INFO:tasks.workunit.client.1.smithi179.stdout:1/795: unlink d3/d5/d2f/d4f/dab/fde 0 2024-04-02T04:48:00.551 INFO:tasks.workunit.client.1.smithi179.stdout:1/796: read - d3/d5/d2f/d36/f93 zero size 2024-04-02T04:48:00.552 INFO:tasks.workunit.client.0.smithi070.stdout:4/672: getdents d1/d9c/d65 0 2024-04-02T04:48:00.554 INFO:tasks.workunit.client.0.smithi070.stdout:3/725: rmdir d10/d17/db2/dbf/dd6 39 2024-04-02T04:48:00.557 INFO:tasks.workunit.client.0.smithi070.stdout:1/679: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/ldc 0 2024-04-02T04:48:00.559 INFO:tasks.workunit.client.0.smithi070.stdout:6/597: write d0/d20/f7e [5100887,32571] 0 2024-04-02T04:48:00.561 INFO:tasks.workunit.client.0.smithi070.stdout:7/651: rmdir d7/dd/df/d7a/d6c/d77/d54 39 2024-04-02T04:48:00.562 INFO:tasks.workunit.client.0.smithi070.stdout:7/652: truncate d7/dd/df/d79/fbb 262424 0 2024-04-02T04:48:00.563 INFO:tasks.workunit.client.1.smithi179.stdout:8/796: mkdir d3/d31/dc3/d11b 0 2024-04-02T04:48:00.565 INFO:tasks.workunit.client.0.smithi070.stdout:0/774: rename d4/d5/da/d2d/f89 to d4/d5/da/d2d/de7/ff0 0 2024-04-02T04:48:00.566 INFO:tasks.workunit.client.0.smithi070.stdout:0/775: fdatasync d4/d5/d21/d6b/fe0 0 2024-04-02T04:48:00.567 INFO:tasks.workunit.client.0.smithi070.stdout:0/776: dread - d4/d5/d58/d60/d83/fd9 zero size 2024-04-02T04:48:00.569 INFO:tasks.workunit.client.0.smithi070.stdout:0/777: write d4/d5/d21/fec [873829,76201] 0 2024-04-02T04:48:00.570 INFO:tasks.workunit.client.1.smithi179.stdout:7/804: creat d1/dd/d23/f119 x:0 0 0 2024-04-02T04:48:00.571 INFO:tasks.workunit.client.0.smithi070.stdout:8/722: dread d2/d8/f6d [0,4194304] 0 2024-04-02T04:48:00.572 INFO:tasks.workunit.client.0.smithi070.stdout:2/774: rmdir d2/d10/d5c/d7a/db0/dd8 39 2024-04-02T04:48:00.573 INFO:tasks.workunit.client.1.smithi179.stdout:3/802: creat d8/ff9 x:0 0 0 2024-04-02T04:48:00.577 INFO:tasks.workunit.client.1.smithi179.stdout:0/859: creat d1/da/df/d56/ffa x:0 0 0 2024-04-02T04:48:00.579 INFO:tasks.workunit.client.1.smithi179.stdout:9/775: readlink d0/d19/d27/d7a/d87/dea/lc7 0 2024-04-02T04:48:00.580 INFO:tasks.workunit.client.0.smithi070.stdout:4/673: creat d1/d3/d85/fe0 x:0 0 0 2024-04-02T04:48:00.584 INFO:tasks.workunit.client.0.smithi070.stdout:3/726: creat d10/d32/d3f/ff9 x:0 0 0 2024-04-02T04:48:00.587 INFO:tasks.workunit.client.0.smithi070.stdout:9/629: dwrite d4/d10/f2c [0,4194304] 0 2024-04-02T04:48:00.587 INFO:tasks.workunit.client.0.smithi070.stdout:1/680: mkdir d0/d54/d64/d6d/ddd 0 2024-04-02T04:48:00.589 INFO:tasks.workunit.client.0.smithi070.stdout:6/598: symlink d0/d43/d80/lb5 0 2024-04-02T04:48:00.593 INFO:tasks.workunit.client.1.smithi179.stdout:6/767: getdents d3/d12/d23/d39/d4e/d6c/dcd 0 2024-04-02T04:48:00.594 INFO:tasks.workunit.client.0.smithi070.stdout:5/865: rename d5/d17/d27/fcd to d5/d17/d50/f10f 0 2024-04-02T04:48:00.596 INFO:tasks.workunit.client.1.smithi179.stdout:3/803: creat d8/d26/d52/d7e/db8/ffa x:0 0 0 2024-04-02T04:48:00.599 INFO:tasks.workunit.client.0.smithi070.stdout:6/599: dwrite d0/d55/d8b/f9e [0,4194304] 0 2024-04-02T04:48:00.600 INFO:tasks.workunit.client.1.smithi179.stdout:5/745: dread d8/d1b/d23/d36/d53/f78 [0,4194304] 0 2024-04-02T04:48:00.602 INFO:tasks.workunit.client.1.smithi179.stdout:5/746: stat d8/d1b/d23/d32/d46/l5b 0 2024-04-02T04:48:00.603 INFO:tasks.workunit.client.1.smithi179.stdout:5/747: write d8/d1b/d23/d32/d4e/f54 [1928087,20283] 0 2024-04-02T04:48:00.605 INFO:tasks.workunit.client.1.smithi179.stdout:2/651: getdents dc/d29/d31/d83/dae/db0 0 2024-04-02T04:48:00.605 INFO:tasks.workunit.client.1.smithi179.stdout:2/652: dread - dc/d29/d31/d83/fab zero size 2024-04-02T04:48:00.606 INFO:tasks.workunit.client.1.smithi179.stdout:8/797: getdents d3/d31/dc3/d10c 0 2024-04-02T04:48:00.607 INFO:tasks.workunit.client.1.smithi179.stdout:2/653: write dc/f17 [2927977,48030] 0 2024-04-02T04:48:00.608 INFO:tasks.workunit.client.1.smithi179.stdout:8/798: chown d3/d32/d21/d50/d65/faa 43664 1 2024-04-02T04:48:00.608 INFO:tasks.workunit.client.1.smithi179.stdout:2/654: readlink dc/d29/d4a/d1e/l59 0 2024-04-02T04:48:00.608 INFO:tasks.workunit.client.0.smithi070.stdout:4/674: creat d1/d3/d9/d5e/fe1 x:0 0 0 2024-04-02T04:48:00.610 INFO:tasks.workunit.client.1.smithi179.stdout:6/768: creat d3/d2c/d85/f111 x:0 0 0 2024-04-02T04:48:00.610 INFO:tasks.workunit.client.1.smithi179.stdout:6/769: chown d3/d42/c88 200370 1 2024-04-02T04:48:00.614 INFO:tasks.workunit.client.1.smithi179.stdout:1/797: write d3/d5/d2f/d4f/d35/f6e [3010608,43801] 0 2024-04-02T04:48:00.614 INFO:tasks.workunit.client.0.smithi070.stdout:3/727: stat d10/d32/d3f/f4d 0 2024-04-02T04:48:00.615 INFO:tasks.workunit.client.1.smithi179.stdout:3/804: creat d8/dd3/d9a/ffb x:0 0 0 2024-04-02T04:48:00.616 INFO:tasks.workunit.client.1.smithi179.stdout:3/805: dread - d8/d57/d63/dc3/fdd zero size 2024-04-02T04:48:00.616 INFO:tasks.workunit.client.0.smithi070.stdout:4/675: dwrite d1/d3/d85/d8e/fac [0,4194304] 0 2024-04-02T04:48:00.618 INFO:tasks.workunit.client.0.smithi070.stdout:0/778: dread d4/d5/d13/d2a/d8d/f8f [0,4194304] 0 2024-04-02T04:48:00.622 INFO:tasks.workunit.client.0.smithi070.stdout:1/681: truncate d0/d9/d1d/d22/d2c/d4f/d77/d8c/faa 1376153 0 2024-04-02T04:48:00.623 INFO:tasks.workunit.client.0.smithi070.stdout:1/682: readlink d0/d54/d64/lb1 0 2024-04-02T04:48:00.626 INFO:tasks.workunit.client.1.smithi179.stdout:2/655: mknod dc/d29/d4a/d1e/d43/ce0 0 2024-04-02T04:48:00.627 INFO:tasks.workunit.client.1.smithi179.stdout:2/656: write dc/d29/d4a/f53 [3630271,101133] 0 2024-04-02T04:48:00.630 INFO:tasks.workunit.client.1.smithi179.stdout:6/770: mkdir d3/d12/d44/d10a/d10e/d112 0 2024-04-02T04:48:00.630 INFO:tasks.workunit.client.1.smithi179.stdout:4/794: sync 2024-04-02T04:48:00.631 INFO:tasks.workunit.client.0.smithi070.stdout:9/630: mknod d4/d10/d12/d27/d83/ccd 0 2024-04-02T04:48:00.631 INFO:tasks.workunit.client.1.smithi179.stdout:7/805: link d1/c19 d1/dd/d16/d4d/c11a 0 2024-04-02T04:48:00.632 INFO:tasks.workunit.client.0.smithi070.stdout:9/631: dread - d4/d10/d11/d7b/d19/fc2 zero size 2024-04-02T04:48:00.633 INFO:tasks.workunit.client.0.smithi070.stdout:3/728: truncate d10/d32/f52 4930992 0 2024-04-02T04:48:00.633 INFO:tasks.workunit.client.1.smithi179.stdout:1/798: creat d3/d5/d2f/d4f/d42/d53/d72/d86/f116 x:0 0 0 2024-04-02T04:48:00.634 INFO:tasks.workunit.client.0.smithi070.stdout:4/676: mkdir d1/d3/d85/d8e/dc8/de2 0 2024-04-02T04:48:00.635 INFO:tasks.workunit.client.1.smithi179.stdout:6/771: dwrite d3/d12/d23/d39/d4e/fbf [0,4194304] 0 2024-04-02T04:48:00.635 INFO:tasks.workunit.client.1.smithi179.stdout:1/799: write d3/d5/d2f/d4f/fe4 [3818740,118187] 0 2024-04-02T04:48:00.636 INFO:tasks.workunit.client.1.smithi179.stdout:0/860: rmdir d1/da/d28/d34/d39/da6 0 2024-04-02T04:48:00.636 INFO:tasks.workunit.client.0.smithi070.stdout:7/653: link d7/dd/l9d d7/dd/df/dca/ld1 0 2024-04-02T04:48:00.637 INFO:tasks.workunit.client.0.smithi070.stdout:9/632: truncate d4/d10/d11/da8/fb5 2019714 0 2024-04-02T04:48:00.638 INFO:tasks.workunit.client.0.smithi070.stdout:8/723: dwrite d2/d8/d23/d24/d5f/f4c [0,4194304] 0 2024-04-02T04:48:00.640 INFO:tasks.workunit.client.0.smithi070.stdout:7/654: write d7/d36/fc6 [38180,127694] 0 2024-04-02T04:48:00.640 INFO:tasks.workunit.client.0.smithi070.stdout:9/633: chown d4/d10/d11/d7b/d19/d82/db7 21698 1 2024-04-02T04:48:00.640 INFO:tasks.workunit.client.0.smithi070.stdout:0/779: rename d4/d5/d13/cca to d4/d3c/cf1 0 2024-04-02T04:48:00.641 INFO:tasks.workunit.client.0.smithi070.stdout:5/866: truncate d5/d17/d50/f10f 779001 0 2024-04-02T04:48:00.643 INFO:tasks.workunit.client.1.smithi179.stdout:5/748: link d8/d1b/d23/d36/d53/d62/da9/fe2 d8/d1b/d23/d32/d46/ff7 0 2024-04-02T04:48:00.644 INFO:tasks.workunit.client.1.smithi179.stdout:1/800: dwrite d3/d5/d2f/d4f/d42/fbd [4194304,4194304] 0 2024-04-02T04:48:00.649 INFO:tasks.workunit.client.1.smithi179.stdout:2/657: creat dc/d29/d4a/d1e/d6c/fe1 x:0 0 0 2024-04-02T04:48:00.650 INFO:tasks.workunit.client.1.smithi179.stdout:2/658: write dc/f44 [780681,99181] 0 2024-04-02T04:48:00.650 INFO:tasks.workunit.client.1.smithi179.stdout:2/659: readlink dc/d29/l96 0 2024-04-02T04:48:00.651 INFO:tasks.workunit.client.1.smithi179.stdout:2/660: dread - dc/d29/d4a/d1e/d43/f98 zero size 2024-04-02T04:48:00.651 INFO:tasks.workunit.client.1.smithi179.stdout:4/795: creat d1/d3/dc/dc6/f10f x:0 0 0 2024-04-02T04:48:00.652 INFO:tasks.workunit.client.1.smithi179.stdout:4/796: truncate d1/d3/fc5 1053523 0 2024-04-02T04:48:00.653 INFO:tasks.workunit.client.1.smithi179.stdout:7/806: mkdir d1/dd/d16/d4d/d11b 0 2024-04-02T04:48:00.656 INFO:tasks.workunit.client.1.smithi179.stdout:6/772: fsync d3/d12/d23/d39/d80/f8f 0 2024-04-02T04:48:00.657 INFO:tasks.workunit.client.1.smithi179.stdout:3/806: symlink d8/d26/d52/lfc 0 2024-04-02T04:48:00.658 INFO:tasks.workunit.client.0.smithi070.stdout:4/677: chown d1/d3/d9/f5f 23082060 1 2024-04-02T04:48:00.658 INFO:tasks.workunit.client.0.smithi070.stdout:4/678: dread - d1/d98/fd6 zero size 2024-04-02T04:48:00.659 INFO:tasks.workunit.client.0.smithi070.stdout:4/679: chown d1/d9c/c51 0 1 2024-04-02T04:48:00.660 INFO:tasks.workunit.client.1.smithi179.stdout:9/776: getdents d0/d19/d27/d7a/d7e/dd4 0 2024-04-02T04:48:00.661 INFO:tasks.workunit.client.1.smithi179.stdout:5/749: symlink d8/d1b/d23/d36/d53/d62/db2/lf8 0 2024-04-02T04:48:00.661 INFO:tasks.workunit.client.1.smithi179.stdout:8/799: link d3/d32/d21/d22/d6b/d91/c43 d3/d32/d21/d45/d10a/c11c 0 2024-04-02T04:48:00.662 INFO:tasks.workunit.client.1.smithi179.stdout:8/800: readlink d3/d31/l54 0 2024-04-02T04:48:00.663 INFO:tasks.workunit.client.0.smithi070.stdout:7/655: symlink d7/dd/df/d7a/dbc/ld2 0 2024-04-02T04:48:00.664 INFO:tasks.workunit.client.0.smithi070.stdout:9/634: creat d4/d10/d11/da8/fce x:0 0 0 2024-04-02T04:48:00.665 INFO:tasks.workunit.client.1.smithi179.stdout:4/797: rmdir d1/d3 39 2024-04-02T04:48:00.669 INFO:tasks.workunit.client.1.smithi179.stdout:0/861: truncate d1/f36 7956183 0 2024-04-02T04:48:00.670 INFO:tasks.workunit.client.1.smithi179.stdout:1/801: mkdir d3/d5/d2f/d4f/d35/d117 0 2024-04-02T04:48:00.671 INFO:tasks.workunit.client.1.smithi179.stdout:8/801: fdatasync d3/d32/d21/d51/d70/f8d 0 2024-04-02T04:48:00.673 INFO:tasks.workunit.client.1.smithi179.stdout:7/807: mknod d1/d9/d27/c11c 0 2024-04-02T04:48:00.675 INFO:tasks.workunit.client.1.smithi179.stdout:0/862: rename d1/da/d28/d34/d39/d77/fd2 to d1/da/d28/d5e/ffb 0 2024-04-02T04:48:00.675 INFO:tasks.workunit.client.1.smithi179.stdout:8/802: write d3/d1b/d2a/d83/fec [230482,40550] 0 2024-04-02T04:48:00.675 INFO:tasks.workunit.client.1.smithi179.stdout:1/802: write d3/d5/fc8 [4167570,84411] 0 2024-04-02T04:48:00.680 INFO:tasks.workunit.client.0.smithi070.stdout:8/724: mkdir d2/d8/d29/d22/d5a/d94/de6/dec 0 2024-04-02T04:48:00.683 INFO:tasks.workunit.client.1.smithi179.stdout:7/808: read d1/dd/f17 [3023398,26005] 0 2024-04-02T04:48:00.683 INFO:tasks.workunit.client.0.smithi070.stdout:1/683: rename d0/d4c/c5f to d0/d9/d1d/d22/d56/cde 0 2024-04-02T04:48:00.685 INFO:tasks.workunit.client.1.smithi179.stdout:4/798: dwrite d1/d3/dc/d27/f7f [0,4194304] 0 2024-04-02T04:48:00.687 INFO:tasks.workunit.client.1.smithi179.stdout:2/661: dread dc/d29/d4a/f8c [0,4194304] 0 2024-04-02T04:48:00.688 INFO:tasks.workunit.client.1.smithi179.stdout:1/803: dwrite d3/d5/d2f/db7/ff4 [0,4194304] 0 2024-04-02T04:48:00.688 INFO:tasks.workunit.client.0.smithi070.stdout:8/725: symlink d2/d8/d23/d24/d66/led 0 2024-04-02T04:48:00.688 INFO:tasks.workunit.client.1.smithi179.stdout:3/807: dread d8/dd3/f97 [0,4194304] 0 2024-04-02T04:48:00.690 INFO:tasks.workunit.client.1.smithi179.stdout:6/773: dread d3/d2d/d43/fef [0,4194304] 0 2024-04-02T04:48:00.692 INFO:tasks.workunit.client.0.smithi070.stdout:0/780: rename d4/d5/d58/da1/fbe to d4/d5/d13/d23/ff2 0 2024-04-02T04:48:00.694 INFO:tasks.workunit.client.0.smithi070.stdout:4/680: dwrite d1/f2d [0,4194304] 0 2024-04-02T04:48:00.695 INFO:tasks.workunit.client.0.smithi070.stdout:1/684: fsync d0/d7/d28/f7a 0 2024-04-02T04:48:00.696 INFO:tasks.workunit.client.0.smithi070.stdout:0/781: chown d4/d5/d13/d2a/d8d/fd3 41718805 1 2024-04-02T04:48:00.698 INFO:tasks.workunit.client.1.smithi179.stdout:0/863: getdents d1/da/d67/de6 0 2024-04-02T04:48:00.699 INFO:tasks.workunit.client.1.smithi179.stdout:2/662: readlink dc/d29/d4a/d4b/l92 0 2024-04-02T04:48:00.700 INFO:tasks.workunit.client.0.smithi070.stdout:4/681: read d1/d3/d85/d8e/dc8/fd7 [516561,9564] 0 2024-04-02T04:48:00.700 INFO:tasks.workunit.client.1.smithi179.stdout:2/663: fsync dc/d29/d4a/f3f 0 2024-04-02T04:48:00.702 INFO:tasks.workunit.client.0.smithi070.stdout:8/726: mknod d2/d8/d29/d22/d5a/d94/cee 0 2024-04-02T04:48:00.703 INFO:tasks.workunit.client.0.smithi070.stdout:8/727: readlink d2/d8/d29/d72/ld9 0 2024-04-02T04:48:00.707 INFO:tasks.workunit.client.0.smithi070.stdout:1/685: read d0/d7/f6c [3265196,126828] 0 2024-04-02T04:48:00.710 INFO:tasks.workunit.client.0.smithi070.stdout:9/635: rename d4/d10/c96 to d4/d10/d11/d7b/d19/d2f/d39/da5/ccf 0 2024-04-02T04:48:00.712 INFO:tasks.workunit.client.1.smithi179.stdout:4/799: rename d1/d3/df/d10/d2f/fa7 to d1/d15/d6e/f110 0 2024-04-02T04:48:00.714 INFO:tasks.workunit.client.0.smithi070.stdout:6/600: write d0/f26 [326080,34346] 0 2024-04-02T04:48:00.719 INFO:tasks.workunit.client.0.smithi070.stdout:8/728: dread d2/d8/d23/d24/f5b [0,4194304] 0 2024-04-02T04:48:00.722 INFO:tasks.workunit.client.1.smithi179.stdout:3/808: mknod d8/d26/d83/dbc/ded/cfd 0 2024-04-02T04:48:00.723 INFO:tasks.workunit.client.1.smithi179.stdout:3/809: chown d8/d57/d63/dc3/fc8 133940579 1 2024-04-02T04:48:00.723 INFO:tasks.workunit.client.0.smithi070.stdout:2/775: truncate d2/d10/d21/f89 2199127 0 2024-04-02T04:48:00.724 INFO:tasks.workunit.client.1.smithi179.stdout:3/810: read - d8/d57/ff4 zero size 2024-04-02T04:48:00.724 INFO:tasks.workunit.client.0.smithi070.stdout:2/776: chown d2/d24/l49 27314 1 2024-04-02T04:48:00.729 INFO:tasks.workunit.client.0.smithi070.stdout:4/682: chown d1/d3/d85/d8e/d99/lc2 3033 1 2024-04-02T04:48:00.730 INFO:tasks.workunit.client.1.smithi179.stdout:1/804: truncate d3/d5/f27 1010989 0 2024-04-02T04:48:00.734 INFO:tasks.workunit.client.0.smithi070.stdout:4/683: dwrite d1/d9c/fa5 [0,4194304] 0 2024-04-02T04:48:00.735 INFO:tasks.workunit.client.1.smithi179.stdout:2/664: rename dc/d29/d4a/d1e/d43/fac to dc/d29/d31/d66/fe2 0 2024-04-02T04:48:00.741 INFO:tasks.workunit.client.1.smithi179.stdout:6/774: symlink d3/d12/d6a/de8/l113 0 2024-04-02T04:48:00.741 INFO:tasks.workunit.client.1.smithi179.stdout:6/775: chown d3/l19 1097 1 2024-04-02T04:48:00.742 INFO:tasks.workunit.client.0.smithi070.stdout:3/729: write d10/d17/db2/dbd/ddb/ff2 [444216,20968] 0 2024-04-02T04:48:00.743 INFO:tasks.workunit.client.1.smithi179.stdout:3/811: fdatasync d8/dd3/fd9 0 2024-04-02T04:48:00.743 INFO:tasks.workunit.client.0.smithi070.stdout:9/636: write d4/d10/d12/f67 [160664,16715] 0 2024-04-02T04:48:00.743 INFO:tasks.workunit.client.0.smithi070.stdout:4/684: dwrite d1/d3/d85/d8e/fc5 [0,4194304] 0 2024-04-02T04:48:00.750 INFO:tasks.workunit.client.1.smithi179.stdout:1/805: dread - d3/d5/d2f/d4f/d42/d55/fcf zero size 2024-04-02T04:48:00.753 INFO:tasks.workunit.client.0.smithi070.stdout:7/656: link d7/l1d d7/dd/df/d79/d46/ld3 0 2024-04-02T04:48:00.754 INFO:tasks.workunit.client.1.smithi179.stdout:5/750: dwrite d8/d1b/d23/d32/d4e/f93 [0,4194304] 0 2024-04-02T04:48:00.755 INFO:tasks.workunit.client.1.smithi179.stdout:5/751: write d8/d1b/d23/d42/dde/fe7 [907482,40499] 0 2024-04-02T04:48:00.756 INFO:tasks.workunit.client.1.smithi179.stdout:5/752: chown f1 1 1 2024-04-02T04:48:00.759 INFO:tasks.workunit.client.1.smithi179.stdout:6/776: creat d3/d12/d44/df1/f114 x:0 0 0 2024-04-02T04:48:00.762 INFO:tasks.workunit.client.1.smithi179.stdout:8/803: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:48:00.764 INFO:tasks.workunit.client.0.smithi070.stdout:0/782: rmdir d4/d5 39 2024-04-02T04:48:00.765 INFO:tasks.workunit.client.1.smithi179.stdout:8/804: write d3/d1b/d2a/d41/d109/f10d [246934,23476] 0 2024-04-02T04:48:00.771 INFO:tasks.workunit.client.1.smithi179.stdout:6/777: dwrite d3/d12/d23/d26/f4f [4194304,4194304] 0 2024-04-02T04:48:00.773 INFO:tasks.workunit.client.1.smithi179.stdout:7/809: dwrite d1/f45 [4194304,4194304] 0 2024-04-02T04:48:00.778 INFO:tasks.workunit.client.1.smithi179.stdout:8/805: read d3/d32/d21/d22/d6b/db1/fbe [789601,106618] 0 2024-04-02T04:48:00.783 INFO:tasks.workunit.client.1.smithi179.stdout:5/753: dwrite d8/d1b/d23/d32/dbe/daf/db7/dad/df1/fc1 [4194304,4194304] 0 2024-04-02T04:48:00.786 INFO:tasks.workunit.client.1.smithi179.stdout:5/754: write d8/d1b/d23/d32/dbe/fcf [1377285,119665] 0 2024-04-02T04:48:00.787 INFO:tasks.workunit.client.0.smithi070.stdout:6/601: truncate d0/d3/d19/f7f 186755 0 2024-04-02T04:48:00.790 INFO:tasks.workunit.client.1.smithi179.stdout:1/806: symlink d3/d5/d2f/d4f/d42/d53/dfe/l118 0 2024-04-02T04:48:00.791 INFO:tasks.workunit.client.1.smithi179.stdout:4/800: dwrite d1/d3/f49 [0,4194304] 0 2024-04-02T04:48:00.797 INFO:tasks.workunit.client.0.smithi070.stdout:7/657: rename d7/dd/df/d34/d68/fc7 to d7/dd/df/dca/fd4 0 2024-04-02T04:48:00.798 INFO:tasks.workunit.client.0.smithi070.stdout:1/686: dwrite d0/d7/d28/d5c/d6e/fd9 [4194304,4194304] 0 2024-04-02T04:48:00.799 INFO:tasks.workunit.client.0.smithi070.stdout:1/687: fsync d0/db4/fd6 0 2024-04-02T04:48:00.800 INFO:tasks.workunit.client.1.smithi179.stdout:0/864: truncate d1/f14 556625 0 2024-04-02T04:48:00.803 INFO:tasks.workunit.client.0.smithi070.stdout:2/777: write d2/d10/d19/f7b [807220,56817] 0 2024-04-02T04:48:00.805 INFO:tasks.workunit.client.1.smithi179.stdout:7/810: symlink d1/d9/d27/d87/d8b/d75/l11d 0 2024-04-02T04:48:00.806 INFO:tasks.workunit.client.1.smithi179.stdout:7/811: truncate d1/dd/f8c 700542 0 2024-04-02T04:48:00.806 INFO:tasks.workunit.client.1.smithi179.stdout:7/812: chown d1/dd/d16/c72 1093 1 2024-04-02T04:48:00.808 INFO:tasks.workunit.client.1.smithi179.stdout:3/812: write d8/d26/d52/fb4 [656862,3040] 0 2024-04-02T04:48:00.809 INFO:tasks.workunit.client.0.smithi070.stdout:3/730: link d10/d17/d51/d73/d9e/dd5/fd8 d10/d17/db2/de2/ffa 0 2024-04-02T04:48:00.809 INFO:tasks.workunit.client.1.smithi179.stdout:9/777: sync 2024-04-02T04:48:00.810 INFO:tasks.workunit.client.1.smithi179.stdout:8/806: mkdir d3/d32/d21/d22/d6b/db1/d104/d11d 0 2024-04-02T04:48:00.810 INFO:tasks.workunit.client.1.smithi179.stdout:9/778: chown d0/d1/d10/c38 591571 1 2024-04-02T04:48:00.811 INFO:tasks.workunit.client.1.smithi179.stdout:9/779: chown d0/d19/d27/d7a/d7e/dd4/le6 40 1 2024-04-02T04:48:00.813 INFO:tasks.workunit.client.0.smithi070.stdout:8/729: creat d2/d8/d29/d22/fef x:0 0 0 2024-04-02T04:48:00.813 INFO:tasks.workunit.client.0.smithi070.stdout:4/685: write d1/d3/f4c [189233,96355] 0 2024-04-02T04:48:00.816 INFO:tasks.workunit.client.1.smithi179.stdout:9/780: dwrite d0/d19/d27/d7a/d7e/fdd [0,4194304] 0 2024-04-02T04:48:00.818 INFO:tasks.workunit.client.0.smithi070.stdout:6/602: dread d0/d20/f23 [0,4194304] 0 2024-04-02T04:48:00.819 INFO:tasks.workunit.client.1.smithi179.stdout:5/755: symlink d8/d7e/lf9 0 2024-04-02T04:48:00.819 INFO:tasks.workunit.client.0.smithi070.stdout:9/637: write d4/d10/d12/d27/d60/d9e/f78 [15928,11124] 0 2024-04-02T04:48:00.829 INFO:tasks.workunit.client.1.smithi179.stdout:0/865: truncate d1/db/f2f 3218495 0 2024-04-02T04:48:00.830 INFO:tasks.workunit.client.1.smithi179.stdout:0/866: truncate d1/db/f7c 4467903 0 2024-04-02T04:48:00.830 INFO:tasks.workunit.client.1.smithi179.stdout:0/867: dread - d1/da/d52/d75/d9a/fe9 zero size 2024-04-02T04:48:00.836 INFO:tasks.workunit.client.1.smithi179.stdout:3/813: rmdir d8/d6d/d94 39 2024-04-02T04:48:00.836 INFO:tasks.workunit.client.1.smithi179.stdout:0/868: dwrite d1/da/df/f8a [0,4194304] 0 2024-04-02T04:48:00.837 INFO:tasks.workunit.client.1.smithi179.stdout:0/869: chown d1/da/l19 100 1 2024-04-02T04:48:00.838 INFO:tasks.workunit.client.1.smithi179.stdout:2/665: sync 2024-04-02T04:48:00.838 INFO:tasks.workunit.client.1.smithi179.stdout:0/870: fsync d1/da/d52/fd4 0 2024-04-02T04:48:00.844 INFO:tasks.workunit.client.0.smithi070.stdout:3/731: rmdir d10/d32/d8a/d8d/d91 39 2024-04-02T04:48:00.847 INFO:tasks.workunit.client.1.smithi179.stdout:9/781: symlink d0/d1/d11/dab/lee 0 2024-04-02T04:48:00.852 INFO:tasks.workunit.client.1.smithi179.stdout:4/801: write d1/d3/df/d10/d2f/d60/f6c [3102234,14304] 0 2024-04-02T04:48:00.852 INFO:tasks.workunit.client.0.smithi070.stdout:9/638: creat d4/d10/d11/d7b/d19/d2f/d8c/fd0 x:0 0 0 2024-04-02T04:48:00.852 INFO:tasks.workunit.client.0.smithi070.stdout:9/639: chown d4/d10/d11/d7b 108924122 1 2024-04-02T04:48:00.857 INFO:tasks.workunit.client.1.smithi179.stdout:4/802: read d1/d3/df/d10/d2f/d7b/f86 [2583845,82327] 0 2024-04-02T04:48:00.859 INFO:tasks.workunit.client.0.smithi070.stdout:0/783: truncate d4/d5/da/fc9 910488 0 2024-04-02T04:48:00.860 INFO:tasks.workunit.client.1.smithi179.stdout:6/778: link d3/d42/c10c d3/d12/d44/d10a/c115 0 2024-04-02T04:48:00.860 INFO:tasks.workunit.client.0.smithi070.stdout:0/784: write d4/d5/d58/d60/f92 [7462305,34787] 0 2024-04-02T04:48:00.861 INFO:tasks.workunit.client.0.smithi070.stdout:2/778: symlink d2/d10/d19/dd7/l101 0 2024-04-02T04:48:00.862 INFO:tasks.workunit.client.0.smithi070.stdout:0/785: write d4/d5/d13/d2a/d29/f59 [5860602,110303] 0 2024-04-02T04:48:00.863 INFO:tasks.workunit.client.0.smithi070.stdout:0/786: readlink d4/d5/d13/d2a/d29/l57 0 2024-04-02T04:48:00.864 INFO:tasks.workunit.client.1.smithi179.stdout:3/814: rename d8/d26/d36/c66 to d8/d26/d52/d7e/db8/cfe 0 2024-04-02T04:48:00.869 INFO:tasks.workunit.client.0.smithi070.stdout:1/688: write d0/d9/d1d/d22/d56/d79/d89/fdb [2841619,93322] 0 2024-04-02T04:48:00.875 INFO:tasks.workunit.client.1.smithi179.stdout:7/813: truncate d1/d9/d27/d87/d8b/f6f 466719 0 2024-04-02T04:48:00.884 INFO:tasks.workunit.client.1.smithi179.stdout:8/807: write d3/d32/d21/d45/d5a/fff [452690,93188] 0 2024-04-02T04:48:00.892 INFO:tasks.workunit.client.1.smithi179.stdout:3/815: mkdir d8/dff 0 2024-04-02T04:48:00.893 INFO:tasks.workunit.client.1.smithi179.stdout:3/816: write d8/d26/d36/f4d [4506269,6078] 0 2024-04-02T04:48:00.894 INFO:tasks.workunit.client.1.smithi179.stdout:3/817: write d8/d57/d63/dc3/fe5 [1019814,85661] 0 2024-04-02T04:48:00.896 INFO:tasks.workunit.client.1.smithi179.stdout:2/666: mkdir dc/d29/d31/d57/d97/dbf/de3 0 2024-04-02T04:48:00.896 INFO:tasks.workunit.client.1.smithi179.stdout:6/779: write d3/d12/d23/d26/d57/f9e [2709064,42095] 0 2024-04-02T04:48:00.897 INFO:tasks.workunit.client.1.smithi179.stdout:6/780: write d3/d12/d23/ffc [330400,114121] 0 2024-04-02T04:48:00.898 INFO:tasks.workunit.client.1.smithi179.stdout:0/871: mknod d1/da/d28/d34/ddd/cfc 0 2024-04-02T04:48:00.899 INFO:tasks.workunit.client.1.smithi179.stdout:0/872: chown d1/da/d52/d75/d9a 32891306 1 2024-04-02T04:48:00.904 INFO:tasks.workunit.client.1.smithi179.stdout:1/807: getdents d3/d5/d2f/d4f/d42/d55 0 2024-04-02T04:48:00.907 INFO:tasks.workunit.client.1.smithi179.stdout:8/808: mknod d3/d32/d21/d45/c11e 0 2024-04-02T04:48:00.909 INFO:tasks.workunit.client.1.smithi179.stdout:4/803: chown d1/d3/df/d10/d2f/d60/da6/cb9 3 1 2024-04-02T04:48:00.910 INFO:tasks.workunit.client.1.smithi179.stdout:1/808: dwrite d3/d5/d96/fb4 [0,4194304] 0 2024-04-02T04:48:00.910 INFO:tasks.workunit.client.1.smithi179.stdout:7/814: write d1/d9/d27/d87/d8b/d75/fbd [1036773,117948] 0 2024-04-02T04:48:00.912 INFO:tasks.workunit.client.1.smithi179.stdout:5/756: rename d8/d1b/f56 to d8/d1b/d23/d30/d73/db3/ffa 0 2024-04-02T04:48:00.912 INFO:tasks.workunit.client.1.smithi179.stdout:0/873: dread d1/da/f4a [0,4194304] 0 2024-04-02T04:48:00.914 INFO:tasks.workunit.client.1.smithi179.stdout:5/757: stat d8/d1b/d23/d32/dbe/daf/ff3 0 2024-04-02T04:48:00.918 INFO:tasks.workunit.client.1.smithi179.stdout:6/781: mknod d3/d2d/da5/c116 0 2024-04-02T04:48:00.919 INFO:tasks.workunit.client.0.smithi070.stdout:8/730: mkdir d2/d8/d29/dc7/df0 0 2024-04-02T04:48:00.920 INFO:tasks.workunit.client.1.smithi179.stdout:0/874: dwrite d1/da/d48/d65/d7d/fd1 [0,4194304] 0 2024-04-02T04:48:00.922 INFO:tasks.workunit.client.1.smithi179.stdout:9/782: link d0/d19/d27/d7a/d82/f98 d0/d19/d27/d7a/d7e/dd4/fef 0 2024-04-02T04:48:00.924 INFO:tasks.workunit.client.1.smithi179.stdout:8/809: symlink d3/d32/d21/d49/df0/l11f 0 2024-04-02T04:48:00.932 INFO:tasks.workunit.client.1.smithi179.stdout:5/758: creat d8/d1b/d23/d32/dbe/daf/db7/dad/df1/ffb x:0 0 0 2024-04-02T04:48:00.934 INFO:tasks.workunit.client.0.smithi070.stdout:1/689: fsync d0/d7/f6a 0 2024-04-02T04:48:00.936 INFO:tasks.workunit.client.0.smithi070.stdout:1/690: truncate d0/d9/d1d/d22/d2c/fb3 4621056 0 2024-04-02T04:48:00.937 INFO:tasks.workunit.client.1.smithi179.stdout:2/667: symlink dc/d29/d31/d57/d97/dbf/de3/le4 0 2024-04-02T04:48:00.937 INFO:tasks.workunit.client.1.smithi179.stdout:2/668: fsync dc/d29/d31/fb9 0 2024-04-02T04:48:00.941 INFO:tasks.workunit.client.1.smithi179.stdout:6/782: mkdir d3/d2c/d85/d117 0 2024-04-02T04:48:00.942 INFO:tasks.workunit.client.0.smithi070.stdout:8/731: mkdir d2/d8/d29/d72/df1 0 2024-04-02T04:48:00.944 INFO:tasks.workunit.client.0.smithi070.stdout:8/732: truncate d2/d8/d29/d72/fd0 1019097 0 2024-04-02T04:48:00.946 INFO:tasks.workunit.client.0.smithi070.stdout:7/658: rename d7/dd/df/d7a/d6c/d77/d54/lac to d7/dd/df/d79/d46/ld5 0 2024-04-02T04:48:00.947 INFO:tasks.workunit.client.0.smithi070.stdout:7/659: chown d7/d4e/d82/cc1 1 1 2024-04-02T04:48:00.949 INFO:tasks.workunit.client.1.smithi179.stdout:8/810: mkdir d3/d32/d21/d22/d120 0 2024-04-02T04:48:00.950 INFO:tasks.workunit.client.1.smithi179.stdout:4/804: write d1/d3/df/d10/fa1 [873097,91916] 0 2024-04-02T04:48:00.955 INFO:tasks.workunit.client.1.smithi179.stdout:1/809: link d3/d5/d2f/d4f/d42/d55/fed d3/d5/d96/f119 0 2024-04-02T04:48:00.955 INFO:tasks.workunit.client.1.smithi179.stdout:7/815: dwrite d1/d9/f1d [0,4194304] 0 2024-04-02T04:48:00.955 INFO:tasks.workunit.client.1.smithi179.stdout:3/818: write d8/d26/d36/f59 [2439292,55391] 0 2024-04-02T04:48:00.955 INFO:tasks.workunit.client.0.smithi070.stdout:3/732: creat d10/d17/db2/ffb x:0 0 0 2024-04-02T04:48:00.957 INFO:tasks.workunit.client.0.smithi070.stdout:5/867: sync 2024-04-02T04:48:00.957 INFO:tasks.workunit.client.1.smithi179.stdout:5/759: mknod d8/d7e/cfc 0 2024-04-02T04:48:00.958 INFO:tasks.workunit.client.0.smithi070.stdout:5/868: readlink d5/d9/d10/d6a/l9c 0 2024-04-02T04:48:00.958 INFO:tasks.workunit.client.0.smithi070.stdout:4/686: getdents d1/d3/d85/da8 0 2024-04-02T04:48:00.960 INFO:tasks.workunit.client.0.smithi070.stdout:6/603: getdents d0/d55/d5c 0 2024-04-02T04:48:00.962 INFO:tasks.workunit.client.1.smithi179.stdout:5/760: read d8/d1b/d23/d32/f3d [688460,18223] 0 2024-04-02T04:48:00.963 INFO:tasks.workunit.client.1.smithi179.stdout:6/783: chown d3/d12/d23/d39/d4e/lb9 15 1 2024-04-02T04:48:00.964 INFO:tasks.workunit.client.0.smithi070.stdout:8/733: write d2/d3/d35/d3a/d4f/d59/f6e [2225154,51673] 0 2024-04-02T04:48:00.966 INFO:tasks.workunit.client.1.smithi179.stdout:5/761: fdatasync d8/d1b/d23/d32/d4e/f59 0 2024-04-02T04:48:00.966 INFO:tasks.workunit.client.1.smithi179.stdout:0/875: mknod d1/da/d52/d75/d9a/dec/cfd 0 2024-04-02T04:48:00.967 INFO:tasks.workunit.client.1.smithi179.stdout:3/819: dwrite d8/dd3/fd9 [0,4194304] 0 2024-04-02T04:48:00.968 INFO:tasks.workunit.client.0.smithi070.stdout:3/733: dread d10/ff1 [0,4194304] 0 2024-04-02T04:48:00.969 INFO:tasks.workunit.client.1.smithi179.stdout:4/805: dread d1/d3/fa [0,4194304] 0 2024-04-02T04:48:00.969 INFO:tasks.workunit.client.1.smithi179.stdout:7/816: dwrite d1/d9/d35/d6b/f105 [0,4194304] 0 2024-04-02T04:48:00.969 INFO:tasks.workunit.client.0.smithi070.stdout:9/640: rename d4/d10/d11/d7b/d19/d2f/d39/fb6 to d4/d10/d11/d7b/d19/d2f/d4d/d92/fd1 0 2024-04-02T04:48:00.969 INFO:tasks.workunit.client.0.smithi070.stdout:1/691: write d0/d7/f81 [7375936,29718] 0 2024-04-02T04:48:00.971 INFO:tasks.workunit.client.1.smithi179.stdout:4/806: stat d1/d3/dc/d27/dae/de5/d102 0 2024-04-02T04:48:00.974 INFO:tasks.workunit.client.1.smithi179.stdout:9/783: creat d0/d19/d27/d7a/dba/ff0 x:0 0 0 2024-04-02T04:48:00.975 INFO:tasks.workunit.client.1.smithi179.stdout:5/762: dread d8/d1b/d23/d32/d46/f7d [0,4194304] 0 2024-04-02T04:48:00.976 INFO:tasks.workunit.client.0.smithi070.stdout:2/779: link d2/d12/d59/fec d2/d10/f102 0 2024-04-02T04:48:00.979 INFO:tasks.workunit.client.1.smithi179.stdout:8/811: fsync d3/d32/f19 0 2024-04-02T04:48:00.981 INFO:tasks.workunit.client.1.smithi179.stdout:9/784: dwrite d0/d1/d10/d6d/d58/f96 [0,4194304] 0 2024-04-02T04:48:00.982 INFO:tasks.workunit.client.1.smithi179.stdout:1/810: rmdir d3/d5/d2f/d65 39 2024-04-02T04:48:00.983 INFO:tasks.workunit.client.1.smithi179.stdout:1/811: stat d3/d5 0 2024-04-02T04:48:00.990 INFO:tasks.workunit.client.1.smithi179.stdout:6/784: fsync d3/d12/d23/d39/fda 0 2024-04-02T04:48:00.990 INFO:tasks.workunit.client.0.smithi070.stdout:5/869: truncate d5/d9/d23/f6e 4591833 0 2024-04-02T04:48:00.992 INFO:tasks.workunit.client.0.smithi070.stdout:5/870: readlink d5/led 0 2024-04-02T04:48:00.999 INFO:tasks.workunit.client.0.smithi070.stdout:8/734: rmdir d2/d8/d23/d24/d5f 39 2024-04-02T04:48:01.000 INFO:tasks.workunit.client.0.smithi070.stdout:8/735: chown d2/d8/d23/d24/d79/ce7 1 1 2024-04-02T04:48:01.001 INFO:tasks.workunit.client.1.smithi179.stdout:3/820: creat d8/d26/d83/dbc/f100 x:0 0 0 2024-04-02T04:48:01.003 INFO:tasks.workunit.client.1.smithi179.stdout:0/876: truncate d1/da/d67/fa1 177422 0 2024-04-02T04:48:01.006 INFO:tasks.workunit.client.0.smithi070.stdout:9/641: unlink d4/d13/l46 0 2024-04-02T04:48:01.007 INFO:tasks.workunit.client.0.smithi070.stdout:7/660: dwrite d7/dd/df/d34/f2a [0,4194304] 0 2024-04-02T04:48:01.008 INFO:tasks.workunit.client.1.smithi179.stdout:5/763: unlink d8/d1b/d23/d32/d46/c82 0 2024-04-02T04:48:01.010 INFO:tasks.workunit.client.0.smithi070.stdout:1/692: dread - d0/d54/d64/d6d/db6/fba zero size 2024-04-02T04:48:01.010 INFO:tasks.workunit.client.0.smithi070.stdout:9/642: write d4/d10/d11/d7b/d19/d2f/d39/f44 [4274985,55234] 0 2024-04-02T04:48:01.011 INFO:tasks.workunit.client.0.smithi070.stdout:1/693: chown d0/db4/fa4 14 1 2024-04-02T04:48:01.013 INFO:tasks.workunit.client.0.smithi070.stdout:1/694: readlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/ldc 0 2024-04-02T04:48:01.013 INFO:tasks.workunit.client.0.smithi070.stdout:1/695: chown d0/d9/lf 4223 1 2024-04-02T04:48:01.016 INFO:tasks.workunit.client.1.smithi179.stdout:8/812: creat d3/d6c/f121 x:0 0 0 2024-04-02T04:48:01.016 INFO:tasks.workunit.client.0.smithi070.stdout:0/787: getdents d4/d5/d58/d60/d83/da5 0 2024-04-02T04:48:01.017 INFO:tasks.workunit.client.0.smithi070.stdout:0/788: readlink d4/d5/d58/d60/d83/lbb 0 2024-04-02T04:48:01.018 INFO:tasks.workunit.client.1.smithi179.stdout:1/812: mknod d3/d5/d2f/d5a/dcc/c11a 0 2024-04-02T04:48:01.024 INFO:tasks.workunit.client.1.smithi179.stdout:0/877: truncate d1/f23 2676189 0 2024-04-02T04:48:01.028 INFO:tasks.workunit.client.0.smithi070.stdout:2/780: write d2/d10/d5c/d7a/db0/fe3 [947526,88020] 0 2024-04-02T04:48:01.030 INFO:tasks.workunit.client.0.smithi070.stdout:3/734: truncate f0 4202977 0 2024-04-02T04:48:01.032 INFO:tasks.workunit.client.1.smithi179.stdout:8/813: fdatasync d3/d32/d21/d50/d56/dad/dbc/fd0 0 2024-04-02T04:48:01.033 INFO:tasks.workunit.client.0.smithi070.stdout:3/735: dread d10/f4f [0,4194304] 0 2024-04-02T04:48:01.035 INFO:tasks.workunit.client.0.smithi070.stdout:7/661: rename d7/dd/df/d7a/dbc/ld2 to d7/d9/ld6 0 2024-04-02T04:48:01.036 INFO:tasks.workunit.client.1.smithi179.stdout:4/807: dread d1/d3/f28 [0,4194304] 0 2024-04-02T04:48:01.041 INFO:tasks.workunit.client.1.smithi179.stdout:3/821: symlink d8/d26/l101 0 2024-04-02T04:48:01.042 INFO:tasks.workunit.client.1.smithi179.stdout:3/822: truncate d8/d57/d63/dc3/fdd 523399 0 2024-04-02T04:48:01.042 INFO:tasks.workunit.client.1.smithi179.stdout:7/817: creat d1/d9/d55/dbf/d9f/f11e x:0 0 0 2024-04-02T04:48:01.044 INFO:tasks.workunit.client.0.smithi070.stdout:0/789: fsync d4/d5/d13/d23/f4d 0 2024-04-02T04:48:01.045 INFO:tasks.workunit.client.1.smithi179.stdout:5/764: truncate d8/d1b/f88 1083369 0 2024-04-02T04:48:01.046 INFO:tasks.workunit.client.0.smithi070.stdout:9/643: write d4/d13/f37 [484143,79363] 0 2024-04-02T04:48:01.049 INFO:tasks.workunit.client.0.smithi070.stdout:0/790: dread d4/d5/da/d2d/f66 [0,4194304] 0 2024-04-02T04:48:01.049 INFO:tasks.workunit.client.1.smithi179.stdout:2/669: sync 2024-04-02T04:48:01.049 INFO:tasks.workunit.client.0.smithi070.stdout:5/871: symlink d5/d17/d27/l110 0 2024-04-02T04:48:01.050 INFO:tasks.workunit.client.1.smithi179.stdout:1/813: symlink d3/d5/d2f/d4f/dab/dcd/l11b 0 2024-04-02T04:48:01.051 INFO:tasks.workunit.client.0.smithi070.stdout:6/604: rmdir d0/d55/d79/d8d/d97 0 2024-04-02T04:48:01.051 INFO:tasks.workunit.client.1.smithi179.stdout:1/814: chown d3/d5/d2f/d4f/d42/d53/f108 458790 1 2024-04-02T04:48:01.052 INFO:tasks.workunit.client.1.smithi179.stdout:4/808: fdatasync d1/d3/dc/d27/dae/f10c 0 2024-04-02T04:48:01.053 INFO:tasks.workunit.client.0.smithi070.stdout:8/736: creat d2/d3/d35/d3a/d4f/de8/ff2 x:0 0 0 2024-04-02T04:48:01.057 INFO:tasks.workunit.client.0.smithi070.stdout:6/605: dwrite d0/d43/d95/f9a [4194304,4194304] 0 2024-04-02T04:48:01.059 INFO:tasks.workunit.client.0.smithi070.stdout:2/781: symlink d2/d10/d19/dd7/l103 0 2024-04-02T04:48:01.065 INFO:tasks.workunit.client.1.smithi179.stdout:5/765: symlink d8/d1b/d23/d36/dcb/lfd 0 2024-04-02T04:48:01.068 INFO:tasks.workunit.client.0.smithi070.stdout:7/662: symlink d7/d36/ld7 0 2024-04-02T04:48:01.074 INFO:tasks.workunit.client.1.smithi179.stdout:4/809: mkdir d1/d3/dc/dc6/d111 0 2024-04-02T04:48:01.075 INFO:tasks.workunit.client.1.smithi179.stdout:4/810: chown d1/d3/dc/d27/d62/f5e 139793 1 2024-04-02T04:48:01.076 INFO:tasks.workunit.client.1.smithi179.stdout:6/785: getdents d3/d12/d23/d39/d80/d93/dd4 0 2024-04-02T04:48:01.076 INFO:tasks.workunit.client.0.smithi070.stdout:3/736: dread d10/d17/f74 [0,4194304] 0 2024-04-02T04:48:01.077 INFO:tasks.workunit.client.0.smithi070.stdout:3/737: read - d10/d17/db2/ff8 zero size 2024-04-02T04:48:01.077 INFO:tasks.workunit.client.0.smithi070.stdout:4/687: link d1/d3/l6 d1/d3/d85/d8e/dc8/le3 0 2024-04-02T04:48:01.080 INFO:tasks.workunit.client.0.smithi070.stdout:5/872: read - d5/d17/d2b/f9a zero size 2024-04-02T04:48:01.085 INFO:tasks.workunit.client.0.smithi070.stdout:2/782: creat d2/d10/d21/d71/f104 x:0 0 0 2024-04-02T04:48:01.085 INFO:tasks.workunit.client.0.smithi070.stdout:2/783: chown d2/d10/d19/dd7/l101 14365 1 2024-04-02T04:48:01.089 INFO:tasks.workunit.client.0.smithi070.stdout:1/696: rmdir d0/db4/dcc 0 2024-04-02T04:48:01.093 INFO:tasks.workunit.client.0.smithi070.stdout:9/644: creat d4/d10/d11/d7b/d19/d2f/d39/d57/da6/fd2 x:0 0 0 2024-04-02T04:48:01.095 INFO:tasks.workunit.client.0.smithi070.stdout:9/645: dread - d4/d10/d11/d7b/d19/d2f/d39/d57/fa9 zero size 2024-04-02T04:48:01.097 INFO:tasks.workunit.client.0.smithi070.stdout:9/646: chown d4/d13/f98 200696522 1 2024-04-02T04:48:01.106 INFO:tasks.workunit.client.0.smithi070.stdout:1/697: creat d0/d4c/da2/fdf x:0 0 0 2024-04-02T04:48:01.106 INFO:tasks.workunit.client.0.smithi070.stdout:1/698: read - d0/d4c/da2/faf zero size 2024-04-02T04:48:01.109 INFO:tasks.workunit.client.0.smithi070.stdout:4/688: creat d1/d3/d85/d8e/dc8/de2/fe4 x:0 0 0 2024-04-02T04:48:01.112 INFO:tasks.workunit.client.1.smithi179.stdout:9/785: sync 2024-04-02T04:48:01.119 INFO:tasks.workunit.client.0.smithi070.stdout:1/699: creat d0/d9/fe0 x:0 0 0 2024-04-02T04:48:01.121 INFO:tasks.workunit.client.0.smithi070.stdout:3/738: creat d10/d32/d8a/ffc x:0 0 0 2024-04-02T04:48:01.122 INFO:tasks.workunit.client.0.smithi070.stdout:3/739: chown d10/d17/f2a 107 1 2024-04-02T04:48:01.126 INFO:tasks.workunit.client.0.smithi070.stdout:9/647: dread - d4/d10/d11/d7b/d19/d2f/d4d/d92/fd1 zero size 2024-04-02T04:48:01.127 INFO:tasks.workunit.client.0.smithi070.stdout:6/606: link d0/l1a d0/d55/d79/lb6 0 2024-04-02T04:48:01.129 INFO:tasks.workunit.client.1.smithi179.stdout:3/823: symlink d8/d26/d83/l102 0 2024-04-02T04:48:01.129 INFO:tasks.workunit.client.0.smithi070.stdout:0/791: dwrite d4/d5/f3f [0,4194304] 0 2024-04-02T04:48:01.131 INFO:tasks.workunit.client.0.smithi070.stdout:1/700: creat d0/db4/fe1 x:0 0 0 2024-04-02T04:48:01.133 INFO:tasks.workunit.client.1.smithi179.stdout:8/814: creat d3/f122 x:0 0 0 2024-04-02T04:48:01.134 INFO:tasks.workunit.client.1.smithi179.stdout:8/815: read - d3/d32/d21/d45/d5a/f101 zero size 2024-04-02T04:48:01.134 INFO:tasks.workunit.client.1.smithi179.stdout:2/670: creat dc/d29/d4a/d1e/d6c/dd2/fe5 x:0 0 0 2024-04-02T04:48:01.135 INFO:tasks.workunit.client.0.smithi070.stdout:5/873: dwrite d5/d9/d23/d30/da2/d75/fd3 [0,4194304] 0 2024-04-02T04:48:01.144 INFO:tasks.workunit.client.0.smithi070.stdout:3/740: chown d10/d32/d8a/d8d/d91 147107885 1 2024-04-02T04:48:01.144 INFO:tasks.workunit.client.0.smithi070.stdout:5/874: write d5/d9/d23/d30/da2/d75/f72 [550503,103394] 0 2024-04-02T04:48:01.145 INFO:tasks.workunit.client.1.smithi179.stdout:9/786: mkdir d0/d1/d11/dab/df1 0 2024-04-02T04:48:01.146 INFO:tasks.workunit.client.0.smithi070.stdout:2/784: dwrite d2/d10/f102 [0,4194304] 0 2024-04-02T04:48:01.147 INFO:tasks.workunit.client.1.smithi179.stdout:0/878: sync 2024-04-02T04:48:01.149 INFO:tasks.workunit.client.0.smithi070.stdout:7/663: dwrite d7/d36/fb9 [0,4194304] 0 2024-04-02T04:48:01.149 INFO:tasks.workunit.client.0.smithi070.stdout:5/875: fdatasync d5/d9/d23/d30/da2/f102 0 2024-04-02T04:48:01.149 INFO:tasks.workunit.client.1.smithi179.stdout:3/824: symlink d8/d57/d63/dc3/l103 0 2024-04-02T04:48:01.154 INFO:tasks.workunit.client.0.smithi070.stdout:9/648: mkdir d4/d10/d11/d7b/d19/d53/d9c/dd3 0 2024-04-02T04:48:01.155 INFO:tasks.workunit.client.1.smithi179.stdout:8/816: creat d3/d32/d21/d22/f123 x:0 0 0 2024-04-02T04:48:01.156 INFO:tasks.workunit.client.0.smithi070.stdout:1/701: creat d0/d54/d64/d6d/db6/fe2 x:0 0 0 2024-04-02T04:48:01.162 INFO:tasks.workunit.client.0.smithi070.stdout:6/607: mknod d0/d3/d19/d31/d61/d7b/da6/cb7 0 2024-04-02T04:48:01.162 INFO:tasks.workunit.client.1.smithi179.stdout:2/671: dwrite dc/d29/d31/f77 [0,4194304] 0 2024-04-02T04:48:01.163 INFO:tasks.workunit.client.0.smithi070.stdout:9/649: dwrite d4/d10/d12/d55/f7e [4194304,4194304] 0 2024-04-02T04:48:01.165 INFO:tasks.workunit.client.0.smithi070.stdout:4/689: write d1/f1e [1707450,126751] 0 2024-04-02T04:48:01.168 INFO:tasks.workunit.client.0.smithi070.stdout:2/785: chown d2/fb 2 1 2024-04-02T04:48:01.169 INFO:tasks.workunit.client.0.smithi070.stdout:9/650: read d4/d10/d11/da8/fb5 [548828,83895] 0 2024-04-02T04:48:01.170 INFO:tasks.workunit.client.1.smithi179.stdout:0/879: mkdir d1/dfe 0 2024-04-02T04:48:01.170 INFO:tasks.workunit.client.0.smithi070.stdout:0/792: dread d4/d5/da/d2d/f4c [0,4194304] 0 2024-04-02T04:48:01.170 INFO:tasks.workunit.client.0.smithi070.stdout:7/664: symlink d7/dd/db5/ld8 0 2024-04-02T04:48:01.171 INFO:tasks.workunit.client.1.smithi179.stdout:3/825: rmdir d8/d26/d52 39 2024-04-02T04:48:01.172 INFO:tasks.workunit.client.0.smithi070.stdout:9/651: read - d4/d10/d11/d7b/d19/d2f/d8c/fd0 zero size 2024-04-02T04:48:01.175 INFO:tasks.workunit.client.1.smithi179.stdout:1/815: rename d3/d5/f27 to d3/d5/d2f/d4f/d35/f11c 0 2024-04-02T04:48:01.180 INFO:tasks.workunit.client.1.smithi179.stdout:2/672: symlink dc/d29/d31/d83/d8d/le6 0 2024-04-02T04:48:01.181 INFO:tasks.workunit.client.1.smithi179.stdout:2/673: fsync dc/d29/d4a/d1e/d8a/fcc 0 2024-04-02T04:48:01.182 INFO:tasks.workunit.client.1.smithi179.stdout:2/674: readlink dc/d29/d4a/d4b/l92 0 2024-04-02T04:48:01.183 INFO:tasks.workunit.client.1.smithi179.stdout:9/787: mknod d0/d1/d11/d29/d86/db0/cf2 0 2024-04-02T04:48:01.183 INFO:tasks.workunit.client.1.smithi179.stdout:4/811: write d1/d3/df/ffc [625196,46024] 0 2024-04-02T04:48:01.184 INFO:tasks.workunit.client.1.smithi179.stdout:4/812: readlink d1/d3/df/d10/l107 0 2024-04-02T04:48:01.185 INFO:tasks.workunit.client.0.smithi070.stdout:6/608: dread d0/d55/d8b/d94/f65 [0,4194304] 0 2024-04-02T04:48:01.187 INFO:tasks.workunit.client.0.smithi070.stdout:3/741: dwrite d10/d32/f63 [0,4194304] 0 2024-04-02T04:48:01.188 INFO:tasks.workunit.client.0.smithi070.stdout:3/742: readlink d10/d32/d3f/l26 0 2024-04-02T04:48:01.189 INFO:tasks.workunit.client.0.smithi070.stdout:3/743: chown d10/d17/f48 70 1 2024-04-02T04:48:01.189 INFO:tasks.workunit.client.0.smithi070.stdout:4/690: rename d1/d3/d85/fe0 to d1/d3/d85/da8/fe5 0 2024-04-02T04:48:01.192 INFO:tasks.workunit.client.0.smithi070.stdout:2/786: fdatasync d2/d10/d5c/d7a/db0/dd8/dd2/fcf 0 2024-04-02T04:48:01.193 INFO:tasks.workunit.client.0.smithi070.stdout:2/787: dread - d2/d24/ff5 zero size 2024-04-02T04:48:01.195 INFO:tasks.workunit.client.1.smithi179.stdout:5/766: getdents d8/d1b/d23/d42 0 2024-04-02T04:48:01.196 INFO:tasks.workunit.client.1.smithi179.stdout:1/816: mknod d3/da8/c11d 0 2024-04-02T04:48:01.198 INFO:tasks.workunit.client.0.smithi070.stdout:7/665: mknod d7/d36/cd9 0 2024-04-02T04:48:01.201 INFO:tasks.workunit.client.1.smithi179.stdout:7/818: rename d1/dd/d23/dab/db0 to d1/d9/d27/d87/d8b/d75/d11f 0 2024-04-02T04:48:01.201 INFO:tasks.workunit.client.1.smithi179.stdout:0/880: write d1/f3b [1744841,63534] 0 2024-04-02T04:48:01.205 INFO:tasks.workunit.client.1.smithi179.stdout:9/788: unlink d0/d19/d27/d7a/d87/daf/fcd 0 2024-04-02T04:48:01.206 INFO:tasks.workunit.client.1.smithi179.stdout:4/813: creat d1/d3/d19/f112 x:0 0 0 2024-04-02T04:48:01.215 INFO:tasks.workunit.client.1.smithi179.stdout:6/786: rename d3/d12/d44/fb3 to d3/d12/d44/df1/f118 0 2024-04-02T04:48:01.216 INFO:tasks.workunit.client.1.smithi179.stdout:6/787: chown d3/d12/d44/df1/f118 160 1 2024-04-02T04:48:01.217 INFO:tasks.workunit.client.1.smithi179.stdout:6/788: write d3/d12/d44/fbb [992872,54374] 0 2024-04-02T04:48:01.217 INFO:tasks.workunit.client.1.smithi179.stdout:2/675: mkdir dc/d29/de7 0 2024-04-02T04:48:01.219 INFO:tasks.workunit.client.1.smithi179.stdout:6/789: fsync d3/d12/d23/d26/dd3/f100 0 2024-04-02T04:48:01.220 INFO:tasks.workunit.client.1.smithi179.stdout:3/826: mknod d8/d26/d52/d7e/ddf/c104 0 2024-04-02T04:48:01.220 INFO:tasks.workunit.client.1.smithi179.stdout:6/790: stat d3/d2c/d85/ffd 0 2024-04-02T04:48:01.225 INFO:tasks.workunit.client.1.smithi179.stdout:4/814: dread d1/d3/dc/d27/f97 [0,4194304] 0 2024-04-02T04:48:01.226 INFO:tasks.workunit.client.1.smithi179.stdout:1/817: read d3/d5/d2f/d4f/d42/d53/d72/d86/fa6 [1125951,54560] 0 2024-04-02T04:48:01.228 INFO:tasks.workunit.client.1.smithi179.stdout:0/881: dwrite d1/da/d28/d81/fab [0,4194304] 0 2024-04-02T04:48:01.228 INFO:tasks.workunit.client.1.smithi179.stdout:4/815: chown d1/d3/dc/dc6/f10f 9 1 2024-04-02T04:48:01.229 INFO:tasks.workunit.client.1.smithi179.stdout:7/819: dwrite d1/d9/d55/dbf/d9f/da8/dda/fc0 [0,4194304] 0 2024-04-02T04:48:01.229 INFO:tasks.workunit.client.1.smithi179.stdout:0/882: fsync d1/f22 0 2024-04-02T04:48:01.232 INFO:tasks.workunit.client.0.smithi070.stdout:3/744: creat d10/d32/d4b/dc5/de0/ffd x:0 0 0 2024-04-02T04:48:01.232 INFO:tasks.workunit.client.0.smithi070.stdout:5/876: creat d5/d17/da7/f111 x:0 0 0 2024-04-02T04:48:01.232 INFO:tasks.workunit.client.0.smithi070.stdout:4/691: dwrite d1/d3/d85/d8e/fc5 [0,4194304] 0 2024-04-02T04:48:01.234 INFO:tasks.workunit.client.1.smithi179.stdout:2/676: symlink dc/d29/d31/d83/dae/le8 0 2024-04-02T04:48:01.236 INFO:tasks.workunit.client.1.smithi179.stdout:8/817: rename d3/d1b/d2a/fdb to d3/d32/d21/d22/d6b/db1/df8/f124 0 2024-04-02T04:48:01.236 INFO:tasks.workunit.client.0.smithi070.stdout:1/702: dwrite d0/d54/d64/d6d/f78 [4194304,4194304] 0 2024-04-02T04:48:01.237 INFO:tasks.workunit.client.1.smithi179.stdout:8/818: fsync d3/d6c/f89 0 2024-04-02T04:48:01.238 INFO:tasks.workunit.client.0.smithi070.stdout:6/609: dread d0/d3/d19/f3f [0,4194304] 0 2024-04-02T04:48:01.238 INFO:tasks.workunit.client.0.smithi070.stdout:2/788: mknod d2/d10/d5c/d7a/db0/c105 0 2024-04-02T04:48:01.242 INFO:tasks.workunit.client.1.smithi179.stdout:5/767: write d8/d1b/f88 [487147,100150] 0 2024-04-02T04:48:01.248 INFO:tasks.workunit.client.0.smithi070.stdout:5/877: chown d5/d17/d2b/f67 30 1 2024-04-02T04:48:01.248 INFO:tasks.workunit.client.0.smithi070.stdout:9/652: dread d4/d10/d12/d27/d60/d9e/f77 [0,4194304] 0 2024-04-02T04:48:01.250 INFO:tasks.workunit.client.1.smithi179.stdout:5/768: dread d8/d1b/d23/d30/f71 [0,4194304] 0 2024-04-02T04:48:01.250 INFO:tasks.workunit.client.1.smithi179.stdout:9/789: write d0/d1/d11/d29/f37 [103242,119520] 0 2024-04-02T04:48:01.252 INFO:tasks.workunit.client.0.smithi070.stdout:7/666: rename d7/d36/cd9 to d7/dd/df/d34/cda 0 2024-04-02T04:48:01.253 INFO:tasks.workunit.client.1.smithi179.stdout:0/883: rmdir d1/da/df/d56/d84 39 2024-04-02T04:48:01.260 INFO:tasks.workunit.client.1.smithi179.stdout:3/827: mknod d8/d26/d52/d7e/de1/c105 0 2024-04-02T04:48:01.262 INFO:tasks.workunit.client.0.smithi070.stdout:2/789: creat d2/d10/d5c/d7a/db0/de4/f106 x:0 0 0 2024-04-02T04:48:01.265 INFO:tasks.workunit.client.0.smithi070.stdout:3/745: dwrite d10/f4f [0,4194304] 0 2024-04-02T04:48:01.266 INFO:tasks.workunit.client.0.smithi070.stdout:4/692: mkdir d1/d3/d9/de6 0 2024-04-02T04:48:01.266 INFO:tasks.workunit.client.1.smithi179.stdout:6/791: fdatasync d3/d12/d23/d39/d4e/d66/fc6 0 2024-04-02T04:48:01.273 INFO:tasks.workunit.client.1.smithi179.stdout:1/818: creat d3/d5/d96/f11e x:0 0 0 2024-04-02T04:48:01.274 INFO:tasks.workunit.client.0.smithi070.stdout:1/703: dread d0/d7/d28/d5c/d6e/fd9 [0,4194304] 0 2024-04-02T04:48:01.275 INFO:tasks.workunit.client.1.smithi179.stdout:9/790: truncate d0/d1/d11/d29/f50 5053462 0 2024-04-02T04:48:01.276 INFO:tasks.workunit.client.1.smithi179.stdout:6/792: dread d3/d2d/da5/fbd [0,4194304] 0 2024-04-02T04:48:01.276 INFO:tasks.workunit.client.0.smithi070.stdout:3/746: truncate d10/d32/d3f/f4d 1998235 0 2024-04-02T04:48:01.278 INFO:tasks.workunit.client.1.smithi179.stdout:0/884: write d1/da/df/d56/d84/fc5 [855121,124521] 0 2024-04-02T04:48:01.278 INFO:tasks.workunit.client.0.smithi070.stdout:4/693: write d1/d9c/d65/f6a [843138,115874] 0 2024-04-02T04:48:01.280 INFO:tasks.workunit.client.0.smithi070.stdout:4/694: chown d1/d3/f4c 0 1 2024-04-02T04:48:01.282 INFO:tasks.workunit.client.0.smithi070.stdout:5/878: link d5/d57/f5d d5/d17/d6c/f112 0 2024-04-02T04:48:01.285 INFO:tasks.workunit.client.0.smithi070.stdout:9/653: write d4/d10/f25 [323935,105004] 0 2024-04-02T04:48:01.286 INFO:tasks.workunit.client.0.smithi070.stdout:9/654: dread d4/d34/fc1 [0,4194304] 0 2024-04-02T04:48:01.292 INFO:tasks.workunit.client.0.smithi070.stdout:7/667: dwrite d7/dd/df/d7a/d6c/d77/d75/fb3 [0,4194304] 0 2024-04-02T04:48:01.294 INFO:tasks.workunit.client.0.smithi070.stdout:4/695: truncate d1/d3/f95 959465 0 2024-04-02T04:48:01.297 INFO:tasks.workunit.client.1.smithi179.stdout:7/820: creat d1/d9/d55/dbf/d9f/f120 x:0 0 0 2024-04-02T04:48:01.298 INFO:tasks.workunit.client.0.smithi070.stdout:2/790: dwrite d2/fb [4194304,4194304] 0 2024-04-02T04:48:01.298 INFO:tasks.workunit.client.0.smithi070.stdout:4/696: fdatasync d1/f12 0 2024-04-02T04:48:01.300 INFO:tasks.workunit.client.0.smithi070.stdout:6/610: getdents d0/d3/d19 0 2024-04-02T04:48:01.301 INFO:tasks.workunit.client.0.smithi070.stdout:9/655: dread d4/d10/d11/f2d [0,4194304] 0 2024-04-02T04:48:01.303 INFO:tasks.workunit.client.0.smithi070.stdout:5/879: rename d5/d9/d10/db9/dba/cd1 to d5/d9/d23/d30/c113 0 2024-04-02T04:48:01.306 INFO:tasks.workunit.client.0.smithi070.stdout:3/747: dwrite d10/ff1 [0,4194304] 0 2024-04-02T04:48:01.306 INFO:tasks.workunit.client.1.smithi179.stdout:9/791: creat d0/d19/d27/d7a/d87/ff3 x:0 0 0 2024-04-02T04:48:01.307 INFO:tasks.workunit.client.1.smithi179.stdout:6/793: truncate d3/d12/d23/f55 3718243 0 2024-04-02T04:48:01.308 INFO:tasks.workunit.client.0.smithi070.stdout:3/748: fdatasync d10/d17/f48 0 2024-04-02T04:48:01.310 INFO:tasks.workunit.client.1.smithi179.stdout:4/816: link d1/d3/df/d10/d2f/d7b/f86 d1/d15/d7c/d98/f113 0 2024-04-02T04:48:01.311 INFO:tasks.workunit.client.0.smithi070.stdout:7/668: mkdir d7/d4e/d82/d57/ddb 0 2024-04-02T04:48:01.312 INFO:tasks.workunit.client.0.smithi070.stdout:7/669: fsync d7/d36/fc6 0 2024-04-02T04:48:01.316 INFO:tasks.workunit.client.1.smithi179.stdout:2/677: creat dc/d29/d4a/fe9 x:0 0 0 2024-04-02T04:48:01.316 INFO:tasks.workunit.client.1.smithi179.stdout:2/678: stat dc/d79/lb3 0 2024-04-02T04:48:01.317 INFO:tasks.workunit.client.1.smithi179.stdout:2/679: truncate dc/fc4 527500 0 2024-04-02T04:48:01.317 INFO:tasks.workunit.client.1.smithi179.stdout:2/680: readlink dc/d29/d31/d57/l99 0 2024-04-02T04:48:01.318 INFO:tasks.workunit.client.1.smithi179.stdout:2/681: readlink dc/d79/lb3 0 2024-04-02T04:48:01.321 INFO:tasks.workunit.client.0.smithi070.stdout:2/791: dread d2/d10/d21/d3d/d42/fad [0,4194304] 0 2024-04-02T04:48:01.322 INFO:tasks.workunit.client.0.smithi070.stdout:2/792: read d2/d12/d59/f8e [405785,9397] 0 2024-04-02T04:48:01.322 INFO:tasks.workunit.client.1.smithi179.stdout:3/828: truncate d8/d26/d36/fb9 844020 0 2024-04-02T04:48:01.323 INFO:tasks.workunit.client.0.smithi070.stdout:2/793: truncate d2/d12/d3e/fac 5224035 0 2024-04-02T04:48:01.323 INFO:tasks.workunit.client.0.smithi070.stdout:6/611: mknod d0/d20/d5f/cb8 0 2024-04-02T04:48:01.325 INFO:tasks.workunit.client.1.smithi179.stdout:8/819: creat d3/d1b/d2a/f125 x:0 0 0 2024-04-02T04:48:01.326 INFO:tasks.workunit.client.0.smithi070.stdout:9/656: mknod d4/cd4 0 2024-04-02T04:48:01.327 INFO:tasks.workunit.client.1.smithi179.stdout:8/820: write d3/d32/d21/d50/d112/f115 [168155,126596] 0 2024-04-02T04:48:01.327 INFO:tasks.workunit.client.0.smithi070.stdout:6/612: dread d0/d20/f34 [0,4194304] 0 2024-04-02T04:48:01.330 INFO:tasks.workunit.client.0.smithi070.stdout:5/880: dread d5/d17/d50/f10f [0,4194304] 0 2024-04-02T04:48:01.332 INFO:tasks.workunit.client.0.smithi070.stdout:5/881: fdatasync d5/d17/d6c/f41 0 2024-04-02T04:48:01.333 INFO:tasks.workunit.client.1.smithi179.stdout:3/829: dread d8/f1b [4194304,4194304] 0 2024-04-02T04:48:01.334 INFO:tasks.workunit.client.1.smithi179.stdout:3/830: fsync d8/dd3/fd9 0 2024-04-02T04:48:01.334 INFO:tasks.workunit.client.1.smithi179.stdout:3/831: dread - d8/fa1 zero size 2024-04-02T04:48:01.337 INFO:tasks.workunit.client.1.smithi179.stdout:1/819: write d3/d5/f8d [995866,84096] 0 2024-04-02T04:48:01.339 INFO:tasks.workunit.client.1.smithi179.stdout:0/885: rename d1/da/d48/d65/d7d/f61 to d1/da/df/ded/fff 0 2024-04-02T04:48:01.341 INFO:tasks.workunit.client.1.smithi179.stdout:4/817: rmdir d1/d3/df/d10/d2f/d7b 39 2024-04-02T04:48:01.342 INFO:tasks.workunit.client.1.smithi179.stdout:4/818: chown d1/d3/df/d10 429 1 2024-04-02T04:48:01.342 INFO:tasks.workunit.client.1.smithi179.stdout:9/792: write d0/d1/d11/d29/d40/faa [15322,27143] 0 2024-04-02T04:48:01.343 INFO:tasks.workunit.client.1.smithi179.stdout:2/682: mkdir dc/d29/d31/d57/dea 0 2024-04-02T04:48:01.344 INFO:tasks.workunit.client.1.smithi179.stdout:4/819: write d1/d93/fbd [4271295,97649] 0 2024-04-02T04:48:01.344 INFO:tasks.workunit.client.1.smithi179.stdout:9/793: write d0/d1/f7 [4892341,70155] 0 2024-04-02T04:48:01.346 INFO:tasks.workunit.client.1.smithi179.stdout:9/794: write d0/d19/f56 [1480181,10962] 0 2024-04-02T04:48:01.348 INFO:tasks.workunit.client.1.smithi179.stdout:9/795: write d0/d19/d27/d7a/d87/ff3 [385017,98803] 0 2024-04-02T04:48:01.349 INFO:tasks.workunit.client.1.smithi179.stdout:6/794: dread d3/d12/d23/d39/d80/faa [0,4194304] 0 2024-04-02T04:48:01.349 INFO:tasks.workunit.client.0.smithi070.stdout:4/697: mkdir d1/de7 0 2024-04-02T04:48:01.351 INFO:tasks.workunit.client.0.smithi070.stdout:4/698: write d1/d3/d78/d91/fa4 [753169,30821] 0 2024-04-02T04:48:01.351 INFO:tasks.workunit.client.0.smithi070.stdout:2/794: rename d2/d10/d53/lf2 to d2/d10/d5c/d7a/db0/dd8/l107 0 2024-04-02T04:48:01.353 INFO:tasks.workunit.client.1.smithi179.stdout:5/769: getdents d8/d1b/d23/d32 0 2024-04-02T04:48:01.353 INFO:tasks.workunit.client.0.smithi070.stdout:7/670: dread d7/dd/df/d7a/d6c/f74 [0,4194304] 0 2024-04-02T04:48:01.354 INFO:tasks.workunit.client.1.smithi179.stdout:5/770: chown d8/d1b/d23/d32/d46/l5b 26391047 1 2024-04-02T04:48:01.354 INFO:tasks.workunit.client.0.smithi070.stdout:6/613: creat d0/d55/d8b/d94/fb9 x:0 0 0 2024-04-02T04:48:01.355 INFO:tasks.workunit.client.0.smithi070.stdout:9/657: write d4/d10/d11/d7b/d19/d2f/d4d/d92/fd1 [687630,114103] 0 2024-04-02T04:48:01.357 INFO:tasks.workunit.client.1.smithi179.stdout:3/832: mkdir d8/dd3/dbb/d106 0 2024-04-02T04:48:01.358 INFO:tasks.workunit.client.1.smithi179.stdout:3/833: chown d8/dd3/fd9 29 1 2024-04-02T04:48:01.358 INFO:tasks.workunit.client.0.smithi070.stdout:3/749: mknod d10/d32/d8a/d8d/cfe 0 2024-04-02T04:48:01.360 INFO:tasks.workunit.client.1.smithi179.stdout:5/771: dwrite d8/d1b/d23/d32/dbe/daf/db7/dad/df1/f8a [0,4194304] 0 2024-04-02T04:48:01.360 INFO:tasks.workunit.client.1.smithi179.stdout:0/886: read d1/f2 [388071,98111] 0 2024-04-02T04:48:01.361 INFO:tasks.workunit.client.1.smithi179.stdout:7/821: rename d1/d9/d35 to d1/d9/dd9/d121 0 2024-04-02T04:48:01.362 INFO:tasks.workunit.client.1.smithi179.stdout:0/887: dread - d1/da/d28/d37/fce zero size 2024-04-02T04:48:01.364 INFO:tasks.workunit.client.0.smithi070.stdout:4/699: chown d1/d9c/d65/c6b 0 1 2024-04-02T04:48:01.370 INFO:tasks.workunit.client.1.smithi179.stdout:2/683: chown dc/d29/l49 315 1 2024-04-02T04:48:01.374 INFO:tasks.workunit.client.1.smithi179.stdout:4/820: creat d1/d3/dc0/f114 x:0 0 0 2024-04-02T04:48:01.377 INFO:tasks.workunit.client.0.smithi070.stdout:6/614: rename d0/d3/d19/f1f to d0/d3/d19/d31/d46/d6b/fba 0 2024-04-02T04:48:01.378 INFO:tasks.workunit.client.0.smithi070.stdout:6/615: dread - d0/d43/f9f zero size 2024-04-02T04:48:01.379 INFO:tasks.workunit.client.1.smithi179.stdout:3/834: dread d8/d26/d52/d7e/db8/fde [0,4194304] 0 2024-04-02T04:48:01.383 INFO:tasks.workunit.client.0.smithi070.stdout:7/671: write d7/dd/df/d34/d68/f70 [3635599,72619] 0 2024-04-02T04:48:01.390 INFO:tasks.workunit.client.0.smithi070.stdout:4/700: write d1/d3/f54 [4983960,97768] 0 2024-04-02T04:48:01.393 INFO:tasks.workunit.client.1.smithi179.stdout:6/795: dwrite d3/d9a/fab [0,4194304] 0 2024-04-02T04:48:01.393 INFO:tasks.workunit.client.1.smithi179.stdout:8/821: rename d3/d6c to d3/d32/d21/d22/d6b/db1/df8/d126 0 2024-04-02T04:48:01.394 INFO:tasks.workunit.client.0.smithi070.stdout:5/882: link d5/led d5/d9/d23/d30/da2/d75/ddf/l114 0 2024-04-02T04:48:01.394 INFO:tasks.workunit.client.1.smithi179.stdout:8/822: dread - d3/d32/d21/d45/feb zero size 2024-04-02T04:48:01.395 INFO:tasks.workunit.client.1.smithi179.stdout:7/822: stat d1/d9/dd9/d121/l5b 0 2024-04-02T04:48:01.397 INFO:tasks.workunit.client.0.smithi070.stdout:7/672: mkdir d7/d4e/d82/d57/ddc 0 2024-04-02T04:48:01.398 INFO:tasks.workunit.client.0.smithi070.stdout:7/673: chown d7/dd/df/f94 2359120 1 2024-04-02T04:48:01.399 INFO:tasks.workunit.client.0.smithi070.stdout:2/795: creat d2/d10/d21/d71/dd4/f108 x:0 0 0 2024-04-02T04:48:01.399 INFO:tasks.workunit.client.0.smithi070.stdout:7/674: readlink d7/dd/l13 0 2024-04-02T04:48:01.406 INFO:tasks.workunit.client.0.smithi070.stdout:5/883: creat d5/d17/d2b/d7b/f115 x:0 0 0 2024-04-02T04:48:01.410 INFO:tasks.workunit.client.1.smithi179.stdout:5/772: creat d8/d1b/d23/d36/d53/dd9/ffe x:0 0 0 2024-04-02T04:48:01.410 INFO:tasks.workunit.client.0.smithi070.stdout:7/675: dread - d7/dd/fc0 zero size 2024-04-02T04:48:01.410 INFO:tasks.workunit.client.0.smithi070.stdout:7/676: chown d7/dd/df/d79/d46/d65/d7b/fb4 75771815 1 2024-04-02T04:48:01.410 INFO:tasks.workunit.client.0.smithi070.stdout:9/658: getdents d4/d13 0 2024-04-02T04:48:01.411 INFO:tasks.workunit.client.0.smithi070.stdout:7/677: write d7/dd/f8d [165974,15475] 0 2024-04-02T04:48:01.411 INFO:tasks.workunit.client.0.smithi070.stdout:2/796: symlink d2/d10/d21/d71/dc1/l109 0 2024-04-02T04:48:01.413 INFO:tasks.workunit.client.1.smithi179.stdout:6/796: unlink d3/l6 0 2024-04-02T04:48:01.414 INFO:tasks.workunit.client.1.smithi179.stdout:6/797: chown d3/d12/d23/d26/d57/c86 0 1 2024-04-02T04:48:01.414 INFO:tasks.workunit.client.1.smithi179.stdout:8/823: mkdir d3/d32/d21/d51/d127 0 2024-04-02T04:48:01.415 INFO:tasks.workunit.client.1.smithi179.stdout:8/824: write d3/d31/f69 [4229174,32165] 0 2024-04-02T04:48:01.420 INFO:tasks.workunit.client.1.smithi179.stdout:3/835: mkdir d8/d57/d8f/d107 0 2024-04-02T04:48:01.424 INFO:tasks.workunit.client.1.smithi179.stdout:1/820: rename d3/d5/d2f/db7/lf7 to d3/d5/d2f/d4f/d42/d53/d72/l11f 0 2024-04-02T04:48:01.428 INFO:tasks.workunit.client.0.smithi070.stdout:6/616: mknod d0/d3/d19/d31/d46/d6b/cbb 0 2024-04-02T04:48:01.429 INFO:tasks.workunit.client.0.smithi070.stdout:5/884: truncate d5/d9/d10/db9/dba/de1/ff4 1719136 0 2024-04-02T04:48:01.430 INFO:tasks.workunit.client.1.smithi179.stdout:6/798: creat d3/d12/d44/d10a/f119 x:0 0 0 2024-04-02T04:48:01.434 INFO:tasks.workunit.client.1.smithi179.stdout:0/888: creat d1/da/d67/f100 x:0 0 0 2024-04-02T04:48:01.435 INFO:tasks.workunit.client.1.smithi179.stdout:5/773: write d8/d1b/d23/d32/f3d [1495322,35176] 0 2024-04-02T04:48:01.436 INFO:tasks.workunit.client.0.smithi070.stdout:9/659: creat d4/d10/d12/d27/d60/dc9/fd5 x:0 0 0 2024-04-02T04:48:01.436 INFO:tasks.workunit.client.0.smithi070.stdout:2/797: dwrite d2/d10/d21/d71/dd4/f108 [0,4194304] 0 2024-04-02T04:48:01.437 INFO:tasks.workunit.client.0.smithi070.stdout:2/798: chown d2/d12/c1f 12576629 1 2024-04-02T04:48:01.438 INFO:tasks.workunit.client.0.smithi070.stdout:9/660: write d4/d10/d11/d7b/d19/d2f/d39/da5/fca [533638,113540] 0 2024-04-02T04:48:01.447 INFO:tasks.workunit.client.0.smithi070.stdout:5/885: dread d5/d9/d23/d8b/de9/fea [0,4194304] 0 2024-04-02T04:48:01.490 INFO:tasks.workunit.client.1.smithi179.stdout:9/796: sync 2024-04-02T04:48:01.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:01 smithi070 ceph-mon[29212]: pgmap v19: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 123 MiB/s rd, 301 MiB/s wr, 644 op/s 2024-04-02T04:48:01.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:01 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:01.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:01 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:01.533 INFO:tasks.workunit.client.0.smithi070.stdout:4/701: getdents d1/d3/d85 0 2024-04-02T04:48:01.534 INFO:tasks.workunit.client.1.smithi179.stdout:8/825: write d3/d32/d21/d50/d65/faa [247694,103246] 0 2024-04-02T04:48:01.535 INFO:tasks.workunit.client.0.smithi070.stdout:7/678: write d7/dd/df/d79/f48 [2571918,63891] 0 2024-04-02T04:48:01.536 INFO:tasks.workunit.client.1.smithi179.stdout:8/826: chown d3/d32/d21/d50/d65/c7a 3071 1 2024-04-02T04:48:01.536 INFO:tasks.workunit.client.1.smithi179.stdout:8/827: write d3/d31/f7f [2467934,114942] 0 2024-04-02T04:48:01.537 INFO:tasks.workunit.client.1.smithi179.stdout:3/836: creat d8/d26/d52/d7e/ddf/f108 x:0 0 0 2024-04-02T04:48:01.544 INFO:tasks.workunit.client.0.smithi070.stdout:9/661: write d4/d10/d12/f62 [3870777,113853] 0 2024-04-02T04:48:01.548 INFO:tasks.workunit.client.1.smithi179.stdout:1/821: dwrite d3/d5/d2f/d4f/d42/d55/fa1 [0,4194304] 0 2024-04-02T04:48:01.548 INFO:tasks.workunit.client.0.smithi070.stdout:5/886: mkdir d5/d9/d10/db9/d116 0 2024-04-02T04:48:01.548 INFO:tasks.workunit.client.1.smithi179.stdout:5/774: creat d8/d1b/d23/d42/dde/fff x:0 0 0 2024-04-02T04:48:01.549 INFO:tasks.workunit.client.1.smithi179.stdout:0/889: write d1/db/d64/f6a [3987733,94770] 0 2024-04-02T04:48:01.549 INFO:tasks.workunit.client.1.smithi179.stdout:1/822: fsync d3/d5/d2f/db7/ff4 0 2024-04-02T04:48:01.552 INFO:tasks.workunit.client.1.smithi179.stdout:0/890: dread - d1/da/df/d56/ffa zero size 2024-04-02T04:48:01.552 INFO:tasks.workunit.client.1.smithi179.stdout:1/823: write d3/d5/d2f/d4f/d42/fae [15069,93693] 0 2024-04-02T04:48:01.553 INFO:tasks.workunit.client.1.smithi179.stdout:2/684: getdents dc/d29/d4a/d1e/d8a 0 2024-04-02T04:48:01.555 INFO:tasks.workunit.client.1.smithi179.stdout:2/685: dread dc/f9f [0,4194304] 0 2024-04-02T04:48:01.555 INFO:tasks.workunit.client.1.smithi179.stdout:1/824: write d3/d5/d2f/d4f/d42/fb1 [61486,79922] 0 2024-04-02T04:48:01.555 INFO:tasks.workunit.client.0.smithi070.stdout:9/662: dwrite d4/d10/d11/d7b/d19/d2f/d39/da5/fca [0,4194304] 0 2024-04-02T04:48:01.560 INFO:tasks.workunit.client.1.smithi179.stdout:9/797: dread d0/d1/f7d [0,4194304] 0 2024-04-02T04:48:01.561 INFO:tasks.workunit.client.1.smithi179.stdout:3/837: rename d8/d57/d63/c7b to d8/d26/d83/d89/d9c/c109 0 2024-04-02T04:48:01.562 INFO:tasks.workunit.client.0.smithi070.stdout:4/702: mknod d1/de7/ce8 0 2024-04-02T04:48:01.565 INFO:tasks.workunit.client.0.smithi070.stdout:9/663: symlink d4/d10/d11/d7b/d19/d2f/d39/da5/ld6 0 2024-04-02T04:48:01.569 INFO:tasks.workunit.client.0.smithi070.stdout:5/887: rename d5/d9/d23/d33/c71 to d5/d9/d23/d30/dde/c117 0 2024-04-02T04:48:01.570 INFO:tasks.workunit.client.0.smithi070.stdout:7/679: write d7/dd/df/d7a/d6c/f9b [4006080,72284] 0 2024-04-02T04:48:01.570 INFO:tasks.workunit.client.0.smithi070.stdout:6/617: truncate d0/d3/d19/d31/d46/d6b/f72 1941092 0 2024-04-02T04:48:01.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:01 smithi179 ceph-mon[36603]: pgmap v19: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 123 MiB/s rd, 301 MiB/s wr, 644 op/s 2024-04-02T04:48:01.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:01 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:01.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:01 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:01.574 INFO:tasks.workunit.client.0.smithi070.stdout:2/799: dread d2/d10/d5c/d7a/db0/dd8/dd2/d90/f9e [0,4194304] 0 2024-04-02T04:48:01.574 INFO:tasks.workunit.client.0.smithi070.stdout:7/680: chown d7/dd/df/ca3 226392135 1 2024-04-02T04:48:01.578 INFO:tasks.workunit.client.1.smithi179.stdout:5/775: unlink d8/d1b/d23/d42/fa4 0 2024-04-02T04:48:01.580 INFO:tasks.workunit.client.0.smithi070.stdout:4/703: dread d1/d3/f39 [0,4194304] 0 2024-04-02T04:48:01.582 INFO:tasks.workunit.client.1.smithi179.stdout:7/823: getdents d1/d9 0 2024-04-02T04:48:01.583 INFO:tasks.workunit.client.0.smithi070.stdout:9/664: fdatasync d4/d10/d11/da8/fb5 0 2024-04-02T04:48:01.584 INFO:tasks.workunit.client.1.smithi179.stdout:8/828: dread d3/d1b/f9a [0,4194304] 0 2024-04-02T04:48:01.585 INFO:tasks.workunit.client.0.smithi070.stdout:8/737: sync 2024-04-02T04:48:01.586 INFO:tasks.workunit.client.1.smithi179.stdout:5/776: dwrite d8/d1b/d23/d32/d4e/d51/f9c [0,4194304] 0 2024-04-02T04:48:01.588 INFO:tasks.workunit.client.0.smithi070.stdout:6/618: creat d0/d43/fbc x:0 0 0 2024-04-02T04:48:01.588 INFO:tasks.workunit.client.1.smithi179.stdout:5/777: dread - d8/d1b/d23/d36/d53/dd9/ffe zero size 2024-04-02T04:48:01.589 INFO:tasks.workunit.client.1.smithi179.stdout:6/799: dwrite d3/fa [0,4194304] 0 2024-04-02T04:48:01.590 INFO:tasks.workunit.client.0.smithi070.stdout:6/619: chown d0/d3/d19/d31/d46/f9b 521 1 2024-04-02T04:48:01.594 INFO:tasks.workunit.client.1.smithi179.stdout:4/821: link d1/d3/df/d10/d2f/d60/fff d1/d15/f115 0 2024-04-02T04:48:01.595 INFO:tasks.workunit.client.0.smithi070.stdout:7/681: dread d7/dd/df/d79/d46/fb2 [0,4194304] 0 2024-04-02T04:48:01.598 INFO:tasks.workunit.client.0.smithi070.stdout:4/704: mkdir d1/d3/d85/d8e/d99/db1/de9 0 2024-04-02T04:48:01.605 INFO:tasks.workunit.client.0.smithi070.stdout:5/888: mkdir d5/d17/da7/d118 0 2024-04-02T04:48:01.610 INFO:tasks.workunit.client.0.smithi070.stdout:2/800: creat d2/d10/d21/d3d/ddc/de8/f10a x:0 0 0 2024-04-02T04:48:01.611 INFO:tasks.workunit.client.0.smithi070.stdout:2/801: chown d2/d10/d5c/c84 1 1 2024-04-02T04:48:01.611 INFO:tasks.workunit.client.1.smithi179.stdout:6/800: dread d3/d12/d23/d2f/f3a [0,4194304] 0 2024-04-02T04:48:01.614 INFO:tasks.workunit.client.1.smithi179.stdout:8/829: mknod d3/d32/d21/d22/c128 0 2024-04-02T04:48:01.618 INFO:tasks.workunit.client.0.smithi070.stdout:2/802: dwrite d2/fb [4194304,4194304] 0 2024-04-02T04:48:01.623 INFO:tasks.workunit.client.1.smithi179.stdout:2/686: truncate dc/d29/d4a/f72 927917 0 2024-04-02T04:48:01.625 INFO:tasks.workunit.client.0.smithi070.stdout:7/682: dread d7/dd/df/d79/d46/d65/f8c [0,4194304] 0 2024-04-02T04:48:01.625 INFO:tasks.workunit.client.0.smithi070.stdout:5/889: symlink d5/d9/d10/d51/l119 0 2024-04-02T04:48:01.627 INFO:tasks.workunit.client.1.smithi179.stdout:4/822: creat d1/d15/d6e/d9c/f116 x:0 0 0 2024-04-02T04:48:01.627 INFO:tasks.workunit.client.1.smithi179.stdout:4/823: dread - d1/d93/d101/f105 zero size 2024-04-02T04:48:01.630 INFO:tasks.workunit.client.0.smithi070.stdout:9/665: dwrite d4/d10/d12/d27/f30 [0,4194304] 0 2024-04-02T04:48:01.633 INFO:tasks.workunit.client.0.smithi070.stdout:8/738: dwrite d2/d3/d52/faf [0,4194304] 0 2024-04-02T04:48:01.637 INFO:tasks.workunit.client.1.smithi179.stdout:9/798: link d0/d19/d1c/f31 d0/d1/d10/de3/ff4 0 2024-04-02T04:48:01.637 INFO:tasks.workunit.client.0.smithi070.stdout:8/739: readlink d2/d8/d23/d24/d79/l57 0 2024-04-02T04:48:01.637 INFO:tasks.workunit.client.0.smithi070.stdout:8/740: chown d2/lc9 506317 1 2024-04-02T04:48:01.638 INFO:tasks.workunit.client.1.smithi179.stdout:7/824: mknod d1/d9/d27/d87/c122 0 2024-04-02T04:48:01.640 INFO:tasks.workunit.client.1.smithi179.stdout:6/801: creat d3/d12/d23/d26/dd3/f11a x:0 0 0 2024-04-02T04:48:01.641 INFO:tasks.workunit.client.0.smithi070.stdout:9/666: dwrite d4/d10/d11/d7b/d19/d53/f59 [0,4194304] 0 2024-04-02T04:48:01.641 INFO:tasks.workunit.client.0.smithi070.stdout:5/890: symlink d5/d17/d6c/l11a 0 2024-04-02T04:48:01.642 INFO:tasks.workunit.client.1.smithi179.stdout:6/802: readlink d3/d12/d6a/de8/l113 0 2024-04-02T04:48:01.642 INFO:tasks.workunit.client.1.smithi179.stdout:5/778: dwrite d8/d1b/d23/d32/dbe/daf/db7/f79 [0,4194304] 0 2024-04-02T04:48:01.650 INFO:tasks.workunit.client.1.smithi179.stdout:2/687: symlink dc/d29/d31/d66/leb 0 2024-04-02T04:48:01.654 INFO:tasks.workunit.client.0.smithi070.stdout:4/705: write d1/d3/f95 [1258349,90705] 0 2024-04-02T04:48:01.657 INFO:tasks.workunit.client.1.smithi179.stdout:4/824: mkdir d1/d3/df/d10/d2f/d60/d71/d117 0 2024-04-02T04:48:01.658 INFO:tasks.workunit.client.1.smithi179.stdout:4/825: fsync d1/d3/df/d10/d2f/d60/d71/ff2 0 2024-04-02T04:48:01.659 INFO:tasks.workunit.client.0.smithi070.stdout:5/891: dread d5/d17/d2b/d7b/f92 [0,4194304] 0 2024-04-02T04:48:01.660 INFO:tasks.workunit.client.1.smithi179.stdout:4/826: chown d1/d3/df/d10/d2f/d60/fff 200 1 2024-04-02T04:48:01.660 INFO:tasks.workunit.client.1.smithi179.stdout:4/827: dread - d1/f108 zero size 2024-04-02T04:48:01.660 INFO:tasks.workunit.client.1.smithi179.stdout:0/891: rename d1/db/d64/lbf to d1/da/d28/l101 0 2024-04-02T04:48:01.661 INFO:tasks.workunit.client.1.smithi179.stdout:4/828: chown d1/d3/df/d10/d2f/d36/l10d 54251926 1 2024-04-02T04:48:01.662 INFO:tasks.workunit.client.0.smithi070.stdout:2/803: mknod d2/d10/c10b 0 2024-04-02T04:48:01.662 INFO:tasks.workunit.client.1.smithi179.stdout:3/838: link d8/l16 d8/d7a/l10a 0 2024-04-02T04:48:01.666 INFO:tasks.workunit.client.0.smithi070.stdout:7/683: write d7/dd/df/d79/f5e [1720625,1879] 0 2024-04-02T04:48:01.667 INFO:tasks.workunit.client.1.smithi179.stdout:7/825: dread - d1/d9/d55/f96 zero size 2024-04-02T04:48:01.669 INFO:tasks.workunit.client.0.smithi070.stdout:4/706: creat d1/d3/d78/dad/dbd/fea x:0 0 0 2024-04-02T04:48:01.669 INFO:tasks.workunit.client.0.smithi070.stdout:5/892: truncate d5/d9/d10/d6a/fd5 28286 0 2024-04-02T04:48:01.670 INFO:tasks.workunit.client.0.smithi070.stdout:4/707: chown d1/de7/ce8 1143246 1 2024-04-02T04:48:01.670 INFO:tasks.workunit.client.1.smithi179.stdout:5/779: dread - d8/d1b/fe1 zero size 2024-04-02T04:48:01.671 INFO:tasks.workunit.client.0.smithi070.stdout:2/804: mknod d2/d10/d21/d3d/ddc/de8/c10c 0 2024-04-02T04:48:01.675 INFO:tasks.workunit.client.1.smithi179.stdout:8/830: fsync d3/d32/d21/d49/fb9 0 2024-04-02T04:48:01.676 INFO:tasks.workunit.client.1.smithi179.stdout:2/688: dread dc/d29/d4a/f53 [0,4194304] 0 2024-04-02T04:48:01.677 INFO:tasks.workunit.client.1.smithi179.stdout:8/831: chown d3/d32/d21/d22/d6b/db1/d104/c113 0 1 2024-04-02T04:48:01.682 INFO:tasks.workunit.client.1.smithi179.stdout:1/825: rename d3/d5/d2f/d4f/d35/c103 to d3/d5/d2f/d4f/d6a/da0/c120 0 2024-04-02T04:48:01.684 INFO:tasks.workunit.client.0.smithi070.stdout:5/893: rmdir d5/d9/d23/d30/da2 39 2024-04-02T04:48:01.685 INFO:tasks.workunit.client.1.smithi179.stdout:6/803: write d3/d12/d23/d26/fcc [68242,81900] 0 2024-04-02T04:48:01.686 INFO:tasks.workunit.client.1.smithi179.stdout:4/829: mkdir d1/d3/df/d10/d2f/d60/d71/dce/d118 0 2024-04-02T04:48:01.687 INFO:tasks.workunit.client.1.smithi179.stdout:4/830: write d1/d3/df/d10/d2f/d60/da6/fb6 [4745955,21322] 0 2024-04-02T04:48:01.688 INFO:tasks.workunit.client.1.smithi179.stdout:4/831: write d1/d3/dc/d26/ff3 [10572,96179] 0 2024-04-02T04:48:01.689 INFO:tasks.workunit.client.1.smithi179.stdout:3/839: mknod d8/d26/d52/d7e/ddf/c10b 0 2024-04-02T04:48:01.690 INFO:tasks.workunit.client.1.smithi179.stdout:4/832: fdatasync d1/d3/dc/f1c 0 2024-04-02T04:48:01.692 INFO:tasks.workunit.client.1.smithi179.stdout:5/780: dread d8/d1b/d23/d30/f5c [0,4194304] 0 2024-04-02T04:48:01.693 INFO:tasks.workunit.client.0.smithi070.stdout:2/805: symlink d2/d10/l10d 0 2024-04-02T04:48:01.694 INFO:tasks.workunit.client.1.smithi179.stdout:2/689: creat dc/d29/d4a/d1e/fec x:0 0 0 2024-04-02T04:48:01.696 INFO:tasks.workunit.client.0.smithi070.stdout:9/667: rename d4/d10/c17 to d4/d10/d11/d7b/d19/cd7 0 2024-04-02T04:48:01.696 INFO:tasks.workunit.client.0.smithi070.stdout:4/708: dread d1/d3/d78/dad/dbd/fc9 [0,4194304] 0 2024-04-02T04:48:01.696 INFO:tasks.workunit.client.1.smithi179.stdout:2/690: read dc/d29/d4a/d4b/f6d [1273570,77726] 0 2024-04-02T04:48:01.699 INFO:tasks.workunit.client.1.smithi179.stdout:7/826: rename d1/d9/d7e/fb8 to d1/d9/dd9/f123 0 2024-04-02T04:48:01.700 INFO:tasks.workunit.client.1.smithi179.stdout:7/827: stat d1/d9/d7e/caa 0 2024-04-02T04:48:01.701 INFO:tasks.workunit.client.1.smithi179.stdout:6/804: mknod d3/d12/d44/d10a/c11b 0 2024-04-02T04:48:01.703 INFO:tasks.workunit.client.0.smithi070.stdout:4/709: mkdir d1/d3/d85/deb 0 2024-04-02T04:48:01.708 INFO:tasks.workunit.client.0.smithi070.stdout:2/806: dwrite d2/d24/f96 [0,4194304] 0 2024-04-02T04:48:01.709 INFO:tasks.workunit.client.0.smithi070.stdout:9/668: dwrite d4/d10/d11/d7b/d19/d82/fc4 [0,4194304] 0 2024-04-02T04:48:01.710 INFO:tasks.workunit.client.1.smithi179.stdout:9/799: creat d0/d1/ff5 x:0 0 0 2024-04-02T04:48:01.713 INFO:tasks.workunit.client.0.smithi070.stdout:5/894: getdents d5/d9/d23/d30/da2/d75/ddf 0 2024-04-02T04:48:01.716 INFO:tasks.workunit.client.0.smithi070.stdout:7/684: rename d7/dd/df/d79/d46/d65/d7b/fab to d7/d9/fdd 0 2024-04-02T04:48:01.720 INFO:tasks.workunit.client.0.smithi070.stdout:4/710: creat d1/d3/d85/d8e/d99/db1/dbb/fec x:0 0 0 2024-04-02T04:48:01.721 INFO:tasks.workunit.client.1.smithi179.stdout:8/832: fdatasync d3/d1b/d2a/f4c 0 2024-04-02T04:48:01.722 INFO:tasks.workunit.client.1.smithi179.stdout:8/833: dread - d3/d1b/d2a/db0/f103 zero size 2024-04-02T04:48:01.724 INFO:tasks.workunit.client.1.smithi179.stdout:8/834: write d3/d32/d21/d50/d56/fcd [2234880,113707] 0 2024-04-02T04:48:01.727 INFO:tasks.workunit.client.0.smithi070.stdout:9/669: mkdir d4/d10/d12/d56/d79/dd8 0 2024-04-02T04:48:01.733 INFO:tasks.workunit.client.1.smithi179.stdout:2/691: write dc/d29/d31/d57/d97/fb6 [618392,54931] 0 2024-04-02T04:48:01.734 INFO:tasks.workunit.client.0.smithi070.stdout:4/711: write d1/d3/d85/da8/fe5 [818632,106687] 0 2024-04-02T04:48:01.735 INFO:tasks.workunit.client.1.smithi179.stdout:6/805: truncate d3/d12/d6a/fc2 64138 0 2024-04-02T04:48:01.737 INFO:tasks.workunit.client.0.smithi070.stdout:2/807: rename d2/d12/lfc to d2/d10/d5c/d7a/db0/dc7/l10e 0 2024-04-02T04:48:01.739 INFO:tasks.workunit.client.0.smithi070.stdout:2/808: write d2/d10/d5c/d7a/db0/fe3 [1284134,67515] 0 2024-04-02T04:48:01.739 INFO:tasks.workunit.client.1.smithi179.stdout:4/833: symlink d1/l119 0 2024-04-02T04:48:01.739 INFO:tasks.workunit.client.1.smithi179.stdout:4/834: dread - d1/d93/fe9 zero size 2024-04-02T04:48:01.740 INFO:tasks.workunit.client.0.smithi070.stdout:2/809: read - d2/d24/ff5 zero size 2024-04-02T04:48:01.740 INFO:tasks.workunit.client.0.smithi070.stdout:9/670: symlink d4/d10/d12/d56/ld9 0 2024-04-02T04:48:01.741 INFO:tasks.workunit.client.1.smithi179.stdout:3/840: creat d8/d26/d52/d7e/def/f10c x:0 0 0 2024-04-02T04:48:01.741 INFO:tasks.workunit.client.1.smithi179.stdout:7/828: dread d1/d9/d27/f69 [0,4194304] 0 2024-04-02T04:48:01.745 INFO:tasks.workunit.client.1.smithi179.stdout:4/835: dwrite d1/d3/df/d10/d2f/d60/f6c [0,4194304] 0 2024-04-02T04:48:01.747 INFO:tasks.workunit.client.0.smithi070.stdout:2/810: mkdir d2/d10/d5c/d10f 0 2024-04-02T04:48:01.749 INFO:tasks.workunit.client.1.smithi179.stdout:0/892: sync 2024-04-02T04:48:01.753 INFO:tasks.workunit.client.0.smithi070.stdout:9/671: creat d4/d10/d11/d7b/d19/d2f/fda x:0 0 0 2024-04-02T04:48:01.753 INFO:tasks.workunit.client.0.smithi070.stdout:4/712: dread d1/d3/d9/f9e [0,4194304] 0 2024-04-02T04:48:01.754 INFO:tasks.workunit.client.1.smithi179.stdout:9/800: dwrite d0/d1/d11/dab/ded/f79 [0,4194304] 0 2024-04-02T04:48:01.754 INFO:tasks.workunit.client.0.smithi070.stdout:2/811: mknod d2/d10/d19/c110 0 2024-04-02T04:48:01.756 INFO:tasks.workunit.client.1.smithi179.stdout:1/826: creat d3/d5/d2f/f121 x:0 0 0 2024-04-02T04:48:01.759 INFO:tasks.workunit.client.1.smithi179.stdout:6/806: mknod d3/d2c/c11c 0 2024-04-02T04:48:01.760 INFO:tasks.workunit.client.1.smithi179.stdout:6/807: readlink d3/d12/d23/d39/d80/d93/ddc/le1 0 2024-04-02T04:48:01.762 INFO:tasks.workunit.client.1.smithi179.stdout:6/808: chown d3/d12/d23/d26/d57/ff7 6 1 2024-04-02T04:48:01.762 INFO:tasks.workunit.client.1.smithi179.stdout:6/809: chown d3/d12/d23/d26/d57/c86 494 1 2024-04-02T04:48:01.763 INFO:tasks.workunit.client.1.smithi179.stdout:6/810: read - d3/d12/d23/d26/d57/fc3 zero size 2024-04-02T04:48:01.764 INFO:tasks.workunit.client.1.smithi179.stdout:6/811: write d3/d2d/d38/d64/f109 [721447,67347] 0 2024-04-02T04:48:01.773 INFO:tasks.workunit.client.1.smithi179.stdout:2/692: dread dc/d29/d4a/d1e/f40 [0,4194304] 0 2024-04-02T04:48:01.785 INFO:tasks.workunit.client.0.smithi070.stdout:0/793: sync 2024-04-02T04:48:01.785 INFO:tasks.workunit.client.0.smithi070.stdout:7/685: getdents d7/dd/df 0 2024-04-02T04:48:01.790 INFO:tasks.workunit.client.1.smithi179.stdout:7/829: rmdir d1/d9/d27/d87/d8b/d75/db9 39 2024-04-02T04:48:01.790 INFO:tasks.workunit.client.0.smithi070.stdout:7/686: chown d7/dd/df/d79/ca8 7 1 2024-04-02T04:48:01.791 INFO:tasks.workunit.client.0.smithi070.stdout:4/713: dwrite d1/d3/d85/da8/fe5 [0,4194304] 0 2024-04-02T04:48:01.793 INFO:tasks.workunit.client.1.smithi179.stdout:5/781: fdatasync d8/d1b/d23/d42/d5a/f6b 0 2024-04-02T04:48:01.793 INFO:tasks.workunit.client.0.smithi070.stdout:9/672: dread d4/d10/d11/d7b/d19/d82/fc4 [0,4194304] 0 2024-04-02T04:48:01.798 INFO:tasks.workunit.client.0.smithi070.stdout:2/812: dwrite d2/fd6 [0,4194304] 0 2024-04-02T04:48:01.800 INFO:tasks.workunit.client.0.smithi070.stdout:2/813: fsync d2/d10/d5c/d7a/db0/dd8/dd2/f9b 0 2024-04-02T04:48:01.802 INFO:tasks.workunit.client.0.smithi070.stdout:0/794: dwrite d4/d5/d13/d17/d98/fdc [0,4194304] 0 2024-04-02T04:48:01.803 INFO:tasks.workunit.client.1.smithi179.stdout:4/836: mknod d1/d93/d101/c11a 0 2024-04-02T04:48:01.805 INFO:tasks.workunit.client.0.smithi070.stdout:7/687: unlink d7/dd/df/d79/d46/d65/f69 0 2024-04-02T04:48:01.805 INFO:tasks.workunit.client.1.smithi179.stdout:0/893: fsync d1/da/d48/d65/d7d/d80/fb0 0 2024-04-02T04:48:01.806 INFO:tasks.workunit.client.0.smithi070.stdout:7/688: chown d7/dd/df/d79/d46/cbe 10 1 2024-04-02T04:48:01.808 INFO:tasks.workunit.client.0.smithi070.stdout:4/714: truncate d1/d9c/f6e 3419385 0 2024-04-02T04:48:01.811 INFO:tasks.workunit.client.1.smithi179.stdout:1/827: rmdir d3/d5/d2f/d4f/d42/df2/db6/dd8 39 2024-04-02T04:48:01.812 INFO:tasks.workunit.client.0.smithi070.stdout:9/673: truncate d4/d10/d11/d7b/d19/d2f/f3c 3307455 0 2024-04-02T04:48:01.813 INFO:tasks.workunit.client.1.smithi179.stdout:6/812: symlink d3/d42/l11d 0 2024-04-02T04:48:01.817 INFO:tasks.workunit.client.1.smithi179.stdout:6/813: dread d3/d9a/fab [0,4194304] 0 2024-04-02T04:48:01.817 INFO:tasks.workunit.client.1.smithi179.stdout:3/841: creat d8/dff/f10d x:0 0 0 2024-04-02T04:48:01.817 INFO:tasks.workunit.client.0.smithi070.stdout:4/715: symlink d1/d9c/d65/d7a/led 0 2024-04-02T04:48:01.818 INFO:tasks.workunit.client.0.smithi070.stdout:4/716: write d1/d3/d9/f24 [8894577,130307] 0 2024-04-02T04:48:01.822 INFO:tasks.workunit.client.1.smithi179.stdout:7/830: creat d1/d9/d55/dbf/d9f/da8/f124 x:0 0 0 2024-04-02T04:48:01.823 INFO:tasks.workunit.client.1.smithi179.stdout:4/837: truncate d1/d3/dc/fdf 486509 0 2024-04-02T04:48:01.825 INFO:tasks.workunit.client.1.smithi179.stdout:0/894: fsync d1/f2 0 2024-04-02T04:48:01.826 INFO:tasks.workunit.client.1.smithi179.stdout:0/895: chown d1/da/dc3 2232202 1 2024-04-02T04:48:01.826 INFO:tasks.workunit.client.1.smithi179.stdout:5/782: dwrite d8/d1b/d23/d36/d53/f78 [0,4194304] 0 2024-04-02T04:48:01.831 INFO:tasks.workunit.client.0.smithi070.stdout:4/717: creat d1/d9c/d65/d7a/fee x:0 0 0 2024-04-02T04:48:01.837 INFO:tasks.workunit.client.1.smithi179.stdout:1/828: creat d3/d85/d112/f122 x:0 0 0 2024-04-02T04:48:01.839 INFO:tasks.workunit.client.0.smithi070.stdout:0/795: link d4/d5/d13/d2a/fea d4/d5/d13/ff3 0 2024-04-02T04:48:01.840 INFO:tasks.workunit.client.0.smithi070.stdout:2/814: dwrite d2/d10/d21/d71/dd4/dce/fff [0,4194304] 0 2024-04-02T04:48:01.840 INFO:tasks.workunit.client.0.smithi070.stdout:0/796: readlink d4/d5/d58/d60/d83/lbb 0 2024-04-02T04:48:01.841 INFO:tasks.workunit.client.0.smithi070.stdout:9/674: write d4/d10/d12/d55/d66/f6d [48964,119961] 0 2024-04-02T04:48:01.844 INFO:tasks.workunit.client.1.smithi179.stdout:2/693: truncate dc/d29/d4a/f56 1122096 0 2024-04-02T04:48:01.845 INFO:tasks.workunit.client.1.smithi179.stdout:2/694: write dc/d29/d31/f41 [2920722,121222] 0 2024-04-02T04:48:01.850 INFO:tasks.workunit.client.0.smithi070.stdout:4/718: creat d1/d3/d85/d8e/d99/db1/fef x:0 0 0 2024-04-02T04:48:01.850 INFO:tasks.workunit.client.0.smithi070.stdout:0/797: read d4/d5/d13/d17/d6e/fdf [1830424,78435] 0 2024-04-02T04:48:01.856 INFO:tasks.workunit.client.1.smithi179.stdout:4/838: creat d1/d3/f11b x:0 0 0 2024-04-02T04:48:01.856 INFO:tasks.workunit.client.1.smithi179.stdout:4/839: chown d1/c9 107295 1 2024-04-02T04:48:01.858 INFO:tasks.workunit.client.1.smithi179.stdout:0/896: creat d1/da/d48/f102 x:0 0 0 2024-04-02T04:48:01.860 INFO:tasks.workunit.client.1.smithi179.stdout:5/783: rename d8/d1b/d23/d30/d73/dba/lc5 to d8/d1b/d23/d42/d5a/l100 0 2024-04-02T04:48:01.862 INFO:tasks.workunit.client.1.smithi179.stdout:8/835: link d3/d1b/d2a/fd8 d3/d32/d21/d22/d6b/d91/da8/dfe/f129 0 2024-04-02T04:48:01.863 INFO:tasks.workunit.client.1.smithi179.stdout:7/831: write d1/d9/d27/d87/d8b/d75/f9a [154366,93505] 0 2024-04-02T04:48:01.863 INFO:tasks.workunit.client.1.smithi179.stdout:9/801: link d0/d19/l89 d0/d1/d11/d29/d80/lf6 0 2024-04-02T04:48:01.865 INFO:tasks.workunit.client.1.smithi179.stdout:1/829: creat d3/d5/d2f/d4f/dab/dcd/f123 x:0 0 0 2024-04-02T04:48:01.869 INFO:tasks.workunit.client.1.smithi179.stdout:3/842: symlink d8/d57/l10e 0 2024-04-02T04:48:01.870 INFO:tasks.workunit.client.1.smithi179.stdout:2/695: fsync dc/f52 0 2024-04-02T04:48:01.871 INFO:tasks.workunit.client.1.smithi179.stdout:6/814: dwrite d3/d12/d23/d2f/f3a [0,4194304] 0 2024-04-02T04:48:01.872 INFO:tasks.workunit.client.0.smithi070.stdout:2/815: dread d2/d10/d21/f89 [0,4194304] 0 2024-04-02T04:48:01.873 INFO:tasks.workunit.client.1.smithi179.stdout:2/696: write dc/d79/fa6 [4790741,48524] 0 2024-04-02T04:48:01.875 INFO:tasks.workunit.client.1.smithi179.stdout:9/802: dwrite d0/f4c [0,4194304] 0 2024-04-02T04:48:01.877 INFO:tasks.workunit.client.1.smithi179.stdout:9/803: chown d0/d1/d11/d29/d40 3 1 2024-04-02T04:48:01.884 INFO:tasks.workunit.client.1.smithi179.stdout:0/897: dread d1/da/df/f12 [4194304,4194304] 0 2024-04-02T04:48:01.884 INFO:tasks.workunit.client.1.smithi179.stdout:5/784: mkdir d8/d1b/d23/d36/d53/d101 0 2024-04-02T04:48:01.887 INFO:tasks.workunit.client.1.smithi179.stdout:8/836: mknod d3/d32/d21/d50/d56/c12a 0 2024-04-02T04:48:01.898 INFO:tasks.workunit.client.1.smithi179.stdout:1/830: write d3/d5/d2f/d4f/d42/df2/fba [2592947,76458] 0 2024-04-02T04:48:01.907 INFO:tasks.workunit.client.1.smithi179.stdout:7/832: write d1/dd/d23/fee [500323,64807] 0 2024-04-02T04:48:01.909 INFO:tasks.workunit.client.1.smithi179.stdout:0/898: symlink d1/db/l103 0 2024-04-02T04:48:01.911 INFO:tasks.workunit.client.1.smithi179.stdout:0/899: readlink d1/da/d28/d37/l6f 0 2024-04-02T04:48:01.912 INFO:tasks.workunit.client.0.smithi070.stdout:9/675: creat d4/d10/d11/d7b/d19/d53/d9c/fdb x:0 0 0 2024-04-02T04:48:01.912 INFO:tasks.workunit.client.1.smithi179.stdout:7/833: dread d1/d9/f1d [0,4194304] 0 2024-04-02T04:48:01.913 INFO:tasks.workunit.client.0.smithi070.stdout:4/719: chown d1/d3/cae 321162 1 2024-04-02T04:48:01.913 INFO:tasks.workunit.client.0.smithi070.stdout:9/676: write d4/d10/d12/d55/d66/f6d [867465,15216] 0 2024-04-02T04:48:01.914 INFO:tasks.workunit.client.0.smithi070.stdout:4/720: write d1/d3/d9/f13 [6146470,57922] 0 2024-04-02T04:48:01.915 INFO:tasks.workunit.client.0.smithi070.stdout:0/798: creat d4/d5/d58/d60/d83/ff4 x:0 0 0 2024-04-02T04:48:01.916 INFO:tasks.workunit.client.1.smithi179.stdout:6/815: symlink d3/d2d/d38/l11e 0 2024-04-02T04:48:01.916 INFO:tasks.workunit.client.0.smithi070.stdout:0/799: readlink d4/d5/d58/d60/d83/da5/lad 0 2024-04-02T04:48:01.917 INFO:tasks.workunit.client.0.smithi070.stdout:3/750: sync 2024-04-02T04:48:01.917 INFO:tasks.workunit.client.0.smithi070.stdout:1/704: sync 2024-04-02T04:48:01.917 INFO:tasks.workunit.client.0.smithi070.stdout:6/620: sync 2024-04-02T04:48:01.918 INFO:tasks.workunit.client.1.smithi179.stdout:2/697: symlink dc/d29/d31/d57/dea/led 0 2024-04-02T04:48:01.918 INFO:tasks.workunit.client.1.smithi179.stdout:4/840: dread d1/d3/dc/d27/d62/f69 [0,4194304] 0 2024-04-02T04:48:01.919 INFO:tasks.workunit.client.0.smithi070.stdout:7/689: rename d7/d9/c49 to d7/d4e/d82/cde 0 2024-04-02T04:48:01.919 INFO:tasks.workunit.client.0.smithi070.stdout:1/705: write d0/d7/f81 [1422942,57099] 0 2024-04-02T04:48:01.921 INFO:tasks.workunit.client.0.smithi070.stdout:2/816: mknod d2/d10/d21/d71/dc1/c111 0 2024-04-02T04:48:01.922 INFO:tasks.workunit.client.1.smithi179.stdout:5/785: getdents d8/d1b/d23/d32/d4e/ddd/de5 0 2024-04-02T04:48:01.923 INFO:tasks.workunit.client.1.smithi179.stdout:8/837: symlink d3/d31/d8f/l12b 0 2024-04-02T04:48:01.924 INFO:tasks.workunit.client.1.smithi179.stdout:5/786: chown d8/d1b/d23/d32/dbe/daf/db7/dad/cb5 31903789 1 2024-04-02T04:48:01.924 INFO:tasks.workunit.client.0.smithi070.stdout:0/800: fsync d4/d5/d13/d17/d72/f90 0 2024-04-02T04:48:01.927 INFO:tasks.workunit.client.0.smithi070.stdout:6/621: creat d0/d55/d8b/d94/fbd x:0 0 0 2024-04-02T04:48:01.928 INFO:tasks.workunit.client.1.smithi179.stdout:6/816: symlink d3/d12/d23/d39/d4e/d6c/dcd/l11f 0 2024-04-02T04:48:01.930 INFO:tasks.workunit.client.0.smithi070.stdout:9/677: rename d4/d10/d11/d7b/d19/d2f/d39/d57/fa9 to d4/d34/fdc 0 2024-04-02T04:48:01.931 INFO:tasks.workunit.client.1.smithi179.stdout:2/698: creat dc/d29/d31/d57/fee x:0 0 0 2024-04-02T04:48:01.932 INFO:tasks.workunit.client.1.smithi179.stdout:2/699: fdatasync dc/d29/d31/f41 0 2024-04-02T04:48:01.932 INFO:tasks.workunit.client.0.smithi070.stdout:7/690: symlink d7/d36/ldf 0 2024-04-02T04:48:01.937 INFO:tasks.workunit.client.0.smithi070.stdout:1/706: unlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/l8d 0 2024-04-02T04:48:01.939 INFO:tasks.workunit.client.0.smithi070.stdout:1/707: dread - d0/d9/d1d/d22/d2c/d4f/fd5 zero size 2024-04-02T04:48:01.939 INFO:tasks.workunit.client.0.smithi070.stdout:6/622: dread d0/f78 [4194304,4194304] 0 2024-04-02T04:48:01.940 INFO:tasks.workunit.client.0.smithi070.stdout:2/817: symlink d2/d10/d5c/d7a/db0/dd8/dd2/d90/l112 0 2024-04-02T04:48:01.943 INFO:tasks.workunit.client.0.smithi070.stdout:4/721: dread d1/d3/d9/f9b [0,4194304] 0 2024-04-02T04:48:01.948 INFO:tasks.workunit.client.1.smithi179.stdout:3/843: rename d8/d26/d52/c73 to d8/d26/c10f 0 2024-04-02T04:48:01.949 INFO:tasks.workunit.client.1.smithi179.stdout:3/844: fdatasync d8/d26/d83/d89/fdc 0 2024-04-02T04:48:01.949 INFO:tasks.workunit.client.1.smithi179.stdout:6/817: mkdir d3/d12/d23/d2f/d120 0 2024-04-02T04:48:01.951 INFO:tasks.workunit.client.0.smithi070.stdout:9/678: write d4/d10/d11/d7b/d19/d2f/fb8 [3569776,129414] 0 2024-04-02T04:48:01.953 INFO:tasks.workunit.client.0.smithi070.stdout:7/691: rmdir d7/dd/db5 39 2024-04-02T04:48:01.954 INFO:tasks.workunit.client.1.smithi179.stdout:2/700: fsync dc/f5a 0 2024-04-02T04:48:01.957 INFO:tasks.workunit.client.0.smithi070.stdout:7/692: dread d7/dd/df/d79/fbb [0,4194304] 0 2024-04-02T04:48:01.959 INFO:tasks.workunit.client.0.smithi070.stdout:2/818: dread d2/d12/d3e/faa [0,4194304] 0 2024-04-02T04:48:01.959 INFO:tasks.workunit.client.1.smithi179.stdout:4/841: unlink d1/d3/df/d10/d2f/ld6 0 2024-04-02T04:48:01.960 INFO:tasks.workunit.client.0.smithi070.stdout:2/819: chown d2/d12/d59/ff9 28240 1 2024-04-02T04:48:01.960 INFO:tasks.workunit.client.1.smithi179.stdout:4/842: stat d1/d3/dc/d27/dae/leb 0 2024-04-02T04:48:01.960 INFO:tasks.workunit.client.0.smithi070.stdout:1/708: dread d0/db4/f73 [0,4194304] 0 2024-04-02T04:48:01.961 INFO:tasks.workunit.client.1.smithi179.stdout:9/804: link d0/d1/lcf d0/d19/d27/d48/lf7 0 2024-04-02T04:48:01.961 INFO:tasks.workunit.client.0.smithi070.stdout:2/820: write d2/d10/d53/ffb [507304,115034] 0 2024-04-02T04:48:01.964 INFO:tasks.workunit.client.1.smithi179.stdout:7/834: creat d1/d9/d55/dbf/d9f/f125 x:0 0 0 2024-04-02T04:48:01.965 INFO:tasks.workunit.client.0.smithi070.stdout:4/722: mkdir d1/d9c/df0 0 2024-04-02T04:48:01.966 INFO:tasks.workunit.client.1.smithi179.stdout:1/831: rename d3/d5/d2f/d4f/dab/dcd/f123 to d3/d5/d2f/d4f/d42/d55/dc5/d10a/f124 0 2024-04-02T04:48:01.970 INFO:tasks.workunit.client.1.smithi179.stdout:6/818: creat d3/d9a/f121 x:0 0 0 2024-04-02T04:48:01.971 INFO:tasks.workunit.client.0.smithi070.stdout:9/679: fdatasync d4/d10/f18 0 2024-04-02T04:48:01.973 INFO:tasks.workunit.client.1.smithi179.stdout:3/845: mknod d8/d26/dcd/de7/c110 0 2024-04-02T04:48:01.979 INFO:tasks.workunit.client.0.smithi070.stdout:7/693: creat d7/dd/db1/fe0 x:0 0 0 2024-04-02T04:48:01.980 INFO:tasks.workunit.client.0.smithi070.stdout:0/801: write d4/d5/d13/d2a/d29/d52/f82 [5031956,78291] 0 2024-04-02T04:48:01.982 INFO:tasks.workunit.client.1.smithi179.stdout:4/843: mknod d1/d93/de6/c11c 0 2024-04-02T04:48:01.988 INFO:tasks.workunit.client.1.smithi179.stdout:7/835: unlink d1/dd/f1c 0 2024-04-02T04:48:01.993 INFO:tasks.workunit.client.1.smithi179.stdout:0/900: rename d1/da/d52/f5d to d1/da/d52/d75/d9a/f104 0 2024-04-02T04:48:01.994 INFO:tasks.workunit.client.0.smithi070.stdout:2/821: unlink d2/d10/d21/d3d/d42/c6f 0 2024-04-02T04:48:01.995 INFO:tasks.workunit.client.1.smithi179.stdout:6/819: fsync d3/ff 0 2024-04-02T04:48:01.997 INFO:tasks.workunit.client.0.smithi070.stdout:4/723: creat d1/d9c/d65/ff1 x:0 0 0 2024-04-02T04:48:01.998 INFO:tasks.workunit.client.1.smithi179.stdout:3/846: read - d8/d6d/d94/fea zero size 2024-04-02T04:48:02.001 INFO:tasks.workunit.client.0.smithi070.stdout:9/680: mknod d4/d10/d12/d27/d60/cdd 0 2024-04-02T04:48:02.002 INFO:tasks.workunit.client.1.smithi179.stdout:4/844: rmdir d1/d3/df/d10/d2f/d36 39 2024-04-02T04:48:02.007 INFO:tasks.workunit.client.0.smithi070.stdout:9/681: dwrite d4/d10/d11/d7b/d19/d82/f8a [0,4194304] 0 2024-04-02T04:48:02.009 INFO:tasks.workunit.client.0.smithi070.stdout:1/709: mkdir d0/d9/d1d/d22/d2c/de3 0 2024-04-02T04:48:02.009 INFO:tasks.workunit.client.0.smithi070.stdout:9/682: truncate d4/d10/d12/d55/f7e 8446351 0 2024-04-02T04:48:02.010 INFO:tasks.workunit.client.0.smithi070.stdout:2/822: truncate d2/d10/d5c/f81 954569 0 2024-04-02T04:48:02.012 INFO:tasks.workunit.client.0.smithi070.stdout:6/623: truncate d0/d3/d19/d31/d46/f9b 379964 0 2024-04-02T04:48:02.013 INFO:tasks.workunit.client.0.smithi070.stdout:6/624: write d0/d20/d5f/f62 [1339113,13493] 0 2024-04-02T04:48:02.018 INFO:tasks.workunit.client.1.smithi179.stdout:0/901: dread d1/da/d48/d65/d7d/fd3 [0,4194304] 0 2024-04-02T04:48:02.019 INFO:tasks.workunit.client.1.smithi179.stdout:6/820: mknod d3/d2c/de0/c122 0 2024-04-02T04:48:02.021 INFO:tasks.workunit.client.1.smithi179.stdout:2/701: truncate dc/d29/d4a/d4b/f51 4112896 0 2024-04-02T04:48:02.022 INFO:tasks.workunit.client.1.smithi179.stdout:2/702: write dc/d79/fa6 [5409912,15711] 0 2024-04-02T04:48:02.022 INFO:tasks.workunit.client.1.smithi179.stdout:3/847: rmdir d8/d6d/d94 39 2024-04-02T04:48:02.025 INFO:tasks.workunit.client.1.smithi179.stdout:3/848: read - d8/d7a/fcf zero size 2024-04-02T04:48:02.027 INFO:tasks.workunit.client.1.smithi179.stdout:7/836: creat d1/d9/d27/d87/d8b/d112/f126 x:0 0 0 2024-04-02T04:48:02.028 INFO:tasks.workunit.client.1.smithi179.stdout:9/805: dwrite d0/d1/d10/f6f [0,4194304] 0 2024-04-02T04:48:02.029 INFO:tasks.workunit.client.1.smithi179.stdout:0/902: mkdir d1/da/d28/d81/d105 0 2024-04-02T04:48:02.030 INFO:tasks.workunit.client.1.smithi179.stdout:0/903: chown d1/da/d52/d75 444765 1 2024-04-02T04:48:02.033 INFO:tasks.workunit.client.1.smithi179.stdout:3/849: symlink d8/dd3/l111 0 2024-04-02T04:48:02.034 INFO:tasks.workunit.client.1.smithi179.stdout:6/821: dread d3/fb [0,4194304] 0 2024-04-02T04:48:02.036 INFO:tasks.workunit.client.1.smithi179.stdout:5/787: rename d8/d1b/d23/d32/dbe/daf/db7/dad/df1/f8a to d8/d1b/d23/db1/f102 0 2024-04-02T04:48:02.040 INFO:tasks.workunit.client.1.smithi179.stdout:2/703: dwrite dc/d29/f6a [4194304,4194304] 0 2024-04-02T04:48:02.040 INFO:tasks.workunit.client.1.smithi179.stdout:7/837: truncate d1/dd/d76/fca 360695 0 2024-04-02T04:48:02.042 INFO:tasks.workunit.client.1.smithi179.stdout:0/904: readlink d1/da/d28/l101 0 2024-04-02T04:48:02.042 INFO:tasks.workunit.client.1.smithi179.stdout:2/704: chown dc/dcb/fcf 4022 1 2024-04-02T04:48:02.043 INFO:tasks.workunit.client.1.smithi179.stdout:7/838: write d1/dd/d16/f48 [886310,59366] 0 2024-04-02T04:48:02.044 INFO:tasks.workunit.client.1.smithi179.stdout:6/822: dwrite d3/f18 [0,4194304] 0 2024-04-02T04:48:02.048 INFO:tasks.workunit.client.1.smithi179.stdout:7/839: readlink d1/d9/d27/d87/d8b/lf7 0 2024-04-02T04:48:02.049 INFO:tasks.workunit.client.1.smithi179.stdout:3/850: fsync d8/d57/d63/f84 0 2024-04-02T04:48:02.049 INFO:tasks.workunit.client.1.smithi179.stdout:3/851: dread - d8/fa1 zero size 2024-04-02T04:48:02.051 INFO:tasks.workunit.client.1.smithi179.stdout:3/852: write d8/d26/d36/f59 [2512910,114374] 0 2024-04-02T04:48:02.054 INFO:tasks.workunit.client.0.smithi070.stdout:6/625: dwrite d0/d3/d19/d31/fb1 [0,4194304] 0 2024-04-02T04:48:02.055 INFO:tasks.workunit.client.0.smithi070.stdout:2/823: rename d2/d10/d21/d3d/df0 to d2/d10/d5c/d7a/db0/de4/d113 0 2024-04-02T04:48:02.056 INFO:tasks.workunit.client.0.smithi070.stdout:1/710: write d0/d7/d1a/d66/f84 [272816,58787] 0 2024-04-02T04:48:02.056 INFO:tasks.workunit.client.0.smithi070.stdout:6/626: chown d0/d20/lab 17637433 1 2024-04-02T04:48:02.068 INFO:tasks.workunit.client.1.smithi179.stdout:4/845: link d1/d3/dc/l83 d1/d3/dc/d27/d62/l11d 0 2024-04-02T04:48:02.068 INFO:tasks.workunit.client.1.smithi179.stdout:9/806: write d0/d1/d11/f51 [472372,31472] 0 2024-04-02T04:48:02.069 INFO:tasks.workunit.client.1.smithi179.stdout:2/705: truncate dc/d29/d4a/d1e/f7b 4129045 0 2024-04-02T04:48:02.085 INFO:tasks.workunit.client.1.smithi179.stdout:8/838: sync 2024-04-02T04:48:02.085 INFO:tasks.workunit.client.1.smithi179.stdout:1/832: sync 2024-04-02T04:48:02.115 INFO:tasks.workunit.client.0.smithi070.stdout:5/895: sync 2024-04-02T04:48:02.115 INFO:tasks.workunit.client.0.smithi070.stdout:8/741: sync 2024-04-02T04:48:02.167 INFO:tasks.workunit.client.0.smithi070.stdout:0/802: dwrite d4/d5/d13/d17/d72/f90 [0,4194304] 0 2024-04-02T04:48:02.168 INFO:tasks.workunit.client.0.smithi070.stdout:4/724: dwrite d1/f3e [0,4194304] 0 2024-04-02T04:48:02.171 INFO:tasks.workunit.client.0.smithi070.stdout:2/824: dwrite d2/fb [4194304,4194304] 0 2024-04-02T04:48:02.174 INFO:tasks.workunit.client.0.smithi070.stdout:2/825: write d2/d10/d5c/d7a/db0/fe3 [305436,81813] 0 2024-04-02T04:48:02.235 INFO:tasks.workunit.client.0.smithi070.stdout:7/694: getdents d7/d4e/d82/d57/dce 0 2024-04-02T04:48:02.238 INFO:tasks.workunit.client.1.smithi179.stdout:5/788: dwrite d8/f18 [4194304,4194304] 0 2024-04-02T04:48:02.239 INFO:tasks.workunit.client.1.smithi179.stdout:5/789: truncate d8/d1b/d23/db1/f102 4503756 0 2024-04-02T04:48:02.243 INFO:tasks.workunit.client.0.smithi070.stdout:8/742: write d2/d8/d23/d24/d5f/feb [462781,125597] 0 2024-04-02T04:48:02.246 INFO:tasks.workunit.client.1.smithi179.stdout:5/790: dwrite d8/d1b/fe1 [0,4194304] 0 2024-04-02T04:48:02.256 INFO:tasks.workunit.client.1.smithi179.stdout:5/791: dwrite d8/d1b/d23/d32/d4b/fb0 [0,4194304] 0 2024-04-02T04:48:02.257 INFO:tasks.workunit.client.1.smithi179.stdout:5/792: chown d8/d1b/d23/d36/f41 64876 1 2024-04-02T04:48:02.257 INFO:tasks.workunit.client.0.smithi070.stdout:4/725: mknod d1/dc3/cf2 0 2024-04-02T04:48:02.258 INFO:tasks.workunit.client.1.smithi179.stdout:5/793: read - d8/d1b/d23/d32/d4e/fb8 zero size 2024-04-02T04:48:02.263 INFO:tasks.workunit.client.1.smithi179.stdout:5/794: dwrite f1 [0,4194304] 0 2024-04-02T04:48:02.277 INFO:tasks.workunit.client.0.smithi070.stdout:2/826: dread d2/d10/d53/f55 [0,4194304] 0 2024-04-02T04:48:02.284 INFO:tasks.workunit.client.0.smithi070.stdout:3/751: sync 2024-04-02T04:48:02.287 INFO:tasks.workunit.client.0.smithi070.stdout:1/711: getdents d0/d7/d28/d83 0 2024-04-02T04:48:02.289 INFO:tasks.workunit.client.0.smithi070.stdout:1/712: chown d0/d54/d64/d6d/db6/fba 64 1 2024-04-02T04:48:02.290 INFO:tasks.workunit.client.0.smithi070.stdout:1/713: chown d0/d9/d1d/d22 36 1 2024-04-02T04:48:02.297 INFO:tasks.workunit.client.0.smithi070.stdout:1/714: dwrite d0/d9/d1d/d22/d2c/d4f/f92 [4194304,4194304] 0 2024-04-02T04:48:02.310 INFO:tasks.workunit.client.0.smithi070.stdout:4/726: mknod d1/dc3/dc4/cf3 0 2024-04-02T04:48:02.315 INFO:tasks.workunit.client.0.smithi070.stdout:0/803: link d4/d5/d13/d2a/d3a/lb9 d4/d5/d13/d2a/d3a/d80/db6/lf5 0 2024-04-02T04:48:02.320 INFO:tasks.workunit.client.0.smithi070.stdout:2/827: creat d2/d10/d21/d71/dd4/d100/f114 x:0 0 0 2024-04-02T04:48:02.323 INFO:tasks.workunit.client.0.smithi070.stdout:1/715: dread d0/d7/d28/f74 [4194304,4194304] 0 2024-04-02T04:48:02.329 INFO:tasks.workunit.client.0.smithi070.stdout:4/727: mknod d1/d3/d85/d8e/d99/db1/dbb/cf4 0 2024-04-02T04:48:02.329 INFO:tasks.workunit.client.0.smithi070.stdout:4/728: stat d1/d3/l55 0 2024-04-02T04:48:02.335 INFO:tasks.workunit.client.0.smithi070.stdout:8/743: getdents d2/d8/d29/d72 0 2024-04-02T04:48:02.345 INFO:tasks.workunit.client.0.smithi070.stdout:9/683: sync 2024-04-02T04:48:02.351 INFO:tasks.workunit.client.0.smithi070.stdout:9/684: creat d4/d10/d12/d27/d68/fde x:0 0 0 2024-04-02T04:48:02.359 INFO:tasks.workunit.client.0.smithi070.stdout:6/627: rename d0/da2/ca4 to d0/d43/cbe 0 2024-04-02T04:48:02.363 INFO:tasks.workunit.client.0.smithi070.stdout:1/716: unlink d0/d4c/c82 0 2024-04-02T04:48:02.366 INFO:tasks.workunit.client.0.smithi070.stdout:4/729: rename d1/d3/d78/cd0 to d1/d3/d78/dad/dbd/cf5 0 2024-04-02T04:48:02.371 INFO:tasks.workunit.client.0.smithi070.stdout:6/628: creat d0/d20/d77/fbf x:0 0 0 2024-04-02T04:48:02.373 INFO:tasks.workunit.client.0.smithi070.stdout:4/730: dwrite d1/f26 [0,4194304] 0 2024-04-02T04:48:02.385 INFO:tasks.workunit.client.0.smithi070.stdout:6/629: stat d0/d3/d19/f47 0 2024-04-02T04:48:02.385 INFO:tasks.workunit.client.1.smithi179.stdout:0/905: truncate d1/db/d64/f99 4979294 0 2024-04-02T04:48:02.387 INFO:tasks.workunit.client.0.smithi070.stdout:9/685: mkdir d4/d10/d12/d56/d79/dd8/ddf 0 2024-04-02T04:48:02.391 INFO:tasks.workunit.client.0.smithi070.stdout:4/731: creat d1/dc3/ff6 x:0 0 0 2024-04-02T04:48:02.393 INFO:tasks.workunit.client.1.smithi179.stdout:4/846: mknod d1/d3/dc/dc6/c11e 0 2024-04-02T04:48:02.394 INFO:tasks.workunit.client.1.smithi179.stdout:4/847: chown d1/d15/d7c/db8 33076 1 2024-04-02T04:48:02.397 INFO:tasks.workunit.client.0.smithi070.stdout:8/744: rename d2/d3/d35/d3a/d4f/cb2 to d2/d8/d29/d22/d5a/d77/cf3 0 2024-04-02T04:48:02.398 INFO:tasks.workunit.client.0.smithi070.stdout:4/732: dwrite d1/d3/f4c [0,4194304] 0 2024-04-02T04:48:02.402 INFO:tasks.workunit.client.1.smithi179.stdout:2/706: mknod dc/d29/d31/d83/dae/db0/cef 0 2024-04-02T04:48:02.403 INFO:tasks.workunit.client.1.smithi179.stdout:2/707: dread - dc/d29/d4a/fad zero size 2024-04-02T04:48:02.404 INFO:tasks.workunit.client.1.smithi179.stdout:2/708: chown dc/d29/d4a/d1e/cd1 2760633 1 2024-04-02T04:48:02.405 INFO:tasks.workunit.client.0.smithi070.stdout:9/686: symlink d4/d10/d11/d7b/d19/d2f/d39/d57/da6/le0 0 2024-04-02T04:48:02.407 INFO:tasks.workunit.client.1.smithi179.stdout:1/833: symlink d3/d5/d2f/db7/l125 0 2024-04-02T04:48:02.409 INFO:tasks.workunit.client.1.smithi179.stdout:1/834: write d3/d5/d2f/d4f/d42/d53/d72/d86/f90 [733608,7543] 0 2024-04-02T04:48:02.410 INFO:tasks.workunit.client.1.smithi179.stdout:1/835: chown d3/d5/d2f/d4f/d42/d53 1779325 1 2024-04-02T04:48:02.424 INFO:tasks.workunit.client.0.smithi070.stdout:9/687: mkdir d4/d34/db9/de1 0 2024-04-02T04:48:02.425 INFO:tasks.workunit.client.0.smithi070.stdout:9/688: write d4/d13/d89/faf [5916281,55358] 0 2024-04-02T04:48:02.425 INFO:tasks.workunit.client.0.smithi070.stdout:4/733: link d1/de7/ce8 d1/d3/d85/d8e/dc8/cf7 0 2024-04-02T04:48:02.427 INFO:tasks.workunit.client.1.smithi179.stdout:6/823: rmdir d3/d2c/d85/d117 0 2024-04-02T04:48:02.431 INFO:tasks.workunit.client.1.smithi179.stdout:0/906: rename d1/da/d28/d81/fab to d1/da/d52/d75/f106 0 2024-04-02T04:48:02.435 INFO:tasks.workunit.client.1.smithi179.stdout:9/807: unlink d0/d1/d11/l46 0 2024-04-02T04:48:02.436 INFO:tasks.workunit.client.1.smithi179.stdout:9/808: write d0/d1/d10/d6d/d58/f96 [177017,97081] 0 2024-04-02T04:48:02.438 INFO:tasks.workunit.client.1.smithi179.stdout:4/848: creat d1/d3/df/d10/d3a/f11f x:0 0 0 2024-04-02T04:48:02.444 INFO:tasks.workunit.client.0.smithi070.stdout:4/734: link d1/d3/f30 d1/dc3/dc4/ff8 0 2024-04-02T04:48:02.446 INFO:tasks.workunit.client.0.smithi070.stdout:4/735: read d1/d9c/f73 [72764,35329] 0 2024-04-02T04:48:02.448 INFO:tasks.workunit.client.0.smithi070.stdout:4/736: read - d1/d3/d85/d8e/d99/db1/dbb/fec zero size 2024-04-02T04:48:02.449 INFO:tasks.workunit.client.0.smithi070.stdout:4/737: dread - d1/dce/fd1 zero size 2024-04-02T04:48:02.449 INFO:tasks.workunit.client.1.smithi179.stdout:8/839: truncate d3/d32/d21/d22/d6b/db1/fbe 160080 0 2024-04-02T04:48:02.453 INFO:tasks.workunit.client.1.smithi179.stdout:3/853: write d8/d26/d52/d7e/db8/fde [1096848,130964] 0 2024-04-02T04:48:02.462 INFO:tasks.workunit.client.1.smithi179.stdout:4/849: creat d1/d15/d7c/d98/f120 x:0 0 0 2024-04-02T04:48:02.464 INFO:tasks.workunit.client.1.smithi179.stdout:7/840: unlink d1/dd/d76/fca 0 2024-04-02T04:48:02.466 INFO:tasks.workunit.client.0.smithi070.stdout:5/896: dwrite d5/d17/f53 [0,4194304] 0 2024-04-02T04:48:02.468 INFO:tasks.workunit.client.1.smithi179.stdout:5/795: rmdir d8/d1b/d23/d30/d73/dee 0 2024-04-02T04:48:02.469 INFO:tasks.workunit.client.1.smithi179.stdout:5/796: readlink d8/d1b/d23/d30/lea 0 2024-04-02T04:48:02.473 INFO:tasks.workunit.client.1.smithi179.stdout:0/907: mkdir d1/da/d48/d65/d107 0 2024-04-02T04:48:02.474 INFO:tasks.workunit.client.0.smithi070.stdout:5/897: dwrite d5/d17/d6c/f41 [0,4194304] 0 2024-04-02T04:48:02.475 INFO:tasks.workunit.client.1.smithi179.stdout:9/809: getdents d0/d9f 0 2024-04-02T04:48:02.475 INFO:tasks.workunit.client.1.smithi179.stdout:0/908: chown d1/da/df/l55 88904 1 2024-04-02T04:48:02.475 INFO:tasks.workunit.client.1.smithi179.stdout:5/797: dwrite d8/fd3 [0,4194304] 0 2024-04-02T04:48:02.477 INFO:tasks.workunit.client.1.smithi179.stdout:0/909: readlink d1/da/d28/d37/l6f 0 2024-04-02T04:48:02.479 INFO:tasks.workunit.client.0.smithi070.stdout:5/898: stat d5/d9/d23/d30/da2/c3c 0 2024-04-02T04:48:02.482 INFO:tasks.workunit.client.1.smithi179.stdout:1/836: creat d3/d5/f126 x:0 0 0 2024-04-02T04:48:02.484 INFO:tasks.workunit.client.1.smithi179.stdout:8/840: link d3/d32/d21/d45/feb d3/d32/d21/d22/d6b/db1/d104/d11d/f12c 0 2024-04-02T04:48:02.485 INFO:tasks.workunit.client.0.smithi070.stdout:5/899: symlink d5/d9/d23/d30/da2/d75/l11b 0 2024-04-02T04:48:02.490 INFO:tasks.workunit.client.0.smithi070.stdout:5/900: chown d5/d17/d6c/fbb 468 1 2024-04-02T04:48:02.491 INFO:tasks.workunit.client.0.smithi070.stdout:5/901: dread - d5/d9/d23/d30/da2/f102 zero size 2024-04-02T04:48:02.495 INFO:tasks.workunit.client.0.smithi070.stdout:5/902: write d5/d9/d23/f106 [5461660,38774] 0 2024-04-02T04:48:02.498 INFO:tasks.workunit.client.1.smithi179.stdout:7/841: creat d1/dd/d16/d4d/d11b/f127 x:0 0 0 2024-04-02T04:48:02.498 INFO:tasks.workunit.client.1.smithi179.stdout:1/837: chown d3/d5/d2f/d36/l38 933 1 2024-04-02T04:48:02.503 INFO:tasks.workunit.client.0.smithi070.stdout:5/903: symlink d5/d17/d27/l11c 0 2024-04-02T04:48:02.504 INFO:tasks.workunit.client.1.smithi179.stdout:5/798: fdatasync d8/d1b/d23/d42/f5d 0 2024-04-02T04:48:02.517 INFO:tasks.workunit.client.1.smithi179.stdout:4/850: getdents d1/d15/d7c/db8 0 2024-04-02T04:48:02.517 INFO:tasks.workunit.client.1.smithi179.stdout:7/842: dwrite d1/d9/d27/d87/d8b/d75/db9/dcf/f114 [0,4194304] 0 2024-04-02T04:48:02.518 INFO:tasks.workunit.client.1.smithi179.stdout:0/910: creat d1/da/d28/f108 x:0 0 0 2024-04-02T04:48:02.519 INFO:tasks.workunit.client.1.smithi179.stdout:5/799: dread d8/d1b/d23/d32/dbe/fcf [0,4194304] 0 2024-04-02T04:48:02.521 INFO:tasks.workunit.client.1.smithi179.stdout:5/800: chown d8/d1b/d23/d30/f34 7 1 2024-04-02T04:48:02.523 INFO:tasks.workunit.client.0.smithi070.stdout:5/904: dread d5/f2f [0,4194304] 0 2024-04-02T04:48:02.523 INFO:tasks.workunit.client.1.smithi179.stdout:7/843: unlink d1/fe2 0 2024-04-02T04:48:02.525 INFO:tasks.workunit.client.1.smithi179.stdout:0/911: fdatasync d1/da/f2d 0 2024-04-02T04:48:02.528 INFO:tasks.workunit.client.0.smithi070.stdout:5/905: creat d5/d9/d23/dc3/f11d x:0 0 0 2024-04-02T04:48:02.531 INFO:tasks.workunit.client.1.smithi179.stdout:7/844: mknod d1/d9/d27/d87/d8b/c128 0 2024-04-02T04:48:02.533 INFO:tasks.workunit.client.0.smithi070.stdout:5/906: creat d5/d17/d6c/f11e x:0 0 0 2024-04-02T04:48:02.538 INFO:tasks.workunit.client.1.smithi179.stdout:7/845: symlink d1/d9/dd9/d121/l129 0 2024-04-02T04:48:02.546 INFO:tasks.workunit.client.0.smithi070.stdout:3/752: write d10/d32/d3f/f4d [306665,124018] 0 2024-04-02T04:48:02.549 INFO:tasks.workunit.client.0.smithi070.stdout:3/753: creat d10/d17/db2/fff x:0 0 0 2024-04-02T04:48:02.555 INFO:tasks.workunit.client.0.smithi070.stdout:2/828: dwrite d2/d10/d5c/d7a/db0/dd8/dd2/fa0 [0,4194304] 0 2024-04-02T04:48:02.558 INFO:tasks.workunit.client.1.smithi179.stdout:3/854: unlink d8/d26/f2c 0 2024-04-02T04:48:02.559 INFO:tasks.workunit.client.1.smithi179.stdout:3/855: fdatasync d8/d26/d52/d7e/db8/fde 0 2024-04-02T04:48:02.560 INFO:tasks.workunit.client.1.smithi179.stdout:3/856: fsync d8/d57/d63/dc3/fc8 0 2024-04-02T04:48:02.560 INFO:tasks.workunit.client.0.smithi070.stdout:2/829: creat d2/d10/d5c/d7a/db0/dd8/dd2/f115 x:0 0 0 2024-04-02T04:48:02.563 INFO:tasks.workunit.client.0.smithi070.stdout:2/830: rmdir d2/d10/d5c/d7a 39 2024-04-02T04:48:02.564 INFO:tasks.workunit.client.0.smithi070.stdout:2/831: rmdir d2/d10/d21 39 2024-04-02T04:48:02.564 INFO:tasks.workunit.client.0.smithi070.stdout:2/832: readlink d2/d24/l49 0 2024-04-02T04:48:02.569 INFO:tasks.workunit.client.0.smithi070.stdout:2/833: getdents d2/d10/d19 0 2024-04-02T04:48:02.572 INFO:tasks.workunit.client.0.smithi070.stdout:2/834: creat d2/d10/d21/d3d/ddc/f116 x:0 0 0 2024-04-02T04:48:02.575 INFO:tasks.workunit.client.0.smithi070.stdout:2/835: mkdir d2/d10/d5c/d7a/db0/dc7/d117 0 2024-04-02T04:48:02.579 INFO:tasks.workunit.client.0.smithi070.stdout:1/717: write d0/d9/d1d/d22/d2c/f76 [5011724,56651] 0 2024-04-02T04:48:02.579 INFO:tasks.workunit.client.0.smithi070.stdout:2/836: dread d2/d24/f96 [0,4194304] 0 2024-04-02T04:48:02.582 INFO:tasks.workunit.client.0.smithi070.stdout:1/718: read d0/d9/f3e [686954,93155] 0 2024-04-02T04:48:02.586 INFO:tasks.workunit.client.0.smithi070.stdout:2/837: truncate d2/d10/d5c/d7a/fda 2318735 0 2024-04-02T04:48:02.587 INFO:tasks.workunit.client.0.smithi070.stdout:2/838: chown d2/d10/d21/d71/dd4/c8b 13981 1 2024-04-02T04:48:02.588 INFO:tasks.workunit.client.0.smithi070.stdout:2/839: readlink d2/d10/d21/d3d/d42/ld5 0 2024-04-02T04:48:02.589 INFO:tasks.workunit.client.0.smithi070.stdout:2/840: dread - d2/d10/d5c/d7a/db0/fdf zero size 2024-04-02T04:48:02.590 INFO:tasks.workunit.client.0.smithi070.stdout:1/719: getdents d0/d9/d1d/d22/d2c/d4f 0 2024-04-02T04:48:02.594 INFO:tasks.workunit.client.0.smithi070.stdout:1/720: truncate d0/d9/d1d/d22/d56/f69 2145222 0 2024-04-02T04:48:02.597 INFO:tasks.workunit.client.0.smithi070.stdout:2/841: dwrite d2/d12/d3e/fac [0,4194304] 0 2024-04-02T04:48:02.598 INFO:tasks.workunit.client.0.smithi070.stdout:6/630: dwrite d0/d20/f23 [0,4194304] 0 2024-04-02T04:48:02.599 INFO:tasks.workunit.client.1.smithi179.stdout:2/709: truncate dc/d29/d4a/d4b/f51 3542244 0 2024-04-02T04:48:02.599 INFO:tasks.workunit.client.0.smithi070.stdout:2/842: write d2/d10/d19/f1e [4569301,116030] 0 2024-04-02T04:48:02.599 INFO:tasks.workunit.client.1.smithi179.stdout:2/710: chown dc/d29/d4a/d1e/d6c/lda 20 1 2024-04-02T04:48:02.601 INFO:tasks.workunit.client.0.smithi070.stdout:1/721: creat d0/d54/d64/fe4 x:0 0 0 2024-04-02T04:48:02.602 INFO:tasks.workunit.client.0.smithi070.stdout:2/843: chown d2/d10/d5c/d7a/db0/dc7 746 1 2024-04-02T04:48:02.607 INFO:tasks.workunit.client.1.smithi179.stdout:2/711: rmdir dc/d29/d4a/d1e 39 2024-04-02T04:48:02.608 INFO:tasks.workunit.client.1.smithi179.stdout:2/712: write dc/d29/d31/d57/fee [743261,89260] 0 2024-04-02T04:48:02.609 INFO:tasks.workunit.client.0.smithi070.stdout:8/745: dwrite d2/d8/d29/db/f13 [4194304,4194304] 0 2024-04-02T04:48:02.609 INFO:tasks.workunit.client.1.smithi179.stdout:5/801: rename d8/d1b/d23/d32/f7f to d8/d1b/d23/d36/d53/d62/f103 0 2024-04-02T04:48:02.618 INFO:tasks.workunit.client.0.smithi070.stdout:6/631: mknod d0/d3/d19/d31/d61/d7b/cc0 0 2024-04-02T04:48:02.621 INFO:tasks.workunit.client.1.smithi179.stdout:4/851: rename d1/d3/df/d10/d2f/d7b/f86 to d1/d3/df/d10/f121 0 2024-04-02T04:48:02.622 INFO:tasks.workunit.client.1.smithi179.stdout:4/852: write d1/d3/dc/d27/d62/fe2 [1692195,28826] 0 2024-04-02T04:48:02.625 INFO:tasks.workunit.client.1.smithi179.stdout:2/713: mknod dc/d29/d4a/d1e/dbe/cf0 0 2024-04-02T04:48:02.625 INFO:tasks.workunit.client.0.smithi070.stdout:9/689: write d4/d10/d11/d7b/d19/f24 [2246559,83234] 0 2024-04-02T04:48:02.626 INFO:tasks.workunit.client.1.smithi179.stdout:2/714: dread - dc/d29/d4a/d1e/fec zero size 2024-04-02T04:48:02.626 INFO:tasks.workunit.client.1.smithi179.stdout:2/715: chown dc/d29/d31 4 1 2024-04-02T04:48:02.627 INFO:tasks.workunit.client.0.smithi070.stdout:2/844: rename d2/d12/fa6 to d2/d10/d53/f118 0 2024-04-02T04:48:02.630 INFO:tasks.workunit.client.0.smithi070.stdout:6/632: unlink d0/d43/fbc 0 2024-04-02T04:48:02.631 INFO:tasks.workunit.client.0.smithi070.stdout:6/633: write d0/d3/d19/d31/fb1 [2802646,35546] 0 2024-04-02T04:48:02.632 INFO:tasks.workunit.client.1.smithi179.stdout:0/912: rename d1/db/l103 to d1/da/d28/d34/ddd/l109 0 2024-04-02T04:48:02.632 INFO:tasks.workunit.client.1.smithi179.stdout:2/716: dwrite dc/d29/f34 [4194304,4194304] 0 2024-04-02T04:48:02.634 INFO:tasks.workunit.client.1.smithi179.stdout:2/717: chown dc/d29/d31/d83/dae/db0/dbc/c74 3 1 2024-04-02T04:48:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:2/845: mknod d2/d10/d21/d71/dc1/c119 0 2024-04-02T04:48:02.638 INFO:tasks.workunit.client.1.smithi179.stdout:2/718: stat dc/d29/d31/d83/dae/ddc 0 2024-04-02T04:48:02.639 INFO:tasks.workunit.client.1.smithi179.stdout:1/838: link d3/d5/de/d33/c52 d3/d85/da2/c127 0 2024-04-02T04:48:02.639 INFO:tasks.workunit.client.1.smithi179.stdout:0/913: dread d1/f3b [0,4194304] 0 2024-04-02T04:48:02.640 INFO:tasks.workunit.client.1.smithi179.stdout:0/914: write d1/da/d48/d65/d7d/fd1 [4220098,46514] 0 2024-04-02T04:48:02.643 INFO:tasks.workunit.client.0.smithi070.stdout:4/738: truncate d1/d3/d78/d91/faa 3228715 0 2024-04-02T04:48:02.646 INFO:tasks.workunit.client.0.smithi070.stdout:6/634: rmdir d0/d55/d8b/d94 39 2024-04-02T04:48:02.652 INFO:tasks.workunit.client.1.smithi179.stdout:1/839: fdatasync d3/d5/d2f/d4f/d42/df2/f8b 0 2024-04-02T04:48:02.652 INFO:tasks.workunit.client.1.smithi179.stdout:1/840: chown d3/d5/d2f/d4f/d42/df2 29 1 2024-04-02T04:48:02.653 INFO:tasks.workunit.client.1.smithi179.stdout:6/824: write d3/d12/d23/f55 [3782363,70698] 0 2024-04-02T04:48:02.656 INFO:tasks.workunit.client.0.smithi070.stdout:4/739: rename d1/d3/d85/da8/fe5 to d1/d3/d85/d8e/d99/ff9 0 2024-04-02T04:48:02.656 INFO:tasks.workunit.client.0.smithi070.stdout:4/740: chown d1/fd 78 1 2024-04-02T04:48:02.658 INFO:tasks.workunit.client.0.smithi070.stdout:8/746: getdents d2/d8/d29/d22/d5a/d94/da2 0 2024-04-02T04:48:02.661 INFO:tasks.workunit.client.0.smithi070.stdout:2/846: link d2/d10/d21/fee d2/d10/d21/d71/dc1/f11a 0 2024-04-02T04:48:02.664 INFO:tasks.workunit.client.0.smithi070.stdout:6/635: mkdir d0/d3/d19/d31/dc1 0 2024-04-02T04:48:02.664 INFO:tasks.workunit.client.0.smithi070.stdout:6/636: readlink d0/d55/l90 0 2024-04-02T04:48:02.664 INFO:tasks.workunit.client.0.smithi070.stdout:6/637: readlink d0/d43/l99 0 2024-04-02T04:48:02.665 INFO:tasks.workunit.client.1.smithi179.stdout:1/841: stat d3/d5/f60 0 2024-04-02T04:48:02.665 INFO:tasks.workunit.client.0.smithi070.stdout:4/741: creat d1/d3/d85/d8e/d99/db1/ffa x:0 0 0 2024-04-02T04:48:02.666 INFO:tasks.workunit.client.1.smithi179.stdout:0/915: mkdir d1/da/d52/d75/d9a/df8/d10a 0 2024-04-02T04:48:02.667 INFO:tasks.workunit.client.1.smithi179.stdout:0/916: stat d1/dfe 0 2024-04-02T04:48:02.668 INFO:tasks.workunit.client.0.smithi070.stdout:7/695: symlink d7/dd/df/le1 0 2024-04-02T04:48:02.670 INFO:tasks.workunit.client.1.smithi179.stdout:7/846: rename d1/dd/f20 to d1/dd/d16/f12a 0 2024-04-02T04:48:02.671 INFO:tasks.workunit.client.0.smithi070.stdout:2/847: mknod d2/d10/d21/d3d/ddc/c11b 0 2024-04-02T04:48:02.671 INFO:tasks.workunit.client.1.smithi179.stdout:8/841: write d3/d32/d21/f66 [117360,108812] 0 2024-04-02T04:48:02.672 INFO:tasks.workunit.client.1.smithi179.stdout:2/719: creat dc/ff1 x:0 0 0 2024-04-02T04:48:02.672 INFO:tasks.workunit.client.1.smithi179.stdout:0/917: dwrite d1/da/d28/f108 [0,4194304] 0 2024-04-02T04:48:02.672 INFO:tasks.workunit.client.1.smithi179.stdout:9/810: write d0/d19/d27/d7a/d7e/dd4/fef [298657,71333] 0 2024-04-02T04:48:02.674 INFO:tasks.workunit.client.1.smithi179.stdout:0/918: fsync d1/da/d67/f100 0 2024-04-02T04:48:02.674 INFO:tasks.workunit.client.0.smithi070.stdout:4/742: creat d1/d3/d85/d8e/dc8/ffb x:0 0 0 2024-04-02T04:48:02.676 INFO:tasks.workunit.client.1.smithi179.stdout:2/720: write dc/dcb/fcf [1535999,36559] 0 2024-04-02T04:48:02.676 INFO:tasks.workunit.client.1.smithi179.stdout:9/811: write d0/d19/d27/fa7 [479768,79208] 0 2024-04-02T04:48:02.677 INFO:tasks.workunit.client.1.smithi179.stdout:8/842: fsync d3/d32/d21/d22/d6b/db1/df8/d126/f121 0 2024-04-02T04:48:02.678 INFO:tasks.workunit.client.0.smithi070.stdout:8/747: creat d2/d8/d29/d22/d5a/d94/da2/dde/ff4 x:0 0 0 2024-04-02T04:48:02.679 INFO:tasks.workunit.client.1.smithi179.stdout:8/843: write d3/d32/d21/d50/d112/f115 [950537,66548] 0 2024-04-02T04:48:02.679 INFO:tasks.workunit.client.0.smithi070.stdout:4/743: dread d1/d3/f39 [0,4194304] 0 2024-04-02T04:48:02.680 INFO:tasks.workunit.client.1.smithi179.stdout:4/853: rename d1/c2a to d1/d93/de6/c122 0 2024-04-02T04:48:02.684 INFO:tasks.workunit.client.0.smithi070.stdout:7/696: truncate d7/dd/df/d79/d46/d65/fa4 238671 0 2024-04-02T04:48:02.684 INFO:tasks.workunit.client.1.smithi179.stdout:7/847: rmdir d1/d9/d55/dbf/d9f/da8/d100 39 2024-04-02T04:48:02.691 INFO:tasks.workunit.client.1.smithi179.stdout:2/721: creat dc/d79/ff2 x:0 0 0 2024-04-02T04:48:02.692 INFO:tasks.workunit.client.1.smithi179.stdout:2/722: dread - dc/f5a zero size 2024-04-02T04:48:02.696 INFO:tasks.workunit.client.0.smithi070.stdout:0/804: creat d4/d5/d13/d2a/ff6 x:0 0 0 2024-04-02T04:48:02.697 INFO:tasks.workunit.client.1.smithi179.stdout:9/812: creat d0/d19/d27/d7a/d84/ff8 x:0 0 0 2024-04-02T04:48:02.698 INFO:tasks.workunit.client.1.smithi179.stdout:9/813: write d0/d19/d27/d7a/d87/dea/fd7 [918936,54636] 0 2024-04-02T04:48:02.699 INFO:tasks.workunit.client.0.smithi070.stdout:8/748: dread d2/d8/d29/f3b [0,4194304] 0 2024-04-02T04:48:02.700 INFO:tasks.workunit.client.0.smithi070.stdout:8/749: chown d2/d8/d29/f3b 3052 1 2024-04-02T04:48:02.702 INFO:tasks.workunit.client.0.smithi070.stdout:5/907: dwrite d5/d57/f5d [0,4194304] 0 2024-04-02T04:48:02.704 INFO:tasks.workunit.client.0.smithi070.stdout:5/908: chown d5/d9/d10/db9/fbd 5154 1 2024-04-02T04:48:02.704 INFO:tasks.workunit.client.0.smithi070.stdout:7/697: mknod d7/dd/df/d7a/d6c/d77/ce2 0 2024-04-02T04:48:02.705 INFO:tasks.workunit.client.1.smithi179.stdout:6/825: rename d3/d12/d23/d2f/f56 to d3/d2d/d38/d64/f123 0 2024-04-02T04:48:02.705 INFO:tasks.workunit.client.1.smithi179.stdout:7/848: creat d1/d9/d7e/f12b x:0 0 0 2024-04-02T04:48:02.706 INFO:tasks.workunit.client.1.smithi179.stdout:9/814: stat d0/d1/d11/dab 0 2024-04-02T04:48:02.708 INFO:tasks.workunit.client.0.smithi070.stdout:0/805: creat d4/d5/d13/d23/ff7 x:0 0 0 2024-04-02T04:48:02.710 INFO:tasks.workunit.client.0.smithi070.stdout:6/638: getdents d0/d43/d80 0 2024-04-02T04:48:02.713 INFO:tasks.workunit.client.0.smithi070.stdout:8/750: dwrite d2/d8/d23/d24/d5f/fe2 [0,4194304] 0 2024-04-02T04:48:02.715 INFO:tasks.workunit.client.0.smithi070.stdout:6/639: fsync d0/d3/d19/d31/f74 0 2024-04-02T04:48:02.715 INFO:tasks.workunit.client.0.smithi070.stdout:3/754: dwrite d10/d32/d8a/d8d/fc1 [0,4194304] 0 2024-04-02T04:48:02.721 INFO:tasks.workunit.client.0.smithi070.stdout:3/755: truncate d10/d32/d4b/dc5/fd0 714522 0 2024-04-02T04:48:02.723 INFO:tasks.workunit.client.0.smithi070.stdout:5/909: write d5/d17/d27/d2e/f91 [299667,83024] 0 2024-04-02T04:48:02.728 INFO:tasks.workunit.client.1.smithi179.stdout:6/826: creat d3/d2d/d38/d64/f124 x:0 0 0 2024-04-02T04:48:02.728 INFO:tasks.workunit.client.0.smithi070.stdout:8/751: dwrite d2/d3/d35/f78 [0,4194304] 0 2024-04-02T04:48:02.734 INFO:tasks.workunit.client.0.smithi070.stdout:1/722: write d0/d54/d64/d6d/fad [360863,81744] 0 2024-04-02T04:48:02.740 INFO:tasks.workunit.client.1.smithi179.stdout:1/842: getdents d3/d5/d2f/d4f/d42/d55/dc5/d10a 0 2024-04-02T04:48:02.741 INFO:tasks.workunit.client.1.smithi179.stdout:2/723: getdents dc/d29/d31/d83/dae/ddc 0 2024-04-02T04:48:02.742 INFO:tasks.workunit.client.1.smithi179.stdout:2/724: truncate dc/d29/d31/fb9 860625 0 2024-04-02T04:48:02.747 INFO:tasks.workunit.client.0.smithi070.stdout:9/690: write d4/d10/d11/d7b/f61 [1029798,66022] 0 2024-04-02T04:48:02.749 INFO:tasks.workunit.client.0.smithi070.stdout:3/756: creat d10/d32/d3f/dc0/f100 x:0 0 0 2024-04-02T04:48:02.749 INFO:tasks.workunit.client.1.smithi179.stdout:4/854: link d1/d3/df/d10/d2f/d60/fef d1/d3/df/d10/d2f/d60/da6/f123 0 2024-04-02T04:48:02.750 INFO:tasks.workunit.client.0.smithi070.stdout:3/757: chown d10/d32/f71 3 1 2024-04-02T04:48:02.752 INFO:tasks.workunit.client.1.smithi179.stdout:6/827: mkdir d3/d125 0 2024-04-02T04:48:02.756 INFO:tasks.workunit.client.0.smithi070.stdout:4/744: rename d1/d3/d85/d8e/d99/ff9 to d1/d3/d85/ddb/ffc 0 2024-04-02T04:48:02.758 INFO:tasks.workunit.client.1.smithi179.stdout:9/815: symlink d0/lf9 0 2024-04-02T04:48:02.758 INFO:tasks.workunit.client.0.smithi070.stdout:9/691: dread d4/d10/f2c [0,4194304] 0 2024-04-02T04:48:02.761 INFO:tasks.workunit.client.0.smithi070.stdout:8/752: symlink d2/d3/d35/d3a/lf5 0 2024-04-02T04:48:02.763 INFO:tasks.workunit.client.1.smithi179.stdout:2/725: creat dc/d29/d31/d83/dae/db0/ff3 x:0 0 0 2024-04-02T04:48:02.768 INFO:tasks.workunit.client.1.smithi179.stdout:6/828: rename d3/d12/d23/d39/d80/fa3 to d3/d2c/de0/f126 0 2024-04-02T04:48:02.768 INFO:tasks.workunit.client.1.smithi179.stdout:2/726: dwrite dc/d29/d31/f3a [4194304,4194304] 0 2024-04-02T04:48:02.770 INFO:tasks.workunit.client.1.smithi179.stdout:5/802: truncate d8/d1b/d23/d32/d46/f7d 1228299 0 2024-04-02T04:48:02.772 INFO:tasks.workunit.client.0.smithi070.stdout:5/910: creat d5/d17/d27/de3/f11f x:0 0 0 2024-04-02T04:48:02.775 INFO:tasks.workunit.client.1.smithi179.stdout:4/855: unlink d1/d3/d19/fe0 0 2024-04-02T04:48:02.776 INFO:tasks.workunit.client.1.smithi179.stdout:9/816: rename d0/d19/d27/d7a/d87/dea to d0/d19/d27/d7a/d7e/db9/dfa 0 2024-04-02T04:48:02.777 INFO:tasks.workunit.client.1.smithi179.stdout:9/817: chown d0/d19/ca6 17909 1 2024-04-02T04:48:02.779 INFO:tasks.workunit.client.1.smithi179.stdout:6/829: symlink d3/d12/d23/d39/d80/d93/dd4/l127 0 2024-04-02T04:48:02.779 INFO:tasks.workunit.client.0.smithi070.stdout:2/848: dwrite d2/d10/d21/f89 [0,4194304] 0 2024-04-02T04:48:02.781 INFO:tasks.workunit.client.0.smithi070.stdout:2/849: write d2/d12/d59/fe5 [584904,121859] 0 2024-04-02T04:48:02.781 INFO:tasks.workunit.client.1.smithi179.stdout:9/818: dread d0/d1/d11/dab/ded/f79 [0,4194304] 0 2024-04-02T04:48:02.781 INFO:tasks.workunit.client.0.smithi070.stdout:8/753: creat d2/d8/d29/d22/dc8/ff6 x:0 0 0 2024-04-02T04:48:02.782 INFO:tasks.workunit.client.1.smithi179.stdout:3/857: sync 2024-04-02T04:48:02.784 INFO:tasks.workunit.client.0.smithi070.stdout:0/806: link d4/d5/d21/f8c d4/d5/da/d2d/de7/ff8 0 2024-04-02T04:48:02.784 INFO:tasks.workunit.client.0.smithi070.stdout:0/807: chown f0 1 1 2024-04-02T04:48:02.785 INFO:tasks.workunit.client.0.smithi070.stdout:0/808: dread - d4/d5/d58/d60/d83/feb zero size 2024-04-02T04:48:02.786 INFO:tasks.workunit.client.1.smithi179.stdout:4/856: unlink d1/d93/de6/c11c 0 2024-04-02T04:48:02.786 INFO:tasks.workunit.client.0.smithi070.stdout:0/809: write d4/d5/f84 [1489325,102075] 0 2024-04-02T04:48:02.788 INFO:tasks.workunit.client.0.smithi070.stdout:0/810: write f0 [4414075,1549] 0 2024-04-02T04:48:02.792 INFO:tasks.workunit.client.0.smithi070.stdout:3/758: symlink d10/d32/d4b/l101 0 2024-04-02T04:48:02.792 INFO:tasks.workunit.client.1.smithi179.stdout:0/919: truncate d1/da/f1b 2227262 0 2024-04-02T04:48:02.794 INFO:tasks.workunit.client.1.smithi179.stdout:6/830: truncate d3/d12/d6a/f75 634328 0 2024-04-02T04:48:02.794 INFO:tasks.workunit.client.1.smithi179.stdout:0/920: write d1/db/d64/f6a [5179428,56611] 0 2024-04-02T04:48:02.795 INFO:tasks.workunit.client.0.smithi070.stdout:5/911: symlink d5/d9/d23/d30/da2/d75/ddf/l120 0 2024-04-02T04:48:02.796 INFO:tasks.workunit.client.0.smithi070.stdout:0/811: dwrite d4/d5/d58/d60/f92 [4194304,4194304] 0 2024-04-02T04:48:02.798 INFO:tasks.workunit.client.0.smithi070.stdout:0/812: fsync d4/d5/f3f 0 2024-04-02T04:48:02.798 INFO:tasks.workunit.client.0.smithi070.stdout:7/698: write d7/dd/df/f94 [1594682,9117] 0 2024-04-02T04:48:02.800 INFO:tasks.workunit.client.1.smithi179.stdout:6/831: dwrite d3/d2d/d43/fef [0,4194304] 0 2024-04-02T04:48:02.802 INFO:tasks.workunit.client.1.smithi179.stdout:3/858: creat d8/d7a/f112 x:0 0 0 2024-04-02T04:48:02.804 INFO:tasks.workunit.client.1.smithi179.stdout:3/859: chown d8/d26/d36/f59 1958743263 1 2024-04-02T04:48:02.806 INFO:tasks.workunit.client.1.smithi179.stdout:4/857: creat d1/d3/dc/d26/d8a/d5d/f124 x:0 0 0 2024-04-02T04:48:02.806 INFO:tasks.workunit.client.1.smithi179.stdout:3/860: dread - d8/d57/d63/fe3 zero size 2024-04-02T04:48:02.807 INFO:tasks.workunit.client.1.smithi179.stdout:4/858: readlink d1/d3/df/d10/d2f/d60/d71/lf9 0 2024-04-02T04:48:02.813 INFO:tasks.workunit.client.0.smithi070.stdout:2/850: unlink d2/d24/l26 0 2024-04-02T04:48:02.815 INFO:tasks.workunit.client.1.smithi179.stdout:0/921: mknod d1/da/d48/d5c/d83/d97/c10b 0 2024-04-02T04:48:02.817 INFO:tasks.workunit.client.1.smithi179.stdout:5/803: creat d8/d1b/d23/f104 x:0 0 0 2024-04-02T04:48:02.818 INFO:tasks.workunit.client.0.smithi070.stdout:6/640: dwrite d0/d55/d8b/f9e [4194304,4194304] 0 2024-04-02T04:48:02.822 INFO:tasks.workunit.client.1.smithi179.stdout:6/832: rename d3/d12/d44/d10a/c11b to d3/d12/d44/d10a/d10e/c128 0 2024-04-02T04:48:02.823 INFO:tasks.workunit.client.1.smithi179.stdout:7/849: truncate d1/d9/dd9/f123 1553292 0 2024-04-02T04:48:02.825 INFO:tasks.workunit.client.1.smithi179.stdout:7/850: dread - d1/dd/d16/d4d/d11b/f127 zero size 2024-04-02T04:48:02.826 INFO:tasks.workunit.client.1.smithi179.stdout:7/851: chown d1/dd/d76/ff4 9 1 2024-04-02T04:48:02.827 INFO:tasks.workunit.client.0.smithi070.stdout:3/759: chown d10/d17/db2/dbd/ddb 2636 1 2024-04-02T04:48:02.828 INFO:tasks.workunit.client.1.smithi179.stdout:7/852: fdatasync d1/d9/d27/d87/d8b/d75/f9a 0 2024-04-02T04:48:02.834 INFO:tasks.workunit.client.0.smithi070.stdout:1/723: dwrite d0/d4c/fb7 [0,4194304] 0 2024-04-02T04:48:02.835 INFO:tasks.workunit.client.0.smithi070.stdout:5/912: dread d5/d17/d2b/d7b/f92 [0,4194304] 0 2024-04-02T04:48:02.845 INFO:tasks.workunit.client.0.smithi070.stdout:8/754: mknod d2/d3/d35/d3a/d4f/d59/dcc/cf7 0 2024-04-02T04:48:02.855 INFO:tasks.workunit.client.1.smithi179.stdout:4/859: dread d1/d3/dc/d26/d8a/f94 [0,4194304] 0 2024-04-02T04:48:02.855 INFO:tasks.workunit.client.0.smithi070.stdout:9/692: write d4/d10/d12/d56/f5b [1024515,113831] 0 2024-04-02T04:48:02.857 INFO:tasks.workunit.client.1.smithi179.stdout:7/853: mknod d1/d9/d27/d87/c12c 0 2024-04-02T04:48:02.860 INFO:tasks.workunit.client.0.smithi070.stdout:3/760: dread d10/d17/f33 [0,4194304] 0 2024-04-02T04:48:02.860 INFO:tasks.workunit.client.1.smithi179.stdout:0/922: creat d1/da/d67/f10c x:0 0 0 2024-04-02T04:48:02.860 INFO:tasks.workunit.client.0.smithi070.stdout:3/761: fsync d10/d32/d3f/f4d 0 2024-04-02T04:48:02.861 INFO:tasks.workunit.client.1.smithi179.stdout:4/860: dwrite d1/d93/d101/f105 [0,4194304] 0 2024-04-02T04:48:02.864 INFO:tasks.workunit.client.0.smithi070.stdout:1/724: unlink d0/db4/fa4 0 2024-04-02T04:48:02.867 INFO:tasks.workunit.client.0.smithi070.stdout:7/699: dwrite d7/d9/fdd [4194304,4194304] 0 2024-04-02T04:48:02.867 INFO:tasks.workunit.client.0.smithi070.stdout:5/913: write d5/d9/d23/d8b/de9/fea [3726515,99918] 0 2024-04-02T04:48:02.869 INFO:tasks.workunit.client.1.smithi179.stdout:6/833: creat d3/d2d/f129 x:0 0 0 2024-04-02T04:48:02.870 INFO:tasks.workunit.client.0.smithi070.stdout:1/725: dread d0/d9/d1d/d22/d2c/fb3 [4194304,4194304] 0 2024-04-02T04:48:02.874 INFO:tasks.workunit.client.0.smithi070.stdout:7/700: write d7/dd/df/d79/fbf [574975,98873] 0 2024-04-02T04:48:02.874 INFO:tasks.workunit.client.0.smithi070.stdout:7/701: chown d7/dd/df/d79/fbb 240 1 2024-04-02T04:48:02.875 INFO:tasks.workunit.client.1.smithi179.stdout:3/861: creat d8/dd3/f113 x:0 0 0 2024-04-02T04:48:02.878 INFO:tasks.workunit.client.0.smithi070.stdout:4/745: getdents d1/d3/d78/d91 0 2024-04-02T04:48:02.879 INFO:tasks.workunit.client.0.smithi070.stdout:4/746: write d1/d9c/f45 [1002195,83369] 0 2024-04-02T04:48:02.882 INFO:tasks.workunit.client.0.smithi070.stdout:8/755: fdatasync d2/d8/d29/fb8 0 2024-04-02T04:48:02.885 INFO:tasks.workunit.client.0.smithi070.stdout:0/813: write d4/d5/da/d2d/f4c [4181686,38253] 0 2024-04-02T04:48:02.890 INFO:tasks.workunit.client.1.smithi179.stdout:3/862: truncate d8/d26/d83/d89/d9c/fb0 997552 0 2024-04-02T04:48:02.895 INFO:tasks.workunit.client.1.smithi179.stdout:5/804: rename d8/d7e to d8/d1b/d23/d32/dbe/daf/dd4/d105 0 2024-04-02T04:48:02.900 INFO:tasks.workunit.client.1.smithi179.stdout:7/854: write d1/dd/d23/f25 [75419,31045] 0 2024-04-02T04:48:02.904 INFO:tasks.workunit.client.1.smithi179.stdout:4/861: rename d1/d3/dc/d7a to d1/d3/df/d10/d2f/d60/da6/d125 0 2024-04-02T04:48:02.906 INFO:tasks.workunit.client.1.smithi179.stdout:5/805: fdatasync d8/d1b/d23/d32/d4e/d51/fdb 0 2024-04-02T04:48:02.908 INFO:tasks.workunit.client.1.smithi179.stdout:4/862: dread d1/d15/d6e/d9c/fe7 [0,4194304] 0 2024-04-02T04:48:02.911 INFO:tasks.workunit.client.1.smithi179.stdout:4/863: read - d1/d93/de6/f10e zero size 2024-04-02T04:48:02.912 INFO:tasks.workunit.client.1.smithi179.stdout:7/855: fdatasync d1/d9/d27/d87/fe0 0 2024-04-02T04:48:02.913 INFO:tasks.workunit.client.1.smithi179.stdout:0/923: dwrite d1/da/f46 [0,4194304] 0 2024-04-02T04:48:02.914 INFO:tasks.workunit.client.1.smithi179.stdout:4/864: write d1/d3/dc/d26/ff3 [251815,106254] 0 2024-04-02T04:48:02.917 INFO:tasks.workunit.client.1.smithi179.stdout:6/834: rename d3/d12/d6a/de8/feb to d3/d12/d44/d10a/d10e/f12a 0 2024-04-02T04:48:02.924 INFO:tasks.workunit.client.1.smithi179.stdout:3/863: dwrite d8/d26/d36/f90 [0,4194304] 0 2024-04-02T04:48:02.928 INFO:tasks.workunit.client.1.smithi179.stdout:0/924: creat d1/f10d x:0 0 0 2024-04-02T04:48:02.934 INFO:tasks.workunit.client.1.smithi179.stdout:4/865: rmdir d1 39 2024-04-02T04:48:02.938 INFO:tasks.workunit.client.1.smithi179.stdout:7/856: rmdir d1/d9/d27/d87/d8b/d75/db9/d8d 39 2024-04-02T04:48:02.940 INFO:tasks.workunit.client.1.smithi179.stdout:5/806: creat d8/d1b/d23/d36/d53/d62/db2/f106 x:0 0 0 2024-04-02T04:48:02.942 INFO:tasks.workunit.client.1.smithi179.stdout:8/844: sync 2024-04-02T04:48:02.946 INFO:tasks.workunit.client.1.smithi179.stdout:6/835: creat d3/d12/d23/d2f/d120/f12b x:0 0 0 2024-04-02T04:48:02.949 INFO:tasks.workunit.client.1.smithi179.stdout:3/864: dwrite d8/d57/d63/fe3 [0,4194304] 0 2024-04-02T04:48:02.952 INFO:tasks.workunit.client.1.smithi179.stdout:8/845: dwrite d3/d32/d21/d22/d6b/d91/f44 [0,4194304] 0 2024-04-02T04:48:02.953 INFO:tasks.workunit.client.1.smithi179.stdout:7/857: creat d1/d9/d27/d87/d8b/d75/db9/ddd/f12d x:0 0 0 2024-04-02T04:48:02.954 INFO:tasks.workunit.client.0.smithi070.stdout:6/641: mkdir d0/d3/dc2 0 2024-04-02T04:48:02.954 INFO:tasks.workunit.client.1.smithi179.stdout:4/866: creat d1/d3/dc/d27/f126 x:0 0 0 2024-04-02T04:48:02.956 INFO:tasks.workunit.client.1.smithi179.stdout:4/867: chown d1/d3/dc/d27/d62/lbb 8803297 1 2024-04-02T04:48:02.956 INFO:tasks.workunit.client.1.smithi179.stdout:3/865: dread d8/d26/d36/fa2 [0,4194304] 0 2024-04-02T04:48:02.957 INFO:tasks.workunit.client.0.smithi070.stdout:9/693: truncate d4/d13/f99 617214 0 2024-04-02T04:48:02.958 INFO:tasks.workunit.client.1.smithi179.stdout:3/866: read - d8/d26/d52/d7e/db8/ffa zero size 2024-04-02T04:48:02.960 INFO:tasks.workunit.client.1.smithi179.stdout:6/836: creat d3/d2c/d85/f12c x:0 0 0 2024-04-02T04:48:02.969 INFO:tasks.workunit.client.0.smithi070.stdout:1/726: creat d0/d54/d64/d6d/fe5 x:0 0 0 2024-04-02T04:48:02.971 INFO:tasks.workunit.client.0.smithi070.stdout:5/914: dread d5/d9/d10/db9/dba/de1/ff4 [0,4194304] 0 2024-04-02T04:48:02.972 INFO:tasks.workunit.client.0.smithi070.stdout:7/702: mkdir d7/dd/df/d34/de3 0 2024-04-02T04:48:02.973 INFO:tasks.workunit.client.1.smithi179.stdout:6/837: mkdir d3/d2d/d43/d12d 0 2024-04-02T04:48:02.974 INFO:tasks.workunit.client.1.smithi179.stdout:0/925: getdents d1/da/d28/d34/d39 0 2024-04-02T04:48:02.977 INFO:tasks.workunit.client.1.smithi179.stdout:2/727: sync 2024-04-02T04:48:02.977 INFO:tasks.workunit.client.1.smithi179.stdout:1/843: sync 2024-04-02T04:48:02.977 INFO:tasks.workunit.client.1.smithi179.stdout:9/819: sync 2024-04-02T04:48:02.978 INFO:tasks.workunit.client.1.smithi179.stdout:7/858: mknod d1/d9/d27/d87/d8b/d75/db9/d8d/c12e 0 2024-04-02T04:48:02.979 INFO:tasks.workunit.client.1.smithi179.stdout:5/807: truncate d8/d1b/d23/d32/dbe/daf/db7/f79 882218 0 2024-04-02T04:48:02.980 INFO:tasks.workunit.client.0.smithi070.stdout:8/756: symlink d2/d8/d29/dc7/lf8 0 2024-04-02T04:48:02.980 INFO:tasks.workunit.client.1.smithi179.stdout:5/808: fdatasync d8/d1b/d23/d32/dbe/daf/db7/dad/df1/f95 0 2024-04-02T04:48:02.981 INFO:tasks.workunit.client.0.smithi070.stdout:8/757: chown d2/d8/d29/f3b 5 1 2024-04-02T04:48:02.981 INFO:tasks.workunit.client.1.smithi179.stdout:4/868: mkdir d1/d15/d127 0 2024-04-02T04:48:02.982 INFO:tasks.workunit.client.0.smithi070.stdout:2/851: link d2/d10/d5c/d7a/l6a d2/d10/d21/d3d/d42/l11c 0 2024-04-02T04:48:02.986 INFO:tasks.workunit.client.1.smithi179.stdout:5/809: dwrite d8/d1b/d23/d32/d4e/d51/f9c [0,4194304] 0 2024-04-02T04:48:02.987 INFO:tasks.workunit.client.1.smithi179.stdout:5/810: dread - d8/d1b/d23/d32/d4e/fb8 zero size 2024-04-02T04:48:02.989 INFO:tasks.workunit.client.0.smithi070.stdout:3/762: dwrite d10/d17/d51/fb0 [0,4194304] 0 2024-04-02T04:48:02.991 INFO:tasks.workunit.client.0.smithi070.stdout:3/763: write d10/f29 [1814002,15411] 0 2024-04-02T04:48:02.994 INFO:tasks.workunit.client.1.smithi179.stdout:9/820: creat d0/d1/d11/d29/d86/ffb x:0 0 0 2024-04-02T04:48:02.996 INFO:tasks.workunit.client.1.smithi179.stdout:9/821: read d0/d1/d10/d6d/d58/f75 [113772,69416] 0 2024-04-02T04:48:02.997 INFO:tasks.workunit.client.1.smithi179.stdout:9/822: readlink d0/lf9 0 2024-04-02T04:48:02.997 INFO:tasks.workunit.client.1.smithi179.stdout:9/823: readlink d0/d19/d27/d7a/d7e/dd4/le6 0 2024-04-02T04:48:02.998 INFO:tasks.workunit.client.1.smithi179.stdout:9/824: read d0/d1/d11/d29/d40/faa [39577,117483] 0 2024-04-02T04:48:02.999 INFO:tasks.workunit.client.1.smithi179.stdout:9/825: readlink d0/d1/d11/d29/d80/l88 0 2024-04-02T04:48:03.002 INFO:tasks.workunit.client.1.smithi179.stdout:4/869: write d1/d3/dc/d27/f97 [528081,110692] 0 2024-04-02T04:48:03.005 INFO:tasks.workunit.client.0.smithi070.stdout:1/727: dwrite d0/d9/d1d/d22/d2c/d4f/fd5 [0,4194304] 0 2024-04-02T04:48:03.005 INFO:tasks.workunit.client.0.smithi070.stdout:7/703: read d7/dd/df/d79/d46/f53 [733151,81467] 0 2024-04-02T04:48:03.007 INFO:tasks.workunit.client.1.smithi179.stdout:6/838: creat d3/d12/d44/d10a/d10e/d112/f12e x:0 0 0 2024-04-02T04:48:03.012 INFO:tasks.workunit.client.1.smithi179.stdout:5/811: chown d8/d1b/d23/f69 0 1 2024-04-02T04:48:03.015 INFO:tasks.workunit.client.1.smithi179.stdout:8/846: getdents d3/d32/d21/d22/d6b/db1 0 2024-04-02T04:48:03.016 INFO:tasks.workunit.client.0.smithi070.stdout:8/758: truncate d2/d3/f61 493668 0 2024-04-02T04:48:03.017 INFO:tasks.workunit.client.1.smithi179.stdout:1/844: symlink d3/d5/d2f/d4f/d42/d55/df3/l128 0 2024-04-02T04:48:03.021 INFO:tasks.workunit.client.1.smithi179.stdout:2/728: mknod dc/d29/d4a/d1e/d8a/dce/cf4 0 2024-04-02T04:48:03.022 INFO:tasks.workunit.client.0.smithi070.stdout:6/642: dwrite d0/d20/f27 [0,4194304] 0 2024-04-02T04:48:03.023 INFO:tasks.workunit.client.0.smithi070.stdout:0/814: mknod d4/d5/d13/d2a/cf9 0 2024-04-02T04:48:03.027 INFO:tasks.workunit.client.1.smithi179.stdout:4/870: dread - d1/d93/d9f/fea zero size 2024-04-02T04:48:03.029 INFO:tasks.workunit.client.1.smithi179.stdout:4/871: chown d1/d15/d6e/f110 1 1 2024-04-02T04:48:03.030 INFO:tasks.workunit.client.1.smithi179.stdout:3/867: truncate d8/dd3/d9a/fd1 3878721 0 2024-04-02T04:48:03.031 INFO:tasks.workunit.client.0.smithi070.stdout:4/747: dwrite d1/d3/d85/d8e/d99/fbc [0,4194304] 0 2024-04-02T04:48:03.032 INFO:tasks.workunit.client.0.smithi070.stdout:3/764: mknod d10/d32/d4b/dc5/de0/c102 0 2024-04-02T04:48:03.033 INFO:tasks.workunit.client.0.smithi070.stdout:3/765: chown d10/d32/d8a/d8d/l98 5 1 2024-04-02T04:48:03.033 INFO:tasks.workunit.client.0.smithi070.stdout:3/766: readlink l3 0 2024-04-02T04:48:03.035 INFO:tasks.workunit.client.0.smithi070.stdout:9/694: mkdir d4/d10/d11/d7b/d19/d2f/d4d/de2 0 2024-04-02T04:48:03.040 INFO:tasks.workunit.client.0.smithi070.stdout:2/852: dwrite d2/d10/d21/d3d/f47 [0,4194304] 0 2024-04-02T04:48:03.041 INFO:tasks.workunit.client.1.smithi179.stdout:0/926: creat d1/da/d48/d5c/f10e x:0 0 0 2024-04-02T04:48:03.042 INFO:tasks.workunit.client.0.smithi070.stdout:9/695: dread d4/d10/d11/d7b/d19/d82/fc4 [0,4194304] 0 2024-04-02T04:48:03.043 INFO:tasks.workunit.client.1.smithi179.stdout:8/847: rmdir d3/d32/d21/d22/d6b/db1/df8/d126/d96 39 2024-04-02T04:48:03.047 INFO:tasks.workunit.client.1.smithi179.stdout:1/845: write d3/d5/f6d [79188,2791] 0 2024-04-02T04:48:03.048 INFO:tasks.workunit.client.1.smithi179.stdout:1/846: dread - d3/d5/d96/f119 zero size 2024-04-02T04:48:03.049 INFO:tasks.workunit.client.1.smithi179.stdout:9/826: symlink d0/d19/d27/d7a/lfc 0 2024-04-02T04:48:03.050 INFO:tasks.workunit.client.1.smithi179.stdout:1/847: write d3/d5/de/fee [2074902,45833] 0 2024-04-02T04:48:03.051 INFO:tasks.workunit.client.1.smithi179.stdout:3/868: creat d8/d26/dcd/de7/f114 x:0 0 0 2024-04-02T04:48:03.054 INFO:tasks.workunit.client.0.smithi070.stdout:8/759: symlink d2/d8/d29/d22/dc8/lf9 0 2024-04-02T04:48:03.057 INFO:tasks.workunit.client.0.smithi070.stdout:5/915: truncate d5/d17/d50/f5a 7191688 0 2024-04-02T04:48:03.059 INFO:tasks.workunit.client.0.smithi070.stdout:6/643: creat d0/d3/d19/d31/d61/fc3 x:0 0 0 2024-04-02T04:48:03.062 INFO:tasks.workunit.client.1.smithi179.stdout:8/848: symlink d3/d32/d21/d22/d6b/d91/da8/dfe/l12d 0 2024-04-02T04:48:03.063 INFO:tasks.workunit.client.1.smithi179.stdout:8/849: chown d3/d32/d21/d50/d56/cb4 71 1 2024-04-02T04:48:03.066 INFO:tasks.workunit.client.0.smithi070.stdout:3/767: mkdir d10/d66/d103 0 2024-04-02T04:48:03.068 INFO:tasks.workunit.client.0.smithi070.stdout:3/768: dread - d10/d17/d51/d73/d9e/dd5/fd8 zero size 2024-04-02T04:48:03.068 INFO:tasks.workunit.client.0.smithi070.stdout:3/769: chown d10/d66/ccd 4 1 2024-04-02T04:48:03.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: Adjusting osd_memory_target on smithi070 to 1216M 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: pgmap v20: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 82 MiB/s rd, 198 MiB/s wr, 429 op/s 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:48:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:48:03.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:48:03.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:02 smithi179 ceph-mon[36603]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:48:03.073 INFO:tasks.workunit.client.0.smithi070.stdout:9/696: fsync d4/d10/d11/f3f 0 2024-04-02T04:48:03.073 INFO:tasks.workunit.client.1.smithi179.stdout:8/850: dwrite d3/d32/d21/d22/d6b/db1/fcb [0,4194304] 0 2024-04-02T04:48:03.076 INFO:tasks.workunit.client.0.smithi070.stdout:2/853: creat d2/d10/d5c/d7a/db0/dd8/dd2/f11d x:0 0 0 2024-04-02T04:48:03.077 INFO:tasks.workunit.client.0.smithi070.stdout:2/854: readlink d2/d10/d5c/d7a/db0/dc7/lf7 0 2024-04-02T04:48:03.078 INFO:tasks.workunit.client.0.smithi070.stdout:4/748: write d1/d3/d78/d91/fa7 [256916,91170] 0 2024-04-02T04:48:03.078 INFO:tasks.workunit.client.1.smithi179.stdout:4/872: mkdir d1/d3/d128 0 2024-04-02T04:48:03.079 INFO:tasks.workunit.client.0.smithi070.stdout:2/855: read - d2/d10/d5c/d7a/db0/dd8/dd2/ffa zero size 2024-04-02T04:48:03.079 INFO:tasks.workunit.client.1.smithi179.stdout:1/848: creat d3/d5/d2f/d4f/d6a/f129 x:0 0 0 2024-04-02T04:48:03.080 INFO:tasks.workunit.client.1.smithi179.stdout:1/849: write d3/f28 [3628541,122838] 0 2024-04-02T04:48:03.082 INFO:tasks.workunit.client.0.smithi070.stdout:7/704: mkdir d7/d9/de4 0 2024-04-02T04:48:03.083 INFO:tasks.workunit.client.0.smithi070.stdout:2/856: dread d2/d10/d21/d71/dd4/fc9 [0,4194304] 0 2024-04-02T04:48:03.083 INFO:tasks.workunit.client.0.smithi070.stdout:7/705: chown d7/d9/fdd 9940107 1 2024-04-02T04:48:03.084 INFO:tasks.workunit.client.0.smithi070.stdout:1/728: symlink d0/d9/d1d/d22/d2c/le6 0 2024-04-02T04:48:03.087 INFO:tasks.workunit.client.1.smithi179.stdout:5/812: rename d8/d1b/d23/d42/la7 to d8/d1b/d23/d32/dbe/daf/db7/de6/l107 0 2024-04-02T04:48:03.089 INFO:tasks.workunit.client.1.smithi179.stdout:0/927: fdatasync d1/da/d48/d65/d7d/d80/fa5 0 2024-04-02T04:48:03.090 INFO:tasks.workunit.client.0.smithi070.stdout:6/644: write d0/d55/d8b/d94/f6c [2950617,70362] 0 2024-04-02T04:48:03.097 INFO:tasks.workunit.client.0.smithi070.stdout:9/697: rename d4/d10/d11/d7b/d19/d53/f8b to d4/d10/d12/d56/d79/dd8/fe3 0 2024-04-02T04:48:03.103 INFO:tasks.workunit.client.1.smithi179.stdout:9/827: dwrite d0/d1/f7d [0,4194304] 0 2024-04-02T04:48:03.111 INFO:tasks.workunit.client.0.smithi070.stdout:8/760: link d2/d8/d29/d22/d5a/d94/da2/dde/ff4 d2/d3/d35/d3a/d4f/de8/ffa 0 2024-04-02T04:48:03.117 INFO:tasks.workunit.client.0.smithi070.stdout:6/645: unlink d0/d43/d95/c9c 0 2024-04-02T04:48:03.118 INFO:tasks.workunit.client.0.smithi070.stdout:6/646: truncate d0/f26 2477713 0 2024-04-02T04:48:03.121 INFO:tasks.workunit.client.0.smithi070.stdout:2/857: write d2/d10/d5c/d7a/fe1 [784735,23695] 0 2024-04-02T04:48:03.124 INFO:tasks.workunit.client.0.smithi070.stdout:2/858: dread - d2/d10/d21/d71/dd4/d100/f114 zero size 2024-04-02T04:48:03.125 INFO:tasks.workunit.client.0.smithi070.stdout:9/698: read d4/d10/d12/d56/d79/dd8/fe3 [3938535,123249] 0 2024-04-02T04:48:03.128 INFO:tasks.workunit.client.0.smithi070.stdout:0/815: link d4/d5/d13/d2a/d3a/c9e d4/d5/d13/cfa 0 2024-04-02T04:48:03.128 INFO:tasks.workunit.client.0.smithi070.stdout:9/699: fsync d4/d10/d12/d56/f5b 0 2024-04-02T04:48:03.130 INFO:tasks.workunit.client.0.smithi070.stdout:6/647: creat d0/d20/d5f/fc4 x:0 0 0 2024-04-02T04:48:03.130 INFO:tasks.workunit.client.0.smithi070.stdout:7/706: dwrite d7/d36/fae [0,4194304] 0 2024-04-02T04:48:03.132 INFO:tasks.workunit.client.0.smithi070.stdout:6/648: write d0/f26 [119850,76751] 0 2024-04-02T04:48:03.138 INFO:tasks.workunit.client.0.smithi070.stdout:5/916: getdents d5/d9/d10/db9/dba 0 2024-04-02T04:48:03.139 INFO:tasks.workunit.client.0.smithi070.stdout:5/917: stat d5/d17/d2b/d7b/l99 0 2024-04-02T04:48:03.141 INFO:tasks.workunit.client.0.smithi070.stdout:5/918: truncate d5/d9/d23/d30/da2/fdd 25844 0 2024-04-02T04:48:03.146 INFO:tasks.workunit.client.0.smithi070.stdout:7/707: chown d7/c43 2234 1 2024-04-02T04:48:03.149 INFO:tasks.workunit.client.0.smithi070.stdout:4/749: link d1/d3/c74 d1/d3/d78/dad/dbd/cfd 0 2024-04-02T04:48:03.152 INFO:tasks.workunit.client.0.smithi070.stdout:1/729: getdents d0/d9/d1d/d22/d56/d79 0 2024-04-02T04:48:03.153 INFO:tasks.workunit.client.1.smithi179.stdout:7/859: sync 2024-04-02T04:48:03.153 INFO:tasks.workunit.client.1.smithi179.stdout:3/869: unlink d8/dd3/d9a/ccb 0 2024-04-02T04:48:03.155 INFO:tasks.workunit.client.1.smithi179.stdout:6/839: getdents d3/d2c/d85 0 2024-04-02T04:48:03.156 INFO:tasks.workunit.client.0.smithi070.stdout:0/816: creat d4/d5/da/dd1/ffb x:0 0 0 2024-04-02T04:48:03.161 INFO:tasks.workunit.client.0.smithi070.stdout:7/708: mkdir d7/dd/df/d7a/d6c/d77/d54/de5 0 2024-04-02T04:48:03.163 INFO:tasks.workunit.client.1.smithi179.stdout:0/928: symlink d1/da/d52/d75/l10f 0 2024-04-02T04:48:03.164 INFO:tasks.workunit.client.0.smithi070.stdout:5/919: dread d5/d9/d23/f6e [0,4194304] 0 2024-04-02T04:48:03.165 INFO:tasks.workunit.client.0.smithi070.stdout:9/700: creat d4/d10/d12/d56/d79/dd8/ddf/fe4 x:0 0 0 2024-04-02T04:48:03.165 INFO:tasks.workunit.client.0.smithi070.stdout:7/709: dread d7/d36/fae [0,4194304] 0 2024-04-02T04:48:03.167 INFO:tasks.workunit.client.0.smithi070.stdout:7/710: read d7/dd/df/d79/fbb [1783565,31267] 0 2024-04-02T04:48:03.167 INFO:tasks.workunit.client.0.smithi070.stdout:4/750: mknod d1/d3/d78/cfe 0 2024-04-02T04:48:03.168 INFO:tasks.workunit.client.0.smithi070.stdout:4/751: write d1/d3/d85/fd3 [959161,17068] 0 2024-04-02T04:48:03.169 INFO:tasks.workunit.client.0.smithi070.stdout:6/649: creat d0/d3/dc2/fc5 x:0 0 0 2024-04-02T04:48:03.170 INFO:tasks.workunit.client.1.smithi179.stdout:8/851: mkdir d3/d32/d21/d22/d6b/db1/df8/d126/d12e 0 2024-04-02T04:48:03.172 INFO:tasks.workunit.client.0.smithi070.stdout:6/650: write d0/d55/d8b/d94/f6c [2780963,45846] 0 2024-04-02T04:48:03.176 INFO:tasks.workunit.client.0.smithi070.stdout:0/817: mkdir d4/d5/d13/d2a/d8d/dfc 0 2024-04-02T04:48:03.176 INFO:tasks.workunit.client.0.smithi070.stdout:7/711: dwrite d7/d36/fae [4194304,4194304] 0 2024-04-02T04:48:03.179 INFO:tasks.workunit.client.0.smithi070.stdout:6/651: dread d0/d3/d19/d31/fb1 [0,4194304] 0 2024-04-02T04:48:03.180 INFO:tasks.workunit.client.0.smithi070.stdout:5/920: write d5/d9/fee [938652,10752] 0 2024-04-02T04:48:03.182 INFO:tasks.workunit.client.1.smithi179.stdout:7/860: creat d1/d9/d27/d87/d8b/d75/db9/d8d/f12f x:0 0 0 2024-04-02T04:48:03.183 INFO:tasks.workunit.client.0.smithi070.stdout:2/859: truncate d2/d10/d5c/d7a/db0/dd8/dd2/fe6 2444866 0 2024-04-02T04:48:03.184 INFO:tasks.workunit.client.0.smithi070.stdout:9/701: chown d4/d10/d11/d7b/d19/cd7 13996 1 2024-04-02T04:48:03.184 INFO:tasks.workunit.client.1.smithi179.stdout:7/861: chown d1/d9/d27/d87/d8b/d75/db9/dcf/f102 15 1 2024-04-02T04:48:03.186 INFO:tasks.workunit.client.0.smithi070.stdout:9/702: write d4/d10/d11/d7b/fa2 [956801,91083] 0 2024-04-02T04:48:03.187 INFO:tasks.workunit.client.1.smithi179.stdout:5/813: fsync d8/d1b/d23/f2b 0 2024-04-02T04:48:03.189 INFO:tasks.workunit.client.0.smithi070.stdout:9/703: write d4/d10/d11/d7b/d19/d2f/d8c/fd0 [93162,67504] 0 2024-04-02T04:48:03.191 INFO:tasks.workunit.client.0.smithi070.stdout:2/860: dwrite d2/d10/f102 [0,4194304] 0 2024-04-02T04:48:03.192 INFO:tasks.workunit.client.0.smithi070.stdout:1/730: truncate d0/d7/d1a/f9a 543886 0 2024-04-02T04:48:03.200 INFO:tasks.workunit.client.1.smithi179.stdout:4/873: dwrite d1/d3/df/d10/d2f/d60/fff [0,4194304] 0 2024-04-02T04:48:03.206 INFO:tasks.workunit.client.1.smithi179.stdout:1/850: creat d3/f12a x:0 0 0 2024-04-02T04:48:03.209 INFO:tasks.workunit.client.1.smithi179.stdout:6/840: dread d3/d12/d23/d39/f71 [0,4194304] 0 2024-04-02T04:48:03.214 INFO:tasks.workunit.client.1.smithi179.stdout:5/814: rmdir d8/d1b/d23/d36/dcb 39 2024-04-02T04:48:03.215 INFO:tasks.workunit.client.1.smithi179.stdout:2/729: sync 2024-04-02T04:48:03.216 INFO:tasks.workunit.client.1.smithi179.stdout:0/929: unlink d1/da/d48/c4e 0 2024-04-02T04:48:03.217 INFO:tasks.workunit.client.0.smithi070.stdout:9/704: creat d4/d10/d12/d55/d66/fe5 x:0 0 0 2024-04-02T04:48:03.218 INFO:tasks.workunit.client.0.smithi070.stdout:9/705: dread - d4/d10/d12/fb3 zero size 2024-04-02T04:48:03.218 INFO:tasks.workunit.client.1.smithi179.stdout:3/870: write d8/d26/d83/d89/d9c/fb0 [2000508,75465] 0 2024-04-02T04:48:03.219 INFO:tasks.workunit.client.1.smithi179.stdout:0/930: read d1/f30 [1286481,110141] 0 2024-04-02T04:48:03.220 INFO:tasks.workunit.client.1.smithi179.stdout:0/931: chown d1/fc1 9614945 1 2024-04-02T04:48:03.220 INFO:tasks.workunit.client.0.smithi070.stdout:2/861: creat d2/d10/d21/d71/dd4/dce/f11e x:0 0 0 2024-04-02T04:48:03.220 INFO:tasks.workunit.client.0.smithi070.stdout:5/921: write d5/d17/d50/f5f [4446744,41796] 0 2024-04-02T04:48:03.221 INFO:tasks.workunit.client.1.smithi179.stdout:9/828: link d0/d1/d11/d3a/f81 d0/d19/d27/d7a/d7e/db9/ffd 0 2024-04-02T04:48:03.222 INFO:tasks.workunit.client.0.smithi070.stdout:5/922: dread - d5/d9/d10/d6a/f10e zero size 2024-04-02T04:48:03.225 INFO:tasks.workunit.client.0.smithi070.stdout:4/752: truncate d1/d9c/f45 3509932 0 2024-04-02T04:48:03.228 INFO:tasks.workunit.client.0.smithi070.stdout:2/862: dwrite d2/d10/d53/f55 [0,4194304] 0 2024-04-02T04:48:03.229 INFO:tasks.workunit.client.0.smithi070.stdout:7/712: mkdir d7/dd/df/d79/de6 0 2024-04-02T04:48:03.232 INFO:tasks.workunit.client.1.smithi179.stdout:6/841: creat d3/d2d/d38/d64/f12f x:0 0 0 2024-04-02T04:48:03.234 INFO:tasks.workunit.client.0.smithi070.stdout:6/652: dread d0/d3/d19/d31/d46/d6b/fba [0,4194304] 0 2024-04-02T04:48:03.234 INFO:tasks.workunit.client.1.smithi179.stdout:7/862: symlink d1/d9/d55/dbf/d9f/da8/l130 0 2024-04-02T04:48:03.236 INFO:tasks.workunit.client.0.smithi070.stdout:9/706: unlink d4/d10/d11/da8/fce 0 2024-04-02T04:48:03.238 INFO:tasks.workunit.client.0.smithi070.stdout:9/707: write d4/d10/d12/d56/f5b [1022682,29719] 0 2024-04-02T04:48:03.240 INFO:tasks.workunit.client.1.smithi179.stdout:5/815: unlink d8/d1b/d23/d36/d53/d62/f75 0 2024-04-02T04:48:03.242 INFO:tasks.workunit.client.0.smithi070.stdout:0/818: creat d4/d5/d21/ffd x:0 0 0 2024-04-02T04:48:03.245 INFO:tasks.workunit.client.1.smithi179.stdout:3/871: unlink d8/fac 0 2024-04-02T04:48:03.249 INFO:tasks.workunit.client.0.smithi070.stdout:4/753: rename d1/d3/d78/dad/dbd/fea to d1/d3/d85/d8e/d99/db1/dbb/fff 0 2024-04-02T04:48:03.252 INFO:tasks.workunit.client.1.smithi179.stdout:8/852: link d3/d31/dc3/lf7 d3/d32/d21/d50/d56/d94/d84/l12f 0 2024-04-02T04:48:03.253 INFO:tasks.workunit.client.1.smithi179.stdout:8/853: chown d3/d32/d21/d50/d56/f6a 401 1 2024-04-02T04:48:03.254 INFO:tasks.workunit.client.0.smithi070.stdout:6/653: mknod d0/d3/d19/d31/d61/d7b/da6/cc6 0 2024-04-02T04:48:03.255 INFO:tasks.workunit.client.1.smithi179.stdout:4/874: creat d1/d3/dc/d26/d8a/d5d/f129 x:0 0 0 2024-04-02T04:48:03.255 INFO:tasks.workunit.client.1.smithi179.stdout:8/854: read d3/d1b/d2a/d83/fec [220656,113366] 0 2024-04-02T04:48:03.256 INFO:tasks.workunit.client.0.smithi070.stdout:9/708: fsync d4/d10/d12/d56/d79/dd8/fe3 0 2024-04-02T04:48:03.257 INFO:tasks.workunit.client.1.smithi179.stdout:1/851: fdatasync d3/d5/f4a 0 2024-04-02T04:48:03.258 INFO:tasks.workunit.client.0.smithi070.stdout:6/654: dread d0/d55/d8b/d94/f53 [0,4194304] 0 2024-04-02T04:48:03.276 INFO:tasks.workunit.client.0.smithi070.stdout:0/819: rmdir d4/d5/d58/d60/d83 39 2024-04-02T04:48:03.279 INFO:tasks.workunit.client.0.smithi070.stdout:7/713: symlink d7/d8e/le7 0 2024-04-02T04:48:03.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:03.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: Adjusting osd_memory_target on smithi070 to 1216M 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: pgmap v20: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 82 MiB/s rd, 198 MiB/s wr, 429 op/s 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:48:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:48:03.281 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:48:03.281 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:48:03.281 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:02 smithi070 ceph-mon[29212]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:48:03.281 INFO:tasks.workunit.client.0.smithi070.stdout:7/714: truncate d7/d36/fb9 4195694 0 2024-04-02T04:48:03.283 INFO:tasks.workunit.client.1.smithi179.stdout:9/829: write d0/d19/d1c/f6b [1686720,117867] 0 2024-04-02T04:48:03.283 INFO:tasks.workunit.client.1.smithi179.stdout:9/830: readlink d0/d19/d27/d7a/lbe 0 2024-04-02T04:48:03.285 INFO:tasks.workunit.client.0.smithi070.stdout:4/754: fdatasync d1/d9c/d65/d7a/fa9 0 2024-04-02T04:48:03.285 INFO:tasks.workunit.client.0.smithi070.stdout:4/755: dread - d1/dc3/ff6 zero size 2024-04-02T04:48:03.286 INFO:tasks.workunit.client.1.smithi179.stdout:5/816: write d8/d1b/d23/d32/f3d [1598129,127534] 0 2024-04-02T04:48:03.290 INFO:tasks.workunit.client.1.smithi179.stdout:2/730: rename dc/d29/d4a/l28 to dc/d29/d31/d83/d8d/lf5 0 2024-04-02T04:48:03.294 INFO:tasks.workunit.client.1.smithi179.stdout:3/872: symlink d8/d26/dcd/l115 0 2024-04-02T04:48:03.298 INFO:tasks.workunit.client.0.smithi070.stdout:6/655: rename d0/d3/d19/d31/d61 to d0/d43/db3/dc7 0 2024-04-02T04:48:03.298 INFO:tasks.workunit.client.0.smithi070.stdout:6/656: read - d0/d43/fae zero size 2024-04-02T04:48:03.299 INFO:tasks.workunit.client.1.smithi179.stdout:7/863: dwrite d1/d9/d27/d87/d8b/d75/db9/fe6 [0,4194304] 0 2024-04-02T04:48:03.299 INFO:tasks.workunit.client.1.smithi179.stdout:7/864: stat d1/d9/d55/f9e 0 2024-04-02T04:48:03.299 INFO:tasks.workunit.client.0.smithi070.stdout:6/657: fdatasync d0/d43/d95/f9a 0 2024-04-02T04:48:03.300 INFO:tasks.workunit.client.0.smithi070.stdout:0/820: stat d4/d5/d21/l5f 0 2024-04-02T04:48:03.302 INFO:tasks.workunit.client.1.smithi179.stdout:4/875: creat d1/d15/d6e/f12a x:0 0 0 2024-04-02T04:48:03.304 INFO:tasks.workunit.client.1.smithi179.stdout:1/852: creat d3/d5/d96/f12b x:0 0 0 2024-04-02T04:48:03.305 INFO:tasks.workunit.client.1.smithi179.stdout:1/853: stat d3/d5/d2f/d5a 0 2024-04-02T04:48:03.308 INFO:tasks.workunit.client.0.smithi070.stdout:6/658: dwrite d0/d55/d8b/d94/fb9 [0,4194304] 0 2024-04-02T04:48:03.309 INFO:tasks.workunit.client.1.smithi179.stdout:9/831: creat d0/d1/d11/dab/ded/ffe x:0 0 0 2024-04-02T04:48:03.310 INFO:tasks.workunit.client.0.smithi070.stdout:6/659: write d0/d55/d8b/d94/fbd [292099,37483] 0 2024-04-02T04:48:03.311 INFO:tasks.workunit.client.0.smithi070.stdout:9/709: creat d4/d34/db9/de1/fe6 x:0 0 0 2024-04-02T04:48:03.312 INFO:tasks.workunit.client.1.smithi179.stdout:5/817: fsync d8/d1b/d23/f2b 0 2024-04-02T04:48:03.315 INFO:tasks.workunit.client.0.smithi070.stdout:5/923: getdents d5 0 2024-04-02T04:48:03.316 INFO:tasks.workunit.client.0.smithi070.stdout:9/710: truncate d4/d10/d11/d7b/d19/d53/d9c/fdb 326066 0 2024-04-02T04:48:03.316 INFO:tasks.workunit.client.1.smithi179.stdout:8/855: rename d3/d32/d21/d22/f2d to d3/d32/d21/d51/d70/f130 0 2024-04-02T04:48:03.318 INFO:tasks.workunit.client.1.smithi179.stdout:2/731: mknod dc/d29/d4a/d4b/cf6 0 2024-04-02T04:48:03.319 INFO:tasks.workunit.client.1.smithi179.stdout:2/732: rename dc to dc/d29/d4a/d1e/d43/df7 22 2024-04-02T04:48:03.320 INFO:tasks.workunit.client.1.smithi179.stdout:3/873: mknod d8/d57/d63/dc3/c116 0 2024-04-02T04:48:03.322 INFO:tasks.workunit.client.0.smithi070.stdout:0/821: mknod d4/d5/d13/d2a/d3a/d80/da8/cfe 0 2024-04-02T04:48:03.322 INFO:tasks.workunit.client.0.smithi070.stdout:4/756: write d1/d3/d9/f9e [4188939,23496] 0 2024-04-02T04:48:03.323 INFO:tasks.workunit.client.0.smithi070.stdout:4/757: chown d1/f2d 166362164 1 2024-04-02T04:48:03.325 INFO:tasks.workunit.client.0.smithi070.stdout:7/715: symlink d7/dd/df/d79/le8 0 2024-04-02T04:48:03.326 INFO:tasks.workunit.client.1.smithi179.stdout:9/832: dread d0/d1/d11/d29/f50 [0,4194304] 0 2024-04-02T04:48:03.328 INFO:tasks.workunit.client.1.smithi179.stdout:9/833: read d0/d19/f56 [1893051,29883] 0 2024-04-02T04:48:03.329 INFO:tasks.workunit.client.1.smithi179.stdout:7/865: creat d1/dd/d16/d4d/f131 x:0 0 0 2024-04-02T04:48:03.329 INFO:tasks.workunit.client.1.smithi179.stdout:0/932: sync 2024-04-02T04:48:03.337 INFO:tasks.workunit.client.1.smithi179.stdout:1/854: rmdir d3/d5/d2f/d4f/d42/d55/df3 39 2024-04-02T04:48:03.341 INFO:tasks.workunit.client.0.smithi070.stdout:4/758: dread d1/d3/d85/ddb/ffc [0,4194304] 0 2024-04-02T04:48:03.341 INFO:tasks.workunit.client.0.smithi070.stdout:7/716: symlink d7/d4e/d82/d57/dce/le9 0 2024-04-02T04:48:03.342 INFO:tasks.workunit.client.1.smithi179.stdout:0/933: dread d1/da/df/f8a [0,4194304] 0 2024-04-02T04:48:03.343 INFO:tasks.workunit.client.0.smithi070.stdout:6/660: write d0/d3/d19/d31/d46/fa9 [710186,35231] 0 2024-04-02T04:48:03.344 INFO:tasks.workunit.client.0.smithi070.stdout:5/924: symlink d5/d17/da7/d118/l121 0 2024-04-02T04:48:03.345 INFO:tasks.workunit.client.0.smithi070.stdout:6/661: truncate d0/d43/f9f 343873 0 2024-04-02T04:48:03.346 INFO:tasks.workunit.client.0.smithi070.stdout:9/711: mknod d4/d10/d11/d7b/d19/d53/d9c/dd3/ce7 0 2024-04-02T04:48:03.346 INFO:tasks.workunit.client.0.smithi070.stdout:6/662: chown d0/d43/c44 9 1 2024-04-02T04:48:03.347 INFO:tasks.workunit.client.1.smithi179.stdout:2/733: symlink dc/d29/d31/d66/lf8 0 2024-04-02T04:48:03.348 INFO:tasks.workunit.client.0.smithi070.stdout:4/759: dread d1/d9c/f79 [0,4194304] 0 2024-04-02T04:48:03.349 INFO:tasks.workunit.client.0.smithi070.stdout:0/822: truncate d4/d5/d58/d60/d83/f94 141333 0 2024-04-02T04:48:03.349 INFO:tasks.workunit.client.1.smithi179.stdout:3/874: mkdir d8/d26/d36/d117 0 2024-04-02T04:48:03.357 INFO:tasks.workunit.client.0.smithi070.stdout:0/823: dwrite d4/d5/d58/d60/f92 [0,4194304] 0 2024-04-02T04:48:03.359 INFO:tasks.workunit.client.1.smithi179.stdout:4/876: mkdir d1/d3/d128/d12b 0 2024-04-02T04:48:03.361 INFO:tasks.workunit.client.0.smithi070.stdout:6/663: rename d0/d3/dc2/fc5 to d0/d3/d19/d31/dc1/fc8 0 2024-04-02T04:48:03.362 INFO:tasks.workunit.client.1.smithi179.stdout:6/842: getdents d3/d2d/da5 0 2024-04-02T04:48:03.365 INFO:tasks.workunit.client.1.smithi179.stdout:1/855: creat d3/d5/de/d33/f12c x:0 0 0 2024-04-02T04:48:03.372 INFO:tasks.workunit.client.0.smithi070.stdout:9/712: dread d4/d10/d12/d55/f7e [4194304,4194304] 0 2024-04-02T04:48:03.372 INFO:tasks.workunit.client.0.smithi070.stdout:5/925: mknod d5/d9/d23/c122 0 2024-04-02T04:48:03.372 INFO:tasks.workunit.client.0.smithi070.stdout:4/760: write d1/d3/f4c [4499948,120290] 0 2024-04-02T04:48:03.373 INFO:tasks.workunit.client.1.smithi179.stdout:8/856: fdatasync d3/f4 0 2024-04-02T04:48:03.374 INFO:tasks.workunit.client.1.smithi179.stdout:2/734: fsync dc/d29/d4a/d1e/dbe/fd8 0 2024-04-02T04:48:03.375 INFO:tasks.workunit.client.1.smithi179.stdout:2/735: readlink dc/d29/d31/l8e 0 2024-04-02T04:48:03.378 INFO:tasks.workunit.client.1.smithi179.stdout:9/834: mkdir d0/d1/d10/de3/dff 0 2024-04-02T04:48:03.378 INFO:tasks.workunit.client.0.smithi070.stdout:7/717: dwrite d7/dd/df/d79/d46/d65/f71 [0,4194304] 0 2024-04-02T04:48:03.381 INFO:tasks.workunit.client.0.smithi070.stdout:7/718: chown d7/dd/df/d7a/d6c/d77/d75/c81 27298 1 2024-04-02T04:48:03.382 INFO:tasks.workunit.client.0.smithi070.stdout:5/926: dread d5/d17/d27/fb7 [0,4194304] 0 2024-04-02T04:48:03.383 INFO:tasks.workunit.client.0.smithi070.stdout:6/664: creat d0/d43/d95/fc9 x:0 0 0 2024-04-02T04:48:03.386 INFO:tasks.workunit.client.0.smithi070.stdout:6/665: write d0/d20/d5f/fc4 [394674,128936] 0 2024-04-02T04:48:03.388 INFO:tasks.workunit.client.0.smithi070.stdout:6/666: truncate d0/d55/d8b/faa 1605558 0 2024-04-02T04:48:03.389 INFO:tasks.workunit.client.0.smithi070.stdout:6/667: read d0/d43/db3/dc7/f66 [3057974,77078] 0 2024-04-02T04:48:03.390 INFO:tasks.workunit.client.1.smithi179.stdout:6/843: symlink d3/d12/d44/d10a/d10e/d112/l130 0 2024-04-02T04:48:03.391 INFO:tasks.workunit.client.0.smithi070.stdout:0/824: write d4/d3c/fa2 [605737,110168] 0 2024-04-02T04:48:03.392 INFO:tasks.workunit.client.0.smithi070.stdout:4/761: symlink d1/d9c/d65/d7a/l100 0 2024-04-02T04:48:03.399 INFO:tasks.workunit.client.1.smithi179.stdout:8/857: dread d3/d1b/d2a/f4c [0,4194304] 0 2024-04-02T04:48:03.399 INFO:tasks.workunit.client.0.smithi070.stdout:5/927: mkdir d5/d9/d23/d30/da2/d75/d123 0 2024-04-02T04:48:03.400 INFO:tasks.workunit.client.1.smithi179.stdout:9/835: creat d0/d1/d11/dab/ded/f100 x:0 0 0 2024-04-02T04:48:03.401 INFO:tasks.workunit.client.0.smithi070.stdout:6/668: mknod d0/d3/dc2/cca 0 2024-04-02T04:48:03.402 INFO:tasks.workunit.client.0.smithi070.stdout:7/719: dwrite d7/d9/f27 [0,4194304] 0 2024-04-02T04:48:03.403 INFO:tasks.workunit.client.0.smithi070.stdout:6/669: chown d0/d20/c87 77810228 1 2024-04-02T04:48:03.403 INFO:tasks.workunit.client.1.smithi179.stdout:7/866: rename l0 to d1/d9/d27/d87/d8b/d75/db9/l132 0 2024-04-02T04:48:03.404 INFO:tasks.workunit.client.0.smithi070.stdout:6/670: chown d0/d20/d77/l81 260183526 1 2024-04-02T04:48:03.404 INFO:tasks.workunit.client.1.smithi179.stdout:7/867: chown d1/d9/d27/d87/d8b/d75/db9/dcf 183836 1 2024-04-02T04:48:03.405 INFO:tasks.workunit.client.1.smithi179.stdout:7/868: read d1/dd/d76/f86 [632391,52762] 0 2024-04-02T04:48:03.408 INFO:tasks.workunit.client.0.smithi070.stdout:4/762: unlink d1/dc3/cf2 0 2024-04-02T04:48:03.410 INFO:tasks.workunit.client.1.smithi179.stdout:5/818: getdents d8/d1b/d23/d32/dbe/daf/db7/dad 0 2024-04-02T04:48:03.413 INFO:tasks.workunit.client.1.smithi179.stdout:8/858: symlink d3/d31/d8f/l131 0 2024-04-02T04:48:03.415 INFO:tasks.workunit.client.1.smithi179.stdout:9/836: rmdir d0/d1/d11/d29 39 2024-04-02T04:48:03.417 INFO:tasks.workunit.client.0.smithi070.stdout:0/825: unlink d4/d5/d58/d60/d83/f94 0 2024-04-02T04:48:03.424 INFO:tasks.workunit.client.1.smithi179.stdout:6/844: write d3/d12/d23/d26/f37 [2768920,7134] 0 2024-04-02T04:48:03.424 INFO:tasks.workunit.client.0.smithi070.stdout:9/713: truncate d4/d10/d12/d55/d66/f6d 454184 0 2024-04-02T04:48:03.426 INFO:tasks.workunit.client.1.smithi179.stdout:1/856: creat d3/d5/d2f/d4f/d42/d53/f12d x:0 0 0 2024-04-02T04:48:03.427 INFO:tasks.workunit.client.0.smithi070.stdout:9/714: dread d4/d10/d11/d7b/d19/d53/d9c/fdb [0,4194304] 0 2024-04-02T04:48:03.428 INFO:tasks.workunit.client.0.smithi070.stdout:7/720: write d7/d36/f62 [4662281,39458] 0 2024-04-02T04:48:03.429 INFO:tasks.workunit.client.0.smithi070.stdout:9/715: read - d4/d10/d11/d7b/d19/fc2 zero size 2024-04-02T04:48:03.429 INFO:tasks.workunit.client.0.smithi070.stdout:7/721: write d7/d36/f62 [3886922,95593] 0 2024-04-02T04:48:03.431 INFO:tasks.workunit.client.1.smithi179.stdout:8/859: mkdir d3/d32/d30/d132 0 2024-04-02T04:48:03.431 INFO:tasks.workunit.client.0.smithi070.stdout:6/671: write d0/d43/f51 [5344627,18555] 0 2024-04-02T04:48:03.435 INFO:tasks.workunit.client.1.smithi179.stdout:4/877: getdents d1/d3/dc/d26 0 2024-04-02T04:48:03.436 INFO:tasks.workunit.client.0.smithi070.stdout:0/826: symlink d4/d5/d13/d2a/d8d/dfc/lff 0 2024-04-02T04:48:03.436 INFO:tasks.workunit.client.0.smithi070.stdout:0/827: readlink d4/l28 0 2024-04-02T04:48:03.438 INFO:tasks.workunit.client.1.smithi179.stdout:6/845: dread d3/d12/f62 [0,4194304] 0 2024-04-02T04:48:03.440 INFO:tasks.workunit.client.0.smithi070.stdout:9/716: fsync d4/fc 0 2024-04-02T04:48:03.442 INFO:tasks.workunit.client.1.smithi179.stdout:1/857: chown d3/d5/d2f/d65/f92 32 1 2024-04-02T04:48:03.443 INFO:tasks.workunit.client.1.smithi179.stdout:9/837: dwrite d0/d19/d27/d7a/d7e/db9/dfa/f66 [0,4194304] 0 2024-04-02T04:48:03.444 INFO:tasks.workunit.client.0.smithi070.stdout:4/763: creat d1/d3/d9/f101 x:0 0 0 2024-04-02T04:48:03.444 INFO:tasks.workunit.client.1.smithi179.stdout:1/858: truncate d3/d5/f8d 1920925 0 2024-04-02T04:48:03.444 INFO:tasks.workunit.client.0.smithi070.stdout:4/764: fsync d1/d3/fd2 0 2024-04-02T04:48:03.445 INFO:tasks.workunit.client.1.smithi179.stdout:8/860: fdatasync d3/d1b/d2a/f3e 0 2024-04-02T04:48:03.445 INFO:tasks.workunit.client.0.smithi070.stdout:4/765: chown d1/d3/d9/d5e/f77 43 1 2024-04-02T04:48:03.446 INFO:tasks.workunit.client.0.smithi070.stdout:6/672: creat d0/d3/d19/d31/d37/fcb x:0 0 0 2024-04-02T04:48:03.446 INFO:tasks.workunit.client.1.smithi179.stdout:8/861: write d3/d32/d21/d22/d6b/db1/fcb [712987,120581] 0 2024-04-02T04:48:03.447 INFO:tasks.workunit.client.1.smithi179.stdout:0/934: sync 2024-04-02T04:48:03.448 INFO:tasks.workunit.client.0.smithi070.stdout:5/928: getdents d5/d9/d10 0 2024-04-02T04:48:03.450 INFO:tasks.workunit.client.1.smithi179.stdout:7/869: rename d1/d9/l26 to d1/d9/d27/d87/d8b/d75/db9/l133 0 2024-04-02T04:48:03.452 INFO:tasks.workunit.client.1.smithi179.stdout:8/862: dwrite d3/d32/d21/d50/d65/faa [0,4194304] 0 2024-04-02T04:48:03.453 INFO:tasks.workunit.client.1.smithi179.stdout:6/846: symlink d3/d12/d44/d10a/l131 0 2024-04-02T04:48:03.455 INFO:tasks.workunit.client.0.smithi070.stdout:9/717: creat d4/d34/db9/fe8 x:0 0 0 2024-04-02T04:48:03.458 INFO:tasks.workunit.client.1.smithi179.stdout:1/859: creat d3/d5/de/d33/f12e x:0 0 0 2024-04-02T04:48:03.462 INFO:tasks.workunit.client.1.smithi179.stdout:6/847: dwrite d3/d12/d23/d2f/f3c [0,4194304] 0 2024-04-02T04:48:03.463 INFO:tasks.workunit.client.0.smithi070.stdout:4/766: creat d1/d9c/f102 x:0 0 0 2024-04-02T04:48:03.465 INFO:tasks.workunit.client.0.smithi070.stdout:0/828: dwrite d4/d5/d13/d2a/d29/d52/f69 [0,4194304] 0 2024-04-02T04:48:03.469 INFO:tasks.workunit.client.1.smithi179.stdout:9/838: rename d0/d19/d27/d7a/dba to d0/d1/d11/dab/ded/d101 0 2024-04-02T04:48:03.474 INFO:tasks.workunit.client.0.smithi070.stdout:9/718: dread d4/f9f [0,4194304] 0 2024-04-02T04:48:03.477 INFO:tasks.workunit.client.1.smithi179.stdout:5/819: getdents d8/d1b/d23/d32 0 2024-04-02T04:48:03.478 INFO:tasks.workunit.client.1.smithi179.stdout:5/820: write d8/d1b/d23/d32/dbe/daf/dd4/d105/fd8 [486622,47964] 0 2024-04-02T04:48:03.479 INFO:tasks.workunit.client.1.smithi179.stdout:3/875: sync 2024-04-02T04:48:03.480 INFO:tasks.workunit.client.0.smithi070.stdout:6/673: symlink d0/d20/d77/d98/lcc 0 2024-04-02T04:48:03.481 INFO:tasks.workunit.client.1.smithi179.stdout:4/878: rmdir d1/d3/df/d10/d2f/d60/da6/df8 0 2024-04-02T04:48:03.484 INFO:tasks.workunit.client.0.smithi070.stdout:0/829: chown d4/d5/c91 564 1 2024-04-02T04:48:03.487 INFO:tasks.workunit.client.1.smithi179.stdout:0/935: rename d1/da/d28/d34/d9c/lf4 to d1/da/d48/d5c/d83/l110 0 2024-04-02T04:48:03.487 INFO:tasks.workunit.client.0.smithi070.stdout:5/929: getdents d5/d9/d23/d30/da2/d75/d123 0 2024-04-02T04:48:03.488 INFO:tasks.workunit.client.0.smithi070.stdout:5/930: chown d5/d17/d6c/lb8 0 1 2024-04-02T04:48:03.489 INFO:tasks.workunit.client.0.smithi070.stdout:7/722: link d7/dd/df/d79/d46/ld3 d7/d8e/lea 0 2024-04-02T04:48:03.493 INFO:tasks.workunit.client.0.smithi070.stdout:9/719: symlink d4/d10/d11/d7b/d19/d2f/d39/d57/da6/le9 0 2024-04-02T04:48:03.494 INFO:tasks.workunit.client.0.smithi070.stdout:9/720: chown d4/d10/d11/d7b/d19/d53/d9c/dd3 2337 1 2024-04-02T04:48:03.495 INFO:tasks.workunit.client.0.smithi070.stdout:6/674: creat d0/d3/d19/d31/d37/fcd x:0 0 0 2024-04-02T04:48:03.496 INFO:tasks.workunit.client.1.smithi179.stdout:5/821: mkdir d8/d1b/d23/d32/d46/d108 0 2024-04-02T04:48:03.497 INFO:tasks.workunit.client.1.smithi179.stdout:4/879: mknod d1/d3/dc/d27/c12c 0 2024-04-02T04:48:03.500 INFO:tasks.workunit.client.1.smithi179.stdout:1/860: rename d3/d5/d2f/d5a/dcc to d3/d5/d2f/d65/d12f 0 2024-04-02T04:48:03.500 INFO:tasks.workunit.client.0.smithi070.stdout:4/767: dwrite d1/d9c/d65/d7a/fa9 [0,4194304] 0 2024-04-02T04:48:03.501 INFO:tasks.workunit.client.0.smithi070.stdout:0/830: dread d4/d5/da/d2d/de7/ff0 [0,4194304] 0 2024-04-02T04:48:03.501 INFO:tasks.workunit.client.0.smithi070.stdout:4/768: stat d1/d9c/df0 0 2024-04-02T04:48:03.502 INFO:tasks.workunit.client.0.smithi070.stdout:5/931: rename d5/d17/da7/d118 to d5/d17/d27/de3/d124 0 2024-04-02T04:48:03.503 INFO:tasks.workunit.client.0.smithi070.stdout:5/932: chown d5/d57/l69 996 1 2024-04-02T04:48:03.504 INFO:tasks.workunit.client.1.smithi179.stdout:0/936: symlink d1/da/d28/d5e/l111 0 2024-04-02T04:48:03.505 INFO:tasks.workunit.client.0.smithi070.stdout:7/723: chown d7/d9/ld6 141 1 2024-04-02T04:48:03.507 INFO:tasks.workunit.client.1.smithi179.stdout:8/863: rmdir d3/d32/d21/d22/d120 0 2024-04-02T04:48:03.508 INFO:tasks.workunit.client.0.smithi070.stdout:9/721: mknod d4/d10/d12/cea 0 2024-04-02T04:48:03.509 INFO:tasks.workunit.client.0.smithi070.stdout:6/675: dread d0/d20/f45 [0,4194304] 0 2024-04-02T04:48:03.509 INFO:tasks.workunit.client.1.smithi179.stdout:6/848: rmdir d3/d12/dde 0 2024-04-02T04:48:03.513 INFO:tasks.workunit.client.1.smithi179.stdout:6/849: dread d3/fa [0,4194304] 0 2024-04-02T04:48:03.516 INFO:tasks.workunit.client.0.smithi070.stdout:5/933: fsync d5/d14/dd8/ddb/fdc 0 2024-04-02T04:48:03.517 INFO:tasks.workunit.client.1.smithi179.stdout:1/861: fdatasync d3/d5/de/fa9 0 2024-04-02T04:48:03.520 INFO:tasks.workunit.client.0.smithi070.stdout:9/722: mknod d4/d10/d12/d27/d60/ceb 0 2024-04-02T04:48:03.521 INFO:tasks.workunit.client.1.smithi179.stdout:0/937: dread d1/fcb [0,4194304] 0 2024-04-02T04:48:03.522 INFO:tasks.workunit.client.1.smithi179.stdout:8/864: unlink d3/c3d 0 2024-04-02T04:48:03.524 INFO:tasks.workunit.client.0.smithi070.stdout:6/676: symlink d0/d55/d5c/lce 0 2024-04-02T04:48:03.527 INFO:tasks.workunit.client.0.smithi070.stdout:6/677: dread d0/d20/f27 [0,4194304] 0 2024-04-02T04:48:03.528 INFO:tasks.workunit.client.0.smithi070.stdout:0/831: unlink d4/d5/d13/d17/f62 0 2024-04-02T04:48:03.530 INFO:tasks.workunit.client.0.smithi070.stdout:4/769: mknod d1/d3/d85/d8e/d99/db1/de9/c103 0 2024-04-02T04:48:03.531 INFO:tasks.workunit.client.0.smithi070.stdout:4/770: write d1/d9c/d65/d7a/f7d [2395223,130123] 0 2024-04-02T04:48:03.531 INFO:tasks.workunit.client.1.smithi179.stdout:4/880: getdents d1/d3/df/d10/d2f/d60/d71/dce/d118 0 2024-04-02T04:48:03.534 INFO:tasks.workunit.client.0.smithi070.stdout:5/934: rmdir d5/d9/d23 39 2024-04-02T04:48:03.534 INFO:tasks.workunit.client.1.smithi179.stdout:5/822: write d8/d1b/d23/d32/d4e/d50/fb9 [436287,103224] 0 2024-04-02T04:48:03.536 INFO:tasks.workunit.client.0.smithi070.stdout:7/724: creat d7/dd/df/d7a/d6c/d77/d54/de5/feb x:0 0 0 2024-04-02T04:48:03.536 INFO:tasks.workunit.client.1.smithi179.stdout:2/736: sync 2024-04-02T04:48:03.537 INFO:tasks.workunit.client.1.smithi179.stdout:1/862: symlink d3/d5/de/d33/l130 0 2024-04-02T04:48:03.542 INFO:tasks.workunit.client.1.smithi179.stdout:8/865: symlink d3/d32/d21/d22/d6b/db1/l133 0 2024-04-02T04:48:03.546 INFO:tasks.workunit.client.1.smithi179.stdout:9/839: rename d0/d19/d27/d7a/d87/daf/dc2/lca to d0/d1/d10/d6d/l102 0 2024-04-02T04:48:03.547 INFO:tasks.workunit.client.1.smithi179.stdout:5/823: dread d8/d1b/d23/d36/d53/d62/da9/fc0 [0,4194304] 0 2024-04-02T04:48:03.549 INFO:tasks.workunit.client.0.smithi070.stdout:5/935: rmdir d5/d17 39 2024-04-02T04:48:03.550 INFO:tasks.workunit.client.1.smithi179.stdout:2/737: mkdir dc/d29/d4a/d1e/d43/df9 0 2024-04-02T04:48:03.552 INFO:tasks.workunit.client.1.smithi179.stdout:9/840: dwrite d0/f4c [0,4194304] 0 2024-04-02T04:48:03.555 INFO:tasks.workunit.client.0.smithi070.stdout:9/723: link d4/d10/f25 d4/d10/d11/d7b/d19/d2f/d4d/d92/fec 0 2024-04-02T04:48:03.560 INFO:tasks.workunit.client.1.smithi179.stdout:8/866: creat d3/d32/d21/d50/dcf/f134 x:0 0 0 2024-04-02T04:48:03.561 INFO:tasks.workunit.client.0.smithi070.stdout:0/832: write d4/d5/d13/ff3 [491466,85209] 0 2024-04-02T04:48:03.561 INFO:tasks.workunit.client.0.smithi070.stdout:9/724: dread d4/d10/d12/d27/d60/d9e/f77 [4194304,4194304] 0 2024-04-02T04:48:03.563 INFO:tasks.workunit.client.0.smithi070.stdout:6/678: dwrite d0/d55/d8b/d94/f65 [0,4194304] 0 2024-04-02T04:48:03.565 INFO:tasks.workunit.client.0.smithi070.stdout:7/725: write d7/dd/df/d7a/d6c/d77/fb0 [282545,101388] 0 2024-04-02T04:48:03.567 INFO:tasks.workunit.client.0.smithi070.stdout:7/726: read - d7/dd/df/fa9 zero size 2024-04-02T04:48:03.567 INFO:tasks.workunit.client.0.smithi070.stdout:4/771: fdatasync d1/d3/d85/d8e/fda 0 2024-04-02T04:48:03.569 INFO:tasks.workunit.client.0.smithi070.stdout:7/727: stat d7/dd/df/d79/d46/d65/lad 0 2024-04-02T04:48:03.570 INFO:tasks.workunit.client.0.smithi070.stdout:7/728: readlink d7/dd/df/d34/l33 0 2024-04-02T04:48:03.572 INFO:tasks.workunit.client.1.smithi179.stdout:9/841: symlink d0/d19/d27/d7a/d87/daf/dc2/l103 0 2024-04-02T04:48:03.573 INFO:tasks.workunit.client.0.smithi070.stdout:9/725: creat d4/d10/d11/da8/fed x:0 0 0 2024-04-02T04:48:03.574 INFO:tasks.workunit.client.1.smithi179.stdout:1/863: truncate d3/d5/d2f/d4f/d35/fa5 871813 0 2024-04-02T04:48:03.576 INFO:tasks.workunit.client.0.smithi070.stdout:6/679: unlink d0/d20/d77/c7c 0 2024-04-02T04:48:03.576 INFO:tasks.workunit.client.1.smithi179.stdout:0/938: dwrite d1/da/d48/d7b/ff0 [0,4194304] 0 2024-04-02T04:48:03.576 INFO:tasks.workunit.client.1.smithi179.stdout:1/864: chown d3/d5/d2f/d4f/d42/df2/fba 4136079 1 2024-04-02T04:48:03.579 INFO:tasks.workunit.client.0.smithi070.stdout:4/772: creat d1/d3/d85/ddb/f104 x:0 0 0 2024-04-02T04:48:03.582 INFO:tasks.workunit.client.1.smithi179.stdout:3/876: rename d8/dd3/l93 to d8/d26/d83/d89/d9c/dcc/l118 0 2024-04-02T04:48:03.582 INFO:tasks.workunit.client.0.smithi070.stdout:5/936: stat d5/d17/d27/l11c 0 2024-04-02T04:48:03.583 INFO:tasks.workunit.client.1.smithi179.stdout:3/877: write d8/f6f [4675101,55958] 0 2024-04-02T04:48:03.583 INFO:tasks.workunit.client.1.smithi179.stdout:3/878: chown d8/d57/d8f/d107 397 1 2024-04-02T04:48:03.584 INFO:tasks.workunit.client.1.smithi179.stdout:5/824: symlink d8/d1b/l109 0 2024-04-02T04:48:03.585 INFO:tasks.workunit.client.1.smithi179.stdout:3/879: write d8/d26/d83/d89/d9c/fb0 [3002850,100669] 0 2024-04-02T04:48:03.585 INFO:tasks.workunit.client.1.smithi179.stdout:2/738: getdents dc/d29/d31/d83/dae/ddc 0 2024-04-02T04:48:03.586 INFO:tasks.workunit.client.0.smithi070.stdout:9/726: mknod d4/cee 0 2024-04-02T04:48:03.589 INFO:tasks.workunit.client.0.smithi070.stdout:0/833: symlink d4/d5/d13/d2a/l100 0 2024-04-02T04:48:03.592 INFO:tasks.workunit.client.1.smithi179.stdout:8/867: dread d3/d32/f19 [0,4194304] 0 2024-04-02T04:48:03.595 INFO:tasks.workunit.client.0.smithi070.stdout:9/727: fsync d4/d10/d11/d7b/d19/d53/d9c/fdb 0 2024-04-02T04:48:03.596 INFO:tasks.workunit.client.1.smithi179.stdout:5/825: rmdir d8/d1b/d23/d32/dbe/daf/db7/dad 39 2024-04-02T04:48:03.597 INFO:tasks.workunit.client.1.smithi179.stdout:5/826: dread - d8/d1b/d23/d32/d46/ff7 zero size 2024-04-02T04:48:03.597 INFO:tasks.workunit.client.0.smithi070.stdout:9/728: write d4/d10/d12/d27/d60/dc9/fd5 [31686,120879] 0 2024-04-02T04:48:03.598 INFO:tasks.workunit.client.0.smithi070.stdout:9/729: chown d4/d10/d12/d56/d79/f85 32 1 2024-04-02T04:48:03.601 INFO:tasks.workunit.client.1.smithi179.stdout:2/739: truncate dc/d29/d31/d57/f62 6904830 0 2024-04-02T04:48:03.603 INFO:tasks.workunit.client.0.smithi070.stdout:4/773: rename d1/d3/d78/d91/lab to d1/de7/l105 0 2024-04-02T04:48:03.605 INFO:tasks.workunit.client.0.smithi070.stdout:5/937: mkdir d5/d17/da7/d125 0 2024-04-02T04:48:03.606 INFO:tasks.workunit.client.1.smithi179.stdout:4/881: getdents d1/d3/d19 0 2024-04-02T04:48:03.611 INFO:tasks.workunit.client.1.smithi179.stdout:7/870: rename d1/d9/fc9 to d1/d9/dd9/d121/d6b/f134 0 2024-04-02T04:48:03.613 INFO:tasks.workunit.client.0.smithi070.stdout:9/730: rmdir d4/d10/d12/d27/d60/dc9 39 2024-04-02T04:48:03.613 INFO:tasks.workunit.client.0.smithi070.stdout:7/729: dwrite d7/dd/df/d79/d46/d65/fa4 [0,4194304] 0 2024-04-02T04:48:03.614 INFO:tasks.workunit.client.0.smithi070.stdout:6/680: write d0/d3/d19/d31/d46/d6b/f70 [3369603,108408] 0 2024-04-02T04:48:03.615 INFO:tasks.workunit.client.1.smithi179.stdout:8/868: mkdir d3/d32/d21/d22/d6b/db1/d104/d135 0 2024-04-02T04:48:03.622 INFO:tasks.workunit.client.1.smithi179.stdout:0/939: truncate d1/da/d48/d5c/d83/fb1 1503554 0 2024-04-02T04:48:03.622 INFO:tasks.workunit.client.0.smithi070.stdout:0/834: dwrite d4/d5/d13/d2a/d3a/d80/f9f [0,4194304] 0 2024-04-02T04:48:03.622 INFO:tasks.workunit.client.0.smithi070.stdout:0/835: stat d4/d5/d13/d2a/ff6 0 2024-04-02T04:48:03.623 INFO:tasks.workunit.client.1.smithi179.stdout:8/869: dwrite d3/d32/d21/d50/f5f [0,4194304] 0 2024-04-02T04:48:03.626 INFO:tasks.workunit.client.1.smithi179.stdout:3/880: mknod d8/d26/d83/d89/c119 0 2024-04-02T04:48:03.627 INFO:tasks.workunit.client.1.smithi179.stdout:3/881: read d8/d57/d63/dc3/fc8 [110396,4268] 0 2024-04-02T04:48:03.628 INFO:tasks.workunit.client.0.smithi070.stdout:7/730: mkdir d7/d8e/dec 0 2024-04-02T04:48:03.628 INFO:tasks.workunit.client.0.smithi070.stdout:7/731: readlink d7/d8e/le7 0 2024-04-02T04:48:03.629 INFO:tasks.workunit.client.1.smithi179.stdout:1/865: unlink d3/d5/d2f/d4f/d35/fa5 0 2024-04-02T04:48:03.630 INFO:tasks.workunit.client.0.smithi070.stdout:9/731: creat d4/d13/d89/fef x:0 0 0 2024-04-02T04:48:03.634 INFO:tasks.workunit.client.0.smithi070.stdout:0/836: stat d4/d5/d13/d2a/d3a/lb9 0 2024-04-02T04:48:03.635 INFO:tasks.workunit.client.1.smithi179.stdout:0/940: mkdir d1/da/d52/d75/d9a/dec/d112 0 2024-04-02T04:48:03.640 INFO:tasks.workunit.client.1.smithi179.stdout:8/870: creat d3/d1b/d2a/d41/d109/f136 x:0 0 0 2024-04-02T04:48:03.641 INFO:tasks.workunit.client.0.smithi070.stdout:7/732: creat d7/dd/df/d7a/dbc/fed x:0 0 0 2024-04-02T04:48:03.646 INFO:tasks.workunit.client.0.smithi070.stdout:6/681: write d0/d3/f3a [2342683,107655] 0 2024-04-02T04:48:03.646 INFO:tasks.workunit.client.0.smithi070.stdout:4/774: link d1/d3/cb6 d1/d3/d85/d8e/d99/db1/dbb/c106 0 2024-04-02T04:48:03.647 INFO:tasks.workunit.client.1.smithi179.stdout:7/871: mknod d1/c135 0 2024-04-02T04:48:03.649 INFO:tasks.workunit.client.0.smithi070.stdout:0/837: unlink d4/d5/d13/f32 0 2024-04-02T04:48:03.651 INFO:tasks.workunit.client.1.smithi179.stdout:0/941: read - d1/da/d48/d7b/fb6 zero size 2024-04-02T04:48:03.652 INFO:tasks.workunit.client.0.smithi070.stdout:7/733: creat d7/dd/df/d7a/d6c/d77/d75/d80/fee x:0 0 0 2024-04-02T04:48:03.652 INFO:tasks.workunit.client.0.smithi070.stdout:5/938: write d5/d9/d10/db9/dd9/fe7 [828973,65433] 0 2024-04-02T04:48:03.658 INFO:tasks.workunit.client.0.smithi070.stdout:6/682: creat d0/d20/fcf x:0 0 0 2024-04-02T04:48:03.661 INFO:tasks.workunit.client.1.smithi179.stdout:6/850: rename d3/d12/f62 to d3/d12/d23/d39/d4e/d66/f132 0 2024-04-02T04:48:03.662 INFO:tasks.workunit.client.0.smithi070.stdout:7/734: mkdir d7/dd/df/def 0 2024-04-02T04:48:03.663 INFO:tasks.workunit.client.0.smithi070.stdout:7/735: write d7/dd/df/d7a/f64 [1740568,95830] 0 2024-04-02T04:48:03.663 INFO:tasks.workunit.client.1.smithi179.stdout:0/942: mkdir d1/da/d28/d37/d113 0 2024-04-02T04:48:03.669 INFO:tasks.workunit.client.0.smithi070.stdout:6/683: mkdir d0/d3/d19/d31/dc1/dd0 0 2024-04-02T04:48:03.670 INFO:tasks.workunit.client.1.smithi179.stdout:8/871: dwrite d3/d32/f5c [4194304,4194304] 0 2024-04-02T04:48:03.671 INFO:tasks.workunit.client.0.smithi070.stdout:5/939: write d5/d17/d27/f4b [3225473,43571] 0 2024-04-02T04:48:03.671 INFO:tasks.workunit.client.1.smithi179.stdout:1/866: dwrite d3/d5/d2f/d4f/d42/d53/d5b/faa [0,4194304] 0 2024-04-02T04:48:03.671 INFO:tasks.workunit.client.1.smithi179.stdout:8/872: chown d3/d32/f5c 3768578 1 2024-04-02T04:48:03.676 INFO:tasks.workunit.client.1.smithi179.stdout:5/827: rename d8/d1b/d23/d36/l87 to d8/d1b/d23/d32/d4e/l10a 0 2024-04-02T04:48:03.676 INFO:tasks.workunit.client.1.smithi179.stdout:6/851: creat d3/d2d/da5/f133 x:0 0 0 2024-04-02T04:48:03.677 INFO:tasks.workunit.client.0.smithi070.stdout:4/775: dwrite d1/d3/d78/d91/faa [0,4194304] 0 2024-04-02T04:48:03.679 INFO:tasks.workunit.client.1.smithi179.stdout:3/882: symlink d8/dd3/d9a/de8/l11a 0 2024-04-02T04:48:03.679 INFO:tasks.workunit.client.0.smithi070.stdout:7/736: rename d7/dd/df/c9f to d7/dd/df/d7a/d6c/d77/d75/cf0 0 2024-04-02T04:48:03.679 INFO:tasks.workunit.client.0.smithi070.stdout:9/732: getdents d4/d10/d11 0 2024-04-02T04:48:03.681 INFO:tasks.workunit.client.1.smithi179.stdout:3/883: write d8/f6f [3330069,48829] 0 2024-04-02T04:48:03.682 INFO:tasks.workunit.client.1.smithi179.stdout:1/867: dwrite d3/d5/d2f/d65/f6b [0,4194304] 0 2024-04-02T04:48:03.683 INFO:tasks.workunit.client.1.smithi179.stdout:9/842: sync 2024-04-02T04:48:03.685 INFO:tasks.workunit.client.0.smithi070.stdout:0/838: creat d4/d5/d13/d2a/d3a/f101 x:0 0 0 2024-04-02T04:48:03.685 INFO:tasks.workunit.client.1.smithi179.stdout:7/872: creat d1/d9/f136 x:0 0 0 2024-04-02T04:48:03.687 INFO:tasks.workunit.client.1.smithi179.stdout:7/873: chown d1/dd/d16/l79 453 1 2024-04-02T04:48:03.687 INFO:tasks.workunit.client.0.smithi070.stdout:0/839: read - d4/d5/d58/d60/d83/ff4 zero size 2024-04-02T04:48:03.687 INFO:tasks.workunit.client.1.smithi179.stdout:7/874: stat d1/dd/d16/l97 0 2024-04-02T04:48:03.688 INFO:tasks.workunit.client.1.smithi179.stdout:7/875: dread - d1/dd/f111 zero size 2024-04-02T04:48:03.688 INFO:tasks.workunit.client.0.smithi070.stdout:7/737: dwrite d7/dd/df/d79/f5e [0,4194304] 0 2024-04-02T04:48:03.689 INFO:tasks.workunit.client.0.smithi070.stdout:5/940: dread - d5/d14/dd8/ddb/fdc zero size 2024-04-02T04:48:03.689 INFO:tasks.workunit.client.1.smithi179.stdout:4/882: rename d1/d3/dc/d26/d8a/f94 to d1/d93/de6/f12d 0 2024-04-02T04:48:03.691 INFO:tasks.workunit.client.0.smithi070.stdout:7/738: chown d7/dd/df/d34/d68 7368 1 2024-04-02T04:48:03.692 INFO:tasks.workunit.client.0.smithi070.stdout:4/776: creat d1/d3/d78/dad/f107 x:0 0 0 2024-04-02T04:48:03.692 INFO:tasks.workunit.client.1.smithi179.stdout:6/852: mknod d3/d12/d23/d39/d4e/d6c/dcd/c134 0 2024-04-02T04:48:03.692 INFO:tasks.workunit.client.0.smithi070.stdout:5/941: write d5/d17/d27/f28 [4901115,91555] 0 2024-04-02T04:48:03.693 INFO:tasks.workunit.client.0.smithi070.stdout:7/739: chown d7/dd/df/fa9 46 1 2024-04-02T04:48:03.693 INFO:tasks.workunit.client.1.smithi179.stdout:6/853: truncate d3/fb 3467531 0 2024-04-02T04:48:03.694 INFO:tasks.workunit.client.0.smithi070.stdout:4/777: chown d1/d3/d85/da8/cc1 2 1 2024-04-02T04:48:03.709 INFO:tasks.workunit.client.1.smithi179.stdout:0/943: rename d1/db/f7c to d1/da/d52/d75/d9a/df8/d10a/f114 0 2024-04-02T04:48:03.709 INFO:tasks.workunit.client.1.smithi179.stdout:7/876: dread d1/d9/f85 [0,4194304] 0 2024-04-02T04:48:03.709 INFO:tasks.workunit.client.0.smithi070.stdout:7/740: write d7/dd/df/fa9 [413980,29541] 0 2024-04-02T04:48:03.710 INFO:tasks.workunit.client.1.smithi179.stdout:4/883: creat d1/d3/d128/f12e x:0 0 0 2024-04-02T04:48:03.717 INFO:tasks.workunit.client.0.smithi070.stdout:9/733: write d4/d10/d12/d27/d60/d9e/fc7 [3354490,845] 0 2024-04-02T04:48:03.723 INFO:tasks.workunit.client.0.smithi070.stdout:0/840: write d4/d5/d58/da1/fcd [251311,49617] 0 2024-04-02T04:48:03.723 INFO:tasks.workunit.client.1.smithi179.stdout:3/884: dwrite d8/d57/d63/f84 [0,4194304] 0 2024-04-02T04:48:03.726 INFO:tasks.workunit.client.0.smithi070.stdout:6/684: dwrite d0/d3/d19/f7f [0,4194304] 0 2024-04-02T04:48:03.727 INFO:tasks.workunit.client.1.smithi179.stdout:9/843: write d0/d1/d11/d29/f50 [2593501,86764] 0 2024-04-02T04:48:03.727 INFO:tasks.workunit.client.0.smithi070.stdout:9/734: dread f3 [0,4194304] 0 2024-04-02T04:48:03.728 INFO:tasks.workunit.client.0.smithi070.stdout:9/735: fsync d4/d13/d89/faf 0 2024-04-02T04:48:03.728 INFO:tasks.workunit.client.0.smithi070.stdout:7/741: write d7/dd/df/d79/d46/f53 [650816,28804] 0 2024-04-02T04:48:03.729 INFO:tasks.workunit.client.1.smithi179.stdout:8/873: rename d3/d31/l29 to d3/d32/d21/d49/df0/l137 0 2024-04-02T04:48:03.730 INFO:tasks.workunit.client.1.smithi179.stdout:8/874: fsync d3/d32/d21/d22/d6b/d91/da8/dfe/f100 0 2024-04-02T04:48:03.731 INFO:tasks.workunit.client.1.smithi179.stdout:8/875: fdatasync d3/f122 0 2024-04-02T04:48:03.732 INFO:tasks.workunit.client.0.smithi070.stdout:5/942: truncate d5/d17/d50/f5f 827286 0 2024-04-02T04:48:03.734 INFO:tasks.workunit.client.1.smithi179.stdout:9/844: dwrite d0/d19/d27/d7a/d7e/fdd [0,4194304] 0 2024-04-02T04:48:03.736 INFO:tasks.workunit.client.0.smithi070.stdout:0/841: mkdir d4/d5/d21/d6b/d102 0 2024-04-02T04:48:03.736 INFO:tasks.workunit.client.1.smithi179.stdout:7/877: stat d1/d9/c34 0 2024-04-02T04:48:03.737 INFO:tasks.workunit.client.0.smithi070.stdout:0/842: stat d4/d5/d13/d17/d98/dd2/lee 0 2024-04-02T04:48:03.738 INFO:tasks.workunit.client.1.smithi179.stdout:9/845: write d0/d19/d27/d7a/d7e/db9/dfa/fd7 [138625,74381] 0 2024-04-02T04:48:03.740 INFO:tasks.workunit.client.0.smithi070.stdout:0/843: write d4/d5/d13/d2a/d3a/d45/fbd [250755,15911] 0 2024-04-02T04:48:03.744 INFO:tasks.workunit.client.0.smithi070.stdout:5/943: rename d5/d17/d50/f10f to d5/d9/d10/db9/dd9/f126 0 2024-04-02T04:48:03.746 INFO:tasks.workunit.client.1.smithi179.stdout:7/878: unlink d1/dd/f17 0 2024-04-02T04:48:03.747 INFO:tasks.workunit.client.0.smithi070.stdout:7/742: dread d7/dd/df/d34/fba [0,4194304] 0 2024-04-02T04:48:03.752 INFO:tasks.workunit.client.1.smithi179.stdout:3/885: symlink d8/d26/d83/l11b 0 2024-04-02T04:48:03.754 INFO:tasks.workunit.client.0.smithi070.stdout:4/778: dread d1/d3/d9/d5e/f77 [0,4194304] 0 2024-04-02T04:48:03.754 INFO:tasks.workunit.client.0.smithi070.stdout:7/743: dwrite d7/d36/f62 [0,4194304] 0 2024-04-02T04:48:03.755 INFO:tasks.workunit.client.1.smithi179.stdout:0/944: dread d1/da/d67/fa8 [0,4194304] 0 2024-04-02T04:48:03.755 INFO:tasks.workunit.client.1.smithi179.stdout:9/846: dread - d0/d1/d11/d29/d86/fb2 zero size 2024-04-02T04:48:03.758 INFO:tasks.workunit.client.1.smithi179.stdout:6/854: rename d3/d2d/d38/d64/cad to d3/d12/d23/d39/dff/c135 0 2024-04-02T04:48:03.758 INFO:tasks.workunit.client.0.smithi070.stdout:7/744: dread d7/dd/df/d79/fbb [0,4194304] 0 2024-04-02T04:48:03.759 INFO:tasks.workunit.client.0.smithi070.stdout:0/844: mknod d4/d5/d21/d6b/c103 0 2024-04-02T04:48:03.761 INFO:tasks.workunit.client.1.smithi179.stdout:7/879: dread - d1/d9/d27/d87/d8b/d75/db9/dcf/f102 zero size 2024-04-02T04:48:03.762 INFO:tasks.workunit.client.1.smithi179.stdout:3/886: write d8/d6d/d94/fea [765697,71480] 0 2024-04-02T04:48:03.763 INFO:tasks.workunit.client.1.smithi179.stdout:9/847: symlink d0/d1/d10/l104 0 2024-04-02T04:48:03.764 INFO:tasks.workunit.client.1.smithi179.stdout:8/876: creat d3/d32/d21/d22/d6b/db1/df8/d126/f138 x:0 0 0 2024-04-02T04:48:03.765 INFO:tasks.workunit.client.1.smithi179.stdout:0/945: symlink d1/da/d28/l115 0 2024-04-02T04:48:03.766 INFO:tasks.workunit.client.1.smithi179.stdout:9/848: rmdir d0/d1/d11/d77 39 2024-04-02T04:48:03.767 INFO:tasks.workunit.client.1.smithi179.stdout:6/855: mknod d3/d2d/d43/d12d/c136 0 2024-04-02T04:48:03.768 INFO:tasks.workunit.client.1.smithi179.stdout:7/880: mknod d1/dd/c137 0 2024-04-02T04:48:03.768 INFO:tasks.workunit.client.1.smithi179.stdout:8/877: symlink d3/d32/d21/d22/d6b/d91/d110/l139 0 2024-04-02T04:48:03.769 INFO:tasks.workunit.client.1.smithi179.stdout:3/887: mkdir d8/d26/d11c 0 2024-04-02T04:48:03.770 INFO:tasks.workunit.client.1.smithi179.stdout:0/946: rename d1/db/f2c to d1/da/d48/d5c/d83/f116 0 2024-04-02T04:48:03.771 INFO:tasks.workunit.client.1.smithi179.stdout:9/849: readlink d0/d1/d10/d6d/l3e 0 2024-04-02T04:48:03.772 INFO:tasks.workunit.client.1.smithi179.stdout:0/947: write d1/da/d48/d5c/d83/fe4 [235608,66003] 0 2024-04-02T04:48:03.773 INFO:tasks.workunit.client.1.smithi179.stdout:8/878: unlink d3/d1b/d2a/db0/cd2 0 2024-04-02T04:48:03.777 INFO:tasks.workunit.client.1.smithi179.stdout:9/850: dwrite d0/d19/d27/dad/fae [0,4194304] 0 2024-04-02T04:48:03.778 INFO:tasks.workunit.client.1.smithi179.stdout:2/740: sync 2024-04-02T04:48:03.779 INFO:tasks.workunit.client.1.smithi179.stdout:7/881: rename d1/d9/d55/dbf/d9f to d1/dd/d76/d138 0 2024-04-02T04:48:03.779 INFO:tasks.workunit.client.1.smithi179.stdout:7/882: chown d1/dd/d76 22462775 1 2024-04-02T04:48:03.781 INFO:tasks.workunit.client.1.smithi179.stdout:9/851: symlink d0/d1/d11/dab/l105 0 2024-04-02T04:48:03.782 INFO:tasks.workunit.client.1.smithi179.stdout:2/741: rmdir dc/d29/d4a/d1e 39 2024-04-02T04:48:03.783 INFO:tasks.workunit.client.1.smithi179.stdout:8/879: symlink d3/d31/dc3/d10c/l13a 0 2024-04-02T04:48:03.784 INFO:tasks.workunit.client.1.smithi179.stdout:8/880: write d3/f122 [1029157,77383] 0 2024-04-02T04:48:03.785 INFO:tasks.workunit.client.1.smithi179.stdout:0/948: rename d1/da/df/d56/ffa to d1/da/d52/d75/d9a/df8/d10a/f117 0 2024-04-02T04:48:03.786 INFO:tasks.workunit.client.1.smithi179.stdout:0/949: readlink d1/da/d28/d37/l6f 0 2024-04-02T04:48:03.787 INFO:tasks.workunit.client.1.smithi179.stdout:7/883: mkdir d1/d9/d27/d87/d8b/d75/db9/dcf/d139 0 2024-04-02T04:48:03.789 INFO:tasks.workunit.client.1.smithi179.stdout:9/852: creat d0/d19/d27/dad/f106 x:0 0 0 2024-04-02T04:48:03.790 INFO:tasks.workunit.client.1.smithi179.stdout:9/853: write d0/d1/d10/d6d/d58/f96 [5233995,39088] 0 2024-04-02T04:48:03.790 INFO:tasks.workunit.client.1.smithi179.stdout:3/888: link d8/dd3/c9e d8/dd3/d9a/de8/c11d 0 2024-04-02T04:48:03.791 INFO:tasks.workunit.client.1.smithi179.stdout:3/889: chown d8/d26/d36 24 1 2024-04-02T04:48:03.803 INFO:tasks.workunit.client.1.smithi179.stdout:2/742: rename dc/d29/d4a/d1e/d8a/cd9 to dc/d29/de7/cfa 0 2024-04-02T04:48:03.803 INFO:tasks.workunit.client.1.smithi179.stdout:0/950: creat d1/da/df/df3/f118 x:0 0 0 2024-04-02T04:48:03.805 INFO:tasks.workunit.client.1.smithi179.stdout:0/951: chown d1/da/d28/d34/ddd 5 1 2024-04-02T04:48:03.808 INFO:tasks.workunit.client.1.smithi179.stdout:7/884: creat d1/f13a x:0 0 0 2024-04-02T04:48:03.810 INFO:tasks.workunit.client.1.smithi179.stdout:9/854: dwrite d0/d1/d11/dab/ded/f100 [0,4194304] 0 2024-04-02T04:48:03.813 INFO:tasks.workunit.client.1.smithi179.stdout:7/885: dread - d1/dd/d76/d138/da8/dda/dea/f108 zero size 2024-04-02T04:48:03.813 INFO:tasks.workunit.client.1.smithi179.stdout:3/890: fsync d8/d26/d36/fb9 0 2024-04-02T04:48:03.815 INFO:tasks.workunit.client.1.smithi179.stdout:2/743: dwrite dc/d29/d4a/d1e/d6c/dd2/fe5 [0,4194304] 0 2024-04-02T04:48:03.816 INFO:tasks.workunit.client.1.smithi179.stdout:2/744: chown dc/ca1 198765010 1 2024-04-02T04:48:03.825 INFO:tasks.workunit.client.0.smithi070.stdout:4/779: mknod d1/d3/d85/d8e/d99/c108 0 2024-04-02T04:48:03.825 INFO:tasks.workunit.client.1.smithi179.stdout:2/745: dwrite dc/f5d [0,4194304] 0 2024-04-02T04:48:03.826 INFO:tasks.workunit.client.0.smithi070.stdout:9/736: truncate d4/d10/d11/d7b/d19/f24 2722836 0 2024-04-02T04:48:03.827 INFO:tasks.workunit.client.0.smithi070.stdout:4/780: dread - d1/d9c/d65/d7a/f9f zero size 2024-04-02T04:48:03.829 INFO:tasks.workunit.client.1.smithi179.stdout:3/891: symlink d8/d26/d83/d89/l11e 0 2024-04-02T04:48:03.830 INFO:tasks.workunit.client.0.smithi070.stdout:6/685: link d0/l21 d0/d3/d19/d31/d46/d6b/ld1 0 2024-04-02T04:48:03.833 INFO:tasks.workunit.client.1.smithi179.stdout:7/886: rename d1/d9/d27/d87/fe3 to d1/d9/d27/d87/d8b/d75/d11f/f13b 0 2024-04-02T04:48:03.834 INFO:tasks.workunit.client.0.smithi070.stdout:0/845: dread - d4/d5/d13/d17/d98/fc0 zero size 2024-04-02T04:48:03.835 INFO:tasks.workunit.client.1.smithi179.stdout:2/746: mknod dc/d29/d31/d57/dea/cfb 0 2024-04-02T04:48:03.837 INFO:tasks.workunit.client.1.smithi179.stdout:0/952: getdents d1/da/d52/d75/dc0 0 2024-04-02T04:48:03.839 INFO:tasks.workunit.client.1.smithi179.stdout:3/892: creat d8/d57/d63/dc3/f11f x:0 0 0 2024-04-02T04:48:03.841 INFO:tasks.workunit.client.0.smithi070.stdout:9/737: rename d4/d10/d11/d7b/d19/d53/d9c to d4/d13/df0 0 2024-04-02T04:48:03.843 INFO:tasks.workunit.client.1.smithi179.stdout:6/856: dwrite d3/d42/fc8 [0,4194304] 0 2024-04-02T04:48:03.848 INFO:tasks.workunit.client.1.smithi179.stdout:2/747: rename dc/d29/d4a/fe9 to dc/d29/d4a/d4b/ffc 0 2024-04-02T04:48:03.849 INFO:tasks.workunit.client.0.smithi070.stdout:6/686: mkdir d0/d43/d95/dd2 0 2024-04-02T04:48:03.849 INFO:tasks.workunit.client.0.smithi070.stdout:6/687: readlink d0/l21 0 2024-04-02T04:48:03.849 INFO:tasks.workunit.client.1.smithi179.stdout:6/857: dwrite d3/fb [0,4194304] 0 2024-04-02T04:48:03.849 INFO:tasks.workunit.client.1.smithi179.stdout:5/828: sync 2024-04-02T04:48:03.861 INFO:tasks.workunit.client.0.smithi070.stdout:7/745: creat d7/ff1 x:0 0 0 2024-04-02T04:48:03.861 INFO:tasks.workunit.client.0.smithi070.stdout:9/738: mkdir d4/d13/df0/dd3/df1 0 2024-04-02T04:48:03.869 INFO:tasks.workunit.client.1.smithi179.stdout:9/855: write d0/d1/d11/f78 [4173704,77396] 0 2024-04-02T04:48:03.870 INFO:tasks.workunit.client.1.smithi179.stdout:0/953: symlink d1/da/d28/d81/d9d/dd8/l119 0 2024-04-02T04:48:03.871 INFO:tasks.workunit.client.1.smithi179.stdout:2/748: dread dc/d29/d4a/f56 [0,4194304] 0 2024-04-02T04:48:03.873 INFO:tasks.workunit.client.1.smithi179.stdout:2/749: write dc/d29/d4a/d1e/d43/f98 [976863,7151] 0 2024-04-02T04:48:03.874 INFO:tasks.workunit.client.1.smithi179.stdout:5/829: rmdir d8/d1b 39 2024-04-02T04:48:03.875 INFO:tasks.workunit.client.1.smithi179.stdout:5/830: read d8/f18 [470522,38609] 0 2024-04-02T04:48:03.876 INFO:tasks.workunit.client.1.smithi179.stdout:6/858: fdatasync d3/d2d/d38/d64/fd6 0 2024-04-02T04:48:03.879 INFO:tasks.workunit.client.1.smithi179.stdout:3/893: unlink d8/dd3/dbb/fbe 0 2024-04-02T04:48:03.881 INFO:tasks.workunit.client.1.smithi179.stdout:1/868: sync 2024-04-02T04:48:03.881 INFO:tasks.workunit.client.1.smithi179.stdout:4/884: sync 2024-04-02T04:48:03.884 INFO:tasks.workunit.client.1.smithi179.stdout:7/887: write d1/d9/d27/d87/d8b/d75/db9/f92 [751361,13267] 0 2024-04-02T04:48:03.887 INFO:tasks.workunit.client.0.smithi070.stdout:0/846: write d4/d5/da/d2d/de7/ff0 [238680,21672] 0 2024-04-02T04:48:03.888 INFO:tasks.workunit.client.1.smithi179.stdout:9/856: truncate d0/d1/d11/d29/d40/f63 30643 0 2024-04-02T04:48:03.888 INFO:tasks.workunit.client.0.smithi070.stdout:0/847: chown d4/d5/d13/d2a/d3a/d45/fc6 12 1 2024-04-02T04:48:03.889 INFO:tasks.workunit.client.1.smithi179.stdout:9/857: write d0/d1/f7b [1026144,120069] 0 2024-04-02T04:48:03.893 INFO:tasks.workunit.client.1.smithi179.stdout:2/750: rename dc/d29/d31/d83/cb5 to dc/d29/d4a/d1e/d8a/dce/cfd 0 2024-04-02T04:48:03.894 INFO:tasks.workunit.client.0.smithi070.stdout:6/688: write d0/d43/db3/dc7/f66 [1398206,38961] 0 2024-04-02T04:48:03.897 INFO:tasks.workunit.client.0.smithi070.stdout:4/781: dwrite d1/dc3/dc4/ff8 [0,4194304] 0 2024-04-02T04:48:03.903 INFO:tasks.workunit.client.0.smithi070.stdout:6/689: dwrite d0/d55/d8b/d94/fbd [0,4194304] 0 2024-04-02T04:48:03.903 INFO:tasks.workunit.client.1.smithi179.stdout:3/894: creat d8/d26/dcd/de7/f120 x:0 0 0 2024-04-02T04:48:03.906 INFO:tasks.workunit.client.1.smithi179.stdout:1/869: fsync d3/d5/d96/f119 0 2024-04-02T04:48:03.907 INFO:tasks.workunit.client.1.smithi179.stdout:1/870: chown d3/d85/ld0 3 1 2024-04-02T04:48:03.910 INFO:tasks.workunit.client.1.smithi179.stdout:4/885: unlink d1/d3/d19/la9 0 2024-04-02T04:48:03.910 INFO:tasks.workunit.client.1.smithi179.stdout:0/954: write d1/da/f3c [461231,23089] 0 2024-04-02T04:48:03.912 INFO:tasks.workunit.client.1.smithi179.stdout:4/886: chown d1/d15/d6e/l96 114340990 1 2024-04-02T04:48:03.913 INFO:tasks.workunit.client.1.smithi179.stdout:4/887: fsync d1/d3/df/d10/d2f/d60/fff 0 2024-04-02T04:48:03.916 INFO:tasks.workunit.client.1.smithi179.stdout:9/858: creat d0/d1/d11/d29/d86/db0/f107 x:0 0 0 2024-04-02T04:48:03.920 INFO:tasks.workunit.client.1.smithi179.stdout:2/751: read - dc/d29/d31/d83/dae/db0/dbc/f65 zero size 2024-04-02T04:48:03.920 INFO:tasks.workunit.client.1.smithi179.stdout:9/859: dread d0/d1/d11/d29/f50 [0,4194304] 0 2024-04-02T04:48:03.920 INFO:tasks.workunit.client.1.smithi179.stdout:7/888: dread d1/d9/f6e [0,4194304] 0 2024-04-02T04:48:03.922 INFO:tasks.workunit.client.1.smithi179.stdout:6/859: mkdir d3/d12/d23/d137 0 2024-04-02T04:48:03.926 INFO:tasks.workunit.client.1.smithi179.stdout:1/871: read d3/d5/d2f/f81 [3398504,5118] 0 2024-04-02T04:48:03.927 INFO:tasks.workunit.client.1.smithi179.stdout:7/889: dwrite d1/d9/d27/d87/d8b/d75/db9/ddd/f12d [0,4194304] 0 2024-04-02T04:48:03.929 INFO:tasks.workunit.client.1.smithi179.stdout:3/895: read d8/d26/f62 [1404395,60017] 0 2024-04-02T04:48:03.933 INFO:tasks.workunit.client.1.smithi179.stdout:4/888: mknod d1/d3/dc/d27/dae/c12f 0 2024-04-02T04:48:03.938 INFO:tasks.workunit.client.1.smithi179.stdout:2/752: creat dc/d29/de7/ffe x:0 0 0 2024-04-02T04:48:03.941 INFO:tasks.workunit.client.1.smithi179.stdout:5/831: creat d8/d1b/d23/d30/d73/db3/f10b x:0 0 0 2024-04-02T04:48:03.944 INFO:tasks.workunit.client.1.smithi179.stdout:9/860: truncate d0/d19/d27/d7a/f8e 237417 0 2024-04-02T04:48:03.946 INFO:tasks.workunit.client.1.smithi179.stdout:6/860: write d3/d12/d44/d10a/d10e/f12a [320587,42395] 0 2024-04-02T04:48:03.948 INFO:tasks.workunit.client.0.smithi070.stdout:7/746: creat d7/dd/df/d7a/ff2 x:0 0 0 2024-04-02T04:48:03.951 INFO:tasks.workunit.client.0.smithi070.stdout:9/739: unlink d4/d10/d12/d56/d79/dd8/fe3 0 2024-04-02T04:48:03.954 INFO:tasks.workunit.client.1.smithi179.stdout:8/881: sync 2024-04-02T04:48:03.954 INFO:tasks.workunit.client.0.smithi070.stdout:0/848: dread - d4/d5/d13/d2a/d8d/fd3 zero size 2024-04-02T04:48:03.964 INFO:tasks.workunit.client.0.smithi070.stdout:7/747: rename d7/d36/f62 to d7/dd/df/d34/d68/ff3 0 2024-04-02T04:48:03.966 INFO:tasks.workunit.client.1.smithi179.stdout:2/753: creat dc/d29/d31/d57/d97/dbf/de3/fff x:0 0 0 2024-04-02T04:48:03.967 INFO:tasks.workunit.client.0.smithi070.stdout:9/740: rmdir d4/d10/d11/d7b/d19/d2f/d4d 39 2024-04-02T04:48:03.969 INFO:tasks.workunit.client.1.smithi179.stdout:5/832: truncate d8/d1b/d23/d32/f99 4837421 0 2024-04-02T04:48:03.971 INFO:tasks.workunit.client.0.smithi070.stdout:0/849: creat d4/d5/d13/d17/d98/dd2/f104 x:0 0 0 2024-04-02T04:48:03.972 INFO:tasks.workunit.client.1.smithi179.stdout:9/861: mkdir d0/d19/d27/d7a/d7e/db9/d108 0 2024-04-02T04:48:03.973 INFO:tasks.workunit.client.1.smithi179.stdout:6/861: creat d3/d2d/d43/f138 x:0 0 0 2024-04-02T04:48:03.975 INFO:tasks.workunit.client.0.smithi070.stdout:6/690: link d0/d20/f23 d0/d43/db3/dc7/d7b/da6/fd3 0 2024-04-02T04:48:03.976 INFO:tasks.workunit.client.1.smithi179.stdout:0/955: creat d1/da/d67/f11a x:0 0 0 2024-04-02T04:48:03.979 INFO:tasks.workunit.client.1.smithi179.stdout:7/890: unlink d1/d9/d7e/f12b 0 2024-04-02T04:48:03.982 INFO:tasks.workunit.client.0.smithi070.stdout:9/741: mknod d4/d13/df0/dd3/cf2 0 2024-04-02T04:48:03.985 INFO:tasks.workunit.client.0.smithi070.stdout:0/850: creat d4/d3c/f105 x:0 0 0 2024-04-02T04:48:03.990 INFO:tasks.workunit.client.1.smithi179.stdout:4/889: getdents d1/d3/df/d10/d2f/d60/d71/dce/d118 0 2024-04-02T04:48:03.991 INFO:tasks.workunit.client.0.smithi070.stdout:6/691: mknod d0/d55/d8b/d94/cd4 0 2024-04-02T04:48:03.996 INFO:tasks.workunit.client.1.smithi179.stdout:5/833: rmdir d8/d1b/d23/d32/dbe/daf/dd4/d105 39 2024-04-02T04:48:03.999 INFO:tasks.workunit.client.0.smithi070.stdout:0/851: symlink d4/d5/d13/d2a/d3a/d80/da8/l106 0 2024-04-02T04:48:04.001 INFO:tasks.workunit.client.0.smithi070.stdout:4/782: getdents d1/d3/d85/d8e/dc8 0 2024-04-02T04:48:04.004 INFO:tasks.workunit.client.0.smithi070.stdout:6/692: mkdir d0/d20/d77/d98/dd5 0 2024-04-02T04:48:04.004 INFO:tasks.workunit.client.0.smithi070.stdout:7/748: write d7/dd/df/d79/d46/d65/f67 [431505,105039] 0 2024-04-02T04:48:04.010 INFO:tasks.workunit.client.1.smithi179.stdout:8/882: mknod d3/d32/d21/d22/d6b/db1/d104/d135/c13b 0 2024-04-02T04:48:04.010 INFO:tasks.workunit.client.1.smithi179.stdout:1/872: truncate d3/d5/d2f/d65/f92 3646612 0 2024-04-02T04:48:04.010 INFO:tasks.workunit.client.1.smithi179.stdout:8/883: stat d3/d32/d21/d50/d56/d94/d84 0 2024-04-02T04:48:04.015 INFO:tasks.workunit.client.0.smithi070.stdout:4/783: dwrite d1/d3/d9/d5e/f77 [4194304,4194304] 0 2024-04-02T04:48:04.015 INFO:tasks.workunit.client.1.smithi179.stdout:3/896: write d8/dd3/f8b [1228019,59273] 0 2024-04-02T04:48:04.016 INFO:tasks.workunit.client.0.smithi070.stdout:9/742: dread d4/d10/d12/d27/d60/f88 [0,4194304] 0 2024-04-02T04:48:04.017 INFO:tasks.workunit.client.0.smithi070.stdout:6/693: symlink d0/d3/d19/d31/d46/ld6 0 2024-04-02T04:48:04.020 INFO:tasks.workunit.client.0.smithi070.stdout:0/852: dread d4/d5/d58/d60/d83/fe4 [0,4194304] 0 2024-04-02T04:48:04.020 INFO:tasks.workunit.client.1.smithi179.stdout:5/834: chown d8/d1b/d23/d42/fcc 766379221 1 2024-04-02T04:48:04.022 INFO:tasks.workunit.client.1.smithi179.stdout:2/754: dwrite dc/d29/d4a/f53 [4194304,4194304] 0 2024-04-02T04:48:04.024 INFO:tasks.workunit.client.0.smithi070.stdout:4/784: rename d1/d3/d9/c1c to d1/de7/c109 0 2024-04-02T04:48:04.025 INFO:tasks.workunit.client.1.smithi179.stdout:7/891: write d1/dd/d23/dab/fdb [727780,111221] 0 2024-04-02T04:48:04.025 INFO:tasks.workunit.client.0.smithi070.stdout:4/785: fdatasync d1/d3/f62 0 2024-04-02T04:48:04.031 INFO:tasks.workunit.client.0.smithi070.stdout:7/749: write d7/dd/f38 [5657339,126808] 0 2024-04-02T04:48:04.031 INFO:tasks.workunit.client.1.smithi179.stdout:9/862: dread d0/d1/d11/d77/f83 [0,4194304] 0 2024-04-02T04:48:04.033 INFO:tasks.workunit.client.1.smithi179.stdout:8/884: creat d3/d32/d21/d49/df0/f13c x:0 0 0 2024-04-02T04:48:04.035 INFO:tasks.workunit.client.0.smithi070.stdout:7/750: dread d7/dd/df/d79/d46/fb2 [0,4194304] 0 2024-04-02T04:48:04.035 INFO:tasks.workunit.client.1.smithi179.stdout:1/873: mkdir d3/d5/d2f/d4f/d42/d53/d5b/d131 0 2024-04-02T04:48:04.036 INFO:tasks.workunit.client.0.smithi070.stdout:6/694: dread d0/d3/d19/d31/fb1 [0,4194304] 0 2024-04-02T04:48:04.038 INFO:tasks.workunit.client.1.smithi179.stdout:4/890: fsync d1/d3/f28 0 2024-04-02T04:48:04.040 INFO:tasks.workunit.client.0.smithi070.stdout:4/786: rename d1/d3/d9/f8b to d1/d3/d85/d8e/dc8/de2/f10a 0 2024-04-02T04:48:04.041 INFO:tasks.workunit.client.0.smithi070.stdout:4/787: write d1/d3/d85/ddb/f104 [594458,100919] 0 2024-04-02T04:48:04.041 INFO:tasks.workunit.client.0.smithi070.stdout:9/743: dread d4/d10/d12/d55/d66/f6d [0,4194304] 0 2024-04-02T04:48:04.044 INFO:tasks.workunit.client.1.smithi179.stdout:4/891: dwrite d1/d3/d19/fa3 [0,4194304] 0 2024-04-02T04:48:04.047 INFO:tasks.workunit.client.1.smithi179.stdout:4/892: write d1/d15/d6e/d9c/f116 [964461,40687] 0 2024-04-02T04:48:04.048 INFO:tasks.workunit.client.1.smithi179.stdout:3/897: dread d8/dd3/f97 [0,4194304] 0 2024-04-02T04:48:04.049 INFO:tasks.workunit.client.0.smithi070.stdout:9/744: dwrite d4/d10/d11/d7b/d19/d2f/fda [0,4194304] 0 2024-04-02T04:48:04.049 INFO:tasks.workunit.client.1.smithi179.stdout:3/898: write d8/d6d/ff3 [140686,127761] 0 2024-04-02T04:48:04.054 INFO:tasks.workunit.client.1.smithi179.stdout:3/899: dwrite d8/dff/f10d [0,4194304] 0 2024-04-02T04:48:04.070 INFO:tasks.workunit.client.0.smithi070.stdout:7/751: dread d7/dd/df/d7a/f64 [0,4194304] 0 2024-04-02T04:48:04.071 INFO:tasks.workunit.client.0.smithi070.stdout:7/752: chown d7/d4e/d82/d57/dce/lcf 12671051 1 2024-04-02T04:48:04.078 INFO:tasks.workunit.client.1.smithi179.stdout:5/835: rmdir d8/d1b/d23/d30/d73/db3 39 2024-04-02T04:48:04.081 INFO:tasks.workunit.client.1.smithi179.stdout:6/862: sync 2024-04-02T04:48:04.084 INFO:tasks.workunit.client.1.smithi179.stdout:0/956: dwrite d1/da/d28/d37/f3f [8388608,4194304] 0 2024-04-02T04:48:04.084 INFO:tasks.workunit.client.1.smithi179.stdout:6/863: write d3/d12/d23/d26/d57/fc3 [977064,35754] 0 2024-04-02T04:48:04.086 INFO:tasks.workunit.client.0.smithi070.stdout:0/853: dwrite d4/d5/d13/fc7 [0,4194304] 0 2024-04-02T04:48:04.086 INFO:tasks.workunit.client.1.smithi179.stdout:7/892: unlink d1/d9/d7e/f81 0 2024-04-02T04:48:04.087 INFO:tasks.workunit.client.0.smithi070.stdout:0/854: dread - d4/d3c/f105 zero size 2024-04-02T04:48:04.094 INFO:tasks.workunit.client.0.smithi070.stdout:9/745: fdatasync d4/d10/d12/d27/d83/f8e 0 2024-04-02T04:48:04.096 INFO:tasks.workunit.client.0.smithi070.stdout:4/788: dread d1/d3/d9/f2a [0,4194304] 0 2024-04-02T04:48:04.096 INFO:tasks.workunit.client.0.smithi070.stdout:6/695: write d0/d3/d19/d31/fb1 [3467662,115276] 0 2024-04-02T04:48:04.099 INFO:tasks.workunit.client.0.smithi070.stdout:7/753: rename d7/c43 to d7/dd/df/d34/cf4 0 2024-04-02T04:48:04.102 INFO:tasks.workunit.client.1.smithi179.stdout:2/755: dread dc/d29/d4a/d1e/f7b [0,4194304] 0 2024-04-02T04:48:04.103 INFO:tasks.workunit.client.1.smithi179.stdout:4/893: creat d1/d3/d19/f130 x:0 0 0 2024-04-02T04:48:04.104 INFO:tasks.workunit.client.0.smithi070.stdout:0/855: truncate d4/d5/d13/d2a/d3a/d45/fc8 2299098 0 2024-04-02T04:48:04.106 INFO:tasks.workunit.client.0.smithi070.stdout:9/746: truncate d4/f9f 944956 0 2024-04-02T04:48:04.107 INFO:tasks.workunit.client.1.smithi179.stdout:2/756: dwrite dc/d79/fa6 [4194304,4194304] 0 2024-04-02T04:48:04.112 INFO:tasks.workunit.client.1.smithi179.stdout:2/757: dread dc/d29/d4a/d1e/f7b [0,4194304] 0 2024-04-02T04:48:04.112 INFO:tasks.workunit.client.0.smithi070.stdout:0/856: mknod d4/d5/da/d2d/de7/c107 0 2024-04-02T04:48:04.113 INFO:tasks.workunit.client.1.smithi179.stdout:8/885: write d3/d32/d21/d50/d56/d94/d84/fa9 [661982,76041] 0 2024-04-02T04:48:04.114 INFO:tasks.workunit.client.0.smithi070.stdout:9/747: readlink d4/d10/d11/d7b/d19/la4 0 2024-04-02T04:48:04.117 INFO:tasks.workunit.client.1.smithi179.stdout:2/758: dwrite dc/ff1 [0,4194304] 0 2024-04-02T04:48:04.124 INFO:tasks.workunit.client.1.smithi179.stdout:2/759: dwrite dc/d29/d31/d83/dae/db0/ff3 [0,4194304] 0 2024-04-02T04:48:04.127 INFO:tasks.workunit.client.0.smithi070.stdout:9/748: stat d4/d10/d11/d7b/d19/lcb 0 2024-04-02T04:48:04.128 INFO:tasks.workunit.client.0.smithi070.stdout:7/754: write d7/dd/df/d79/f31 [1673489,82095] 0 2024-04-02T04:48:04.131 INFO:tasks.workunit.client.0.smithi070.stdout:6/696: dwrite d0/d55/d8b/d94/f53 [0,4194304] 0 2024-04-02T04:48:04.139 INFO:tasks.workunit.client.0.smithi070.stdout:4/789: getdents d1/d3 0 2024-04-02T04:48:04.139 INFO:tasks.workunit.client.0.smithi070.stdout:9/749: mknod d4/d10/d11/d7b/d19/d2f/d8c/cf3 0 2024-04-02T04:48:04.142 INFO:tasks.workunit.client.0.smithi070.stdout:0/857: creat d4/d5/d58/d60/f108 x:0 0 0 2024-04-02T04:48:04.142 INFO:tasks.workunit.client.0.smithi070.stdout:6/697: dread d0/d3/d19/d31/d46/d6b/f6d [0,4194304] 0 2024-04-02T04:48:04.144 INFO:tasks.workunit.client.0.smithi070.stdout:6/698: write d0/d55/d8b/f9e [1575839,74827] 0 2024-04-02T04:48:04.145 INFO:tasks.workunit.client.0.smithi070.stdout:6/699: readlink d0/d3/l18 0 2024-04-02T04:48:04.149 INFO:tasks.workunit.client.0.smithi070.stdout:9/750: rename d4/d10/d11/d7b/d19/d2f/d39/d57/c7f to d4/d10/d12/d27/cf4 0 2024-04-02T04:48:04.150 INFO:tasks.workunit.client.0.smithi070.stdout:9/751: dread - d4/d10/d11/d7b/d19/fc2 zero size 2024-04-02T04:48:04.152 INFO:tasks.workunit.client.0.smithi070.stdout:9/752: dread d4/d10/d12/d55/d66/f6d [0,4194304] 0 2024-04-02T04:48:04.155 INFO:tasks.workunit.client.0.smithi070.stdout:0/858: unlink d4/d5/d13/d17/d98/dd2/lee 0 2024-04-02T04:48:04.158 INFO:tasks.workunit.client.0.smithi070.stdout:9/753: dwrite d4/d10/d11/d7b/d19/d2f/d39/f44 [0,4194304] 0 2024-04-02T04:48:04.163 INFO:tasks.workunit.client.0.smithi070.stdout:4/790: link d1/d3/l6 d1/de7/l10b 0 2024-04-02T04:48:04.168 INFO:tasks.workunit.client.0.smithi070.stdout:7/755: getdents d7/dd/df/d7a 0 2024-04-02T04:48:04.170 INFO:tasks.workunit.client.0.smithi070.stdout:9/754: truncate d4/d10/d12/d27/f5c 3025722 0 2024-04-02T04:48:04.173 INFO:tasks.workunit.client.0.smithi070.stdout:0/859: truncate d4/d5/d13/d43/f53 13164 0 2024-04-02T04:48:04.181 INFO:tasks.workunit.client.0.smithi070.stdout:9/755: rmdir d4/d10/d12/d27/d60 39 2024-04-02T04:48:04.188 INFO:tasks.workunit.client.0.smithi070.stdout:9/756: dwrite d4/d10/d12/fb3 [0,4194304] 0 2024-04-02T04:48:04.188 INFO:tasks.workunit.client.0.smithi070.stdout:4/791: truncate d1/d3/f62 2871338 0 2024-04-02T04:48:04.190 INFO:tasks.workunit.client.0.smithi070.stdout:4/792: chown d1/d9c/l56 108068205 1 2024-04-02T04:48:04.192 INFO:tasks.workunit.client.0.smithi070.stdout:7/756: unlink d7/d36/f72 0 2024-04-02T04:48:04.195 INFO:tasks.workunit.client.0.smithi070.stdout:6/700: dwrite d0/d3/d19/f3f [0,4194304] 0 2024-04-02T04:48:04.200 INFO:tasks.workunit.client.0.smithi070.stdout:0/860: truncate d4/d5/da/f63 1161089 0 2024-04-02T04:48:04.204 INFO:tasks.workunit.client.0.smithi070.stdout:0/861: write d4/d5/f3f [1818548,120409] 0 2024-04-02T04:48:04.205 INFO:tasks.workunit.client.1.smithi179.stdout:0/957: mknod d1/da/df/ded/c11b 0 2024-04-02T04:48:04.205 INFO:tasks.workunit.client.0.smithi070.stdout:4/793: dwrite d1/d3/d85/d8e/fc5 [0,4194304] 0 2024-04-02T04:48:04.207 INFO:tasks.workunit.client.1.smithi179.stdout:6/864: mkdir d3/d12/d23/d2f/d120/d139 0 2024-04-02T04:48:04.208 INFO:tasks.workunit.client.1.smithi179.stdout:6/865: truncate d3/d2d/f129 424981 0 2024-04-02T04:48:04.208 INFO:tasks.workunit.client.0.smithi070.stdout:7/757: dwrite d7/ff1 [0,4194304] 0 2024-04-02T04:48:04.209 INFO:tasks.workunit.client.0.smithi070.stdout:9/757: dread d4/d10/f2c [0,4194304] 0 2024-04-02T04:48:04.209 INFO:tasks.workunit.client.1.smithi179.stdout:6/866: write d3/d12/d44/fbb [832352,53152] 0 2024-04-02T04:48:04.210 INFO:tasks.workunit.client.0.smithi070.stdout:4/794: chown d1/d9c/f88 785 1 2024-04-02T04:48:04.211 INFO:tasks.workunit.client.1.smithi179.stdout:6/867: write d3/d12/d23/d2f/f3c [4665749,28501] 0 2024-04-02T04:48:04.211 INFO:tasks.workunit.client.1.smithi179.stdout:7/893: fsync d1/d9/fd0 0 2024-04-02T04:48:04.212 INFO:tasks.workunit.client.0.smithi070.stdout:4/795: chown d1/d9c/d65/l7e 6 1 2024-04-02T04:48:04.212 INFO:tasks.workunit.client.1.smithi179.stdout:6/868: readlink d3/d12/d23/d26/l101 0 2024-04-02T04:48:04.213 INFO:tasks.workunit.client.1.smithi179.stdout:6/869: dread - d3/d12/d23/d39/d80/d93/dd4/fb1 zero size 2024-04-02T04:48:04.214 INFO:tasks.workunit.client.1.smithi179.stdout:9/863: creat d0/d1/d11/dab/df1/f109 x:0 0 0 2024-04-02T04:48:04.215 INFO:tasks.workunit.client.0.smithi070.stdout:7/758: truncate d7/dd/df/d7a/dbc/fed 420663 0 2024-04-02T04:48:04.215 INFO:tasks.workunit.client.1.smithi179.stdout:9/864: fsync d0/d1/d11/dab/ded/f100 0 2024-04-02T04:48:04.224 INFO:tasks.workunit.client.0.smithi070.stdout:9/758: dwrite d4/d10/d11/d7b/d19/d53/f59 [0,4194304] 0 2024-04-02T04:48:04.224 INFO:tasks.workunit.client.0.smithi070.stdout:4/796: dread - d1/d3/d85/d8e/d99/db1/dbb/fec zero size 2024-04-02T04:48:04.234 INFO:tasks.workunit.client.1.smithi179.stdout:5/836: dwrite d8/d1b/d23/d36/d53/d62/da9/fe2 [0,4194304] 0 2024-04-02T04:48:04.240 INFO:tasks.workunit.client.1.smithi179.stdout:5/837: dwrite d8/d1b/d23/d42/fcc [0,4194304] 0 2024-04-02T04:48:04.245 INFO:tasks.workunit.client.0.smithi070.stdout:7/759: creat d7/dd/df/d7a/d6c/d77/d75/ff5 x:0 0 0 2024-04-02T04:48:04.247 INFO:tasks.workunit.client.0.smithi070.stdout:4/797: mknod d1/d98/c10c 0 2024-04-02T04:48:04.249 INFO:tasks.workunit.client.1.smithi179.stdout:2/760: mkdir dc/d29/d31/d57/dea/d100 0 2024-04-02T04:48:04.254 INFO:tasks.workunit.client.1.smithi179.stdout:0/958: mkdir d1/da/d52/d11c 0 2024-04-02T04:48:04.255 INFO:tasks.workunit.client.0.smithi070.stdout:4/798: dwrite d1/d9c/d65/d7a/f9f [0,4194304] 0 2024-04-02T04:48:04.265 INFO:tasks.workunit.client.0.smithi070.stdout:9/759: rename d4/d10/d11/da8 to d4/d13/d89/df5 0 2024-04-02T04:48:04.267 INFO:tasks.workunit.client.1.smithi179.stdout:7/894: rmdir d1/dd 39 2024-04-02T04:48:04.270 INFO:tasks.workunit.client.1.smithi179.stdout:6/870: creat d3/d12/d44/f13a x:0 0 0 2024-04-02T04:48:04.271 INFO:tasks.workunit.client.0.smithi070.stdout:6/701: rmdir d0/d43/d95/dd2 0 2024-04-02T04:48:04.271 INFO:tasks.workunit.client.0.smithi070.stdout:6/702: chown d0/d55/d8b 2 1 2024-04-02T04:48:04.273 INFO:tasks.workunit.client.0.smithi070.stdout:0/862: truncate d4/d5/da/fc9 450081 0 2024-04-02T04:48:04.276 INFO:tasks.workunit.client.1.smithi179.stdout:9/865: rmdir d0/d1/d10/d6d 39 2024-04-02T04:48:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:03 smithi070 ceph-mon[29212]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:48:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:03 smithi070 ceph-mon[29212]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:48:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:03 smithi070 ceph-mon[29212]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:48:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:03 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:03 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:03 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:03 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:03 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.283 INFO:tasks.workunit.client.0.smithi070.stdout:4/799: dread d1/d3/d9/f53 [0,4194304] 0 2024-04-02T04:48:04.287 INFO:tasks.workunit.client.0.smithi070.stdout:0/863: creat d4/d3c/f109 x:0 0 0 2024-04-02T04:48:04.294 INFO:tasks.workunit.client.0.smithi070.stdout:9/760: truncate d4/d10/d12/d27/d60/dc9/fd5 845800 0 2024-04-02T04:48:04.296 INFO:tasks.workunit.client.0.smithi070.stdout:9/761: chown d4/d10/d12/caa 49 1 2024-04-02T04:48:04.297 INFO:tasks.workunit.client.0.smithi070.stdout:4/800: mkdir d1/dc3/dc4/d10d 0 2024-04-02T04:48:04.297 INFO:tasks.workunit.client.0.smithi070.stdout:7/760: dwrite d7/dd/df/d79/d46/d65/f67 [0,4194304] 0 2024-04-02T04:48:04.298 INFO:tasks.workunit.client.0.smithi070.stdout:4/801: chown d1/d3/d85/d8e/fc6 3816256 1 2024-04-02T04:48:04.299 INFO:tasks.workunit.client.1.smithi179.stdout:4/894: truncate d1/d15/d6e/f80 703386 0 2024-04-02T04:48:04.301 INFO:tasks.workunit.client.0.smithi070.stdout:4/802: truncate d1/d3/d85/d8e/d99/db1/fef 708528 0 2024-04-02T04:48:04.302 INFO:tasks.workunit.client.1.smithi179.stdout:8/886: truncate d3/d32/d21/d50/d56/d94/d84/fe9 3298690 0 2024-04-02T04:48:04.303 INFO:tasks.workunit.client.1.smithi179.stdout:0/959: symlink d1/da/df/l11d 0 2024-04-02T04:48:04.310 INFO:tasks.workunit.client.1.smithi179.stdout:1/874: getdents d3/d5/d96 0 2024-04-02T04:48:04.311 INFO:tasks.workunit.client.0.smithi070.stdout:7/761: creat d7/dd/df/d7a/d6c/d77/d75/d80/ff6 x:0 0 0 2024-04-02T04:48:04.312 INFO:tasks.workunit.client.1.smithi179.stdout:2/761: dwrite dc/d29/d4a/faa [0,4194304] 0 2024-04-02T04:48:04.313 INFO:tasks.workunit.client.1.smithi179.stdout:3/900: link d8/d57/d63/lf5 d8/d6d/l121 0 2024-04-02T04:48:04.317 INFO:tasks.workunit.client.1.smithi179.stdout:5/838: mkdir d8/d1b/d23/d42/dde/d10c 0 2024-04-02T04:48:04.319 INFO:tasks.workunit.client.0.smithi070.stdout:9/762: write d4/d13/fbd [372628,53859] 0 2024-04-02T04:48:04.320 INFO:tasks.workunit.client.0.smithi070.stdout:6/703: getdents d0/d43/db3/dc7/d7b 0 2024-04-02T04:48:04.321 INFO:tasks.workunit.client.0.smithi070.stdout:9/763: read d4/d10/d12/d27/f97 [3026816,107543] 0 2024-04-02T04:48:04.322 INFO:tasks.workunit.client.1.smithi179.stdout:3/901: dwrite d8/dd3/f8b [0,4194304] 0 2024-04-02T04:48:04.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:03 smithi179 ceph-mon[36603]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:48:04.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:03 smithi179 ceph-mon[36603]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:48:04.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:03 smithi179 ceph-mon[36603]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:48:04.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:03 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:03 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:03 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:03 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:03 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:04.323 INFO:tasks.workunit.client.0.smithi070.stdout:4/803: write d1/d3/d9/f29 [3027550,28198] 0 2024-04-02T04:48:04.326 INFO:tasks.workunit.client.1.smithi179.stdout:7/895: mkdir d1/dd/d76/d13c 0 2024-04-02T04:48:04.327 INFO:tasks.workunit.client.1.smithi179.stdout:8/887: dread d3/d32/d21/d50/d56/d94/d84/fbd [0,4194304] 0 2024-04-02T04:48:04.329 INFO:tasks.workunit.client.1.smithi179.stdout:7/896: write d1/d9/d27/d87/d8b/d75/fbd [667964,12858] 0 2024-04-02T04:48:04.329 INFO:tasks.workunit.client.0.smithi070.stdout:6/704: creat d0/d43/db3/fd7 x:0 0 0 2024-04-02T04:48:04.330 INFO:tasks.workunit.client.0.smithi070.stdout:9/764: creat d4/d10/d11/d7b/d19/d2f/d39/d57/da6/ff6 x:0 0 0 2024-04-02T04:48:04.332 INFO:tasks.workunit.client.1.smithi179.stdout:1/875: rmdir d3/d5/d2f/d4f 39 2024-04-02T04:48:04.333 INFO:tasks.workunit.client.0.smithi070.stdout:0/864: getdents d4/d5/da/d2d 0 2024-04-02T04:48:04.334 INFO:tasks.workunit.client.1.smithi179.stdout:2/762: rmdir dc/d29/d4a/d1e/d6c 39 2024-04-02T04:48:04.335 INFO:tasks.workunit.client.1.smithi179.stdout:2/763: chown dc/f89 0 1 2024-04-02T04:48:04.336 INFO:tasks.workunit.client.0.smithi070.stdout:4/804: dread d1/f12 [0,4194304] 0 2024-04-02T04:48:04.337 INFO:tasks.workunit.client.0.smithi070.stdout:4/805: chown d1/dc3/ff6 18713 1 2024-04-02T04:48:04.338 INFO:tasks.workunit.client.0.smithi070.stdout:6/705: mkdir d0/d20/dd8 0 2024-04-02T04:48:04.339 INFO:tasks.workunit.client.0.smithi070.stdout:6/706: truncate d0/d3/d19/d31/dc1/fc8 556143 0 2024-04-02T04:48:04.340 INFO:tasks.workunit.client.1.smithi179.stdout:2/764: dwrite dc/d29/d31/d57/d97/fc2 [0,4194304] 0 2024-04-02T04:48:04.343 INFO:tasks.workunit.client.0.smithi070.stdout:9/765: rename d4/d10/d11/d7b/d19/f48 to d4/ff7 0 2024-04-02T04:48:04.344 INFO:tasks.workunit.client.1.smithi179.stdout:4/895: mkdir d1/d3/df/d106/d131 0 2024-04-02T04:48:04.344 INFO:tasks.workunit.client.1.smithi179.stdout:4/896: fsync d1/d3/dc/f1c 0 2024-04-02T04:48:04.345 INFO:tasks.workunit.client.0.smithi070.stdout:0/865: unlink d4/d5/d21/f8c 0 2024-04-02T04:48:04.347 INFO:tasks.workunit.client.1.smithi179.stdout:0/960: write d1/db/fd7 [286924,72399] 0 2024-04-02T04:48:04.351 INFO:tasks.workunit.client.0.smithi070.stdout:7/762: getdents d7/dd/df/d7a/d6c/d77/d75/d80 0 2024-04-02T04:48:04.353 INFO:tasks.workunit.client.0.smithi070.stdout:6/707: creat d0/d20/d5f/fd9 x:0 0 0 2024-04-02T04:48:04.355 INFO:tasks.workunit.client.0.smithi070.stdout:9/766: rename d4/d10/d11/d7b/l1f to d4/d10/d12/d55/d66/lf8 0 2024-04-02T04:48:04.359 INFO:tasks.workunit.client.0.smithi070.stdout:0/866: creat d4/d5/d13/d2a/d3a/d45/f10a x:0 0 0 2024-04-02T04:48:04.360 INFO:tasks.workunit.client.0.smithi070.stdout:9/767: dread d4/d10/d11/f4e [4194304,4194304] 0 2024-04-02T04:48:04.360 INFO:tasks.workunit.client.1.smithi179.stdout:8/888: creat d3/d32/d21/d22/d6b/f13d x:0 0 0 2024-04-02T04:48:04.363 INFO:tasks.workunit.client.1.smithi179.stdout:9/866: creat d0/d19/d27/d7a/f10a x:0 0 0 2024-04-02T04:48:04.364 INFO:tasks.workunit.client.1.smithi179.stdout:3/902: dread d8/d26/d83/d89/fab [0,4194304] 0 2024-04-02T04:48:04.364 INFO:tasks.workunit.client.1.smithi179.stdout:9/867: readlink d0/d1/d11/d3a/l9c 0 2024-04-02T04:48:04.365 INFO:tasks.workunit.client.1.smithi179.stdout:5/839: write d8/d1b/d23/d32/dbe/daf/fbf [2908847,4787] 0 2024-04-02T04:48:04.366 INFO:tasks.workunit.client.1.smithi179.stdout:7/897: creat d1/d9/d27/d87/d8b/d75/db9/dcf/f13d x:0 0 0 2024-04-02T04:48:04.367 INFO:tasks.workunit.client.1.smithi179.stdout:0/961: dread d1/db/f8e [0,4194304] 0 2024-04-02T04:48:04.368 INFO:tasks.workunit.client.0.smithi070.stdout:0/867: unlink d4/d5/d13/d17/d98/fc0 0 2024-04-02T04:48:04.370 INFO:tasks.workunit.client.0.smithi070.stdout:7/763: dwrite d7/dd/df/d7a/d6c/d77/d75/fb3 [0,4194304] 0 2024-04-02T04:48:04.370 INFO:tasks.workunit.client.0.smithi070.stdout:0/868: truncate d4/d5/f84 2568272 0 2024-04-02T04:48:04.371 INFO:tasks.workunit.client.1.smithi179.stdout:9/868: dread d0/d1/f7b [4194304,4194304] 0 2024-04-02T04:48:04.371 INFO:tasks.workunit.client.1.smithi179.stdout:0/962: dread d1/da/f2d [0,4194304] 0 2024-04-02T04:48:04.372 INFO:tasks.workunit.client.0.smithi070.stdout:9/768: fsync d4/d10/f25 0 2024-04-02T04:48:04.372 INFO:tasks.workunit.client.1.smithi179.stdout:0/963: dread - d1/da/d52/d75/d9a/fe9 zero size 2024-04-02T04:48:04.373 INFO:tasks.workunit.client.1.smithi179.stdout:1/876: chown d3/d5/d2f/d4f/d42/d53/d72/d86/f10f 1080 1 2024-04-02T04:48:04.380 INFO:tasks.workunit.client.1.smithi179.stdout:4/897: dread d1/d15/d6e/f80 [0,4194304] 0 2024-04-02T04:48:04.381 INFO:tasks.workunit.client.0.smithi070.stdout:0/869: creat d4/d5/d13/d2a/d8d/dfc/f10b x:0 0 0 2024-04-02T04:48:04.386 INFO:tasks.workunit.client.0.smithi070.stdout:4/806: getdents d1/d9c 0 2024-04-02T04:48:04.390 INFO:tasks.workunit.client.0.smithi070.stdout:9/769: rename d4/d34/fdc to d4/d13/df0/ff9 0 2024-04-02T04:48:04.392 INFO:tasks.workunit.client.1.smithi179.stdout:3/903: creat d8/d7a/f122 x:0 0 0 2024-04-02T04:48:04.392 INFO:tasks.workunit.client.1.smithi179.stdout:3/904: readlink d8/d26/d83/l102 0 2024-04-02T04:48:04.393 INFO:tasks.workunit.client.0.smithi070.stdout:7/764: mkdir d7/dd/db5/df7 0 2024-04-02T04:48:04.400 INFO:tasks.workunit.client.0.smithi070.stdout:4/807: stat d1/l6c 0 2024-04-02T04:48:04.402 INFO:tasks.workunit.client.1.smithi179.stdout:7/898: chown d1/d9/d27/d87/l95 383896 1 2024-04-02T04:48:04.404 INFO:tasks.workunit.client.1.smithi179.stdout:8/889: write d3/d32/d21/d22/d6b/db1/fbe [1097811,23010] 0 2024-04-02T04:48:04.406 INFO:tasks.workunit.client.0.smithi070.stdout:7/765: rename d7/dd/l9d to d7/dd/df/d7a/d6c/d77/d54/lf8 0 2024-04-02T04:48:04.407 INFO:tasks.workunit.client.1.smithi179.stdout:6/871: link d3/d12/d6a/fc2 d3/d12/d23/d39/d80/d93/dd0/f13b 0 2024-04-02T04:48:04.410 INFO:tasks.workunit.client.0.smithi070.stdout:0/870: write d4/d5/d13/d2a/d29/f5d [1278102,106151] 0 2024-04-02T04:48:04.411 INFO:tasks.workunit.client.1.smithi179.stdout:9/869: creat d0/d1/d10/de3/f10b x:0 0 0 2024-04-02T04:48:04.412 INFO:tasks.workunit.client.1.smithi179.stdout:9/870: fdatasync d0/d1/d10/de3/f10b 0 2024-04-02T04:48:04.412 INFO:tasks.workunit.client.0.smithi070.stdout:4/808: dread d1/f26 [0,4194304] 0 2024-04-02T04:48:04.413 INFO:tasks.workunit.client.1.smithi179.stdout:0/964: unlink d1/da/d28/l101 0 2024-04-02T04:48:04.414 INFO:tasks.workunit.client.0.smithi070.stdout:7/766: creat d7/dd/df/d7a/dbc/ff9 x:0 0 0 2024-04-02T04:48:04.416 INFO:tasks.workunit.client.1.smithi179.stdout:4/898: write d1/d3/f1b [4800774,57154] 0 2024-04-02T04:48:04.419 INFO:tasks.workunit.client.0.smithi070.stdout:9/770: rmdir d4/d13/df0/dd3/df1 0 2024-04-02T04:48:04.420 INFO:tasks.workunit.client.0.smithi070.stdout:4/809: mknod d1/d3/d85/d8e/d99/db1/c10e 0 2024-04-02T04:48:04.422 INFO:tasks.workunit.client.0.smithi070.stdout:4/810: chown d1/dc3/cd5 7887 1 2024-04-02T04:48:04.424 INFO:tasks.workunit.client.1.smithi179.stdout:3/905: rename d8/l35 to d8/dd3/dbb/d106/l123 0 2024-04-02T04:48:04.427 INFO:tasks.workunit.client.1.smithi179.stdout:9/871: dread d0/d1/d11/dab/ded/f69 [0,4194304] 0 2024-04-02T04:48:04.428 INFO:tasks.workunit.client.0.smithi070.stdout:9/771: mknod d4/d10/d11/d7b/d19/d53/cfa 0 2024-04-02T04:48:04.429 INFO:tasks.workunit.client.0.smithi070.stdout:9/772: readlink d4/d10/d11/l75 0 2024-04-02T04:48:04.430 INFO:tasks.workunit.client.0.smithi070.stdout:4/811: dwrite d1/d9c/f102 [0,4194304] 0 2024-04-02T04:48:04.430 INFO:tasks.workunit.client.0.smithi070.stdout:7/767: dread d7/dd/df/d79/d46/d65/f8c [0,4194304] 0 2024-04-02T04:48:04.430 INFO:tasks.workunit.client.1.smithi179.stdout:5/840: symlink d8/d1b/d23/d36/d53/d62/l10d 0 2024-04-02T04:48:04.432 INFO:tasks.workunit.client.0.smithi070.stdout:9/773: write d4/d10/d11/f42 [763109,83638] 0 2024-04-02T04:48:04.437 INFO:tasks.workunit.client.1.smithi179.stdout:6/872: creat d3/d2c/de0/f13c x:0 0 0 2024-04-02T04:48:04.438 INFO:tasks.workunit.client.0.smithi070.stdout:4/812: write d1/d3/d9/f9b [3395230,91163] 0 2024-04-02T04:48:04.439 INFO:tasks.workunit.client.0.smithi070.stdout:7/768: dread - d7/d4e/f89 zero size 2024-04-02T04:48:04.441 INFO:tasks.workunit.client.0.smithi070.stdout:4/813: dread - d1/d3/d85/d8e/d99/fd4 zero size 2024-04-02T04:48:04.442 INFO:tasks.workunit.client.0.smithi070.stdout:0/871: dwrite d4/d5/d13/d17/d6e/fbc [0,4194304] 0 2024-04-02T04:48:04.443 INFO:tasks.workunit.client.0.smithi070.stdout:4/814: read - d1/d3/d78/dad/f107 zero size 2024-04-02T04:48:04.445 INFO:tasks.workunit.client.0.smithi070.stdout:0/872: chown d4/d5/d58/d60/d83/ff4 32250703 1 2024-04-02T04:48:04.445 INFO:tasks.workunit.client.0.smithi070.stdout:0/873: chown d4/d5/da/d2d 3 1 2024-04-02T04:48:04.446 INFO:tasks.workunit.client.1.smithi179.stdout:1/877: mkdir d3/d85/d132 0 2024-04-02T04:48:04.448 INFO:tasks.workunit.client.0.smithi070.stdout:0/874: chown d4/d5/d13/d2a/f20 4608625 1 2024-04-02T04:48:04.454 INFO:tasks.workunit.client.1.smithi179.stdout:7/899: dwrite d1/d9/d55/f61 [0,4194304] 0 2024-04-02T04:48:04.454 INFO:tasks.workunit.client.0.smithi070.stdout:4/815: rename d1/d3/d9/f1a to d1/dc3/dc4/d10d/f10f 0 2024-04-02T04:48:04.456 INFO:tasks.workunit.client.0.smithi070.stdout:0/875: symlink d4/d5/d13/d2a/d3a/d80/da8/l10c 0 2024-04-02T04:48:04.461 INFO:tasks.workunit.client.0.smithi070.stdout:9/774: write d4/d13/f99 [540325,13874] 0 2024-04-02T04:48:04.463 INFO:tasks.workunit.client.1.smithi179.stdout:5/841: chown d8/d1b/d23/db1/f102 199373 1 2024-04-02T04:48:04.469 INFO:tasks.workunit.client.0.smithi070.stdout:7/769: dwrite d7/d4e/f89 [0,4194304] 0 2024-04-02T04:48:04.469 INFO:tasks.workunit.client.1.smithi179.stdout:6/873: rmdir d3/d12/d44/df1 39 2024-04-02T04:48:04.470 INFO:tasks.workunit.client.1.smithi179.stdout:3/906: write d8/d6d/fb6 [1458170,99611] 0 2024-04-02T04:48:04.471 INFO:tasks.workunit.client.0.smithi070.stdout:0/876: dwrite d4/d5/d58/d60/d83/fd9 [0,4194304] 0 2024-04-02T04:48:04.473 INFO:tasks.workunit.client.0.smithi070.stdout:4/816: mkdir d1/d3/d85/deb/d110 0 2024-04-02T04:48:04.474 INFO:tasks.workunit.client.0.smithi070.stdout:7/770: dread d7/dd/df/d79/d46/fb2 [0,4194304] 0 2024-04-02T04:48:04.475 INFO:tasks.workunit.client.0.smithi070.stdout:7/771: fdatasync d7/dd/df/d34/f2a 0 2024-04-02T04:48:04.476 INFO:tasks.workunit.client.0.smithi070.stdout:7/772: stat d7/dd/df/d34/de3 0 2024-04-02T04:48:04.477 INFO:tasks.workunit.client.0.smithi070.stdout:9/775: mkdir d4/d10/d11/d7b/d19/d2f/d39/d57/dfb 0 2024-04-02T04:48:04.477 INFO:tasks.workunit.client.1.smithi179.stdout:2/765: getdents dc/d29/d31/d57/d97/dbf/de3 0 2024-04-02T04:48:04.478 INFO:tasks.workunit.client.0.smithi070.stdout:4/817: mknod d1/d98/c111 0 2024-04-02T04:48:04.479 INFO:tasks.workunit.client.1.smithi179.stdout:7/900: symlink d1/d9/d55/dbf/l13e 0 2024-04-02T04:48:04.482 INFO:tasks.workunit.client.1.smithi179.stdout:9/872: mknod d0/d1/d10/d6d/d58/c10c 0 2024-04-02T04:48:04.485 INFO:tasks.workunit.client.0.smithi070.stdout:7/773: rmdir d7/dd/df/d79/d46/d65 39 2024-04-02T04:48:04.485 INFO:tasks.workunit.client.1.smithi179.stdout:8/890: link d3/d32/d21/d50/d56/d94/f60 d3/d31/dc3/f13e 0 2024-04-02T04:48:04.487 INFO:tasks.workunit.client.0.smithi070.stdout:9/776: read d4/d10/d12/d27/f5c [2196654,42911] 0 2024-04-02T04:48:04.488 INFO:tasks.workunit.client.1.smithi179.stdout:6/874: creat d3/d12/d23/d39/dff/f13d x:0 0 0 2024-04-02T04:48:04.488 INFO:tasks.workunit.client.0.smithi070.stdout:4/818: mknod d1/dc3/dc4/d10d/c112 0 2024-04-02T04:48:04.489 INFO:tasks.workunit.client.1.smithi179.stdout:3/907: mkdir d8/dff/d124 0 2024-04-02T04:48:04.491 INFO:tasks.workunit.client.0.smithi070.stdout:0/877: rename d4/d5/d21/cb4 to d4/d5/da/d2d/c10d 0 2024-04-02T04:48:04.492 INFO:tasks.workunit.client.0.smithi070.stdout:7/774: write d7/dd/df/d7a/f64 [9230005,75986] 0 2024-04-02T04:48:04.494 INFO:tasks.workunit.client.0.smithi070.stdout:9/777: fsync d4/d10/d11/f14 0 2024-04-02T04:48:04.494 INFO:tasks.workunit.client.0.smithi070.stdout:7/775: chown d7/dd/df/d7a/d6c/d77/d54/ld0 14 1 2024-04-02T04:48:04.496 INFO:tasks.workunit.client.1.smithi179.stdout:4/899: link d1/d15/d7c/db8/cbf d1/d93/d9f/c132 0 2024-04-02T04:48:04.498 INFO:tasks.workunit.client.0.smithi070.stdout:0/878: read d4/d5/da/d2d/f66 [6783,91442] 0 2024-04-02T04:48:04.499 INFO:tasks.workunit.client.1.smithi179.stdout:7/901: fsync d1/dd/fc3 0 2024-04-02T04:48:04.499 INFO:tasks.workunit.client.0.smithi070.stdout:0/879: dread - d4/d5/d13/d17/d98/dd2/f104 zero size 2024-04-02T04:48:04.502 INFO:tasks.workunit.client.1.smithi179.stdout:5/842: creat d8/d1b/d23/d32/d4e/ddd/de5/f10e x:0 0 0 2024-04-02T04:48:04.502 INFO:tasks.workunit.client.1.smithi179.stdout:4/900: dwrite d1/d3/dc/d27/d62/fe2 [0,4194304] 0 2024-04-02T04:48:04.503 INFO:tasks.workunit.client.1.smithi179.stdout:6/875: dread d3/d9a/fe7 [0,4194304] 0 2024-04-02T04:48:04.505 INFO:tasks.workunit.client.1.smithi179.stdout:9/873: rename d0/d1/d11/dab/ded/f100 to d0/d19/d27/d48/f10d 0 2024-04-02T04:48:04.505 INFO:tasks.workunit.client.1.smithi179.stdout:6/876: write d3/d2c/de0/f13c [1025794,27177] 0 2024-04-02T04:48:04.506 INFO:tasks.workunit.client.0.smithi070.stdout:4/819: rename d1/d3/c52 to d1/d3/d85/deb/d110/c113 0 2024-04-02T04:48:04.514 INFO:tasks.workunit.client.0.smithi070.stdout:0/880: symlink d4/d5/d13/d2a/d8d/dfc/l10e 0 2024-04-02T04:48:04.515 INFO:tasks.workunit.client.0.smithi070.stdout:0/881: write d4/d5/d13/d17/d6e/fbc [217468,59394] 0 2024-04-02T04:48:04.515 INFO:tasks.workunit.client.1.smithi179.stdout:3/908: mknod d8/d7a/c125 0 2024-04-02T04:48:04.517 INFO:tasks.workunit.client.0.smithi070.stdout:7/776: mkdir d7/d4e/d82/d57/ddc/dfa 0 2024-04-02T04:48:04.519 INFO:tasks.workunit.client.1.smithi179.stdout:1/878: creat d3/f133 x:0 0 0 2024-04-02T04:48:04.519 INFO:tasks.workunit.client.1.smithi179.stdout:2/766: dwrite dc/d29/fa3 [0,4194304] 0 2024-04-02T04:48:04.520 INFO:tasks.workunit.client.1.smithi179.stdout:1/879: readlink d3/d5/de/d33/l9e 0 2024-04-02T04:48:04.520 INFO:tasks.workunit.client.1.smithi179.stdout:0/965: link d1/db/d64/f6a d1/da/d48/d7b/f11e 0 2024-04-02T04:48:04.522 INFO:tasks.workunit.client.0.smithi070.stdout:4/820: fdatasync d1/d3/d9/f64 0 2024-04-02T04:48:04.523 INFO:tasks.workunit.client.0.smithi070.stdout:4/821: write d1/d3/f30 [998428,27231] 0 2024-04-02T04:48:04.529 INFO:tasks.workunit.client.1.smithi179.stdout:7/902: mknod d1/d9/d27/d87/d8b/d75/db9/dcf/c13f 0 2024-04-02T04:48:04.530 INFO:tasks.workunit.client.0.smithi070.stdout:0/882: truncate d4/d5/d13/d23/f36 405806 0 2024-04-02T04:48:04.531 INFO:tasks.workunit.client.0.smithi070.stdout:7/777: mknod d7/dd/df/dca/cfb 0 2024-04-02T04:48:04.532 INFO:tasks.workunit.client.1.smithi179.stdout:5/843: fsync d8/d1b/d23/d30/d73/f81 0 2024-04-02T04:48:04.535 INFO:tasks.workunit.client.1.smithi179.stdout:4/901: rename d1/c8 to d1/d3/df/d10/d2f/d60/da6/c133 0 2024-04-02T04:48:04.536 INFO:tasks.workunit.client.0.smithi070.stdout:9/778: truncate d4/d10/d11/d7b/d19/d82/f8a 3289179 0 2024-04-02T04:48:04.537 INFO:tasks.workunit.client.1.smithi179.stdout:6/877: fdatasync d3/d2d/f65 0 2024-04-02T04:48:04.539 INFO:tasks.workunit.client.0.smithi070.stdout:4/822: rename d1/d3/f95 to d1/d9c/df0/f114 0 2024-04-02T04:48:04.540 INFO:tasks.workunit.client.1.smithi179.stdout:9/874: mknod d0/d19/d27/d7a/d7e/dd4/c10e 0 2024-04-02T04:48:04.541 INFO:tasks.workunit.client.1.smithi179.stdout:9/875: write d0/d19/d27/fa7 [926584,42970] 0 2024-04-02T04:48:04.542 INFO:tasks.workunit.client.1.smithi179.stdout:9/876: chown d0/d1/d11/d29/d80/lf6 727697440 1 2024-04-02T04:48:04.547 INFO:tasks.workunit.client.1.smithi179.stdout:3/909: read - d8/d26/d83/fd7 zero size 2024-04-02T04:48:04.549 INFO:tasks.workunit.client.1.smithi179.stdout:2/767: chown dc/d29/d4a/d1e/d6c/lda 4638 1 2024-04-02T04:48:04.550 INFO:tasks.workunit.client.1.smithi179.stdout:0/966: mkdir d1/da/dc3/d11f 0 2024-04-02T04:48:04.551 INFO:tasks.workunit.client.1.smithi179.stdout:1/880: creat d3/d5/d2f/d4f/dab/dcd/f134 x:0 0 0 2024-04-02T04:48:04.553 INFO:tasks.workunit.client.1.smithi179.stdout:5/844: unlink d8/d1b/d23/f2b 0 2024-04-02T04:48:04.553 INFO:tasks.workunit.client.1.smithi179.stdout:3/910: dwrite d8/d26/d52/d7e/db8/ffa [0,4194304] 0 2024-04-02T04:48:04.553 INFO:tasks.workunit.client.1.smithi179.stdout:1/881: chown d3/d5/d2f/d36/c8c 16797 1 2024-04-02T04:48:04.553 INFO:tasks.workunit.client.1.smithi179.stdout:4/902: symlink d1/d15/d7c/d98/l134 0 2024-04-02T04:48:04.555 INFO:tasks.workunit.client.1.smithi179.stdout:3/911: read d8/d26/d36/fa2 [592750,69007] 0 2024-04-02T04:48:04.606 INFO:tasks.workunit.client.0.smithi070.stdout:8/761: sync 2024-04-02T04:48:04.607 INFO:tasks.workunit.client.0.smithi070.stdout:3/770: sync 2024-04-02T04:48:04.608 INFO:tasks.workunit.client.0.smithi070.stdout:9/779: rmdir d4/d10/d12/d56/d79/dd8 39 2024-04-02T04:48:04.610 INFO:tasks.workunit.client.0.smithi070.stdout:4/823: dwrite d1/dce/fd1 [0,4194304] 0 2024-04-02T04:48:04.612 INFO:tasks.workunit.client.0.smithi070.stdout:4/824: chown d1/d3/d9/f22 221984 1 2024-04-02T04:48:04.614 INFO:tasks.workunit.client.0.smithi070.stdout:0/883: creat d4/d5/d13/d2a/d3a/f10f x:0 0 0 2024-04-02T04:48:04.616 INFO:tasks.workunit.client.0.smithi070.stdout:4/825: read d1/d9c/f45 [2712284,67705] 0 2024-04-02T04:48:04.626 INFO:tasks.workunit.client.1.smithi179.stdout:1/882: mknod d3/da8/c135 0 2024-04-02T04:48:04.627 INFO:tasks.workunit.client.0.smithi070.stdout:9/780: dread d4/d10/d12/d27/d83/f8e [0,4194304] 0 2024-04-02T04:48:04.629 INFO:tasks.workunit.client.1.smithi179.stdout:6/878: mknod d3/d12/d23/d39/d80/d93/c13e 0 2024-04-02T04:48:04.629 INFO:tasks.workunit.client.1.smithi179.stdout:9/877: dread d0/d19/d27/fe0 [0,4194304] 0 2024-04-02T04:48:04.630 INFO:tasks.workunit.client.1.smithi179.stdout:5/845: dwrite d8/d1b/f98 [0,4194304] 0 2024-04-02T04:48:04.631 INFO:tasks.workunit.client.1.smithi179.stdout:4/903: fdatasync d1/d3/fa 0 2024-04-02T04:48:04.632 INFO:tasks.workunit.client.0.smithi070.stdout:7/778: dread d7/d36/fb9 [0,4194304] 0 2024-04-02T04:48:04.633 INFO:tasks.workunit.client.1.smithi179.stdout:4/904: truncate d1/d15/d7c/d98/f120 465227 0 2024-04-02T04:48:04.634 INFO:tasks.workunit.client.0.smithi070.stdout:3/771: rename d10/d17/f33 to d10/d17/d51/d73/d8f/f104 0 2024-04-02T04:48:04.635 INFO:tasks.workunit.client.0.smithi070.stdout:0/884: getdents d4/d5/d58/d60/d83/da5 0 2024-04-02T04:48:04.636 INFO:tasks.workunit.client.1.smithi179.stdout:8/891: getdents d3/d1b/d2a/d41/d109 0 2024-04-02T04:48:04.642 INFO:tasks.workunit.client.1.smithi179.stdout:0/967: mkdir d1/db/d64/d120 0 2024-04-02T04:48:04.644 INFO:tasks.workunit.client.0.smithi070.stdout:3/772: chown d10/d17/c25 10886355 1 2024-04-02T04:48:04.648 INFO:tasks.workunit.client.1.smithi179.stdout:7/903: dwrite d1/d9/f1d [0,4194304] 0 2024-04-02T04:48:04.655 INFO:tasks.workunit.client.0.smithi070.stdout:8/762: dwrite d2/d3/f9 [0,4194304] 0 2024-04-02T04:48:04.657 INFO:tasks.workunit.client.0.smithi070.stdout:3/773: unlink d10/d32/d8a/d8d/f9a 0 2024-04-02T04:48:04.658 INFO:tasks.workunit.client.1.smithi179.stdout:4/905: write d1/d15/d6e/f110 [224480,128100] 0 2024-04-02T04:48:04.659 INFO:tasks.workunit.client.0.smithi070.stdout:0/885: write d4/d3c/f6a [3264628,66330] 0 2024-04-02T04:48:04.659 INFO:tasks.workunit.client.0.smithi070.stdout:3/774: truncate d10/d32/d4b/dc5/de0/ffd 457811 0 2024-04-02T04:48:04.662 INFO:tasks.workunit.client.1.smithi179.stdout:8/892: creat d3/d32/d21/d22/da5/f13f x:0 0 0 2024-04-02T04:48:04.663 INFO:tasks.workunit.client.1.smithi179.stdout:8/893: readlink d3/d32/d21/d22/d6b/d91/l90 0 2024-04-02T04:48:04.663 INFO:tasks.workunit.client.0.smithi070.stdout:4/826: getdents d1/d3/d9 0 2024-04-02T04:48:04.664 INFO:tasks.workunit.client.1.smithi179.stdout:8/894: stat d3/d32/d21/d45/d5a/fff 0 2024-04-02T04:48:04.666 INFO:tasks.workunit.client.1.smithi179.stdout:2/768: creat dc/f101 x:0 0 0 2024-04-02T04:48:04.667 INFO:tasks.workunit.client.1.smithi179.stdout:0/968: dread - d1/da/d48/d65/d7d/d80/f9e zero size 2024-04-02T04:48:04.668 INFO:tasks.workunit.client.0.smithi070.stdout:4/827: dread d1/d3/d9/d5e/f77 [0,4194304] 0 2024-04-02T04:48:04.669 INFO:tasks.workunit.client.0.smithi070.stdout:8/763: mkdir d2/d3/d35/d3a/dfb 0 2024-04-02T04:48:04.670 INFO:tasks.workunit.client.0.smithi070.stdout:8/764: fdatasync d2/d8/d29/db/f13 0 2024-04-02T04:48:04.672 INFO:tasks.workunit.client.1.smithi179.stdout:6/879: rename d3/d12/d23/d39/dff/c135 to d3/d12/d23/d39/d80/c13f 0 2024-04-02T04:48:04.672 INFO:tasks.workunit.client.0.smithi070.stdout:3/775: symlink d10/d17/d51/d73/d9e/dd5/l105 0 2024-04-02T04:48:04.674 INFO:tasks.workunit.client.1.smithi179.stdout:2/769: symlink dc/d79/l102 0 2024-04-02T04:48:04.675 INFO:tasks.workunit.client.0.smithi070.stdout:4/828: creat d1/dc3/dc4/d10d/f115 x:0 0 0 2024-04-02T04:48:04.677 INFO:tasks.workunit.client.1.smithi179.stdout:1/883: creat d3/d5/de/f136 x:0 0 0 2024-04-02T04:48:04.679 INFO:tasks.workunit.client.1.smithi179.stdout:9/878: creat d0/d19/d27/f10f x:0 0 0 2024-04-02T04:48:04.681 INFO:tasks.workunit.client.1.smithi179.stdout:4/906: rename d1/d3/dc/d27/d10b to d1/d3/dc/d27/d62/d100/d135 0 2024-04-02T04:48:04.681 INFO:tasks.workunit.client.0.smithi070.stdout:3/776: chown d10/d17/c40 65991 1 2024-04-02T04:48:04.682 INFO:tasks.workunit.client.1.smithi179.stdout:6/880: creat d3/d12/d6a/f140 x:0 0 0 2024-04-02T04:48:04.683 INFO:tasks.workunit.client.1.smithi179.stdout:2/770: mknod dc/d29/d4a/d1e/d6c/dd2/c103 0 2024-04-02T04:48:04.684 INFO:tasks.workunit.client.0.smithi070.stdout:4/829: rename d1/f86 to d1/d3/d85/deb/f116 0 2024-04-02T04:48:04.685 INFO:tasks.workunit.client.1.smithi179.stdout:0/969: truncate d1/da/d48/d65/d7d/f7a 5668741 0 2024-04-02T04:48:04.686 INFO:tasks.workunit.client.1.smithi179.stdout:0/970: chown d1/da/d28/d34/d9c/ccf 65275 1 2024-04-02T04:48:04.688 INFO:tasks.workunit.client.1.smithi179.stdout:0/971: dread d1/da/f2d [0,4194304] 0 2024-04-02T04:48:04.689 INFO:tasks.workunit.client.0.smithi070.stdout:7/779: truncate d7/d36/fae 7938838 0 2024-04-02T04:48:04.690 INFO:tasks.workunit.client.1.smithi179.stdout:4/907: stat d1/d3/l5c 0 2024-04-02T04:48:04.691 INFO:tasks.workunit.client.0.smithi070.stdout:8/765: link d2/d8/d29/d22/d5a/le3 d2/d8/d29/d72/df1/lfc 0 2024-04-02T04:48:04.693 INFO:tasks.workunit.client.1.smithi179.stdout:8/895: rename d3/d31/d8f/l12b to d3/d1b/d2a/db0/l140 0 2024-04-02T04:48:04.696 INFO:tasks.workunit.client.1.smithi179.stdout:2/771: creat dc/d29/d4a/d1e/d8a/dce/f104 x:0 0 0 2024-04-02T04:48:04.698 INFO:tasks.workunit.client.1.smithi179.stdout:3/912: dwrite d8/d26/f34 [0,4194304] 0 2024-04-02T04:48:04.702 INFO:tasks.workunit.client.1.smithi179.stdout:5/846: dread d8/d1b/f37 [0,4194304] 0 2024-04-02T04:48:04.703 INFO:tasks.workunit.client.1.smithi179.stdout:0/972: rename d1/da/d48/d65/d7d/d80 to d1/da/d48/d65/d7d/d121 0 2024-04-02T04:48:04.704 INFO:tasks.workunit.client.0.smithi070.stdout:7/780: rename d7/dd/df/d79/d46/d65/f67 to d7/dd/df/ffc 0 2024-04-02T04:48:04.705 INFO:tasks.workunit.client.1.smithi179.stdout:0/973: read d1/f30 [2480791,54582] 0 2024-04-02T04:48:04.707 INFO:tasks.workunit.client.1.smithi179.stdout:4/908: creat d1/d3/df/d10/d2f/d60/da6/d125/f136 x:0 0 0 2024-04-02T04:48:04.707 INFO:tasks.workunit.client.1.smithi179.stdout:4/909: truncate d1/d93/fe4 744937 0 2024-04-02T04:48:04.709 INFO:tasks.workunit.client.1.smithi179.stdout:1/884: rename d3/f95 to d3/d5/d2f/d36/f137 0 2024-04-02T04:48:04.710 INFO:tasks.workunit.client.1.smithi179.stdout:0/974: dread d1/da/f93 [0,4194304] 0 2024-04-02T04:48:04.711 INFO:tasks.workunit.client.0.smithi070.stdout:3/777: getdents d10/d17/d51/d73 0 2024-04-02T04:48:04.712 INFO:tasks.workunit.client.0.smithi070.stdout:3/778: chown d10/d32/d8a/d8d 143 1 2024-04-02T04:48:04.713 INFO:tasks.workunit.client.0.smithi070.stdout:0/886: dwrite d4/d5/da/fc9 [0,4194304] 0 2024-04-02T04:48:04.716 INFO:tasks.workunit.client.0.smithi070.stdout:8/766: dread d2/d8/d23/d24/d5f/fe2 [0,4194304] 0 2024-04-02T04:48:04.717 INFO:tasks.workunit.client.1.smithi179.stdout:2/772: creat dc/d29/d31/d57/d97/f105 x:0 0 0 2024-04-02T04:48:04.719 INFO:tasks.workunit.client.0.smithi070.stdout:7/781: symlink d7/dd/db1/lfd 0 2024-04-02T04:48:04.719 INFO:tasks.workunit.client.1.smithi179.stdout:2/773: chown dc/d29/d31/d83/dae/db0/cef 0 1 2024-04-02T04:48:04.721 INFO:tasks.workunit.client.1.smithi179.stdout:9/879: creat d0/d1/d10/d6d/f110 x:0 0 0 2024-04-02T04:48:04.723 INFO:tasks.workunit.client.1.smithi179.stdout:2/774: dread dc/ff1 [0,4194304] 0 2024-04-02T04:48:04.729 INFO:tasks.workunit.client.0.smithi070.stdout:4/830: dwrite d1/f93 [0,4194304] 0 2024-04-02T04:48:04.730 INFO:tasks.workunit.client.0.smithi070.stdout:0/887: creat d4/d5/d58/d60/d83/da5/f110 x:0 0 0 2024-04-02T04:48:04.731 INFO:tasks.workunit.client.0.smithi070.stdout:0/888: write f0 [4832438,57910] 0 2024-04-02T04:48:04.732 INFO:tasks.workunit.client.0.smithi070.stdout:9/781: write d4/d10/d11/d7b/d19/d82/f8a [2909400,54323] 0 2024-04-02T04:48:04.732 INFO:tasks.workunit.client.1.smithi179.stdout:4/910: fsync d1/d15/d7c/fc2 0 2024-04-02T04:48:04.732 INFO:tasks.workunit.client.1.smithi179.stdout:8/896: dread d3/d32/d21/d51/f55 [0,4194304] 0 2024-04-02T04:48:04.732 INFO:tasks.workunit.client.0.smithi070.stdout:4/831: write d1/d3/d9/f24 [5220282,44977] 0 2024-04-02T04:48:04.733 INFO:tasks.workunit.client.1.smithi179.stdout:4/911: chown d1/d3/d19/f130 3153 1 2024-04-02T04:48:04.734 INFO:tasks.workunit.client.0.smithi070.stdout:4/832: write d1/dc3/dc4/ff8 [708199,62574] 0 2024-04-02T04:48:04.736 INFO:tasks.workunit.client.1.smithi179.stdout:8/897: dread d3/d32/d21/d51/f55 [0,4194304] 0 2024-04-02T04:48:04.739 INFO:tasks.workunit.client.0.smithi070.stdout:7/782: mknod d7/dd/df/d79/d46/d65/cfe 0 2024-04-02T04:48:04.741 INFO:tasks.workunit.client.0.smithi070.stdout:4/833: dwrite d1/fd [4194304,4194304] 0 2024-04-02T04:48:04.743 INFO:tasks.workunit.client.1.smithi179.stdout:1/885: creat d3/d5/d2f/d4f/d42/d53/dfe/f138 x:0 0 0 2024-04-02T04:48:04.748 INFO:tasks.workunit.client.0.smithi070.stdout:4/834: dwrite d1/dc3/dc4/ff8 [0,4194304] 0 2024-04-02T04:48:04.750 INFO:tasks.workunit.client.0.smithi070.stdout:4/835: truncate d1/dc3/dc4/d10d/f115 589971 0 2024-04-02T04:48:04.761 INFO:tasks.workunit.client.1.smithi179.stdout:0/975: dread d1/da/d48/d65/d7d/f6c [0,4194304] 0 2024-04-02T04:48:04.808 INFO:tasks.workunit.client.1.smithi179.stdout:7/904: sync 2024-04-02T04:48:04.815 INFO:tasks.workunit.client.1.smithi179.stdout:3/913: write d8/d26/d83/fa0 [1043762,6657] 0 2024-04-02T04:48:04.817 INFO:tasks.workunit.client.0.smithi070.stdout:0/889: unlink d4/d5/d13/d2a/d8d/fd3 0 2024-04-02T04:48:04.819 INFO:tasks.workunit.client.0.smithi070.stdout:0/890: write d4/d5/d58/d60/f108 [621621,80125] 0 2024-04-02T04:48:04.821 INFO:tasks.workunit.client.0.smithi070.stdout:8/767: write d2/d3/d52/f3c [3130805,111703] 0 2024-04-02T04:48:04.822 INFO:tasks.workunit.client.1.smithi179.stdout:2/775: dread dc/f52 [0,4194304] 0 2024-04-02T04:48:04.823 INFO:tasks.workunit.client.1.smithi179.stdout:5/847: dwrite d8/d1b/d23/d36/fca [0,4194304] 0 2024-04-02T04:48:04.824 INFO:tasks.workunit.client.0.smithi070.stdout:8/768: truncate d2/d8/d29/d22/d5a/d94/da2/dde/ff4 103023 0 2024-04-02T04:48:04.826 INFO:tasks.workunit.client.0.smithi070.stdout:8/769: stat d2/d8/d23/d24/d66/led 0 2024-04-02T04:48:04.826 INFO:tasks.workunit.client.1.smithi179.stdout:4/912: mknod d1/d3/dc/d27/d62/d100/c137 0 2024-04-02T04:48:04.827 INFO:tasks.workunit.client.1.smithi179.stdout:2/776: dread dc/d29/d4a/d1e/f7b [0,4194304] 0 2024-04-02T04:48:04.829 INFO:tasks.workunit.client.1.smithi179.stdout:2/777: dread dc/fc4 [0,4194304] 0 2024-04-02T04:48:04.830 INFO:tasks.workunit.client.0.smithi070.stdout:3/779: dwrite ff [0,4194304] 0 2024-04-02T04:48:04.834 INFO:tasks.workunit.client.0.smithi070.stdout:4/836: rename d1/d3/d9/f6f to d1/d9c/df0/f117 0 2024-04-02T04:48:04.834 INFO:tasks.workunit.client.0.smithi070.stdout:4/837: chown d1/d3/d85/d8e/d99/db1/lbf 6042 1 2024-04-02T04:48:04.835 INFO:tasks.workunit.client.1.smithi179.stdout:0/976: unlink d1/da/d48/d7b/f11e 0 2024-04-02T04:48:04.836 INFO:tasks.workunit.client.1.smithi179.stdout:0/977: write d1/da/d67/f10c [925712,48988] 0 2024-04-02T04:48:04.837 INFO:tasks.workunit.client.1.smithi179.stdout:7/905: mknod d1/dd/d16/d4d/c140 0 2024-04-02T04:48:04.838 INFO:tasks.workunit.client.0.smithi070.stdout:0/891: dread d4/d3c/f77 [0,4194304] 0 2024-04-02T04:48:04.841 INFO:tasks.workunit.client.0.smithi070.stdout:0/892: fdatasync d4/d5/d13/d17/d6e/fbc 0 2024-04-02T04:48:04.842 INFO:tasks.workunit.client.0.smithi070.stdout:9/782: link d4/d10/d12/d56/l72 d4/d10/d11/d7b/d19/d82/lfc 0 2024-04-02T04:48:04.843 INFO:tasks.workunit.client.0.smithi070.stdout:1/731: sync 2024-04-02T04:48:04.843 INFO:tasks.workunit.client.0.smithi070.stdout:5/944: sync 2024-04-02T04:48:04.843 INFO:tasks.workunit.client.0.smithi070.stdout:2/863: sync 2024-04-02T04:48:04.844 INFO:tasks.workunit.client.0.smithi070.stdout:4/838: dwrite d1/d3/d85/d8e/d99/db1/ffa [0,4194304] 0 2024-04-02T04:48:04.847 INFO:tasks.workunit.client.0.smithi070.stdout:0/893: truncate d4/d5/d13/d2a/d3a/d45/f10a 99928 0 2024-04-02T04:48:04.848 INFO:tasks.workunit.client.1.smithi179.stdout:6/881: rename d3/d12/d23/d26/d57/c60 to d3/d12/d23/d39/d80/d93/c141 0 2024-04-02T04:48:04.849 INFO:tasks.workunit.client.0.smithi070.stdout:0/894: chown d4/d5/d13/d2a/d3a/d80/db6/lf5 352198 1 2024-04-02T04:48:04.851 INFO:tasks.workunit.client.0.smithi070.stdout:5/945: dread d5/d9/d10/db9/dba/de1/ff4 [0,4194304] 0 2024-04-02T04:48:04.854 INFO:tasks.workunit.client.1.smithi179.stdout:8/898: mkdir d3/d31/dc3/d11b/d141 0 2024-04-02T04:48:04.858 INFO:tasks.workunit.client.1.smithi179.stdout:1/886: creat d3/d5/d2f/d4f/d42/df2/db6/dd8/f139 x:0 0 0 2024-04-02T04:48:04.866 INFO:tasks.workunit.client.0.smithi070.stdout:3/780: write d10/d17/db2/de2/ffa [1003503,23935] 0 2024-04-02T04:48:04.866 INFO:tasks.workunit.client.1.smithi179.stdout:9/880: creat d0/d19/f111 x:0 0 0 2024-04-02T04:48:04.866 INFO:tasks.workunit.client.1.smithi179.stdout:3/914: dread d8/d26/d83/d89/d9c/fc2 [0,4194304] 0 2024-04-02T04:48:04.867 INFO:tasks.workunit.client.1.smithi179.stdout:3/915: chown d8/d26/d52/c61 10 1 2024-04-02T04:48:04.867 INFO:tasks.workunit.client.0.smithi070.stdout:8/770: link d2/d8/d29/d22/c2f d2/d3/cfd 0 2024-04-02T04:48:04.872 INFO:tasks.workunit.client.1.smithi179.stdout:4/913: fdatasync d1/d93/de6/f12d 0 2024-04-02T04:48:04.874 INFO:tasks.workunit.client.0.smithi070.stdout:0/895: rename d4/d5/d58/fbf to d4/d5/d13/d2a/d3a/d45/f111 0 2024-04-02T04:48:04.877 INFO:tasks.workunit.client.0.smithi070.stdout:4/839: dread d1/d3/d85/deb/f116 [0,4194304] 0 2024-04-02T04:48:04.877 INFO:tasks.workunit.client.1.smithi179.stdout:6/882: unlink d3/d12/d23/d26/f4f 0 2024-04-02T04:48:04.878 INFO:tasks.workunit.client.0.smithi070.stdout:4/840: chown d1/d3/l7 505061 1 2024-04-02T04:48:04.878 INFO:tasks.workunit.client.0.smithi070.stdout:3/781: dread d10/d17/db2/dbd/fce [0,4194304] 0 2024-04-02T04:48:04.879 INFO:tasks.workunit.client.1.smithi179.stdout:8/899: symlink d3/d32/d21/d51/l142 0 2024-04-02T04:48:04.882 INFO:tasks.workunit.client.1.smithi179.stdout:5/848: write d8/d1b/d23/db1/fc6 [618315,124796] 0 2024-04-02T04:48:04.883 INFO:tasks.workunit.client.0.smithi070.stdout:9/783: dwrite d4/d13/df0/fdb [0,4194304] 0 2024-04-02T04:48:04.884 INFO:tasks.workunit.client.1.smithi179.stdout:0/978: dwrite d1/da/d48/d65/d7d/d121/fa5 [0,4194304] 0 2024-04-02T04:48:04.885 INFO:tasks.workunit.client.1.smithi179.stdout:9/881: creat d0/d19/d27/d48/f112 x:0 0 0 2024-04-02T04:48:04.888 INFO:tasks.workunit.client.1.smithi179.stdout:3/916: creat d8/d7a/f126 x:0 0 0 2024-04-02T04:48:04.888 INFO:tasks.workunit.client.1.smithi179.stdout:3/917: write d8/d57/d63/dc3/fe5 [1243034,70700] 0 2024-04-02T04:48:04.890 INFO:tasks.workunit.client.1.smithi179.stdout:4/914: fdatasync d1/d3/dc/d27/d62/f54 0 2024-04-02T04:48:04.893 INFO:tasks.workunit.client.1.smithi179.stdout:6/883: rmdir d3/d12/d23 39 2024-04-02T04:48:04.895 INFO:tasks.workunit.client.1.smithi179.stdout:1/887: getdents d3/d85/d132 0 2024-04-02T04:48:04.896 INFO:tasks.workunit.client.1.smithi179.stdout:3/918: dwrite d8/d26/d36/fa2 [0,4194304] 0 2024-04-02T04:48:04.897 INFO:tasks.workunit.client.1.smithi179.stdout:5/849: truncate d8/f18 5319718 0 2024-04-02T04:48:04.901 INFO:tasks.workunit.client.1.smithi179.stdout:7/906: symlink d1/dd/d76/d138/da8/d100/d107/l141 0 2024-04-02T04:48:04.902 INFO:tasks.workunit.client.1.smithi179.stdout:0/979: mkdir d1/da/d52/d122 0 2024-04-02T04:48:04.904 INFO:tasks.workunit.client.1.smithi179.stdout:9/882: fdatasync d0/d1/d10/d6d/fd8 0 2024-04-02T04:48:04.911 INFO:tasks.workunit.client.1.smithi179.stdout:8/900: mknod d3/d32/d21/d50/d56/d94/d84/d107/d117/c143 0 2024-04-02T04:48:04.913 INFO:tasks.workunit.client.1.smithi179.stdout:2/778: getdents dc/d29/d31/d57 0 2024-04-02T04:48:04.914 INFO:tasks.workunit.client.1.smithi179.stdout:6/884: write d3/d2d/d43/dd7/f107 [7821018,34960] 0 2024-04-02T04:48:04.916 INFO:tasks.workunit.client.1.smithi179.stdout:6/885: read d3/f18 [4071896,101931] 0 2024-04-02T04:48:04.922 INFO:tasks.workunit.client.1.smithi179.stdout:5/850: unlink d8/d1b/d23/db1/fc6 0 2024-04-02T04:48:04.922 INFO:tasks.workunit.client.1.smithi179.stdout:1/888: dread d3/d5/d2f/d4f/d42/d55/fa1 [0,4194304] 0 2024-04-02T04:48:04.923 INFO:tasks.workunit.client.1.smithi179.stdout:1/889: chown d3/d5/d2f/d4f/dab 1185 1 2024-04-02T04:48:04.928 INFO:tasks.workunit.client.1.smithi179.stdout:4/915: mknod d1/d3/df/d106/d131/c138 0 2024-04-02T04:48:04.928 INFO:tasks.workunit.client.1.smithi179.stdout:4/916: dread - d1/d3/dc/d26/d8a/d5d/f129 zero size 2024-04-02T04:48:04.929 INFO:tasks.workunit.client.1.smithi179.stdout:8/901: symlink d3/d32/d21/d22/d6b/d91/da8/l144 0 2024-04-02T04:48:04.942 INFO:tasks.workunit.client.0.smithi070.stdout:8/771: truncate d2/d3/d35/d3a/d4f/f87 2708757 0 2024-04-02T04:48:04.943 INFO:tasks.workunit.client.1.smithi179.stdout:7/907: mkdir d1/dd/d76/d13c/d142 0 2024-04-02T04:48:04.945 INFO:tasks.workunit.client.1.smithi179.stdout:5/851: dwrite d8/d1b/d23/db1/f102 [0,4194304] 0 2024-04-02T04:48:04.946 INFO:tasks.workunit.client.0.smithi070.stdout:4/841: mkdir d1/d9c/d118 0 2024-04-02T04:48:04.947 INFO:tasks.workunit.client.0.smithi070.stdout:4/842: fsync d1/d9c/f102 0 2024-04-02T04:48:04.951 INFO:tasks.workunit.client.1.smithi179.stdout:4/917: mkdir d1/d3/df/d106/d139 0 2024-04-02T04:48:04.952 INFO:tasks.workunit.client.1.smithi179.stdout:0/980: dwrite d1/da/d48/d7b/fc7 [0,4194304] 0 2024-04-02T04:48:04.953 INFO:tasks.workunit.client.1.smithi179.stdout:0/981: stat d1/da/l19 0 2024-04-02T04:48:04.955 INFO:tasks.workunit.client.1.smithi179.stdout:2/779: symlink dc/d29/d4a/l106 0 2024-04-02T04:48:04.956 INFO:tasks.workunit.client.0.smithi070.stdout:1/732: link d0/d9/fc d0/d54/d64/d6d/db6/fe7 0 2024-04-02T04:48:04.956 INFO:tasks.workunit.client.1.smithi179.stdout:2/780: write f2 [3986317,90103] 0 2024-04-02T04:48:04.957 INFO:tasks.workunit.client.0.smithi070.stdout:1/733: chown d0/d7/d1a/f75 48 1 2024-04-02T04:48:04.957 INFO:tasks.workunit.client.1.smithi179.stdout:8/902: dread d3/d32/d21/d49/f72 [0,4194304] 0 2024-04-02T04:48:04.959 INFO:tasks.workunit.client.1.smithi179.stdout:6/886: symlink d3/d12/d23/d39/d80/l142 0 2024-04-02T04:48:04.961 INFO:tasks.workunit.client.1.smithi179.stdout:6/887: write d3/d12/d23/d2f/d120/f12b [1015768,86992] 0 2024-04-02T04:48:04.962 INFO:tasks.workunit.client.1.smithi179.stdout:6/888: write d3/d12/d23/d26/d57/fc3 [186195,72962] 0 2024-04-02T04:48:04.965 INFO:tasks.workunit.client.1.smithi179.stdout:9/883: sync 2024-04-02T04:48:04.966 INFO:tasks.workunit.client.0.smithi070.stdout:0/896: write d4/d5/d13/d2a/d29/d52/fa0 [1640818,35724] 0 2024-04-02T04:48:04.966 INFO:tasks.workunit.client.0.smithi070.stdout:3/782: symlink d10/d66/d103/l106 0 2024-04-02T04:48:04.971 INFO:tasks.workunit.client.0.smithi070.stdout:4/843: creat d1/d3/d85/deb/d110/f119 x:0 0 0 2024-04-02T04:48:04.973 INFO:tasks.workunit.client.1.smithi179.stdout:1/890: creat d3/d5/d2f/d4f/d35/d117/f13a x:0 0 0 2024-04-02T04:48:04.973 INFO:tasks.workunit.client.0.smithi070.stdout:5/946: getdents d5/d9/d23/d30/da2/d75/d123 0 2024-04-02T04:48:04.974 INFO:tasks.workunit.client.1.smithi179.stdout:1/891: truncate d3/d5/d2f/f121 73733 0 2024-04-02T04:48:04.974 INFO:tasks.workunit.client.1.smithi179.stdout:4/918: dread d1/fc3 [0,4194304] 0 2024-04-02T04:48:04.975 INFO:tasks.workunit.client.1.smithi179.stdout:5/852: symlink d8/d1b/d23/d32/d4e/d51/l10f 0 2024-04-02T04:48:04.975 INFO:tasks.workunit.client.1.smithi179.stdout:4/919: chown d1/d93/d9f/fea 12277809 1 2024-04-02T04:48:04.975 INFO:tasks.workunit.client.0.smithi070.stdout:2/864: getdents d2/d10/d19/dd7 0 2024-04-02T04:48:04.981 INFO:tasks.workunit.client.0.smithi070.stdout:2/865: dread d2/d10/d5c/d7a/fab [0,4194304] 0 2024-04-02T04:48:04.984 INFO:tasks.workunit.client.1.smithi179.stdout:0/982: creat d1/da/d48/d65/d7d/f123 x:0 0 0 2024-04-02T04:48:04.985 INFO:tasks.workunit.client.0.smithi070.stdout:0/897: rmdir d4/d5/d13/d17/d72 39 2024-04-02T04:48:04.986 INFO:tasks.workunit.client.0.smithi070.stdout:0/898: readlink d4/d5/d13/d2a/d29/lef 0 2024-04-02T04:48:04.986 INFO:tasks.workunit.client.1.smithi179.stdout:8/903: fdatasync d3/d32/ff6 0 2024-04-02T04:48:04.987 INFO:tasks.workunit.client.0.smithi070.stdout:0/899: stat d4/d5/d13/d2a/d8d/dfc/l10e 0 2024-04-02T04:48:04.988 INFO:tasks.workunit.client.0.smithi070.stdout:3/783: mkdir d10/d17/d51/d73/d9e/d107 0 2024-04-02T04:48:04.990 INFO:tasks.workunit.client.0.smithi070.stdout:4/844: creat d1/dc3/dc4/f11a x:0 0 0 2024-04-02T04:48:04.991 INFO:tasks.workunit.client.1.smithi179.stdout:3/919: link d8/d26/d83/dbc/lf2 d8/d6d/l127 0 2024-04-02T04:48:04.993 INFO:tasks.workunit.client.1.smithi179.stdout:3/920: read d8/d57/d63/fe3 [56944,97048] 0 2024-04-02T04:48:04.993 INFO:tasks.workunit.client.0.smithi070.stdout:3/784: dwrite d10/f29 [0,4194304] 0 2024-04-02T04:48:04.997 INFO:tasks.workunit.client.1.smithi179.stdout:6/889: rmdir d3/d12/d23/d26/d9b 39 2024-04-02T04:48:04.999 INFO:tasks.workunit.client.0.smithi070.stdout:1/734: dwrite d0/d7/d1a/f75 [0,4194304] 0 2024-04-02T04:48:05.001 INFO:tasks.workunit.client.0.smithi070.stdout:8/772: truncate d2/d3/d35/d3a/d4f/d59/f90 3315809 0 2024-04-02T04:48:05.004 INFO:tasks.workunit.client.1.smithi179.stdout:9/884: mkdir d0/d1/d11/dab/d113 0 2024-04-02T04:48:05.009 INFO:tasks.workunit.client.0.smithi070.stdout:0/900: mkdir d4/d5/d13/d2a/d3a/d45/d112 0 2024-04-02T04:48:05.015 INFO:tasks.workunit.client.1.smithi179.stdout:0/983: fdatasync d1/da/df/f8a 0 2024-04-02T04:48:05.015 INFO:tasks.workunit.client.0.smithi070.stdout:5/947: write d5/d17/d6c/f6d [145078,69203] 0 2024-04-02T04:48:05.020 INFO:tasks.workunit.client.1.smithi179.stdout:1/892: dread d3/d5/d2f/d4f/d42/d55/fd6 [0,4194304] 0 2024-04-02T04:48:05.020 INFO:tasks.workunit.client.0.smithi070.stdout:1/735: rmdir d0/d54/d64 39 2024-04-02T04:48:05.022 INFO:tasks.workunit.client.0.smithi070.stdout:2/866: getdents d2/d10/d5c/d7a/db0/de4/d113 0 2024-04-02T04:48:05.023 INFO:tasks.workunit.client.0.smithi070.stdout:2/867: stat d2/d10/d21/d3d/d42/c5a 0 2024-04-02T04:48:05.025 INFO:tasks.workunit.client.0.smithi070.stdout:0/901: rmdir d4/d5/da/d2d 39 2024-04-02T04:48:05.025 INFO:tasks.workunit.client.1.smithi179.stdout:1/893: dwrite d3/d5/de/d33/f12c [0,4194304] 0 2024-04-02T04:48:05.028 INFO:tasks.workunit.client.0.smithi070.stdout:0/902: truncate d4/d5/d13/d2a/d8d/dfc/f10b 1028922 0 2024-04-02T04:48:05.031 INFO:tasks.workunit.client.1.smithi179.stdout:7/908: dwrite d1/d9/d27/d87/fbe [0,4194304] 0 2024-04-02T04:48:05.032 INFO:tasks.workunit.client.0.smithi070.stdout:5/948: rename d5/d9/d10/d6a/d7d/fc8 to d5/d9/d23/d30/dde/f127 0 2024-04-02T04:48:05.033 INFO:tasks.workunit.client.1.smithi179.stdout:6/890: dwrite d3/f10b [4194304,4194304] 0 2024-04-02T04:48:05.036 INFO:tasks.workunit.client.1.smithi179.stdout:5/853: symlink d8/d1b/d23/db1/l110 0 2024-04-02T04:48:05.037 INFO:tasks.workunit.client.1.smithi179.stdout:4/920: getdents d1/d3/df/d10/d2f/d60/d71/dce/d118 0 2024-04-02T04:48:05.039 INFO:tasks.workunit.client.1.smithi179.stdout:2/781: truncate dc/d29/d31/d83/dae/db0/dbc/f65 3926031 0 2024-04-02T04:48:05.044 INFO:tasks.workunit.client.1.smithi179.stdout:4/921: truncate d1/d15/d6e/f12a 659678 0 2024-04-02T04:48:05.044 INFO:tasks.workunit.client.0.smithi070.stdout:1/736: dwrite d0/d54/d64/d6d/fe5 [0,4194304] 0 2024-04-02T04:48:05.045 INFO:tasks.workunit.client.0.smithi070.stdout:0/903: mknod d4/d5/d13/d17/d98/dd2/c113 0 2024-04-02T04:48:05.045 INFO:tasks.workunit.client.1.smithi179.stdout:7/909: dwrite d1/d9/dd9/d121/d6b/fff [0,4194304] 0 2024-04-02T04:48:05.047 INFO:tasks.workunit.client.1.smithi179.stdout:3/921: link d8/d26/d83/d89/d9c/dcc/fe2 d8/d57/d63/f128 0 2024-04-02T04:48:05.048 INFO:tasks.workunit.client.1.smithi179.stdout:3/922: write d8/d57/d63/dc3/f11f [608896,125432] 0 2024-04-02T04:48:05.049 INFO:tasks.workunit.client.1.smithi179.stdout:3/923: dread - d8/d7a/f112 zero size 2024-04-02T04:48:05.049 INFO:tasks.workunit.client.0.smithi070.stdout:3/785: dwrite d10/d32/d4b/dc5/fed [0,4194304] 0 2024-04-02T04:48:05.052 INFO:tasks.workunit.client.0.smithi070.stdout:6/708: sync 2024-04-02T04:48:05.053 INFO:tasks.workunit.client.0.smithi070.stdout:5/949: unlink d5/d17/f32 0 2024-04-02T04:48:05.057 INFO:tasks.workunit.client.0.smithi070.stdout:5/950: chown d5/d17/lc6 285 1 2024-04-02T04:48:05.058 INFO:tasks.workunit.client.0.smithi070.stdout:0/904: creat d4/d3c/f114 x:0 0 0 2024-04-02T04:48:05.058 INFO:tasks.workunit.client.0.smithi070.stdout:2/868: dread d2/d10/d5c/d7a/fda [0,4194304] 0 2024-04-02T04:48:05.059 INFO:tasks.workunit.client.1.smithi179.stdout:8/904: sync 2024-04-02T04:48:05.060 INFO:tasks.workunit.client.0.smithi070.stdout:1/737: mknod d0/d7/d28/d83/ce8 0 2024-04-02T04:48:05.070 INFO:tasks.workunit.client.0.smithi070.stdout:0/905: mknod d4/d5/d21/d6b/c115 0 2024-04-02T04:48:05.071 INFO:tasks.workunit.client.0.smithi070.stdout:8/773: truncate d2/f7f 2967595 0 2024-04-02T04:48:05.072 INFO:tasks.workunit.client.1.smithi179.stdout:7/910: mknod d1/dd/d76/d13c/c143 0 2024-04-02T04:48:05.072 INFO:tasks.workunit.client.0.smithi070.stdout:5/951: dwrite d5/d17/d6c/fe6 [0,4194304] 0 2024-04-02T04:48:05.073 INFO:tasks.workunit.client.1.smithi179.stdout:2/782: dread dc/f17 [0,4194304] 0 2024-04-02T04:48:05.073 INFO:tasks.workunit.client.0.smithi070.stdout:0/906: chown d4/d5/d13/d2a/d3a/d80/f9f 16660 1 2024-04-02T04:48:05.074 INFO:tasks.workunit.client.0.smithi070.stdout:2/869: mknod d2/d12/d3e/c11f 0 2024-04-02T04:48:05.075 INFO:tasks.workunit.client.1.smithi179.stdout:0/984: dread d1/db/d64/f99 [0,4194304] 0 2024-04-02T04:48:05.075 INFO:tasks.workunit.client.1.smithi179.stdout:3/924: chown d8/dd3/fd9 1 1 2024-04-02T04:48:05.080 INFO:tasks.workunit.client.0.smithi070.stdout:4/845: dwrite d1/d3/d9/f22 [0,4194304] 0 2024-04-02T04:48:05.081 INFO:tasks.workunit.client.1.smithi179.stdout:1/894: mknod d3/d5/d2f/d4f/d42/d55/c13b 0 2024-04-02T04:48:05.083 INFO:tasks.workunit.client.1.smithi179.stdout:6/891: creat d3/d12/d23/d137/f143 x:0 0 0 2024-04-02T04:48:05.084 INFO:tasks.workunit.client.1.smithi179.stdout:6/892: chown d3/d12/d23/d26/dd3/f100 921 1 2024-04-02T04:48:05.084 INFO:tasks.workunit.client.1.smithi179.stdout:7/911: dwrite d1/d9/d27/d87/d8b/d75/db9/dcf/f114 [0,4194304] 0 2024-04-02T04:48:05.085 INFO:tasks.workunit.client.0.smithi070.stdout:6/709: unlink d0/d3/d19/l48 0 2024-04-02T04:48:05.086 INFO:tasks.workunit.client.1.smithi179.stdout:7/912: dread - d1/dd/d76/d138/da8/f124 zero size 2024-04-02T04:48:05.089 INFO:tasks.workunit.client.1.smithi179.stdout:9/885: truncate d0/d1/d10/d6d/fd0 2840662 0 2024-04-02T04:48:05.090 INFO:tasks.workunit.client.1.smithi179.stdout:9/886: write d0/d19/d27/d7a/f10a [680314,122885] 0 2024-04-02T04:48:05.092 INFO:tasks.workunit.client.1.smithi179.stdout:8/905: creat d3/d31/dc3/f145 x:0 0 0 2024-04-02T04:48:05.093 INFO:tasks.workunit.client.1.smithi179.stdout:8/906: chown d3/d32/d21/d22/d6b/d91/da8/l144 1 1 2024-04-02T04:48:05.096 INFO:tasks.workunit.client.0.smithi070.stdout:0/907: creat d4/d5/d13/d2a/d29/d52/f116 x:0 0 0 2024-04-02T04:48:05.098 INFO:tasks.workunit.client.1.smithi179.stdout:8/907: dwrite d3/d32/d21/d50/dcf/f134 [0,4194304] 0 2024-04-02T04:48:05.104 INFO:tasks.workunit.client.0.smithi070.stdout:0/908: dwrite d4/d5/d13/d2a/d3a/f10f [0,4194304] 0 2024-04-02T04:48:05.104 INFO:tasks.workunit.client.0.smithi070.stdout:3/786: link d10/d17/d51/d73/c81 d10/d32/d3f/dc0/c108 0 2024-04-02T04:48:05.105 INFO:tasks.workunit.client.0.smithi070.stdout:3/787: fsync d10/d32/dd1/fdf 0 2024-04-02T04:48:05.112 INFO:tasks.workunit.client.0.smithi070.stdout:1/738: write d0/f2a [5256139,12117] 0 2024-04-02T04:48:05.125 INFO:tasks.workunit.client.0.smithi070.stdout:2/870: write d2/d10/d21/d3d/d42/fad [318564,80070] 0 2024-04-02T04:48:05.127 INFO:tasks.workunit.client.0.smithi070.stdout:8/774: dwrite d2/f48 [0,4194304] 0 2024-04-02T04:48:05.129 INFO:tasks.workunit.client.1.smithi179.stdout:2/783: write dc/d29/d4a/d4b/f61 [96287,117249] 0 2024-04-02T04:48:05.131 INFO:tasks.workunit.client.0.smithi070.stdout:8/775: truncate d2/fea 316566 0 2024-04-02T04:48:05.131 INFO:tasks.workunit.client.0.smithi070.stdout:5/952: dwrite d5/d17/d2b/f8d [0,4194304] 0 2024-04-02T04:48:05.136 INFO:tasks.workunit.client.1.smithi179.stdout:0/985: dwrite d1/f57 [0,4194304] 0 2024-04-02T04:48:05.136 INFO:tasks.workunit.client.0.smithi070.stdout:6/710: dwrite d0/f78 [0,4194304] 0 2024-04-02T04:48:05.146 INFO:tasks.workunit.client.0.smithi070.stdout:6/711: dwrite d0/d3/d19/f3f [4194304,4194304] 0 2024-04-02T04:48:05.149 INFO:tasks.workunit.client.0.smithi070.stdout:6/712: read d0/d3/f3a [8207,98177] 0 2024-04-02T04:48:05.149 INFO:tasks.workunit.client.0.smithi070.stdout:2/871: mkdir d2/d10/d21/d3d/ddc/de8/d120 0 2024-04-02T04:48:05.155 INFO:tasks.workunit.client.0.smithi070.stdout:5/953: dread - d5/d17/d6c/ffc zero size 2024-04-02T04:48:05.158 INFO:tasks.workunit.client.0.smithi070.stdout:2/872: dwrite d2/d10/d5c/d7a/db0/fdf [0,4194304] 0 2024-04-02T04:48:05.160 INFO:tasks.workunit.client.0.smithi070.stdout:8/776: creat d2/d8/d29/d22/d5a/d94/da2/dde/ffe x:0 0 0 2024-04-02T04:48:05.163 INFO:tasks.workunit.client.0.smithi070.stdout:3/788: creat d10/d17/d51/d73/d9e/d107/f109 x:0 0 0 2024-04-02T04:48:05.165 INFO:tasks.workunit.client.0.smithi070.stdout:1/739: write d0/d7/d28/f74 [995711,28079] 0 2024-04-02T04:48:05.169 INFO:tasks.workunit.client.0.smithi070.stdout:9/784: sync 2024-04-02T04:48:05.169 INFO:tasks.workunit.client.0.smithi070.stdout:7/783: sync 2024-04-02T04:48:05.172 INFO:tasks.workunit.client.0.smithi070.stdout:6/713: creat d0/d55/d5c/fda x:0 0 0 2024-04-02T04:48:05.175 INFO:tasks.workunit.client.0.smithi070.stdout:0/909: rmdir d4/d5/d13/d2a/d3a/d80/da8/ddb 0 2024-04-02T04:48:05.180 INFO:tasks.workunit.client.0.smithi070.stdout:8/777: creat d2/d3/d35/d3a/d4f/de8/fff x:0 0 0 2024-04-02T04:48:05.182 INFO:tasks.workunit.client.0.smithi070.stdout:2/873: dread d2/f32 [0,4194304] 0 2024-04-02T04:48:05.182 INFO:tasks.workunit.client.0.smithi070.stdout:0/910: dwrite d4/d5/d13/fc7 [0,4194304] 0 2024-04-02T04:48:05.182 INFO:tasks.workunit.client.0.smithi070.stdout:3/789: creat d10/d32/d3f/f10a x:0 0 0 2024-04-02T04:48:05.188 INFO:tasks.workunit.client.0.smithi070.stdout:4/846: rmdir d1/d3/d9/de6 0 2024-04-02T04:48:05.190 INFO:tasks.workunit.client.0.smithi070.stdout:2/874: dwrite d2/d10/d5c/d7a/db0/dd8/dd2/fcf [0,4194304] 0 2024-04-02T04:48:05.199 INFO:tasks.workunit.client.0.smithi070.stdout:6/714: rmdir d0/d55/d5c 39 2024-04-02T04:48:05.200 INFO:tasks.workunit.client.0.smithi070.stdout:6/715: write d0/d3/d19/d31/f4e [9039321,91152] 0 2024-04-02T04:48:05.201 INFO:tasks.workunit.client.0.smithi070.stdout:6/716: fsync d0/d43/db3/dc7/fc3 0 2024-04-02T04:48:05.203 INFO:tasks.workunit.client.0.smithi070.stdout:5/954: mknod d5/d9/d10/db9/d116/c128 0 2024-04-02T04:48:05.206 INFO:tasks.workunit.client.0.smithi070.stdout:8/778: dread - d2/d3/d35/fb3 zero size 2024-04-02T04:48:05.208 INFO:tasks.workunit.client.0.smithi070.stdout:8/779: chown d2/d3/d35/d3a/d4f/c85 3575090 1 2024-04-02T04:48:05.209 INFO:tasks.workunit.client.0.smithi070.stdout:1/740: dwrite d0/d54/d64/fa9 [4194304,4194304] 0 2024-04-02T04:48:05.213 INFO:tasks.workunit.client.0.smithi070.stdout:0/911: rename d4/d3c/f6a to d4/d5/d13/d2a/d3a/d80/db6/f117 0 2024-04-02T04:48:05.218 INFO:tasks.workunit.client.0.smithi070.stdout:4/847: symlink d1/d3/d78/d91/l11b 0 2024-04-02T04:48:05.220 INFO:tasks.workunit.client.0.smithi070.stdout:7/784: creat d7/d8e/fff x:0 0 0 2024-04-02T04:48:05.221 INFO:tasks.workunit.client.0.smithi070.stdout:7/785: read - d7/dd/df/d7a/fc4 zero size 2024-04-02T04:48:05.226 INFO:tasks.workunit.client.0.smithi070.stdout:3/790: write d10/d32/f36 [9353554,42505] 0 2024-04-02T04:48:05.229 INFO:tasks.workunit.client.0.smithi070.stdout:6/717: rmdir d0/d43/db3/dc7/d7b/da6 39 2024-04-02T04:48:05.235 INFO:tasks.workunit.client.0.smithi070.stdout:2/875: write d2/d12/d59/fb2 [444458,21887] 0 2024-04-02T04:48:05.241 INFO:tasks.workunit.client.0.smithi070.stdout:8/780: dread d2/d8/d29/d22/fe1 [0,4194304] 0 2024-04-02T04:48:05.242 INFO:tasks.workunit.client.0.smithi070.stdout:8/781: write d2/f48 [360795,42886] 0 2024-04-02T04:48:05.245 INFO:tasks.workunit.client.0.smithi070.stdout:9/785: getdents d4/d10/d11/d7b/d19/d2f/d4d/de2 0 2024-04-02T04:48:05.246 INFO:tasks.workunit.client.0.smithi070.stdout:1/741: write d0/d7/d28/d83/fbd [418479,82279] 0 2024-04-02T04:48:05.247 INFO:tasks.workunit.client.1.smithi179.stdout:7/913: write d1/d9/d55/f96 [991196,22747] 0 2024-04-02T04:48:05.248 INFO:tasks.workunit.client.0.smithi070.stdout:6/718: creat d0/d3/d19/d31/d37/fdb x:0 0 0 2024-04-02T04:48:05.248 INFO:tasks.workunit.client.1.smithi179.stdout:8/908: dread d3/d32/d21/d50/d56/d94/d99/fb6 [0,4194304] 0 2024-04-02T04:48:05.249 INFO:tasks.workunit.client.1.smithi179.stdout:6/893: dread d3/d12/d23/d26/d57/f8a [0,4194304] 0 2024-04-02T04:48:05.249 INFO:tasks.workunit.client.1.smithi179.stdout:2/784: fdatasync dc/d29/d31/d66/fe2 0 2024-04-02T04:48:05.250 INFO:tasks.workunit.client.0.smithi070.stdout:6/719: truncate d0/d3/d19/d31/d46/d6b/f6d 773509 0 2024-04-02T04:48:05.251 INFO:tasks.workunit.client.0.smithi070.stdout:5/955: rename d5/d9/d10/d6a/fd5 to d5/d9/d10/db9/dba/de1/f129 0 2024-04-02T04:48:05.253 INFO:tasks.workunit.client.1.smithi179.stdout:0/986: dread d1/da/df/f8a [4194304,4194304] 0 2024-04-02T04:48:05.256 INFO:tasks.workunit.client.1.smithi179.stdout:5/854: getdents d8/d1b/d23/d36/d53/d62 0 2024-04-02T04:48:05.257 INFO:tasks.workunit.client.0.smithi070.stdout:8/782: mknod d2/d8/d23/d24/d66/c100 0 2024-04-02T04:48:05.261 INFO:tasks.workunit.client.1.smithi179.stdout:4/922: rename d1/d3/dc/fdf to d1/f13a 0 2024-04-02T04:48:05.261 INFO:tasks.workunit.client.0.smithi070.stdout:1/742: chown d0/d7/d1a/d95 2074780386 1 2024-04-02T04:48:05.261 INFO:tasks.workunit.client.0.smithi070.stdout:1/743: chown d0/d54/f63 91328 1 2024-04-02T04:48:05.262 INFO:tasks.workunit.client.1.smithi179.stdout:1/895: write d3/d85/da2/fb5 [988435,128985] 0 2024-04-02T04:48:05.267 INFO:tasks.workunit.client.0.smithi070.stdout:7/786: rename d7/dd/df/f32 to d7/dd/df/dca/f100 0 2024-04-02T04:48:05.268 INFO:tasks.workunit.client.0.smithi070.stdout:7/787: stat d7/dd/db5/df7 0 2024-04-02T04:48:05.268 INFO:tasks.workunit.client.0.smithi070.stdout:4/848: dwrite d1/d3/d9/f2a [0,4194304] 0 2024-04-02T04:48:05.269 INFO:tasks.workunit.client.1.smithi179.stdout:7/914: creat d1/d9/dd9/d121/d6b/f144 x:0 0 0 2024-04-02T04:48:05.269 INFO:tasks.workunit.client.1.smithi179.stdout:6/894: read - d3/d2d/d38/ffe zero size 2024-04-02T04:48:05.270 INFO:tasks.workunit.client.0.smithi070.stdout:7/788: write d7/d8e/fff [934714,121256] 0 2024-04-02T04:48:05.271 INFO:tasks.workunit.client.0.smithi070.stdout:0/912: link d4/d3c/lc3 d4/d5/d21/d6b/d102/l118 0 2024-04-02T04:48:05.271 INFO:tasks.workunit.client.0.smithi070.stdout:0/913: fsync d4/d5/d13/d17/d98/fdc 0 2024-04-02T04:48:05.276 INFO:tasks.workunit.client.1.smithi179.stdout:3/925: rename d8/d6d/ff3 to d8/d26/d52/d7e/def/f129 0 2024-04-02T04:48:05.279 INFO:tasks.workunit.client.0.smithi070.stdout:2/876: link d2/d12/d59/ccc d2/d10/d21/d71/dc1/c121 0 2024-04-02T04:48:05.281 INFO:tasks.workunit.client.0.smithi070.stdout:2/877: read d2/d10/d21/d71/f9f [2871155,95007] 0 2024-04-02T04:48:05.282 INFO:tasks.workunit.client.1.smithi179.stdout:1/896: mkdir d3/d5/d2f/d4f/d42/d53/d72/d13c 0 2024-04-02T04:48:05.282 INFO:tasks.workunit.client.1.smithi179.stdout:2/785: write dc/d29/d4a/d1e/d43/f47 [60011,103911] 0 2024-04-02T04:48:05.282 INFO:tasks.workunit.client.0.smithi070.stdout:2/878: chown d2/fd6 20436 1 2024-04-02T04:48:05.282 INFO:tasks.workunit.client.0.smithi070.stdout:2/879: stat d2/d10/d21/d3d/d42/l11c 0 2024-04-02T04:48:05.282 INFO:tasks.workunit.client.0.smithi070.stdout:6/720: creat d0/d20/dd8/fdc x:0 0 0 2024-04-02T04:48:05.283 INFO:tasks.workunit.client.1.smithi179.stdout:8/909: symlink d3/d32/l146 0 2024-04-02T04:48:05.285 INFO:tasks.workunit.client.1.smithi179.stdout:7/915: mkdir d1/d9/dd9/d121/d145 0 2024-04-02T04:48:05.287 INFO:tasks.workunit.client.0.smithi070.stdout:9/786: rename d4/d10/d11/c4c to d4/d10/d12/cfd 0 2024-04-02T04:48:05.289 INFO:tasks.workunit.client.1.smithi179.stdout:5/855: mkdir d8/d1b/d23/d32/dbe/daf/db7/dad/d111 0 2024-04-02T04:48:05.289 INFO:tasks.workunit.client.1.smithi179.stdout:5/856: write d8/d1b/d23/d36/fca [3189013,84809] 0 2024-04-02T04:48:05.293 INFO:tasks.workunit.client.1.smithi179.stdout:9/887: rename d0/d19/d27/d7a/d84/fb6 to d0/d19/d27/d7a/d7e/dd4/f114 0 2024-04-02T04:48:05.297 INFO:tasks.workunit.client.1.smithi179.stdout:1/897: unlink d3/d5/d2f/f81 0 2024-04-02T04:48:05.303 INFO:tasks.workunit.client.1.smithi179.stdout:7/916: creat d1/d9/d27/d87/d8b/d75/d11f/f146 x:0 0 0 2024-04-02T04:48:05.305 INFO:tasks.workunit.client.1.smithi179.stdout:2/786: write dc/d29/d4a/d1e/d6c/f9e [840527,37469] 0 2024-04-02T04:48:05.306 INFO:tasks.workunit.client.1.smithi179.stdout:5/857: creat d8/d1b/d23/d32/d4e/d50/f112 x:0 0 0 2024-04-02T04:48:05.308 INFO:tasks.workunit.client.1.smithi179.stdout:4/923: dwrite d1/d3/df/d10/d3a/ff0 [0,4194304] 0 2024-04-02T04:48:05.308 INFO:tasks.workunit.client.1.smithi179.stdout:5/858: stat d8/d1b/d23/d36/d53/d62/da9/fc3 0 2024-04-02T04:48:05.309 INFO:tasks.workunit.client.1.smithi179.stdout:5/859: stat d8/d1b/d23/d32/c84 0 2024-04-02T04:48:05.309 INFO:tasks.workunit.client.1.smithi179.stdout:6/895: rename d3/d12/d23/d39/d4e/d66/fc1 to d3/d2c/de0/f144 0 2024-04-02T04:48:05.311 INFO:tasks.workunit.client.1.smithi179.stdout:8/910: mkdir d3/d32/d21/d50/d56/d94/d84/de4/d147 0 2024-04-02T04:48:05.311 INFO:tasks.workunit.client.0.smithi070.stdout:5/956: mknod d5/d17/c12a 0 2024-04-02T04:48:05.312 INFO:tasks.workunit.client.0.smithi070.stdout:4/849: mknod d1/d3/c11c 0 2024-04-02T04:48:05.312 INFO:tasks.workunit.client.1.smithi179.stdout:7/917: truncate d1/fd2 51248 0 2024-04-02T04:48:05.312 INFO:tasks.workunit.client.0.smithi070.stdout:3/791: getdents d10/d32/d4b/dc5/de0 0 2024-04-02T04:48:05.315 INFO:tasks.workunit.client.1.smithi179.stdout:9/888: rename d0/d1/d11/d29/d80/lf6 to d0/d19/d27/d7a/d87/daf/l115 0 2024-04-02T04:48:05.317 INFO:tasks.workunit.client.1.smithi179.stdout:1/898: link d3/d5/f126 d3/d5/d2f/d65/f13d 0 2024-04-02T04:48:05.318 INFO:tasks.workunit.client.0.smithi070.stdout:4/850: stat d1/d3/f62 0 2024-04-02T04:48:05.319 INFO:tasks.workunit.client.0.smithi070.stdout:4/851: fdatasync d1/d3/d9/f22 0 2024-04-02T04:48:05.319 INFO:tasks.workunit.client.0.smithi070.stdout:8/783: link d2/d8/d29/d22/d5a/d94/da2/fc3 d2/d3/d35/d3a/d4f/d59/dcc/f101 0 2024-04-02T04:48:05.322 INFO:tasks.workunit.client.1.smithi179.stdout:1/899: dwrite d3/d5/d2f/d4f/d42/d53/d72/d86/f90 [0,4194304] 0 2024-04-02T04:48:05.322 INFO:tasks.workunit.client.1.smithi179.stdout:2/787: truncate dc/d29/d31/d83/dae/db0/dbc/f65 535715 0 2024-04-02T04:48:05.323 INFO:tasks.workunit.client.1.smithi179.stdout:1/900: readlink d3/d5/d2f/d65/l98 0 2024-04-02T04:48:05.324 INFO:tasks.workunit.client.1.smithi179.stdout:2/788: write dc/d29/d4a/d1e/d6c/fe1 [491735,6034] 0 2024-04-02T04:48:05.324 INFO:tasks.workunit.client.1.smithi179.stdout:5/860: rename d8/d1b/l109 to d8/d1b/d23/d32/dbe/daf/db7/dad/l113 0 2024-04-02T04:48:05.328 INFO:tasks.workunit.client.1.smithi179.stdout:5/861: dread d8/d1b/d23/db1/f102 [0,4194304] 0 2024-04-02T04:48:05.329 INFO:tasks.workunit.client.1.smithi179.stdout:1/901: creat d3/dbe/f13e x:0 0 0 2024-04-02T04:48:05.332 INFO:tasks.workunit.client.1.smithi179.stdout:8/911: rename d3/d32/d21/d45/d5a to d3/d1b/d2a/d41/d148 0 2024-04-02T04:48:05.338 INFO:tasks.workunit.client.1.smithi179.stdout:5/862: dread - d8/d1b/d23/d32/d46/fdc zero size 2024-04-02T04:48:05.342 INFO:tasks.workunit.client.1.smithi179.stdout:5/863: read d8/d1b/d23/d32/d4e/f59 [2951116,91903] 0 2024-04-02T04:48:05.342 INFO:tasks.workunit.client.1.smithi179.stdout:6/896: write d3/d2c/d85/fe6 [489160,71900] 0 2024-04-02T04:48:05.342 INFO:tasks.workunit.client.1.smithi179.stdout:4/924: truncate d1/d3/f1b 2815434 0 2024-04-02T04:48:05.342 INFO:tasks.workunit.client.1.smithi179.stdout:7/918: write d1/fc [3873731,129725] 0 2024-04-02T04:48:05.343 INFO:tasks.workunit.client.1.smithi179.stdout:1/902: mkdir d3/d5/d2f/d4f/d35/d13f 0 2024-04-02T04:48:05.347 INFO:tasks.workunit.client.1.smithi179.stdout:5/864: read d8/f6c [157091,117299] 0 2024-04-02T04:48:05.348 INFO:tasks.workunit.client.1.smithi179.stdout:5/865: chown d8/d1b/d23/d36/d53/d62/db2/lf8 7 1 2024-04-02T04:48:05.349 INFO:tasks.workunit.client.1.smithi179.stdout:5/866: fdatasync d8/d1b/d23/d32/f3d 0 2024-04-02T04:48:05.350 INFO:tasks.workunit.client.1.smithi179.stdout:5/867: write d8/d1b/d23/d32/d4e/f93 [2081235,30818] 0 2024-04-02T04:48:05.350 INFO:tasks.workunit.client.1.smithi179.stdout:1/903: fsync d3/d5/d96/f119 0 2024-04-02T04:48:05.352 INFO:tasks.workunit.client.1.smithi179.stdout:5/868: write d8/d1b/d23/d32/d46/ff7 [4534892,46445] 0 2024-04-02T04:48:05.353 INFO:tasks.workunit.client.1.smithi179.stdout:5/869: read - d8/d1b/d23/d42/dde/fff zero size 2024-04-02T04:48:05.354 INFO:tasks.workunit.client.1.smithi179.stdout:4/925: mknod d1/d3/dc0/c13b 0 2024-04-02T04:48:05.354 INFO:tasks.workunit.client.1.smithi179.stdout:9/889: truncate d0/d1/d11/d29/f37 750099 0 2024-04-02T04:48:05.355 INFO:tasks.workunit.client.1.smithi179.stdout:5/870: readlink d8/d1b/d23/d30/ld1 0 2024-04-02T04:48:05.358 INFO:tasks.workunit.client.1.smithi179.stdout:2/789: dwrite dc/d29/d4a/f75 [0,4194304] 0 2024-04-02T04:48:05.362 INFO:tasks.workunit.client.1.smithi179.stdout:1/904: creat d3/d5/d2f/d4f/d6a/f140 x:0 0 0 2024-04-02T04:48:05.364 INFO:tasks.workunit.client.1.smithi179.stdout:8/912: truncate d3/d32/d21/d50/d65/faa 2309836 0 2024-04-02T04:48:05.365 INFO:tasks.workunit.client.1.smithi179.stdout:8/913: stat d3/d31/dc3/f13e 0 2024-04-02T04:48:05.365 INFO:tasks.workunit.client.1.smithi179.stdout:9/890: mknod d0/d19/d27/d7a/d87/daf/c116 0 2024-04-02T04:48:05.366 INFO:tasks.workunit.client.1.smithi179.stdout:9/891: chown d0/d1/f7b 2 1 2024-04-02T04:48:05.367 INFO:tasks.workunit.client.1.smithi179.stdout:7/919: rename d1/dd/d23/f119 to d1/d9/d27/d87/d8b/d75/db9/f147 0 2024-04-02T04:48:05.370 INFO:tasks.workunit.client.1.smithi179.stdout:1/905: mknod d3/d5/de/d33/c141 0 2024-04-02T04:48:05.372 INFO:tasks.workunit.client.1.smithi179.stdout:5/871: mknod d8/d1b/d23/d32/d4e/d50/dae/c114 0 2024-04-02T04:48:05.372 INFO:tasks.workunit.client.1.smithi179.stdout:6/897: dwrite d3/d42/f102 [0,4194304] 0 2024-04-02T04:48:05.372 INFO:tasks.workunit.client.1.smithi179.stdout:2/790: dread dc/d29/d4a/d1e/d43/f47 [0,4194304] 0 2024-04-02T04:48:05.372 INFO:tasks.workunit.client.1.smithi179.stdout:1/906: fdatasync d3/d5/d2f/d4f/d6a/f129 0 2024-04-02T04:48:05.376 INFO:tasks.workunit.client.1.smithi179.stdout:8/914: rmdir d3/d32/d21/d22 39 2024-04-02T04:48:05.377 INFO:tasks.workunit.client.1.smithi179.stdout:8/915: fdatasync d3/d32/d21/d49/df0/f13c 0 2024-04-02T04:48:05.378 INFO:tasks.workunit.client.1.smithi179.stdout:6/898: mkdir d3/d42/dbe/d145 0 2024-04-02T04:48:05.379 INFO:tasks.workunit.client.1.smithi179.stdout:6/899: chown d3/d12/d23/d39/d4e/d6c/l5c 737 1 2024-04-02T04:48:05.380 INFO:tasks.workunit.client.1.smithi179.stdout:1/907: mknod d3/d5/d2f/d4f/d42/d53/dfe/c142 0 2024-04-02T04:48:05.381 INFO:tasks.workunit.client.1.smithi179.stdout:0/987: sync 2024-04-02T04:48:05.382 INFO:tasks.workunit.client.1.smithi179.stdout:8/916: symlink d3/d1b/d2a/db0/l149 0 2024-04-02T04:48:05.382 INFO:tasks.workunit.client.1.smithi179.stdout:1/908: dread - d3/d5/d2f/d65/f13d zero size 2024-04-02T04:48:05.383 INFO:tasks.workunit.client.1.smithi179.stdout:1/909: chown d3/d5/d2f/d4f/d6a/da0/dd4 30434 1 2024-04-02T04:48:05.386 INFO:tasks.workunit.client.1.smithi179.stdout:2/791: link dc/d29/d31/d57/d97/f105 dc/d29/de7/f107 0 2024-04-02T04:48:05.388 INFO:tasks.workunit.client.1.smithi179.stdout:9/892: rename d0/d1/d11/dab/ded/fb7 to d0/d1/d10/f117 0 2024-04-02T04:48:05.393 INFO:tasks.workunit.client.1.smithi179.stdout:6/900: symlink d3/d12/d23/d39/d4e/d6c/l146 0 2024-04-02T04:48:05.394 INFO:tasks.workunit.client.1.smithi179.stdout:2/792: mknod dc/d29/d31/d83/c108 0 2024-04-02T04:48:05.395 INFO:tasks.workunit.client.1.smithi179.stdout:9/893: symlink d0/d1/d11/dab/ded/d101/l118 0 2024-04-02T04:48:05.398 INFO:tasks.workunit.client.1.smithi179.stdout:1/910: unlink d3/d5/caf 0 2024-04-02T04:48:05.401 INFO:tasks.workunit.client.1.smithi179.stdout:2/793: rename dc/d29/d4a/d4b/l60 to dc/d29/d31/d57/d97/dbf/de3/l109 0 2024-04-02T04:48:05.401 INFO:tasks.workunit.client.1.smithi179.stdout:2/794: readlink dc/d79/lb3 0 2024-04-02T04:48:05.402 INFO:tasks.workunit.client.1.smithi179.stdout:8/917: creat d3/d32/d21/d22/d6b/d91/da8/d11a/f14a x:0 0 0 2024-04-02T04:48:05.404 INFO:tasks.workunit.client.1.smithi179.stdout:1/911: fsync d3/d5/d2f/d4f/d35/fbf 0 2024-04-02T04:48:05.405 INFO:tasks.workunit.client.1.smithi179.stdout:0/988: getdents d1/da/d28/d37 0 2024-04-02T04:48:05.409 INFO:tasks.workunit.client.1.smithi179.stdout:6/901: creat d3/d12/d23/d39/d4e/f147 x:0 0 0 2024-04-02T04:48:05.410 INFO:tasks.workunit.client.1.smithi179.stdout:6/902: fsync d3/d2c/d85/fe6 0 2024-04-02T04:48:05.410 INFO:tasks.workunit.client.1.smithi179.stdout:1/912: dwrite d3/d5/d2f/d4f/d35/f6e [0,4194304] 0 2024-04-02T04:48:05.411 INFO:tasks.workunit.client.1.smithi179.stdout:6/903: write d3/d12/d23/d39/dff/f13d [596464,98714] 0 2024-04-02T04:48:05.414 INFO:tasks.workunit.client.1.smithi179.stdout:0/989: mkdir d1/da/d48/d5c/d83/d97/d124 0 2024-04-02T04:48:05.416 INFO:tasks.workunit.client.1.smithi179.stdout:2/795: getdents dc/d29/d31/d83/dae 0 2024-04-02T04:48:05.418 INFO:tasks.workunit.client.1.smithi179.stdout:6/904: creat d3/d12/d23/d39/f148 x:0 0 0 2024-04-02T04:48:05.420 INFO:tasks.workunit.client.1.smithi179.stdout:0/990: creat d1/da/dc3/f125 x:0 0 0 2024-04-02T04:48:05.420 INFO:tasks.workunit.client.1.smithi179.stdout:2/796: creat dc/d29/d31/d57/f10a x:0 0 0 2024-04-02T04:48:05.423 INFO:tasks.workunit.client.1.smithi179.stdout:2/797: dread dc/d29/d31/fb9 [0,4194304] 0 2024-04-02T04:48:05.424 INFO:tasks.workunit.client.1.smithi179.stdout:2/798: fsync dc/d29/d31/f3a 0 2024-04-02T04:48:05.424 INFO:tasks.workunit.client.1.smithi179.stdout:6/905: dwrite d3/d2c/de0/f13c [0,4194304] 0 2024-04-02T04:48:05.425 INFO:tasks.workunit.client.1.smithi179.stdout:6/906: chown d3/d2d/da5 3730 1 2024-04-02T04:48:05.427 INFO:tasks.workunit.client.1.smithi179.stdout:2/799: unlink dc/d29/d4a/f56 0 2024-04-02T04:48:05.430 INFO:tasks.workunit.client.1.smithi179.stdout:6/907: dwrite d3/d2c/de0/f13c [0,4194304] 0 2024-04-02T04:48:05.431 INFO:tasks.workunit.client.1.smithi179.stdout:2/800: rename dc/d29/d4a/d1e/dbe to dc/d29/d10b 0 2024-04-02T04:48:05.432 INFO:tasks.workunit.client.1.smithi179.stdout:6/908: mknod d3/d2d/d43/c149 0 2024-04-02T04:48:05.433 INFO:tasks.workunit.client.1.smithi179.stdout:2/801: unlink dc/d29/d31/d83/dae/db0/dbc/c74 0 2024-04-02T04:48:05.434 INFO:tasks.workunit.client.1.smithi179.stdout:2/802: chown dc/d29/d31/f77 305482189 1 2024-04-02T04:48:05.435 INFO:tasks.workunit.client.1.smithi179.stdout:6/909: rename d3/ff to d3/d12/d44/d10a/d10e/d112/f14a 0 2024-04-02T04:48:05.436 INFO:tasks.workunit.client.1.smithi179.stdout:2/803: mkdir dc/d29/d4a/d4b/d10c 0 2024-04-02T04:48:05.446 INFO:tasks.workunit.client.1.smithi179.stdout:6/910: dread d3/d12/d23/d39/d4e/fbf [4194304,4194304] 0 2024-04-02T04:48:05.447 INFO:tasks.workunit.client.1.smithi179.stdout:6/911: chown d3/d12/d23/d39/d4e 19160999 1 2024-04-02T04:48:05.448 INFO:tasks.workunit.client.1.smithi179.stdout:6/912: truncate d3/d2d/d38/ffe 864089 0 2024-04-02T04:48:05.494 INFO:tasks.workunit.client.0.smithi070.stdout:5/957: rmdir d5/d9/d10/db9/dd9 39 2024-04-02T04:48:05.494 INFO:tasks.workunit.client.0.smithi070.stdout:8/784: stat d2/d8/d23/d24/d66/led 0 2024-04-02T04:48:05.495 INFO:tasks.workunit.client.0.smithi070.stdout:2/880: fdatasync d2/d10/d21/fc2 0 2024-04-02T04:48:05.497 INFO:tasks.workunit.client.0.smithi070.stdout:6/721: unlink d0/d3/d19/l67 0 2024-04-02T04:48:05.499 INFO:tasks.workunit.client.0.smithi070.stdout:3/792: creat d10/d17/d51/d73/d9e/f10b x:0 0 0 2024-04-02T04:48:05.499 INFO:tasks.workunit.client.0.smithi070.stdout:0/914: truncate d4/d5/d13/d2a/d3a/d45/fbd 1510480 0 2024-04-02T04:48:05.501 INFO:tasks.workunit.client.0.smithi070.stdout:4/852: mknod d1/d3/d85/da8/c11d 0 2024-04-02T04:48:05.501 INFO:tasks.workunit.client.0.smithi070.stdout:3/793: chown d10/d17/d51/d73/d9e/d107 1111977656 1 2024-04-02T04:48:05.501 INFO:tasks.workunit.client.0.smithi070.stdout:1/744: getdents d0/d9/d1d/d22 0 2024-04-02T04:48:05.504 INFO:tasks.workunit.client.0.smithi070.stdout:5/958: truncate d5/d9/d23/d33/feb 364574 0 2024-04-02T04:48:05.505 INFO:tasks.workunit.client.0.smithi070.stdout:6/722: chown d0/c2c 386904429 1 2024-04-02T04:48:05.507 INFO:tasks.workunit.client.1.smithi179.stdout:4/926: dread d1/d3/dc/dc1/ff1 [0,4194304] 0 2024-04-02T04:48:05.507 INFO:tasks.workunit.client.0.smithi070.stdout:9/787: link d4/d10/d12/d56/ld9 d4/lfe 0 2024-04-02T04:48:05.508 INFO:tasks.workunit.client.0.smithi070.stdout:5/959: write d5/d9/d23/dc3/f11d [268767,31731] 0 2024-04-02T04:48:05.509 INFO:tasks.workunit.client.0.smithi070.stdout:6/723: chown d0/d3/d19/d31/f36 127 1 2024-04-02T04:48:05.509 INFO:tasks.workunit.client.0.smithi070.stdout:3/794: unlink d10/d17/l44 0 2024-04-02T04:48:05.511 INFO:tasks.workunit.client.0.smithi070.stdout:4/853: unlink d1/d3/d85/da8/c11d 0 2024-04-02T04:48:05.512 INFO:tasks.workunit.client.0.smithi070.stdout:8/785: mkdir d2/d8/d29/d72/df1/d102 0 2024-04-02T04:48:05.513 INFO:tasks.workunit.client.0.smithi070.stdout:8/786: fdatasync d2/d8/d29/db/f13 0 2024-04-02T04:48:05.515 INFO:tasks.workunit.client.0.smithi070.stdout:1/745: fsync d0/d9/d1d/d22/f2d 0 2024-04-02T04:48:05.516 INFO:tasks.workunit.client.0.smithi070.stdout:1/746: chown d0/d7/d28 3803410 1 2024-04-02T04:48:05.519 INFO:tasks.workunit.client.0.smithi070.stdout:2/881: dread d2/d10/d53/f55 [0,4194304] 0 2024-04-02T04:48:05.520 INFO:tasks.workunit.client.0.smithi070.stdout:4/854: dwrite d1/d3/d85/d8e/d99/db1/dbb/fec [0,4194304] 0 2024-04-02T04:48:05.523 INFO:tasks.workunit.client.0.smithi070.stdout:9/788: rmdir d4/d10/d12/d56/d79 39 2024-04-02T04:48:05.524 INFO:tasks.workunit.client.1.smithi179.stdout:3/926: sync 2024-04-02T04:48:05.524 INFO:tasks.workunit.client.0.smithi070.stdout:3/795: mknod d10/d32/d3f/c10c 0 2024-04-02T04:48:05.524 INFO:tasks.workunit.client.0.smithi070.stdout:9/789: read d4/d10/d12/f67 [231838,5239] 0 2024-04-02T04:48:05.525 INFO:tasks.workunit.client.0.smithi070.stdout:3/796: stat d10/d17/f2a 0 2024-04-02T04:48:05.526 INFO:tasks.workunit.client.1.smithi179.stdout:4/927: rmdir d1/d3/d128/d12b 0 2024-04-02T04:48:05.528 INFO:tasks.workunit.client.1.smithi179.stdout:3/927: creat d8/d26/d36/f12a x:0 0 0 2024-04-02T04:48:05.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:05 smithi070 ceph-mon[29212]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-04-02T04:48:05.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:05 smithi070 ceph-mon[29212]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-04-02T04:48:05.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:05 smithi070 ceph-mon[29212]: pgmap v21: 65 pgs: 65 active+clean; 3.0 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 92 MiB/s rd, 217 MiB/s wr, 509 op/s 2024-04-02T04:48:05.530 INFO:tasks.workunit.client.1.smithi179.stdout:4/928: creat d1/d93/f13c x:0 0 0 2024-04-02T04:48:05.530 INFO:tasks.workunit.client.1.smithi179.stdout:5/872: write d8/d1b/d23/d32/dbe/daf/db7/f79 [733665,74793] 0 2024-04-02T04:48:05.531 INFO:tasks.workunit.client.0.smithi070.stdout:8/787: mknod d2/d3/d35/d3a/d4f/d59/c103 0 2024-04-02T04:48:05.531 INFO:tasks.workunit.client.1.smithi179.stdout:4/929: dread - d1/d3/f11b zero size 2024-04-02T04:48:05.537 INFO:tasks.workunit.client.1.smithi179.stdout:5/873: symlink d8/d1b/d23/d30/d73/dba/l115 0 2024-04-02T04:48:05.540 INFO:tasks.workunit.client.1.smithi179.stdout:4/930: symlink d1/d15/d6e/l13d 0 2024-04-02T04:48:05.541 INFO:tasks.workunit.client.0.smithi070.stdout:4/855: symlink d1/d3/d85/ddb/l11e 0 2024-04-02T04:48:05.543 INFO:tasks.workunit.client.0.smithi070.stdout:5/960: symlink d5/d9/d10/da9/l12b 0 2024-04-02T04:48:05.544 INFO:tasks.workunit.client.1.smithi179.stdout:4/931: dwrite d1/d15/f56 [0,4194304] 0 2024-04-02T04:48:05.545 INFO:tasks.workunit.client.1.smithi179.stdout:4/932: write d1/d3/dc/d27/f126 [705442,73075] 0 2024-04-02T04:48:05.548 INFO:tasks.workunit.client.0.smithi070.stdout:3/797: unlink d10/d17/d4c/fa3 0 2024-04-02T04:48:05.550 INFO:tasks.workunit.client.1.smithi179.stdout:5/874: dread d8/d1b/d23/d32/d4e/fab [0,4194304] 0 2024-04-02T04:48:05.550 INFO:tasks.workunit.client.1.smithi179.stdout:9/894: dwrite d0/f90 [0,4194304] 0 2024-04-02T04:48:05.550 INFO:tasks.workunit.client.1.smithi179.stdout:7/920: sync 2024-04-02T04:48:05.551 INFO:tasks.workunit.client.1.smithi179.stdout:2/804: sync 2024-04-02T04:48:05.552 INFO:tasks.workunit.client.1.smithi179.stdout:7/921: chown d1/dd/d16/c72 0 1 2024-04-02T04:48:05.552 INFO:tasks.workunit.client.1.smithi179.stdout:4/933: rename d1/d3/dc/d26/d8a/f4a to d1/d3/dc/d27/d62/d100/f13e 0 2024-04-02T04:48:05.553 INFO:tasks.workunit.client.1.smithi179.stdout:8/918: write d3/d32/d21/f25 [4894710,120777] 0 2024-04-02T04:48:05.559 INFO:tasks.workunit.client.0.smithi070.stdout:1/747: symlink d0/d9/d1d/d22/d56/dc1/le9 0 2024-04-02T04:48:05.559 INFO:tasks.workunit.client.0.smithi070.stdout:8/788: write d2/d8/d23/d24/d5f/fe2 [1263269,27807] 0 2024-04-02T04:48:05.560 INFO:tasks.workunit.client.1.smithi179.stdout:5/875: creat d8/d1b/d23/d32/d4e/d50/f116 x:0 0 0 2024-04-02T04:48:05.560 INFO:tasks.workunit.client.1.smithi179.stdout:8/919: dwrite d3/d32/d21/d22/f123 [0,4194304] 0 2024-04-02T04:48:05.563 INFO:tasks.workunit.client.1.smithi179.stdout:1/913: truncate d3/d5/d2f/d4f/d42/d53/d77/f97 834907 0 2024-04-02T04:48:05.563 INFO:tasks.workunit.client.1.smithi179.stdout:5/876: chown d8/d1b/d23/d32/d4e/d51/l10f 51 1 2024-04-02T04:48:05.566 INFO:tasks.workunit.client.1.smithi179.stdout:7/922: mknod d1/dd/d76/d138/da8/dda/dea/c148 0 2024-04-02T04:48:05.567 INFO:tasks.workunit.client.0.smithi070.stdout:4/856: fsync d1/d3/d85/d8e/dc8/fd7 0 2024-04-02T04:48:05.569 INFO:tasks.workunit.client.1.smithi179.stdout:4/934: creat d1/d3/dc0/f13f x:0 0 0 2024-04-02T04:48:05.571 INFO:tasks.workunit.client.1.smithi179.stdout:4/935: write d1/d3/dc/f1c [3146914,78072] 0 2024-04-02T04:48:05.572 INFO:tasks.workunit.client.1.smithi179.stdout:4/936: stat d1/d3/df/d10/c46 0 2024-04-02T04:48:05.574 INFO:tasks.workunit.client.1.smithi179.stdout:0/991: dwrite d1/da/df/fa2 [0,4194304] 0 2024-04-02T04:48:05.574 INFO:tasks.workunit.client.1.smithi179.stdout:0/992: dread - d1/da/d67/f11a zero size 2024-04-02T04:48:05.576 INFO:tasks.workunit.client.1.smithi179.stdout:0/993: truncate d1/da/d67/de6/fea 962022 0 2024-04-02T04:48:05.576 INFO:tasks.workunit.client.0.smithi070.stdout:6/724: creat d0/d3/d19/d31/fdd x:0 0 0 2024-04-02T04:48:05.583 INFO:tasks.workunit.client.0.smithi070.stdout:6/725: dwrite d0/d20/fcf [0,4194304] 0 2024-04-02T04:48:05.584 INFO:tasks.workunit.client.0.smithi070.stdout:6/726: chown d0/d20/d77/fbf 11838888 1 2024-04-02T04:48:05.586 INFO:tasks.workunit.client.1.smithi179.stdout:1/914: creat d3/d5/d2f/d4f/d42/df2/db6/dd8/f143 x:0 0 0 2024-04-02T04:48:05.588 INFO:tasks.workunit.client.1.smithi179.stdout:6/913: write d3/d42/f7e [1313415,42702] 0 2024-04-02T04:48:05.588 INFO:tasks.workunit.client.1.smithi179.stdout:6/914: chown d3/l19 853563 1 2024-04-02T04:48:05.590 INFO:tasks.workunit.client.1.smithi179.stdout:7/923: creat d1/dd/d76/d138/da8/d100/f149 x:0 0 0 2024-04-02T04:48:05.594 INFO:tasks.workunit.client.0.smithi070.stdout:1/748: dread d0/d9/d1d/d22/d2c/d4f/d77/d8c/faa [0,4194304] 0 2024-04-02T04:48:05.599 INFO:tasks.workunit.client.1.smithi179.stdout:4/937: mkdir d1/d15/d7c/d140 0 2024-04-02T04:48:05.599 INFO:tasks.workunit.client.0.smithi070.stdout:5/961: symlink d5/d57/l12c 0 2024-04-02T04:48:05.600 INFO:tasks.workunit.client.1.smithi179.stdout:4/938: chown d1/d3/dc/dc6/f10f 980636 1 2024-04-02T04:48:05.601 INFO:tasks.workunit.client.1.smithi179.stdout:8/920: creat d3/d32/d21/d22/d6b/db1/d104/d11d/f14b x:0 0 0 2024-04-02T04:48:05.604 INFO:tasks.workunit.client.1.smithi179.stdout:5/877: symlink d8/d1b/d23/l117 0 2024-04-02T04:48:05.606 INFO:tasks.workunit.client.1.smithi179.stdout:6/915: symlink d3/d42/l14b 0 2024-04-02T04:48:05.607 INFO:tasks.workunit.client.0.smithi070.stdout:2/882: getdents d2/d10/d21/d71/dd4/d100 0 2024-04-02T04:48:05.609 INFO:tasks.workunit.client.1.smithi179.stdout:3/928: dwrite d8/d26/f56 [0,4194304] 0 2024-04-02T04:48:05.610 INFO:tasks.workunit.client.1.smithi179.stdout:3/929: chown d8/d6d/l127 3 1 2024-04-02T04:48:05.611 INFO:tasks.workunit.client.1.smithi179.stdout:3/930: readlink d8/d57/d63/dc3/l103 0 2024-04-02T04:48:05.611 INFO:tasks.workunit.client.1.smithi179.stdout:3/931: readlink d8/d57/d8f/la7 0 2024-04-02T04:48:05.612 INFO:tasks.workunit.client.1.smithi179.stdout:3/932: stat d8/d26/f62 0 2024-04-02T04:48:05.612 INFO:tasks.workunit.client.1.smithi179.stdout:3/933: stat d8/d26/dcd/de7/f120 0 2024-04-02T04:48:05.612 INFO:tasks.workunit.client.1.smithi179.stdout:3/934: fsync d8/ff9 0 2024-04-02T04:48:05.612 INFO:tasks.workunit.client.0.smithi070.stdout:1/749: mkdir d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/dea 0 2024-04-02T04:48:05.614 INFO:tasks.workunit.client.1.smithi179.stdout:0/994: mknod d1/da/d48/d65/c126 0 2024-04-02T04:48:05.615 INFO:tasks.workunit.client.0.smithi070.stdout:5/962: creat d5/d9/d23/dc3/f12d x:0 0 0 2024-04-02T04:48:05.616 INFO:tasks.workunit.client.1.smithi179.stdout:9/895: write d0/d19/d1c/f31 [4255097,10682] 0 2024-04-02T04:48:05.616 INFO:tasks.workunit.client.0.smithi070.stdout:5/963: chown d5/d9/d10/db9/dba/l10c 36732 1 2024-04-02T04:48:05.617 INFO:tasks.workunit.client.1.smithi179.stdout:9/896: chown d0/d19/d27/d7a/d84/ff8 77704544 1 2024-04-02T04:48:05.618 INFO:tasks.workunit.client.1.smithi179.stdout:8/921: creat d3/d32/d21/d50/d56/f14c x:0 0 0 2024-04-02T04:48:05.618 INFO:tasks.workunit.client.1.smithi179.stdout:2/805: dwrite dc/f89 [0,4194304] 0 2024-04-02T04:48:05.620 INFO:tasks.workunit.client.0.smithi070.stdout:6/727: truncate d0/d20/f23 3434999 0 2024-04-02T04:48:05.625 INFO:tasks.workunit.client.1.smithi179.stdout:9/897: dwrite d0/d19/d27/dad/fae [4194304,4194304] 0 2024-04-02T04:48:05.626 INFO:tasks.workunit.client.0.smithi070.stdout:1/750: mkdir d0/d7/d28/d83/deb 0 2024-04-02T04:48:05.628 INFO:tasks.workunit.client.1.smithi179.stdout:6/916: mkdir d3/d2d/da5/d14c 0 2024-04-02T04:48:05.630 INFO:tasks.workunit.client.1.smithi179.stdout:7/924: mknod d1/d9/d27/d87/c14a 0 2024-04-02T04:48:05.631 INFO:tasks.workunit.client.0.smithi070.stdout:9/790: truncate d4/d10/d11/d7b/d19/d53/f59 7523644 0 2024-04-02T04:48:05.633 INFO:tasks.workunit.client.1.smithi179.stdout:3/935: symlink d8/d26/d52/d7e/ddf/l12b 0 2024-04-02T04:48:05.633 INFO:tasks.workunit.client.0.smithi070.stdout:8/789: write d2/d3/d35/d3a/d4f/d59/f8f [848789,100869] 0 2024-04-02T04:48:05.635 INFO:tasks.workunit.client.0.smithi070.stdout:6/728: symlink d0/d3/d19/d31/dc1/lde 0 2024-04-02T04:48:05.636 INFO:tasks.workunit.client.0.smithi070.stdout:3/798: dwrite d10/d17/d4c/f84 [0,4194304] 0 2024-04-02T04:48:05.637 INFO:tasks.workunit.client.0.smithi070.stdout:8/790: write d2/f48 [4041372,8736] 0 2024-04-02T04:48:05.638 INFO:tasks.workunit.client.0.smithi070.stdout:3/799: chown d10/d32/d4b/l6e 394072737 1 2024-04-02T04:48:05.639 INFO:tasks.workunit.client.1.smithi179.stdout:0/995: fsync d1/da/d52/fe8 0 2024-04-02T04:48:05.640 INFO:tasks.workunit.client.1.smithi179.stdout:0/996: chown d1/da/d28/d37/f95 31278 1 2024-04-02T04:48:05.641 INFO:tasks.workunit.client.1.smithi179.stdout:4/939: mknod d1/d3/dc/c141 0 2024-04-02T04:48:05.642 INFO:tasks.workunit.client.1.smithi179.stdout:4/940: chown d1/d3/df/d10/d2f/d60/d71/lf9 363717 1 2024-04-02T04:48:05.645 INFO:tasks.workunit.client.1.smithi179.stdout:2/806: symlink dc/d29/d4a/d1e/d6c/dd2/l10d 0 2024-04-02T04:48:05.651 INFO:tasks.workunit.client.1.smithi179.stdout:3/936: creat d8/d26/d52/d7e/db8/f12c x:0 0 0 2024-04-02T04:48:05.652 INFO:tasks.workunit.client.1.smithi179.stdout:3/937: chown d8/c2e 3 1 2024-04-02T04:48:05.653 INFO:tasks.workunit.client.1.smithi179.stdout:2/807: write dc/ff1 [4002309,128682] 0 2024-04-02T04:48:05.653 INFO:tasks.workunit.client.1.smithi179.stdout:3/938: chown d8/d26/dcd/de7/c110 3715 1 2024-04-02T04:48:05.654 INFO:tasks.workunit.client.1.smithi179.stdout:2/808: truncate dc/fe 4209324 0 2024-04-02T04:48:05.654 INFO:tasks.workunit.client.1.smithi179.stdout:1/915: rename d3/d5/d2f/d4f/d42/df2/db6/dd8/f143 to d3/d5/f144 0 2024-04-02T04:48:05.655 INFO:tasks.workunit.client.1.smithi179.stdout:2/809: chown dc/d29/d4a/d1e/d8a/dce 7 1 2024-04-02T04:48:05.655 INFO:tasks.workunit.client.1.smithi179.stdout:9/898: link d0/d19/d27/fe0 d0/d19/d27/d7a/d87/daf/dc2/f119 0 2024-04-02T04:48:05.656 INFO:tasks.workunit.client.1.smithi179.stdout:8/922: link d3/d32/d21/d22/d6b/db1/df8/f124 d3/d32/d21/d50/d56/d94/d84/de4/f14d 0 2024-04-02T04:48:05.656 INFO:tasks.workunit.client.1.smithi179.stdout:2/810: stat dc/d29/d4a/d1e/d43 0 2024-04-02T04:48:05.657 INFO:tasks.workunit.client.1.smithi179.stdout:4/941: rename d1/d3/dc/d26/d8a/d5d/fbc to d1/d3/df/d10/d2f/d60/d71/dce/d118/f142 0 2024-04-02T04:48:05.659 INFO:tasks.workunit.client.1.smithi179.stdout:4/942: rmdir d1/d15 39 2024-04-02T04:48:05.660 INFO:tasks.workunit.client.1.smithi179.stdout:9/899: symlink d0/d1/d11/dab/d113/l11a 0 2024-04-02T04:48:05.661 INFO:tasks.workunit.client.1.smithi179.stdout:8/923: rename d3/d32/d21/d50/d56/d94/d84/fe9 to d3/d1b/d2a/d41/d148/f14e 0 2024-04-02T04:48:05.661 INFO:tasks.workunit.client.1.smithi179.stdout:1/916: dread d3/da8/fb3 [0,4194304] 0 2024-04-02T04:48:05.666 INFO:tasks.workunit.client.0.smithi070.stdout:4/857: rename d1/d3/d9/f101 to d1/d3/d9/f11f 0 2024-04-02T04:48:05.667 INFO:tasks.workunit.client.0.smithi070.stdout:6/729: mknod d0/d43/db3/cdf 0 2024-04-02T04:48:05.669 INFO:tasks.workunit.client.0.smithi070.stdout:4/858: fdatasync d1/d3/d85/d8e/fd9 0 2024-04-02T04:48:05.669 INFO:tasks.workunit.client.0.smithi070.stdout:2/883: creat d2/d10/d21/d71/dd4/f122 x:0 0 0 2024-04-02T04:48:05.670 INFO:tasks.workunit.client.0.smithi070.stdout:3/800: mkdir d10/d17/d51/d73/d9e/d107/d10d 0 2024-04-02T04:48:05.671 INFO:tasks.workunit.client.0.smithi070.stdout:9/791: mknod d4/d10/d11/d7b/d19/d2f/cff 0 2024-04-02T04:48:05.673 INFO:tasks.workunit.client.0.smithi070.stdout:3/801: write d10/d32/d4b/dc5/fd0 [530632,68034] 0 2024-04-02T04:48:05.673 INFO:tasks.workunit.client.0.smithi070.stdout:5/964: getdents d5/d14/dd8/ddb 0 2024-04-02T04:48:05.677 INFO:tasks.workunit.client.0.smithi070.stdout:4/859: truncate d1/d9c/f6e 3435527 0 2024-04-02T04:48:05.678 INFO:tasks.workunit.client.0.smithi070.stdout:4/860: fsync d1/dc3/dc4/ff8 0 2024-04-02T04:48:05.679 INFO:tasks.workunit.client.0.smithi070.stdout:3/802: dread d10/d32/d4b/dc5/fed [0,4194304] 0 2024-04-02T04:48:05.680 INFO:tasks.workunit.client.0.smithi070.stdout:5/965: rmdir d5/d9/d23/d30/dde 39 2024-04-02T04:48:05.682 INFO:tasks.workunit.client.0.smithi070.stdout:6/730: creat d0/d55/d8b/fe0 x:0 0 0 2024-04-02T04:48:05.684 INFO:tasks.workunit.client.0.smithi070.stdout:9/792: chown d4/d10/d11/d7b/d19/d53/f59 35 1 2024-04-02T04:48:05.689 INFO:tasks.workunit.client.0.smithi070.stdout:4/861: rename d1/d3/d85/d8e/d99/db1/fdf to d1/d3/d85/deb/d110/f120 0 2024-04-02T04:48:05.694 INFO:tasks.workunit.client.0.smithi070.stdout:9/793: mknod d4/d13/df0/c100 0 2024-04-02T04:48:05.697 INFO:tasks.workunit.client.0.smithi070.stdout:1/751: write d0/d4c/f70 [601858,94349] 0 2024-04-02T04:48:05.698 INFO:tasks.workunit.client.0.smithi070.stdout:5/966: symlink d5/d9/d23/d30/dde/l12e 0 2024-04-02T04:48:05.699 INFO:tasks.workunit.client.0.smithi070.stdout:6/731: truncate d0/d3/d19/d31/d46/d6b/f72 1183947 0 2024-04-02T04:48:05.701 INFO:tasks.workunit.client.0.smithi070.stdout:9/794: rename d4/d13/f51 to d4/d10/d12/d55/d66/f101 0 2024-04-02T04:48:05.701 INFO:tasks.workunit.client.0.smithi070.stdout:9/795: dread - d4/d10/d11/d7b/d19/f50 zero size 2024-04-02T04:48:05.706 INFO:tasks.workunit.client.0.smithi070.stdout:5/967: truncate d5/d17/d50/fd7 1462875 0 2024-04-02T04:48:05.706 INFO:tasks.workunit.client.0.smithi070.stdout:9/796: mkdir d4/d10/d12/d27/d60/dc9/d102 0 2024-04-02T04:48:05.707 INFO:tasks.workunit.client.0.smithi070.stdout:2/884: rmdir d2/d10/d21/d71/dd4 39 2024-04-02T04:48:05.712 INFO:tasks.workunit.client.0.smithi070.stdout:8/791: dwrite d2/d8/d29/d22/fdd [0,4194304] 0 2024-04-02T04:48:05.714 INFO:tasks.workunit.client.0.smithi070.stdout:9/797: creat d4/d10/d11/d7b/d19/d2f/d8c/f103 x:0 0 0 2024-04-02T04:48:05.715 INFO:tasks.workunit.client.0.smithi070.stdout:9/798: chown d4/d10/d11/d7b/f2b 42429 1 2024-04-02T04:48:05.720 INFO:tasks.workunit.client.1.smithi179.stdout:3/939: read d8/dd3/fd9 [2385620,87303] 0 2024-04-02T04:48:05.720 INFO:tasks.workunit.client.1.smithi179.stdout:6/917: dread d3/d42/f54 [0,4194304] 0 2024-04-02T04:48:05.723 INFO:tasks.workunit.client.0.smithi070.stdout:8/792: dread d2/d3/d35/d3a/d4f/fab [0,4194304] 0 2024-04-02T04:48:05.723 INFO:tasks.workunit.client.0.smithi070.stdout:5/968: dread d5/ff [8388608,4194304] 0 2024-04-02T04:48:05.724 INFO:tasks.workunit.client.1.smithi179.stdout:6/918: write d3/d12/d23/d39/f148 [166854,13494] 0 2024-04-02T04:48:05.724 INFO:tasks.workunit.client.0.smithi070.stdout:6/732: link d0/d55/d8b/cb0 d0/d55/d8b/ce1 0 2024-04-02T04:48:05.724 INFO:tasks.workunit.client.0.smithi070.stdout:6/733: dread - d0/d43/fae zero size 2024-04-02T04:48:05.727 INFO:tasks.workunit.client.1.smithi179.stdout:6/919: read - d3/d9a/f121 zero size 2024-04-02T04:48:05.727 INFO:tasks.workunit.client.1.smithi179.stdout:3/940: truncate d8/d26/d52/d7e/db8/f12c 738391 0 2024-04-02T04:48:05.727 INFO:tasks.workunit.client.1.smithi179.stdout:1/917: creat d3/d5/d2f/db7/f145 x:0 0 0 2024-04-02T04:48:05.730 INFO:tasks.workunit.client.1.smithi179.stdout:8/924: creat d3/d32/d21/d50/dcf/f14f x:0 0 0 2024-04-02T04:48:05.730 INFO:tasks.workunit.client.1.smithi179.stdout:5/878: dwrite d8/f18 [0,4194304] 0 2024-04-02T04:48:05.731 INFO:tasks.workunit.client.0.smithi070.stdout:5/969: rename d5/d17/c1a to d5/d9/d23/d30/dde/c12f 0 2024-04-02T04:48:05.731 INFO:tasks.workunit.client.1.smithi179.stdout:3/941: stat d8/dd3/f8b 0 2024-04-02T04:48:05.731 INFO:tasks.workunit.client.1.smithi179.stdout:7/925: dwrite d1/fc4 [0,4194304] 0 2024-04-02T04:48:05.731 INFO:tasks.workunit.client.1.smithi179.stdout:6/920: chown d3/l19 474803 1 2024-04-02T04:48:05.735 INFO:tasks.workunit.client.1.smithi179.stdout:6/921: chown d3/d42/fc8 78198158 1 2024-04-02T04:48:05.736 INFO:tasks.workunit.client.0.smithi070.stdout:6/734: creat d0/d20/d5f/fe2 x:0 0 0 2024-04-02T04:48:05.736 INFO:tasks.workunit.client.1.smithi179.stdout:5/879: chown d8/d1b/d23/d32/dbe/daf/db7/f79 1 1 2024-04-02T04:48:05.737 INFO:tasks.workunit.client.1.smithi179.stdout:4/943: dwrite d1/d3/df/d10/d2f/d7b/f91 [0,4194304] 0 2024-04-02T04:48:05.738 INFO:tasks.workunit.client.1.smithi179.stdout:8/925: write d3/d32/d21/d22/d6b/db1/df8/d126/f121 [163300,42684] 0 2024-04-02T04:48:05.739 INFO:tasks.workunit.client.1.smithi179.stdout:5/880: chown d8/f6c 118 1 2024-04-02T04:48:05.740 INFO:tasks.workunit.client.0.smithi070.stdout:8/793: creat d2/d8/d29/dc7/df0/f104 x:0 0 0 2024-04-02T04:48:05.741 INFO:tasks.workunit.client.1.smithi179.stdout:5/881: write d8/d1b/d23/d32/d4e/fb8 [624882,93772] 0 2024-04-02T04:48:05.744 INFO:tasks.workunit.client.0.smithi070.stdout:3/803: dwrite d10/d32/d8a/fe5 [0,4194304] 0 2024-04-02T04:48:05.745 INFO:tasks.workunit.client.1.smithi179.stdout:7/926: dwrite d1/dd/d76/d138/da8/d104/f117 [0,4194304] 0 2024-04-02T04:48:05.747 INFO:tasks.workunit.client.1.smithi179.stdout:4/944: dwrite d1/d93/de6/f10e [0,4194304] 0 2024-04-02T04:48:05.750 INFO:tasks.workunit.client.0.smithi070.stdout:4/862: write d1/d3/d85/deb/f116 [2762375,104876] 0 2024-04-02T04:48:05.750 INFO:tasks.workunit.client.1.smithi179.stdout:9/900: getdents d0/d1/d10/d6d 0 2024-04-02T04:48:05.751 INFO:tasks.workunit.client.0.smithi070.stdout:9/799: link d4/d10/d11/d7b/d19/f50 d4/d10/d11/d7b/d19/d82/f104 0 2024-04-02T04:48:05.752 INFO:tasks.workunit.client.1.smithi179.stdout:0/997: sync 2024-04-02T04:48:05.767 INFO:tasks.workunit.client.0.smithi070.stdout:1/752: dwrite d0/d54/f87 [0,4194304] 0 2024-04-02T04:48:05.768 INFO:tasks.workunit.client.1.smithi179.stdout:0/998: dread d1/f22 [0,4194304] 0 2024-04-02T04:48:05.769 INFO:tasks.workunit.client.1.smithi179.stdout:0/999: chown d1/da/d52/d75/d9a 368142 1 2024-04-02T04:48:05.773 INFO:tasks.workunit.client.0.smithi070.stdout:2/885: truncate d2/d10/d53/f55 1448928 0 2024-04-02T04:48:05.776 INFO:tasks.workunit.client.0.smithi070.stdout:3/804: rmdir d10/d32/d8a/d8d 39 2024-04-02T04:48:05.780 INFO:tasks.workunit.client.0.smithi070.stdout:5/970: creat d5/d105/f130 x:0 0 0 2024-04-02T04:48:05.782 INFO:tasks.workunit.client.0.smithi070.stdout:4/863: dread d1/f12 [0,4194304] 0 2024-04-02T04:48:05.788 INFO:tasks.workunit.client.1.smithi179.stdout:6/922: dread d3/d12/d23/d39/d4e/d66/f132 [0,4194304] 0 2024-04-02T04:48:05.789 INFO:tasks.workunit.client.1.smithi179.stdout:6/923: readlink d3/d2d/l31 0 2024-04-02T04:48:05.789 INFO:tasks.workunit.client.0.smithi070.stdout:1/753: mkdir d0/d9/d94/dec 0 2024-04-02T04:48:05.790 INFO:tasks.workunit.client.1.smithi179.stdout:9/901: mknod d0/d19/d27/d7a/d7e/db9/dfa/c11b 0 2024-04-02T04:48:05.791 INFO:tasks.workunit.client.1.smithi179.stdout:2/811: sync 2024-04-02T04:48:05.792 INFO:tasks.workunit.client.1.smithi179.stdout:2/812: write dc/f44 [3551526,74348] 0 2024-04-02T04:48:05.793 INFO:tasks.workunit.client.0.smithi070.stdout:2/886: rename d2/d12/d3e/fac to d2/d10/d21/d71/dc1/f123 0 2024-04-02T04:48:05.794 INFO:tasks.workunit.client.1.smithi179.stdout:8/926: mkdir d3/d31/d150 0 2024-04-02T04:48:05.795 INFO:tasks.workunit.client.1.smithi179.stdout:8/927: write d3/d32/d21/d22/f123 [4004824,1918] 0 2024-04-02T04:48:05.796 INFO:tasks.workunit.client.1.smithi179.stdout:8/928: readlink d3/d1b/d2a/d41/d148/ld7 0 2024-04-02T04:48:05.797 INFO:tasks.workunit.client.1.smithi179.stdout:1/918: creat d3/d5/d2f/d5a/f146 x:0 0 0 2024-04-02T04:48:05.799 INFO:tasks.workunit.client.0.smithi070.stdout:2/887: dwrite d2/fb [4194304,4194304] 0 2024-04-02T04:48:05.800 INFO:tasks.workunit.client.1.smithi179.stdout:1/919: dread - d3/d5/d2f/d4f/dab/dcd/f134 zero size 2024-04-02T04:48:05.801 INFO:tasks.workunit.client.1.smithi179.stdout:3/942: dwrite d8/d26/d52/d7e/db8/fba [0,4194304] 0 2024-04-02T04:48:05.802 INFO:tasks.workunit.client.1.smithi179.stdout:8/929: dread d3/d32/d21/d50/d56/d94/d84/fa9 [0,4194304] 0 2024-04-02T04:48:05.804 INFO:tasks.workunit.client.0.smithi070.stdout:9/800: symlink d4/d10/d11/d7b/d19/l105 0 2024-04-02T04:48:05.808 INFO:tasks.workunit.client.0.smithi070.stdout:8/794: dwrite d2/d3/d35/d3a/d4f/de8/ffa [0,4194304] 0 2024-04-02T04:48:05.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:05 smithi179 ceph-mon[36603]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-04-02T04:48:05.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:05 smithi179 ceph-mon[36603]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-04-02T04:48:05.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:05 smithi179 ceph-mon[36603]: pgmap v21: 65 pgs: 65 active+clean; 3.0 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 92 MiB/s rd, 217 MiB/s wr, 509 op/s 2024-04-02T04:48:05.823 INFO:tasks.workunit.client.1.smithi179.stdout:2/813: unlink dc/d29/d31/d57/dea/cfb 0 2024-04-02T04:48:05.828 INFO:tasks.workunit.client.0.smithi070.stdout:5/971: truncate d5/d17/d50/f5f 1104630 0 2024-04-02T04:48:05.829 INFO:tasks.workunit.client.1.smithi179.stdout:4/945: dread d1/d15/d7c/d98/f113 [0,4194304] 0 2024-04-02T04:48:05.830 INFO:tasks.workunit.client.1.smithi179.stdout:7/927: write d1/d9/f12 [3085334,5194] 0 2024-04-02T04:48:05.831 INFO:tasks.workunit.client.0.smithi070.stdout:3/805: dwrite d10/d32/d3f/f6d [0,4194304] 0 2024-04-02T04:48:05.833 INFO:tasks.workunit.client.1.smithi179.stdout:1/920: fsync d3/d5/d2f/d4f/d42/df2/f7e 0 2024-04-02T04:48:05.834 INFO:tasks.workunit.client.1.smithi179.stdout:7/928: dread d1/dd/d23/dab/fba [0,4194304] 0 2024-04-02T04:48:05.836 INFO:tasks.workunit.client.0.smithi070.stdout:4/864: dwrite d1/d9c/d65/fb4 [0,4194304] 0 2024-04-02T04:48:05.840 INFO:tasks.workunit.client.0.smithi070.stdout:1/754: dwrite d0/d9/d1d/d22/d56/fbf [4194304,4194304] 0 2024-04-02T04:48:05.842 INFO:tasks.workunit.client.1.smithi179.stdout:9/902: dwrite d0/d1/d11/d29/f50 [4194304,4194304] 0 2024-04-02T04:48:05.845 INFO:tasks.workunit.client.1.smithi179.stdout:8/930: symlink d3/d1b/d2a/db0/l151 0 2024-04-02T04:48:05.854 INFO:tasks.workunit.client.1.smithi179.stdout:3/943: write d8/d26/d36/fb9 [566252,17815] 0 2024-04-02T04:48:05.855 INFO:tasks.workunit.client.0.smithi070.stdout:2/888: unlink d2/d10/d21/d71/dd4/dce/fff 0 2024-04-02T04:48:05.857 INFO:tasks.workunit.client.1.smithi179.stdout:6/924: write d3/d2c/de0/f126 [362697,119434] 0 2024-04-02T04:48:05.858 INFO:tasks.workunit.client.1.smithi179.stdout:6/925: stat d3/d12/d23/d26/c68 0 2024-04-02T04:48:05.859 INFO:tasks.workunit.client.1.smithi179.stdout:5/882: getdents d8/d1b/d23/d32/dbe 0 2024-04-02T04:48:05.861 INFO:tasks.workunit.client.0.smithi070.stdout:4/865: creat d1/d3/d85/d8e/dc8/f121 x:0 0 0 2024-04-02T04:48:05.863 INFO:tasks.workunit.client.1.smithi179.stdout:1/921: fsync d3/d5/d96/f119 0 2024-04-02T04:48:05.863 INFO:tasks.workunit.client.0.smithi070.stdout:1/755: rmdir d0/d7/d1a 39 2024-04-02T04:48:05.871 INFO:tasks.workunit.client.1.smithi179.stdout:2/814: rename dc/d29/d31/d83/d8d/lf5 to dc/d29/d31/d83/dae/db0/dbc/l10e 0 2024-04-02T04:48:05.872 INFO:tasks.workunit.client.0.smithi070.stdout:2/889: dread d2/d10/d21/d3d/d42/fad [0,4194304] 0 2024-04-02T04:48:05.874 INFO:tasks.workunit.client.0.smithi070.stdout:3/806: chown d10/d32/d8a/d8d/d91/cc8 2925334 1 2024-04-02T04:48:05.877 INFO:tasks.workunit.client.0.smithi070.stdout:4/866: creat d1/d9c/d65/d7a/f122 x:0 0 0 2024-04-02T04:48:05.878 INFO:tasks.workunit.client.1.smithi179.stdout:9/903: write d0/d19/d27/d7a/d7e/fb8 [1537058,47898] 0 2024-04-02T04:48:05.878 INFO:tasks.workunit.client.1.smithi179.stdout:6/926: dread d3/d2c/de0/f144 [0,4194304] 0 2024-04-02T04:48:05.879 INFO:tasks.workunit.client.1.smithi179.stdout:8/931: dread d3/d31/f9e [0,4194304] 0 2024-04-02T04:48:05.880 INFO:tasks.workunit.client.1.smithi179.stdout:8/932: chown d3/d32/d21/d22/da5/f13f 300168 1 2024-04-02T04:48:05.881 INFO:tasks.workunit.client.0.smithi070.stdout:9/801: dwrite d4/d10/d11/d7b/d19/d2f/f3c [0,4194304] 0 2024-04-02T04:48:05.882 INFO:tasks.workunit.client.1.smithi179.stdout:1/922: chown d3/d5/f25 6780956 1 2024-04-02T04:48:05.892 INFO:tasks.workunit.client.0.smithi070.stdout:4/867: dread d1/d3/d78/dad/dbd/fc9 [0,4194304] 0 2024-04-02T04:48:05.892 INFO:tasks.workunit.client.1.smithi179.stdout:7/929: rename d1/dd/d16 to d1/dd/d76/d138/da8/dda/d14b 0 2024-04-02T04:48:05.892 INFO:tasks.workunit.client.1.smithi179.stdout:5/883: mknod d8/d1b/c118 0 2024-04-02T04:48:05.893 INFO:tasks.workunit.client.1.smithi179.stdout:8/933: mknod d3/d1b/d2a/d41/d148/c152 0 2024-04-02T04:48:05.893 INFO:tasks.workunit.client.1.smithi179.stdout:6/927: dwrite d3/d2c/d85/f12c [0,4194304] 0 2024-04-02T04:48:05.893 INFO:tasks.workunit.client.1.smithi179.stdout:1/923: unlink d3/d5/d2f/d4f/cf1 0 2024-04-02T04:48:05.895 INFO:tasks.workunit.client.1.smithi179.stdout:4/946: getdents d1/d93/de6 0 2024-04-02T04:48:05.896 INFO:tasks.workunit.client.1.smithi179.stdout:5/884: write d8/d1b/d23/d32/d4e/f54 [2853172,64461] 0 2024-04-02T04:48:05.900 INFO:tasks.workunit.client.1.smithi179.stdout:1/924: fsync d3/d5/fc8 0 2024-04-02T04:48:05.903 INFO:tasks.workunit.client.1.smithi179.stdout:6/928: dwrite d3/f7f [0,4194304] 0 2024-04-02T04:48:05.904 INFO:tasks.workunit.client.1.smithi179.stdout:8/934: creat d3/d32/d21/d49/f153 x:0 0 0 2024-04-02T04:48:05.909 INFO:tasks.workunit.client.1.smithi179.stdout:4/947: read d1/d3/f1b [2364380,61349] 0 2024-04-02T04:48:05.910 INFO:tasks.workunit.client.1.smithi179.stdout:5/885: write d8/d1b/f3b [3490741,39614] 0 2024-04-02T04:48:05.911 INFO:tasks.workunit.client.1.smithi179.stdout:5/886: chown d8/d1b/d23/d32/d4e/d50/l57 15535 1 2024-04-02T04:48:05.917 INFO:tasks.workunit.client.1.smithi179.stdout:7/930: mkdir d1/d9/dd9/d121/d145/d14c 0 2024-04-02T04:48:05.921 INFO:tasks.workunit.client.1.smithi179.stdout:6/929: fsync d3/d9a/fe7 0 2024-04-02T04:48:05.921 INFO:tasks.workunit.client.1.smithi179.stdout:6/930: stat d3/d12/d23/d39/d4e/d6c/cb4 0 2024-04-02T04:48:05.922 INFO:tasks.workunit.client.1.smithi179.stdout:8/935: rmdir d3/d32/d21/d22/da5 39 2024-04-02T04:48:05.922 INFO:tasks.workunit.client.1.smithi179.stdout:6/931: write d3/d12/d23/d26/d57/f9e [182626,96119] 0 2024-04-02T04:48:05.923 INFO:tasks.workunit.client.1.smithi179.stdout:8/936: stat d3/d31/dc3/d10c/l13a 0 2024-04-02T04:48:05.923 INFO:tasks.workunit.client.1.smithi179.stdout:6/932: readlink d3/d12/d23/d26/dd3/l10d 0 2024-04-02T04:48:05.924 INFO:tasks.workunit.client.1.smithi179.stdout:6/933: chown d3/d125 11 1 2024-04-02T04:48:05.924 INFO:tasks.workunit.client.1.smithi179.stdout:6/934: chown d3/d42/c88 385297 1 2024-04-02T04:48:05.925 INFO:tasks.workunit.client.1.smithi179.stdout:4/948: fdatasync d1/d3/dc/dc1/fd2 0 2024-04-02T04:48:05.926 INFO:tasks.workunit.client.1.smithi179.stdout:4/949: chown d1/d3/f11b 0 1 2024-04-02T04:48:05.928 INFO:tasks.workunit.client.1.smithi179.stdout:7/931: symlink d1/d9/d55/dbf/l14d 0 2024-04-02T04:48:05.932 INFO:tasks.workunit.client.1.smithi179.stdout:8/937: rmdir d3/d32/d21/d22/d6b/d91 39 2024-04-02T04:48:05.936 INFO:tasks.workunit.client.1.smithi179.stdout:1/925: write d3/d5/d96/f119 [127824,7902] 0 2024-04-02T04:48:05.938 INFO:tasks.workunit.client.1.smithi179.stdout:3/944: rename d8/d26/d83/d89/d9c/c109 to d8/d26/c12d 0 2024-04-02T04:48:05.939 INFO:tasks.workunit.client.1.smithi179.stdout:3/945: write d8/d6d/d94/fea [1046240,748] 0 2024-04-02T04:48:05.940 INFO:tasks.workunit.client.1.smithi179.stdout:7/932: fsync d1/d9/d55/fd7 0 2024-04-02T04:48:05.948 INFO:tasks.workunit.client.1.smithi179.stdout:2/815: rename dc/ff1 to dc/d29/d31/d57/f10f 0 2024-04-02T04:48:05.951 INFO:tasks.workunit.client.0.smithi070.stdout:8/795: rename d2/d8/d29/d22/dc8/lf9 to d2/d8/d23/d24/l105 0 2024-04-02T04:48:05.953 INFO:tasks.workunit.client.1.smithi179.stdout:3/946: mknod d8/d6d/d94/c12e 0 2024-04-02T04:48:05.954 INFO:tasks.workunit.client.1.smithi179.stdout:7/933: symlink d1/d9/d27/d87/d8b/d112/l14e 0 2024-04-02T04:48:05.956 INFO:tasks.workunit.client.1.smithi179.stdout:7/934: write d1/dd/d76/d138/da8/dda/dea/f108 [434361,26813] 0 2024-04-02T04:48:05.956 INFO:tasks.workunit.client.0.smithi070.stdout:1/756: mknod d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/ced 0 2024-04-02T04:48:05.957 INFO:tasks.workunit.client.1.smithi179.stdout:8/938: write d3/d32/d21/d49/df0/df9/f102 [513828,105107] 0 2024-04-02T04:48:05.959 INFO:tasks.workunit.client.0.smithi070.stdout:2/890: truncate d2/f9c 2768644 0 2024-04-02T04:48:05.962 INFO:tasks.workunit.client.0.smithi070.stdout:2/891: dread d2/d10/d21/d71/f76 [0,4194304] 0 2024-04-02T04:48:05.963 INFO:tasks.workunit.client.1.smithi179.stdout:1/926: dwrite d3/d5/d2f/fb0 [0,4194304] 0 2024-04-02T04:48:05.964 INFO:tasks.workunit.client.1.smithi179.stdout:5/887: rename d8/d1b/d23/d36/fca to d8/d1b/d23/d32/d46/f119 0 2024-04-02T04:48:05.965 INFO:tasks.workunit.client.1.smithi179.stdout:2/816: mkdir dc/d29/d10b/d110 0 2024-04-02T04:48:05.966 INFO:tasks.workunit.client.1.smithi179.stdout:2/817: fsync dc/d79/ff2 0 2024-04-02T04:48:05.970 INFO:tasks.workunit.client.1.smithi179.stdout:6/935: link d3/d12/d23/d39/d80/d93/lba d3/df2/l14d 0 2024-04-02T04:48:05.973 INFO:tasks.workunit.client.1.smithi179.stdout:5/888: dwrite d8/d1b/d23/d32/d4e/fb8 [0,4194304] 0 2024-04-02T04:48:05.976 INFO:tasks.workunit.client.1.smithi179.stdout:5/889: read d8/d1b/d23/d32/d46/f119 [562157,55692] 0 2024-04-02T04:48:05.976 INFO:tasks.workunit.client.1.smithi179.stdout:4/950: rename d1/d93/d101/f105 to d1/d15/d7c/d98/f143 0 2024-04-02T04:48:05.978 INFO:tasks.workunit.client.1.smithi179.stdout:1/927: read d3/d5/f4a [2408881,55453] 0 2024-04-02T04:48:05.980 INFO:tasks.workunit.client.1.smithi179.stdout:6/936: dwrite d3/d2d/d38/d64/f109 [0,4194304] 0 2024-04-02T04:48:05.983 INFO:tasks.workunit.client.0.smithi070.stdout:8/796: truncate d2/d8/d23/d24/d66/f6a 220518 0 2024-04-02T04:48:05.986 INFO:tasks.workunit.client.1.smithi179.stdout:2/818: mknod dc/d29/d4a/d1e/d43/c111 0 2024-04-02T04:48:05.991 INFO:tasks.workunit.client.1.smithi179.stdout:7/935: dread d1/d9/d7e/ff2 [0,4194304] 0 2024-04-02T04:48:05.994 INFO:tasks.workunit.client.0.smithi070.stdout:9/802: truncate d4/d10/d11/d7b/f61 1869944 0 2024-04-02T04:48:05.995 INFO:tasks.workunit.client.1.smithi179.stdout:8/939: dwrite d3/d32/d21/d49/f9f [0,4194304] 0 2024-04-02T04:48:05.996 INFO:tasks.workunit.client.1.smithi179.stdout:9/904: sync 2024-04-02T04:48:05.997 INFO:tasks.workunit.client.0.smithi070.stdout:4/868: fdatasync d1/d3/d9/d5e/f87 0 2024-04-02T04:48:05.999 INFO:tasks.workunit.client.1.smithi179.stdout:3/947: link d8/d26/d83/d89/d9c/ca4 d8/d26/d52/d7e/db8/c12f 0 2024-04-02T04:48:06.000 INFO:tasks.workunit.client.1.smithi179.stdout:3/948: chown d8/d26/d52/d7e/def/f10c 0 1 2024-04-02T04:48:06.001 INFO:tasks.workunit.client.1.smithi179.stdout:3/949: dread - d8/d26/d52/d7e/ddf/f108 zero size 2024-04-02T04:48:06.002 INFO:tasks.workunit.client.0.smithi070.stdout:5/972: rename d5/d9/d23/d30/da2/d75/l11b to d5/d17/d27/l131 0 2024-04-02T04:48:06.004 INFO:tasks.workunit.client.0.smithi070.stdout:7/789: sync 2024-04-02T04:48:06.007 INFO:tasks.workunit.client.1.smithi179.stdout:3/950: dwrite d8/dd3/f113 [0,4194304] 0 2024-04-02T04:48:06.007 INFO:tasks.workunit.client.0.smithi070.stdout:8/797: truncate d2/d3/d52/d62/f95 2337659 0 2024-04-02T04:48:06.009 INFO:tasks.workunit.client.0.smithi070.stdout:1/757: truncate d0/d7/d1a/f9a 1135218 0 2024-04-02T04:48:06.011 INFO:tasks.workunit.client.0.smithi070.stdout:3/807: link d10/d17/f4a d10/d17/db2/dbf/dd2/f10e 0 2024-04-02T04:48:06.013 INFO:tasks.workunit.client.1.smithi179.stdout:6/937: symlink d3/d2c/de0/l14e 0 2024-04-02T04:48:06.017 INFO:tasks.workunit.client.0.smithi070.stdout:9/803: fdatasync d4/d10/d12/d55/d66/f7c 0 2024-04-02T04:48:06.024 INFO:tasks.workunit.client.0.smithi070.stdout:2/892: dwrite d2/d10/d5c/f81 [0,4194304] 0 2024-04-02T04:48:06.026 INFO:tasks.workunit.client.0.smithi070.stdout:4/869: creat d1/d9c/f123 x:0 0 0 2024-04-02T04:48:06.027 INFO:tasks.workunit.client.0.smithi070.stdout:2/893: chown d2/d10/d19/dd7/l103 182191171 1 2024-04-02T04:48:06.028 INFO:tasks.workunit.client.1.smithi179.stdout:6/938: dread d3/d9a/fe7 [0,4194304] 0 2024-04-02T04:48:06.028 INFO:tasks.workunit.client.1.smithi179.stdout:8/940: read d3/d32/d21/d22/f62 [69981,125745] 0 2024-04-02T04:48:06.028 INFO:tasks.workunit.client.0.smithi070.stdout:5/973: fsync d5/d17/d2b/d7b/ff9 0 2024-04-02T04:48:06.029 INFO:tasks.workunit.client.1.smithi179.stdout:8/941: fsync d3/d1b/d2a/d41/d109/f136 0 2024-04-02T04:48:06.030 INFO:tasks.workunit.client.0.smithi070.stdout:5/974: readlink d5/d9/d10/db9/dba/lf1 0 2024-04-02T04:48:06.030 INFO:tasks.workunit.client.1.smithi179.stdout:9/905: creat d0/d1/d11/d29/d86/db0/f11c x:0 0 0 2024-04-02T04:48:06.033 INFO:tasks.workunit.client.1.smithi179.stdout:1/928: dread d3/d5/f1d [0,4194304] 0 2024-04-02T04:48:06.035 INFO:tasks.workunit.client.1.smithi179.stdout:4/951: write d1/d3/df/f51 [2644022,16061] 0 2024-04-02T04:48:06.037 INFO:tasks.workunit.client.0.smithi070.stdout:1/758: truncate d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/fb2 679843 0 2024-04-02T04:48:06.039 INFO:tasks.workunit.client.0.smithi070.stdout:3/808: rename d10/d66/cc7 to d10/d17/db2/de2/c10f 0 2024-04-02T04:48:06.045 INFO:tasks.workunit.client.1.smithi179.stdout:6/939: chown d3/d12/d23/d39/d80/c13f 6989812 1 2024-04-02T04:48:06.045 INFO:tasks.workunit.client.1.smithi179.stdout:5/890: write d8/d1b/d23/d32/d4e/d50/f68 [2172207,101894] 0 2024-04-02T04:48:06.048 INFO:tasks.workunit.client.1.smithi179.stdout:9/906: chown d0/d19/d27/d7a/d87/daf/dc2/ce8 1372321704 1 2024-04-02T04:48:06.051 INFO:tasks.workunit.client.1.smithi179.stdout:1/929: rename d3/d85/da2/fb5 to d3/d5/d2f/d4f/d42/d53/dfe/f147 0 2024-04-02T04:48:06.052 INFO:tasks.workunit.client.1.smithi179.stdout:3/951: mknod d8/d57/c130 0 2024-04-02T04:48:06.053 INFO:tasks.workunit.client.1.smithi179.stdout:3/952: chown d8/d26/d52/d7e/db8 206608 1 2024-04-02T04:48:06.053 INFO:tasks.workunit.client.1.smithi179.stdout:4/952: mkdir d1/d3/d128/d144 0 2024-04-02T04:48:06.054 INFO:tasks.workunit.client.1.smithi179.stdout:4/953: chown d1/d3/dc/dc1/ff4 119944 1 2024-04-02T04:48:06.055 INFO:tasks.workunit.client.0.smithi070.stdout:7/790: mknod d7/dd/df/d79/d46/c101 0 2024-04-02T04:48:06.057 INFO:tasks.workunit.client.1.smithi179.stdout:7/936: creat d1/d9/d27/d87/d8b/d75/db9/f14f x:0 0 0 2024-04-02T04:48:06.058 INFO:tasks.workunit.client.0.smithi070.stdout:8/798: write d2/d8/d29/d22/d5a/f7e [4945915,52430] 0 2024-04-02T04:48:06.058 INFO:tasks.workunit.client.1.smithi179.stdout:8/942: dread d3/f27 [0,4194304] 0 2024-04-02T04:48:06.059 INFO:tasks.workunit.client.1.smithi179.stdout:5/891: symlink d8/d1b/d23/d32/d4e/d50/l11a 0 2024-04-02T04:48:06.060 INFO:tasks.workunit.client.0.smithi070.stdout:1/759: mkdir d0/d54/d64/dee 0 2024-04-02T04:48:06.061 INFO:tasks.workunit.client.1.smithi179.stdout:9/907: symlink d0/d19/d27/d7a/d87/daf/dc2/l11d 0 2024-04-02T04:48:06.062 INFO:tasks.workunit.client.1.smithi179.stdout:9/908: write d0/d1/d11/d29/d40/f49 [5660862,31163] 0 2024-04-02T04:48:06.067 INFO:tasks.workunit.client.1.smithi179.stdout:3/953: creat d8/dd3/d9a/de8/f131 x:0 0 0 2024-04-02T04:48:06.068 INFO:tasks.workunit.client.0.smithi070.stdout:9/804: write d4/f8 [985936,42365] 0 2024-04-02T04:48:06.070 INFO:tasks.workunit.client.0.smithi070.stdout:2/894: mknod d2/c124 0 2024-04-02T04:48:06.071 INFO:tasks.workunit.client.1.smithi179.stdout:2/819: dwrite dc/d29/d4a/faa [0,4194304] 0 2024-04-02T04:48:06.072 INFO:tasks.workunit.client.1.smithi179.stdout:7/937: chown d1/dd/d23/dab/cf9 94189644 1 2024-04-02T04:48:06.085 INFO:tasks.workunit.client.0.smithi070.stdout:3/809: link d10/d17/d51/fb0 d10/d17/db2/de2/f110 0 2024-04-02T04:48:06.086 INFO:tasks.workunit.client.1.smithi179.stdout:3/954: chown d8/c25 44 1 2024-04-02T04:48:06.087 INFO:tasks.workunit.client.0.smithi070.stdout:9/805: fdatasync d4/d10/d11/f4e 0 2024-04-02T04:48:06.090 INFO:tasks.workunit.client.0.smithi070.stdout:0/915: sync 2024-04-02T04:48:06.090 INFO:tasks.workunit.client.0.smithi070.stdout:6/735: sync 2024-04-02T04:48:06.090 INFO:tasks.workunit.client.0.smithi070.stdout:2/895: mkdir d2/d10/d21/d71/dc1/d125 0 2024-04-02T04:48:06.092 INFO:tasks.workunit.client.0.smithi070.stdout:0/916: fsync d4/d5/d13/d2a/d29/f5d 0 2024-04-02T04:48:06.094 INFO:tasks.workunit.client.1.smithi179.stdout:9/909: dwrite d0/d1/d11/dab/fbb [0,4194304] 0 2024-04-02T04:48:06.094 INFO:tasks.workunit.client.0.smithi070.stdout:5/975: creat d5/d9/f132 x:0 0 0 2024-04-02T04:48:06.096 INFO:tasks.workunit.client.1.smithi179.stdout:6/940: creat d3/d12/f14f x:0 0 0 2024-04-02T04:48:06.097 INFO:tasks.workunit.client.0.smithi070.stdout:0/917: dread d4/d5/d13/d2a/fea [0,4194304] 0 2024-04-02T04:48:06.098 INFO:tasks.workunit.client.0.smithi070.stdout:7/791: link d7/dd/df/d34/d68/ff3 d7/dd/df/d7a/dbc/f102 0 2024-04-02T04:48:06.098 INFO:tasks.workunit.client.0.smithi070.stdout:0/918: chown d4/d5/d13/d2a/d29/l57 60829 1 2024-04-02T04:48:06.101 INFO:tasks.workunit.client.1.smithi179.stdout:7/938: unlink d1/dd/d76/d138/da8/dda/d14b/d4d/f131 0 2024-04-02T04:48:06.103 INFO:tasks.workunit.client.0.smithi070.stdout:8/799: mkdir d2/d8/d23/d24/d106 0 2024-04-02T04:48:06.104 INFO:tasks.workunit.client.1.smithi179.stdout:8/943: mknod d3/d32/d21/d50/d56/d94/c154 0 2024-04-02T04:48:06.108 INFO:tasks.workunit.client.0.smithi070.stdout:3/810: rename d10/d17/db2/de2 to d10/d66/d103/d111 0 2024-04-02T04:48:06.108 INFO:tasks.workunit.client.0.smithi070.stdout:4/870: link d1/d9c/df0/f117 d1/d9c/f124 0 2024-04-02T04:48:06.109 INFO:tasks.workunit.client.1.smithi179.stdout:4/954: dwrite d1/d3/df/d10/d2f/d36/fdb [4194304,4194304] 0 2024-04-02T04:48:06.112 INFO:tasks.workunit.client.1.smithi179.stdout:2/820: dwrite dc/f52 [0,4194304] 0 2024-04-02T04:48:06.116 INFO:tasks.workunit.client.0.smithi070.stdout:4/871: dwrite d1/d9c/d65/d7a/f122 [0,4194304] 0 2024-04-02T04:48:06.122 INFO:tasks.workunit.client.1.smithi179.stdout:7/939: dread d1/dd/d76/d138/da8/dda/fc0 [0,4194304] 0 2024-04-02T04:48:06.128 INFO:tasks.workunit.client.0.smithi070.stdout:7/792: creat d7/dd/df/d34/d68/f103 x:0 0 0 2024-04-02T04:48:06.128 INFO:tasks.workunit.client.0.smithi070.stdout:1/760: dwrite d0/d9/d1d/d22/d56/f69 [0,4194304] 0 2024-04-02T04:48:06.129 INFO:tasks.workunit.client.0.smithi070.stdout:1/761: readlink d0/lae 0 2024-04-02T04:48:06.136 INFO:tasks.workunit.client.0.smithi070.stdout:2/896: dwrite d2/d24/f8f [0,4194304] 0 2024-04-02T04:48:06.137 INFO:tasks.workunit.client.0.smithi070.stdout:3/811: fdatasync d10/d17/d4c/f62 0 2024-04-02T04:48:06.138 INFO:tasks.workunit.client.0.smithi070.stdout:2/897: dread - d2/d10/d5c/d7a/db0/de4/f106 zero size 2024-04-02T04:48:06.139 INFO:tasks.workunit.client.0.smithi070.stdout:2/898: chown d2/d10/d21/d71/caf 26107 1 2024-04-02T04:48:06.141 INFO:tasks.workunit.client.0.smithi070.stdout:9/806: link d4/d10/d12/d55/d66/fe5 d4/f106 0 2024-04-02T04:48:06.148 INFO:tasks.workunit.client.0.smithi070.stdout:4/872: symlink d1/d3/d85/deb/d110/l125 0 2024-04-02T04:48:06.153 INFO:tasks.workunit.client.0.smithi070.stdout:8/800: dwrite d2/d8/d29/d22/fef [0,4194304] 0 2024-04-02T04:48:06.161 INFO:tasks.workunit.client.0.smithi070.stdout:1/762: rmdir d0/d9/d1d/d22/d56/d79/d89 39 2024-04-02T04:48:06.164 INFO:tasks.workunit.client.0.smithi070.stdout:6/736: truncate d0/d55/d8b/d94/fb9 2461708 0 2024-04-02T04:48:06.165 INFO:tasks.workunit.client.0.smithi070.stdout:5/976: rename d5/cd2 to d5/d9/d23/d30/da2/d75/ddf/c133 0 2024-04-02T04:48:06.166 INFO:tasks.workunit.client.0.smithi070.stdout:5/977: chown d5/d17/d2b/d7b/l8e 549158976 1 2024-04-02T04:48:06.171 INFO:tasks.workunit.client.0.smithi070.stdout:9/807: mkdir d4/d10/d12/d27/d60/d9e/d107 0 2024-04-02T04:48:06.173 INFO:tasks.workunit.client.0.smithi070.stdout:4/873: write d1/d9c/f88 [745384,328] 0 2024-04-02T04:48:06.174 INFO:tasks.workunit.client.0.smithi070.stdout:7/793: dwrite d7/dd/df/d79/d46/d65/d7b/fb8 [0,4194304] 0 2024-04-02T04:48:06.176 INFO:tasks.workunit.client.0.smithi070.stdout:8/801: creat d2/d8/d29/d22/d5a/d9e/f107 x:0 0 0 2024-04-02T04:48:06.177 INFO:tasks.workunit.client.0.smithi070.stdout:0/919: link d4/d5/cce d4/d5/d13/d2a/d3a/d45/d112/c119 0 2024-04-02T04:48:06.178 INFO:tasks.workunit.client.0.smithi070.stdout:0/920: dread - d4/d5/d58/d60/d83/feb zero size 2024-04-02T04:48:06.182 INFO:tasks.workunit.client.0.smithi070.stdout:6/737: creat d0/d3/dc2/fe3 x:0 0 0 2024-04-02T04:48:06.187 INFO:tasks.workunit.client.0.smithi070.stdout:5/978: fdatasync d5/d14/dd8/ddb/fdc 0 2024-04-02T04:48:06.189 INFO:tasks.workunit.client.0.smithi070.stdout:3/812: truncate d10/d17/d51/d73/d8f/f104 852212 0 2024-04-02T04:48:06.192 INFO:tasks.workunit.client.0.smithi070.stdout:2/899: truncate d2/d10/d5c/d7a/db0/de4/ff1 411781 0 2024-04-02T04:48:06.194 INFO:tasks.workunit.client.0.smithi070.stdout:4/874: rmdir d1/d3/d9/d5e 39 2024-04-02T04:48:06.195 INFO:tasks.workunit.client.0.smithi070.stdout:4/875: write d1/d9c/f102 [132027,112521] 0 2024-04-02T04:48:06.196 INFO:tasks.workunit.client.0.smithi070.stdout:4/876: stat d1/d9c/d65/d7a/f122 0 2024-04-02T04:48:06.198 INFO:tasks.workunit.client.0.smithi070.stdout:9/808: rename d4/d10/d12/d27/f97 to d4/d10/d12/d27/d68/f108 0 2024-04-02T04:48:06.204 INFO:tasks.workunit.client.0.smithi070.stdout:9/809: dwrite d4/d10/d12/d27/f30 [0,4194304] 0 2024-04-02T04:48:06.207 INFO:tasks.workunit.client.0.smithi070.stdout:6/738: rmdir d0/d43/d95 39 2024-04-02T04:48:06.219 INFO:tasks.workunit.client.0.smithi070.stdout:4/877: mknod d1/d3/d78/dad/dbd/c126 0 2024-04-02T04:48:06.220 INFO:tasks.workunit.client.0.smithi070.stdout:8/802: write d2/d3/d35/d3a/d4f/d59/dcc/f101 [548820,102523] 0 2024-04-02T04:48:06.222 INFO:tasks.workunit.client.0.smithi070.stdout:7/794: mknod d7/dd/df/d79/c104 0 2024-04-02T04:48:06.227 INFO:tasks.workunit.client.0.smithi070.stdout:1/763: creat d0/d9/d1d/d22/d2c/fef x:0 0 0 2024-04-02T04:48:06.228 INFO:tasks.workunit.client.0.smithi070.stdout:1/764: write d0/d54/d64/fe4 [987389,91228] 0 2024-04-02T04:48:06.230 INFO:tasks.workunit.client.0.smithi070.stdout:9/810: dread d4/d10/d12/d55/d66/f7c [0,4194304] 0 2024-04-02T04:48:06.230 INFO:tasks.workunit.client.0.smithi070.stdout:1/765: read d0/d4c/f70 [1577,129988] 0 2024-04-02T04:48:06.231 INFO:tasks.workunit.client.1.smithi179.stdout:1/930: rename d3/d5/d2f/d4f/d6a/da0/dd4 to d3/d5/d148 0 2024-04-02T04:48:06.232 INFO:tasks.workunit.client.0.smithi070.stdout:0/921: creat d4/d5/d13/d2a/d3a/d45/d112/f11a x:0 0 0 2024-04-02T04:48:06.233 INFO:tasks.workunit.client.1.smithi179.stdout:3/955: fdatasync d8/d26/d36/f38 0 2024-04-02T04:48:06.237 INFO:tasks.workunit.client.0.smithi070.stdout:2/900: dwrite d2/d10/d5c/fbe [0,4194304] 0 2024-04-02T04:48:06.245 INFO:tasks.workunit.client.1.smithi179.stdout:8/944: dread - d3/d32/d21/d50/d65/ff3 zero size 2024-04-02T04:48:06.245 INFO:tasks.workunit.client.0.smithi070.stdout:6/739: dread d0/d43/d95/f9a [4194304,4194304] 0 2024-04-02T04:48:06.250 INFO:tasks.workunit.client.0.smithi070.stdout:5/979: write d5/d17/d2b/f89 [532964,122797] 0 2024-04-02T04:48:06.251 INFO:tasks.workunit.client.0.smithi070.stdout:7/795: creat d7/d36/f105 x:0 0 0 2024-04-02T04:48:06.257 INFO:tasks.workunit.client.1.smithi179.stdout:2/821: dread dc/d29/d31/fb9 [0,4194304] 0 2024-04-02T04:48:06.258 INFO:tasks.workunit.client.0.smithi070.stdout:0/922: mknod d4/d5/d13/d2a/d8d/c11b 0 2024-04-02T04:48:06.258 INFO:tasks.workunit.client.1.smithi179.stdout:2/822: chown dc/d29/d31/d66/fc5 84373235 1 2024-04-02T04:48:06.259 INFO:tasks.workunit.client.0.smithi070.stdout:8/803: dwrite d2/d8/d29/d22/f7a [0,4194304] 0 2024-04-02T04:48:06.260 INFO:tasks.workunit.client.0.smithi070.stdout:0/923: chown d4/d5/da/dd1/ffb 9 1 2024-04-02T04:48:06.260 INFO:tasks.workunit.client.1.smithi179.stdout:6/941: rename d3/d2d/d38/d64 to d3/d2d/d38/db5/d150 0 2024-04-02T04:48:06.262 INFO:tasks.workunit.client.1.smithi179.stdout:1/931: symlink d3/d5/d2f/d4f/d42/l149 0 2024-04-02T04:48:06.265 INFO:tasks.workunit.client.1.smithi179.stdout:3/956: creat d8/d26/d52/d7e/def/f132 x:0 0 0 2024-04-02T04:48:06.266 INFO:tasks.workunit.client.0.smithi070.stdout:8/804: dread d2/d8/d23/d24/d79/f4e [0,4194304] 0 2024-04-02T04:48:06.269 INFO:tasks.workunit.client.0.smithi070.stdout:6/740: symlink d0/d55/d79/d8d/le4 0 2024-04-02T04:48:06.270 INFO:tasks.workunit.client.0.smithi070.stdout:6/741: readlink d0/d43/d80/l8e 0 2024-04-02T04:48:06.270 INFO:tasks.workunit.client.0.smithi070.stdout:1/766: dread d0/d9/d1d/f5e [0,4194304] 0 2024-04-02T04:48:06.274 INFO:tasks.workunit.client.1.smithi179.stdout:5/892: getdents d8/d1b/d23/d42/d5a 0 2024-04-02T04:48:06.278 INFO:tasks.workunit.client.0.smithi070.stdout:7/796: mkdir d7/dd/df/d106 0 2024-04-02T04:48:06.282 INFO:tasks.workunit.client.0.smithi070.stdout:0/924: symlink d4/d3c/l11c 0 2024-04-02T04:48:06.286 INFO:tasks.workunit.client.1.smithi179.stdout:1/932: dread d3/d5/de/d33/fc0 [0,4194304] 0 2024-04-02T04:48:06.286 INFO:tasks.workunit.client.0.smithi070.stdout:5/980: dwrite d5/d17/d27/f7c [0,4194304] 0 2024-04-02T04:48:06.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/901: creat d2/d10/d5c/d10f/f126 x:0 0 0 2024-04-02T04:48:06.288 INFO:tasks.workunit.client.0.smithi070.stdout:3/813: getdents d10/d17/db2/dbd/ddb 0 2024-04-02T04:48:06.289 INFO:tasks.workunit.client.1.smithi179.stdout:9/910: creat d0/d1/f11e x:0 0 0 2024-04-02T04:48:06.290 INFO:tasks.workunit.client.0.smithi070.stdout:3/814: chown d10/l69 1 1 2024-04-02T04:48:06.290 INFO:tasks.workunit.client.0.smithi070.stdout:0/925: dwrite d4/d5/f3f [4194304,4194304] 0 2024-04-02T04:48:06.290 INFO:tasks.workunit.client.1.smithi179.stdout:4/955: dwrite d1/d3/df/d10/d2f/d60/fef [0,4194304] 0 2024-04-02T04:48:06.292 INFO:tasks.workunit.client.1.smithi179.stdout:8/945: unlink d3/d32/d21/d22/d6b/db1/df8/d126/d96/fa1 0 2024-04-02T04:48:06.294 INFO:tasks.workunit.client.0.smithi070.stdout:2/902: dread d2/d10/d21/d3d/d42/fad [0,4194304] 0 2024-04-02T04:48:06.294 INFO:tasks.workunit.client.1.smithi179.stdout:7/940: dwrite d1/d9/d7e/f8a [0,4194304] 0 2024-04-02T04:48:06.294 INFO:tasks.workunit.client.1.smithi179.stdout:5/893: mknod d8/d1b/d23/d32/d46/c11b 0 2024-04-02T04:48:06.296 INFO:tasks.workunit.client.0.smithi070.stdout:2/903: chown d2/d10/d21/d71/dd4/d100 310 1 2024-04-02T04:48:06.296 INFO:tasks.workunit.client.0.smithi070.stdout:4/878: creat d1/f127 x:0 0 0 2024-04-02T04:48:06.300 INFO:tasks.workunit.client.1.smithi179.stdout:6/942: dread d3/d2d/d43/f47 [0,4194304] 0 2024-04-02T04:48:06.300 INFO:tasks.workunit.client.1.smithi179.stdout:5/894: write d8/d1b/d23/d32/f3d [1579964,115886] 0 2024-04-02T04:48:06.309 INFO:tasks.workunit.client.0.smithi070.stdout:5/981: creat d5/d9/d10/d6a/d7d/f134 x:0 0 0 2024-04-02T04:48:06.314 INFO:tasks.workunit.client.0.smithi070.stdout:1/767: dread d0/d7/f10 [0,4194304] 0 2024-04-02T04:48:06.316 INFO:tasks.workunit.client.0.smithi070.stdout:1/768: read d0/d9/d1d/d22/d2c/d4f/d77/d8c/faa [107187,26818] 0 2024-04-02T04:48:06.317 INFO:tasks.workunit.client.0.smithi070.stdout:1/769: chown d0/d4c/f72 12656123 1 2024-04-02T04:48:06.322 INFO:tasks.workunit.client.1.smithi179.stdout:5/895: mknod d8/d1b/d23/d36/d53/c11c 0 2024-04-02T04:48:06.322 INFO:tasks.workunit.client.1.smithi179.stdout:3/957: write d8/d57/fb3 [230631,35079] 0 2024-04-02T04:48:06.322 INFO:tasks.workunit.client.1.smithi179.stdout:6/943: truncate d3/d12/d6a/f6e 2204224 0 2024-04-02T04:48:06.322 INFO:tasks.workunit.client.1.smithi179.stdout:3/958: write d8/d26/d36/fa2 [4844778,118853] 0 2024-04-02T04:48:06.323 INFO:tasks.workunit.client.1.smithi179.stdout:3/959: write d8/dd3/f113 [3661897,51688] 0 2024-04-02T04:48:06.323 INFO:tasks.workunit.client.0.smithi070.stdout:4/879: mkdir d1/d3/d78/dad/dbd/d128 0 2024-04-02T04:48:06.324 INFO:tasks.workunit.client.0.smithi070.stdout:3/815: dread d10/d17/db2/dbd/fdd [0,4194304] 0 2024-04-02T04:48:06.325 INFO:tasks.workunit.client.1.smithi179.stdout:3/960: chown d8/d6d/d94/c12e 149 1 2024-04-02T04:48:06.325 INFO:tasks.workunit.client.0.smithi070.stdout:7/797: dwrite d7/dd/df/d7a/d6c/f74 [0,4194304] 0 2024-04-02T04:48:06.326 INFO:tasks.workunit.client.0.smithi070.stdout:3/816: chown d10/d32/d3f/f6d 0 1 2024-04-02T04:48:06.327 INFO:tasks.workunit.client.1.smithi179.stdout:9/911: rename d0/d19/c3d to d0/d19/d27/d7a/dcb/c11f 0 2024-04-02T04:48:06.327 INFO:tasks.workunit.client.0.smithi070.stdout:3/817: readlink d10/d32/d4b/l83 0 2024-04-02T04:48:06.329 INFO:tasks.workunit.client.1.smithi179.stdout:1/933: dwrite d3/d85/da2/fea [0,4194304] 0 2024-04-02T04:48:06.331 INFO:tasks.workunit.client.0.smithi070.stdout:8/805: creat d2/d8/d29/d22/f108 x:0 0 0 2024-04-02T04:48:06.332 INFO:tasks.workunit.client.1.smithi179.stdout:7/941: mkdir d1/d150 0 2024-04-02T04:48:06.334 INFO:tasks.workunit.client.0.smithi070.stdout:6/742: dwrite d0/d55/d5c/f60 [0,4194304] 0 2024-04-02T04:48:06.337 INFO:tasks.workunit.client.1.smithi179.stdout:9/912: dwrite d0/d1/d11/f78 [0,4194304] 0 2024-04-02T04:48:06.337 INFO:tasks.workunit.client.1.smithi179.stdout:6/944: unlink d3/d2d/d38/db5/d150/f109 0 2024-04-02T04:48:06.338 INFO:tasks.workunit.client.0.smithi070.stdout:0/926: rename d4/d5/da/f63 to d4/d3c/f11d 0 2024-04-02T04:48:06.338 INFO:tasks.workunit.client.1.smithi179.stdout:7/942: write d1/d9/d7e/f8a [4774635,16795] 0 2024-04-02T04:48:06.339 INFO:tasks.workunit.client.1.smithi179.stdout:8/946: dwrite d3/d32/d21/d50/d65/ff3 [0,4194304] 0 2024-04-02T04:48:06.340 INFO:tasks.workunit.client.1.smithi179.stdout:7/943: chown d1/dd/d76/d138/da8 117324 1 2024-04-02T04:48:06.343 INFO:tasks.workunit.client.0.smithi070.stdout:2/904: mkdir d2/d10/d21/d127 0 2024-04-02T04:48:06.346 INFO:tasks.workunit.client.1.smithi179.stdout:4/956: rename d1/d3/dc/d27/dae to d1/d15/d7c/d145 0 2024-04-02T04:48:06.347 INFO:tasks.workunit.client.0.smithi070.stdout:0/927: dwrite d4/d5/d13/d17/d6e/fbc [4194304,4194304] 0 2024-04-02T04:48:06.348 INFO:tasks.workunit.client.1.smithi179.stdout:9/913: dwrite d0/d1/d11/f78 [0,4194304] 0 2024-04-02T04:48:06.348 INFO:tasks.workunit.client.0.smithi070.stdout:0/928: chown d4/d5/d13/d2a/d29 1 1 2024-04-02T04:48:06.349 INFO:tasks.workunit.client.0.smithi070.stdout:4/880: unlink d1/d3/d78/dad/dbd/c126 0 2024-04-02T04:48:06.351 INFO:tasks.workunit.client.1.smithi179.stdout:1/934: readlink d3/d5/d2f/d4f/l99 0 2024-04-02T04:48:06.355 INFO:tasks.workunit.client.1.smithi179.stdout:9/914: dwrite d0/d1/d10/f6f [0,4194304] 0 2024-04-02T04:48:06.365 INFO:tasks.workunit.client.0.smithi070.stdout:6/743: unlink d0/d3/l29 0 2024-04-02T04:48:06.366 INFO:tasks.workunit.client.0.smithi070.stdout:6/744: readlink d0/d20/l32 0 2024-04-02T04:48:06.372 INFO:tasks.workunit.client.0.smithi070.stdout:1/770: fdatasync d0/d54/d64/d6d/db6/fe7 0 2024-04-02T04:48:06.374 INFO:tasks.workunit.client.0.smithi070.stdout:6/745: dwrite d0/d43/f51 [0,4194304] 0 2024-04-02T04:48:06.377 INFO:tasks.workunit.client.0.smithi070.stdout:2/905: symlink d2/d10/d21/d3d/d42/l128 0 2024-04-02T04:48:06.380 INFO:tasks.workunit.client.1.smithi179.stdout:3/961: mknod d8/d26/d83/c133 0 2024-04-02T04:48:06.385 INFO:tasks.workunit.client.0.smithi070.stdout:5/982: write d5/f56 [6145228,95008] 0 2024-04-02T04:48:06.386 INFO:tasks.workunit.client.0.smithi070.stdout:4/881: creat d1/d3/d85/deb/d110/f129 x:0 0 0 2024-04-02T04:48:06.389 INFO:tasks.workunit.client.0.smithi070.stdout:7/798: mkdir d7/dd/df/d79/d46/d107 0 2024-04-02T04:48:06.391 INFO:tasks.workunit.client.1.smithi179.stdout:1/935: unlink d3/d5/d2f/d4f/d42/d53/d72/d86/f10f 0 2024-04-02T04:48:06.392 INFO:tasks.workunit.client.0.smithi070.stdout:3/818: symlink d10/d17/d51/l112 0 2024-04-02T04:48:06.392 INFO:tasks.workunit.client.1.smithi179.stdout:1/936: chown d3/d5/d2f/d4f/d6a 133065960 1 2024-04-02T04:48:06.394 INFO:tasks.workunit.client.1.smithi179.stdout:9/915: write d0/d19/d27/d7a/dcb/feb [825197,57209] 0 2024-04-02T04:48:06.394 INFO:tasks.workunit.client.1.smithi179.stdout:6/945: dwrite f2 [0,4194304] 0 2024-04-02T04:48:06.396 INFO:tasks.workunit.client.0.smithi070.stdout:8/806: rename d2/d8/d29/d72/df1/lfc to d2/d8/d23/d24/d106/l109 0 2024-04-02T04:48:06.397 INFO:tasks.workunit.client.0.smithi070.stdout:8/807: readlink d2/d3/d52/l11 0 2024-04-02T04:48:06.399 INFO:tasks.workunit.client.0.smithi070.stdout:1/771: dread - d0/d7/fd4 zero size 2024-04-02T04:48:06.399 INFO:tasks.workunit.client.0.smithi070.stdout:8/808: chown d2/d3/d35/d3a/d4f/c69 4 1 2024-04-02T04:48:06.400 INFO:tasks.workunit.client.1.smithi179.stdout:4/957: getdents d1/d93/de6 0 2024-04-02T04:48:06.400 INFO:tasks.workunit.client.1.smithi179.stdout:8/947: dwrite d3/d1b/d2a/d41/d109/f116 [0,4194304] 0 2024-04-02T04:48:06.402 INFO:tasks.workunit.client.1.smithi179.stdout:1/937: chown d3/d5/c17 141891 1 2024-04-02T04:48:06.406 INFO:tasks.workunit.client.1.smithi179.stdout:9/916: mkdir d0/d1/d11/d29/d80/d120 0 2024-04-02T04:48:06.407 INFO:tasks.workunit.client.1.smithi179.stdout:9/917: chown d0/d1/d11/d3a/l9c 117747 1 2024-04-02T04:48:06.407 INFO:tasks.workunit.client.1.smithi179.stdout:9/918: readlink d0/d1/l14 0 2024-04-02T04:48:06.410 INFO:tasks.workunit.client.1.smithi179.stdout:4/958: symlink d1/d3/dc/d26/d8a/l146 0 2024-04-02T04:48:06.410 INFO:tasks.workunit.client.1.smithi179.stdout:2/823: sync 2024-04-02T04:48:06.411 INFO:tasks.workunit.client.1.smithi179.stdout:1/938: truncate d3/d5/f25 608916 0 2024-04-02T04:48:06.412 INFO:tasks.workunit.client.1.smithi179.stdout:7/944: rename d1/dd/d76/d138/da8/dda/d14b/d4d/fb6 to d1/d9/d27/d87/d8b/d75/f151 0 2024-04-02T04:48:06.413 INFO:tasks.workunit.client.1.smithi179.stdout:8/948: symlink d3/d31/d150/l155 0 2024-04-02T04:48:06.415 INFO:tasks.workunit.client.1.smithi179.stdout:2/824: creat dc/d29/d31/d57/d97/dbf/f112 x:0 0 0 2024-04-02T04:48:06.416 INFO:tasks.workunit.client.1.smithi179.stdout:2/825: stat dc/d29/d4a/d1e/d6c/dd2 0 2024-04-02T04:48:06.416 INFO:tasks.workunit.client.1.smithi179.stdout:7/945: creat d1/d9/d27/d87/d8b/d75/d11f/f152 x:0 0 0 2024-04-02T04:48:06.418 INFO:tasks.workunit.client.1.smithi179.stdout:7/946: truncate d1/dd/d76/d138/f120 274053 0 2024-04-02T04:48:06.418 INFO:tasks.workunit.client.1.smithi179.stdout:8/949: creat d3/d31/dc3/d11b/f156 x:0 0 0 2024-04-02T04:48:06.419 INFO:tasks.workunit.client.1.smithi179.stdout:7/947: stat d1/d9/f85 0 2024-04-02T04:48:06.420 INFO:tasks.workunit.client.1.smithi179.stdout:2/826: rename dc/l13 to dc/d29/d31/d83/dae/db0/l113 0 2024-04-02T04:48:06.422 INFO:tasks.workunit.client.1.smithi179.stdout:7/948: mknod d1/dd/d76/d138/da8/dda/d14b/d4d/c153 0 2024-04-02T04:48:06.423 INFO:tasks.workunit.client.1.smithi179.stdout:7/949: fsync d1/d9/d27/d87/d8b/d75/fbd 0 2024-04-02T04:48:06.424 INFO:tasks.workunit.client.1.smithi179.stdout:8/950: dwrite d3/d32/d21/d50/dcf/f134 [0,4194304] 0 2024-04-02T04:48:06.427 INFO:tasks.workunit.client.1.smithi179.stdout:7/950: creat d1/dd/d76/d138/da8/d104/f154 x:0 0 0 2024-04-02T04:48:06.430 INFO:tasks.workunit.client.1.smithi179.stdout:7/951: mknod d1/d9/d27/d87/d8b/d75/db9/d8d/c155 0 2024-04-02T04:48:06.430 INFO:tasks.workunit.client.1.smithi179.stdout:8/951: dwrite d3/d32/d21/d22/d6b/db1/df8/d126/f121 [0,4194304] 0 2024-04-02T04:48:06.431 INFO:tasks.workunit.client.1.smithi179.stdout:7/952: mknod d1/d9/d55/c156 0 2024-04-02T04:48:06.433 INFO:tasks.workunit.client.1.smithi179.stdout:8/952: truncate d3/d32/d21/d22/d6b/db1/d104/d11d/f12c 53415 0 2024-04-02T04:48:06.437 INFO:tasks.workunit.client.1.smithi179.stdout:8/953: dwrite d3/f122 [0,4194304] 0 2024-04-02T04:48:06.440 INFO:tasks.workunit.client.1.smithi179.stdout:8/954: creat d3/d31/f157 x:0 0 0 2024-04-02T04:48:06.485 INFO:tasks.workunit.client.1.smithi179.stdout:5/896: sync 2024-04-02T04:48:06.485 INFO:tasks.workunit.client.1.smithi179.stdout:5/897: readlink d8/d1b/d23/d32/d46/le3 0 2024-04-02T04:48:06.500 INFO:tasks.workunit.client.0.smithi070.stdout:6/746: creat d0/d20/d77/d98/fe5 x:0 0 0 2024-04-02T04:48:06.501 INFO:tasks.workunit.client.0.smithi070.stdout:1/772: stat d0/d7/d28/d83/ce8 0 2024-04-02T04:48:06.501 INFO:tasks.workunit.client.0.smithi070.stdout:2/906: rmdir d2/d10/d5c/d7a/db0/dd8/dd2 39 2024-04-02T04:48:06.503 INFO:tasks.workunit.client.0.smithi070.stdout:0/929: dwrite d4/d5/d13/d2a/fea [0,4194304] 0 2024-04-02T04:48:06.503 INFO:tasks.workunit.client.1.smithi179.stdout:2/827: dread dc/d29/d31/faf [0,4194304] 0 2024-04-02T04:48:06.503 INFO:tasks.workunit.client.1.smithi179.stdout:8/955: rename d3/d32/d21/d22/d6b/d91/d110/l139 to d3/d1b/d2a/db0/l158 0 2024-04-02T04:48:06.504 INFO:tasks.workunit.client.0.smithi070.stdout:4/882: creat d1/de7/f12a x:0 0 0 2024-04-02T04:48:06.506 INFO:tasks.workunit.client.0.smithi070.stdout:1/773: read d0/d9/f3e [393734,55490] 0 2024-04-02T04:48:06.506 INFO:tasks.workunit.client.1.smithi179.stdout:8/956: truncate d3/d1b/d2a/f4c 797798 0 2024-04-02T04:48:06.507 INFO:tasks.workunit.client.0.smithi070.stdout:8/809: dread - d2/d8/d29/fe4 zero size 2024-04-02T04:48:06.509 INFO:tasks.workunit.client.0.smithi070.stdout:8/810: chown d2/d3/d52/l11 506024 1 2024-04-02T04:48:06.514 INFO:tasks.workunit.client.1.smithi179.stdout:8/957: mkdir d3/d32/d21/d22/d6b/d91/d110/d159 0 2024-04-02T04:48:06.517 INFO:tasks.workunit.client.0.smithi070.stdout:3/819: getdents d10/d32/d4b/dc5/de0 0 2024-04-02T04:48:06.518 INFO:tasks.workunit.client.1.smithi179.stdout:3/962: sync 2024-04-02T04:48:06.519 INFO:tasks.workunit.client.1.smithi179.stdout:3/963: fsync d8/d26/d52/d7e/db8/fde 0 2024-04-02T04:48:06.521 INFO:tasks.workunit.client.1.smithi179.stdout:6/946: write d3/d12/d23/d39/fda [624261,17991] 0 2024-04-02T04:48:06.521 INFO:tasks.workunit.client.1.smithi179.stdout:2/828: read dc/f50 [1931520,124810] 0 2024-04-02T04:48:06.521 INFO:tasks.workunit.client.1.smithi179.stdout:8/958: chown d3/d32/d21/d22/d6b/l119 7415443 1 2024-04-02T04:48:06.522 INFO:tasks.workunit.client.0.smithi070.stdout:5/983: link d5/d9/d23/d30/da2/cb5 d5/d14/c135 0 2024-04-02T04:48:06.522 INFO:tasks.workunit.client.1.smithi179.stdout:3/964: dread - d8/d26/d52/d7e/def/f10c zero size 2024-04-02T04:48:06.523 INFO:tasks.workunit.client.0.smithi070.stdout:2/907: symlink d2/d10/d21/d71/l129 0 2024-04-02T04:48:06.524 INFO:tasks.workunit.client.0.smithi070.stdout:2/908: chown d2/d10/d5c/d7a/db0/cd9 11 1 2024-04-02T04:48:06.524 INFO:tasks.workunit.client.1.smithi179.stdout:2/829: write dc/d29/d31/d83/dae/db0/dbc/f65 [1069151,87680] 0 2024-04-02T04:48:06.526 INFO:tasks.workunit.client.0.smithi070.stdout:8/811: creat d2/d8/d29/d22/d5a/d94/da2/f10a x:0 0 0 2024-04-02T04:48:06.539 INFO:tasks.workunit.client.0.smithi070.stdout:7/799: dwrite d7/dd/df/d7a/f5a [0,4194304] 0 2024-04-02T04:48:06.539 INFO:tasks.workunit.client.0.smithi070.stdout:7/800: stat d7/dd/df/d7a/d6c/f6d 0 2024-04-02T04:48:06.539 INFO:tasks.workunit.client.0.smithi070.stdout:4/883: mkdir d1/d3/d9/d5e/d12b 0 2024-04-02T04:48:06.540 INFO:tasks.workunit.client.1.smithi179.stdout:5/898: dread d8/d1b/d23/d32/f99 [0,4194304] 0 2024-04-02T04:48:06.540 INFO:tasks.workunit.client.1.smithi179.stdout:9/919: write d0/d1/d11/d29/f37 [1288540,112663] 0 2024-04-02T04:48:06.540 INFO:tasks.workunit.client.1.smithi179.stdout:6/947: fsync d3/d12/d23/d39/d80/faa 0 2024-04-02T04:48:06.540 INFO:tasks.workunit.client.1.smithi179.stdout:9/920: fsync d0/d19/d27/fa7 0 2024-04-02T04:48:06.540 INFO:tasks.workunit.client.1.smithi179.stdout:6/948: stat d3/d2d/d43/f4b 0 2024-04-02T04:48:06.541 INFO:tasks.workunit.client.1.smithi179.stdout:3/965: dread - d8/d57/ff4 zero size 2024-04-02T04:48:06.541 INFO:tasks.workunit.client.1.smithi179.stdout:5/899: mkdir d8/d1b/d23/d32/dbe/daf/dec/d11d 0 2024-04-02T04:48:06.542 INFO:tasks.workunit.client.0.smithi070.stdout:0/930: creat d4/d5/d58/d60/f11e x:0 0 0 2024-04-02T04:48:06.543 INFO:tasks.workunit.client.0.smithi070.stdout:0/931: chown d4/d5/d21/fec 739597 1 2024-04-02T04:48:06.543 INFO:tasks.workunit.client.1.smithi179.stdout:4/959: dwrite d1/f42 [0,4194304] 0 2024-04-02T04:48:06.545 INFO:tasks.workunit.client.0.smithi070.stdout:1/774: creat d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/dea/ff0 x:0 0 0 2024-04-02T04:48:06.545 INFO:tasks.workunit.client.1.smithi179.stdout:1/939: dwrite d3/d5/d2f/d4f/d42/d53/f5e [0,4194304] 0 2024-04-02T04:48:06.545 INFO:tasks.workunit.client.1.smithi179.stdout:7/953: write d1/d9/d27/d87/fe0 [1631762,108474] 0 2024-04-02T04:48:06.546 INFO:tasks.workunit.client.1.smithi179.stdout:3/966: dread d8/d26/d52/d7e/def/f129 [0,4194304] 0 2024-04-02T04:48:06.548 INFO:tasks.workunit.client.1.smithi179.stdout:3/967: chown d8/d26/d83/l102 8292559 1 2024-04-02T04:48:06.557 INFO:tasks.workunit.client.1.smithi179.stdout:9/921: symlink d0/d1/d11/dab/ded/l121 0 2024-04-02T04:48:06.559 INFO:tasks.workunit.client.0.smithi070.stdout:7/801: rmdir d7/dd/df/d7a/d6c/d77/d75/d80 39 2024-04-02T04:48:06.563 INFO:tasks.workunit.client.0.smithi070.stdout:3/820: symlink d10/d17/d51/d73/d9e/d107/d10d/l113 0 2024-04-02T04:48:06.565 INFO:tasks.workunit.client.0.smithi070.stdout:5/984: mknod d5/d9/d23/d30/da2/c136 0 2024-04-02T04:48:06.566 INFO:tasks.workunit.client.0.smithi070.stdout:6/747: truncate d0/d55/d8b/d94/f53 3787470 0 2024-04-02T04:48:06.566 INFO:tasks.workunit.client.1.smithi179.stdout:8/959: dwrite d3/d32/d21/d49/fb9 [0,4194304] 0 2024-04-02T04:48:06.567 INFO:tasks.workunit.client.0.smithi070.stdout:6/748: readlink d0/d43/d80/l8e 0 2024-04-02T04:48:06.567 INFO:tasks.workunit.client.1.smithi179.stdout:8/960: fdatasync d3/d31/f157 0 2024-04-02T04:48:06.568 INFO:tasks.workunit.client.1.smithi179.stdout:1/940: creat d3/d5/d2f/d4f/dab/f14a x:0 0 0 2024-04-02T04:48:06.569 INFO:tasks.workunit.client.0.smithi070.stdout:0/932: unlink d4/d5/d13/d2a/d3a/d80/f9f 0 2024-04-02T04:48:06.572 INFO:tasks.workunit.client.1.smithi179.stdout:7/954: stat d1/dd/d23/dab/leb 0 2024-04-02T04:48:06.573 INFO:tasks.workunit.client.0.smithi070.stdout:4/884: dread d1/d3/d9/f9e [0,4194304] 0 2024-04-02T04:48:06.575 INFO:tasks.workunit.client.0.smithi070.stdout:2/909: write d2/d10/d21/d3d/f9d [1323541,92666] 0 2024-04-02T04:48:06.577 INFO:tasks.workunit.client.1.smithi179.stdout:8/961: dwrite d3/d32/d21/d49/df0/df9/f102 [0,4194304] 0 2024-04-02T04:48:06.577 INFO:tasks.workunit.client.0.smithi070.stdout:7/802: rename d7/dd/df/d7a/d6c/d77/l2b to d7/dd/df/d7a/d6c/d77/d54/de5/l108 0 2024-04-02T04:48:06.581 INFO:tasks.workunit.client.0.smithi070.stdout:3/821: truncate d10/d17/d4c/f62 2406118 0 2024-04-02T04:48:06.581 INFO:tasks.workunit.client.1.smithi179.stdout:5/900: symlink d8/d1b/d23/d36/d53/d101/l11e 0 2024-04-02T04:48:06.585 INFO:tasks.workunit.client.1.smithi179.stdout:4/960: dread d1/d3/dc/d26/d8a/f82 [0,4194304] 0 2024-04-02T04:48:06.586 INFO:tasks.workunit.client.1.smithi179.stdout:6/949: dwrite d3/d12/d23/d26/d57/fa2 [0,4194304] 0 2024-04-02T04:48:06.586 INFO:tasks.workunit.client.0.smithi070.stdout:8/812: dwrite d2/d8/d23/d24/d5f/dc2/fd7 [0,4194304] 0 2024-04-02T04:48:06.587 INFO:tasks.workunit.client.0.smithi070.stdout:8/813: readlink d2/d3/lbb 0 2024-04-02T04:48:06.588 INFO:tasks.workunit.client.0.smithi070.stdout:1/775: mkdir d0/d9/d1d/d22/d56/df1 0 2024-04-02T04:48:06.590 INFO:tasks.workunit.client.0.smithi070.stdout:9/811: sync 2024-04-02T04:48:06.593 INFO:tasks.workunit.client.0.smithi070.stdout:4/885: mknod d1/d9c/d65/c12c 0 2024-04-02T04:48:06.596 INFO:tasks.workunit.client.1.smithi179.stdout:9/922: link d0/d1/d10/d6d/d58/f96 d0/d19/d27/d7a/d7e/db9/d108/f122 0 2024-04-02T04:48:06.597 INFO:tasks.workunit.client.0.smithi070.stdout:2/910: truncate d2/d10/d21/d71/dd4/fc4 353942 0 2024-04-02T04:48:06.599 INFO:tasks.workunit.client.1.smithi179.stdout:7/955: rename d1/dd/d76/d138/da8/dda/d14b/d4d/ld5 to d1/d9/d27/d87/d8b/d75/db9/d8d/l157 0 2024-04-02T04:48:06.601 INFO:tasks.workunit.client.0.smithi070.stdout:6/749: rename d0/d43/db3/dc7/d7b to d0/d20/d5f/de6 0 2024-04-02T04:48:06.601 INFO:tasks.workunit.client.0.smithi070.stdout:6/750: chown d0/d20/d77 9 1 2024-04-02T04:48:06.602 INFO:tasks.workunit.client.1.smithi179.stdout:8/962: write d3/d32/d21/d50/d56/f6a [2142008,45674] 0 2024-04-02T04:48:06.602 INFO:tasks.workunit.client.0.smithi070.stdout:6/751: dread - d0/d20/d5f/fd9 zero size 2024-04-02T04:48:06.608 INFO:tasks.workunit.client.1.smithi179.stdout:1/941: mknod d3/d5/d2f/c14b 0 2024-04-02T04:48:06.610 INFO:tasks.workunit.client.1.smithi179.stdout:4/961: truncate d1/d3/d19/fa3 483984 0 2024-04-02T04:48:06.611 INFO:tasks.workunit.client.0.smithi070.stdout:8/814: creat d2/d8/d29/db/f10b x:0 0 0 2024-04-02T04:48:06.612 INFO:tasks.workunit.client.0.smithi070.stdout:8/815: readlink d2/d8/d23/d24/d66/led 0 2024-04-02T04:48:06.614 INFO:tasks.workunit.client.0.smithi070.stdout:1/776: truncate d0/d4c/f72 573590 0 2024-04-02T04:48:06.615 INFO:tasks.workunit.client.1.smithi179.stdout:9/923: symlink d0/d19/d27/d7a/d7e/dd4/l123 0 2024-04-02T04:48:06.617 INFO:tasks.workunit.client.0.smithi070.stdout:9/812: mkdir d4/d34/db9/d109 0 2024-04-02T04:48:06.619 INFO:tasks.workunit.client.0.smithi070.stdout:4/886: creat d1/d9c/d65/f12d x:0 0 0 2024-04-02T04:48:06.620 INFO:tasks.workunit.client.1.smithi179.stdout:7/956: chown d1/d9/l118 132085951 1 2024-04-02T04:48:06.623 INFO:tasks.workunit.client.1.smithi179.stdout:8/963: read d3/d32/f6e [3061782,51583] 0 2024-04-02T04:48:06.624 INFO:tasks.workunit.client.0.smithi070.stdout:0/933: rename d4/d5/d58/d60/f11e to d4/d3c/f11f 0 2024-04-02T04:48:06.625 INFO:tasks.workunit.client.1.smithi179.stdout:5/901: write d8/d1b/d23/d32/dbe/daf/db7/f77 [1653964,60507] 0 2024-04-02T04:48:06.626 INFO:tasks.workunit.client.1.smithi179.stdout:5/902: write d8/d1b/d23/d36/d53/d62/da9/ff0 [846767,28347] 0 2024-04-02T04:48:06.626 INFO:tasks.workunit.client.1.smithi179.stdout:1/942: fsync d3/d5/d2f/f47 0 2024-04-02T04:48:06.628 INFO:tasks.workunit.client.0.smithi070.stdout:7/803: truncate d7/dd/df/d34/d68/ff3 4618357 0 2024-04-02T04:48:06.628 INFO:tasks.workunit.client.1.smithi179.stdout:2/830: sync 2024-04-02T04:48:06.630 INFO:tasks.workunit.client.1.smithi179.stdout:4/962: mkdir d1/d3/dc0/d147 0 2024-04-02T04:48:06.632 INFO:tasks.workunit.client.0.smithi070.stdout:5/985: creat d5/d9/d23/d30/da2/f137 x:0 0 0 2024-04-02T04:48:06.632 INFO:tasks.workunit.client.0.smithi070.stdout:5/986: stat d5/d9/d10 0 2024-04-02T04:48:06.634 INFO:tasks.workunit.client.1.smithi179.stdout:3/968: getdents d8/d26 0 2024-04-02T04:48:06.636 INFO:tasks.workunit.client.1.smithi179.stdout:9/924: mkdir d0/d19/d27/d48/d124 0 2024-04-02T04:48:06.637 INFO:tasks.workunit.client.0.smithi070.stdout:2/911: write d2/d12/f38 [3776536,109147] 0 2024-04-02T04:48:06.640 INFO:tasks.workunit.client.1.smithi179.stdout:7/957: symlink d1/d9/d27/d87/d8b/d75/d11f/l158 0 2024-04-02T04:48:06.646 INFO:tasks.workunit.client.1.smithi179.stdout:3/969: dread d8/d26/d36/fa2 [0,4194304] 0 2024-04-02T04:48:06.649 INFO:tasks.workunit.client.1.smithi179.stdout:9/925: mknod d0/d19/d27/dad/c125 0 2024-04-02T04:48:06.650 INFO:tasks.workunit.client.0.smithi070.stdout:0/934: unlink d4/d5/d13/d23/fc5 0 2024-04-02T04:48:06.653 INFO:tasks.workunit.client.1.smithi179.stdout:6/950: rename d3/d12/d23/d39/d4e/d66 to d3/d12/d151 0 2024-04-02T04:48:06.654 INFO:tasks.workunit.client.1.smithi179.stdout:5/903: dread d8/d1b/f26 [0,4194304] 0 2024-04-02T04:48:06.655 INFO:tasks.workunit.client.1.smithi179.stdout:8/964: dread d3/d32/d21/d50/d56/dad/dbc/fd0 [0,4194304] 0 2024-04-02T04:48:06.655 INFO:tasks.workunit.client.0.smithi070.stdout:5/987: symlink d5/d14/dd8/l138 0 2024-04-02T04:48:06.656 INFO:tasks.workunit.client.1.smithi179.stdout:1/943: mknod d3/d85/d132/c14c 0 2024-04-02T04:48:06.658 INFO:tasks.workunit.client.1.smithi179.stdout:6/951: dread d3/d12/d23/d26/d57/fa2 [0,4194304] 0 2024-04-02T04:48:06.663 INFO:tasks.workunit.client.0.smithi070.stdout:9/813: symlink d4/d10/d11/d7b/d19/d2f/d4d/de2/l10a 0 2024-04-02T04:48:06.668 INFO:tasks.workunit.client.1.smithi179.stdout:1/944: dread d3/d5/d2f/d4f/d35/f6e [0,4194304] 0 2024-04-02T04:48:06.668 INFO:tasks.workunit.client.1.smithi179.stdout:8/965: creat d3/d32/d21/d22/d6b/db1/df8/f15a x:0 0 0 2024-04-02T04:48:06.669 INFO:tasks.workunit.client.0.smithi070.stdout:8/816: truncate d2/d3/d35/d3a/d4f/d59/f8f 782852 0 2024-04-02T04:48:06.672 INFO:tasks.workunit.client.1.smithi179.stdout:3/970: creat d8/d26/d36/d117/f134 x:0 0 0 2024-04-02T04:48:06.673 INFO:tasks.workunit.client.1.smithi179.stdout:6/952: mknod d3/d12/d6a/c152 0 2024-04-02T04:48:06.677 INFO:tasks.workunit.client.0.smithi070.stdout:5/988: unlink d5/d9/d23/f106 0 2024-04-02T04:48:06.678 INFO:tasks.workunit.client.0.smithi070.stdout:1/777: link d0/d9/fc d0/db4/d9f/ff2 0 2024-04-02T04:48:06.682 INFO:tasks.workunit.client.1.smithi179.stdout:5/904: mkdir d8/d1b/d23/d11f 0 2024-04-02T04:48:06.687 INFO:tasks.workunit.client.1.smithi179.stdout:5/905: dwrite d8/d1b/d23/d32/d4e/d51/fdb [0,4194304] 0 2024-04-02T04:48:06.754 INFO:tasks.workunit.client.1.smithi179.stdout:2/831: sync 2024-04-02T04:48:06.803 INFO:tasks.workunit.client.1.smithi179.stdout:4/963: sync 2024-04-02T04:48:06.811 INFO:tasks.workunit.client.0.smithi070.stdout:9/814: mkdir d4/d10b 0 2024-04-02T04:48:06.812 INFO:tasks.workunit.client.0.smithi070.stdout:7/804: truncate d7/dd/df/f94 688831 0 2024-04-02T04:48:06.813 INFO:tasks.workunit.client.1.smithi179.stdout:8/966: creat d3/d32/d21/d50/d56/dad/dbc/f15b x:0 0 0 2024-04-02T04:48:06.814 INFO:tasks.workunit.client.0.smithi070.stdout:6/752: getdents d0/d20/d77/d98 0 2024-04-02T04:48:06.817 INFO:tasks.workunit.client.0.smithi070.stdout:6/753: dread d0/d20/fcf [0,4194304] 0 2024-04-02T04:48:06.866 INFO:tasks.workunit.client.0.smithi070.stdout:8/817: truncate d2/d8/d23/d24/f5b 538714 0 2024-04-02T04:48:06.869 INFO:tasks.workunit.client.1.smithi179.stdout:6/953: symlink d3/d12/d23/d26/d57/l153 0 2024-04-02T04:48:06.872 INFO:tasks.workunit.client.1.smithi179.stdout:9/926: rename d0/d19/d27/d7a/dcb/feb to d0/d19/d27/d7a/f126 0 2024-04-02T04:48:06.872 INFO:tasks.workunit.client.1.smithi179.stdout:7/958: link d1/dd/d76/d138/da8/dda/d14b/d4d/la6 d1/d9/d27/d87/d8b/l159 0 2024-04-02T04:48:06.875 INFO:tasks.workunit.client.0.smithi070.stdout:9/815: dread - d4/d10/d11/d7b/d19/f50 zero size 2024-04-02T04:48:06.876 INFO:tasks.workunit.client.1.smithi179.stdout:5/906: fsync d8/d1b/d23/d32/f99 0 2024-04-02T04:48:06.877 INFO:tasks.workunit.client.0.smithi070.stdout:2/912: dwrite d2/d10/d5c/d7a/db0/de4/ff1 [0,4194304] 0 2024-04-02T04:48:06.878 INFO:tasks.workunit.client.0.smithi070.stdout:7/805: symlink d7/dd/df/d7a/d6c/d77/d54/de5/l109 0 2024-04-02T04:48:06.879 INFO:tasks.workunit.client.1.smithi179.stdout:2/832: symlink dc/d29/d31/d83/dae/db0/l114 0 2024-04-02T04:48:06.880 INFO:tasks.workunit.client.0.smithi070.stdout:7/806: dread - d7/dd/df/d7a/dbc/ff9 zero size 2024-04-02T04:48:06.882 INFO:tasks.workunit.client.1.smithi179.stdout:1/945: symlink d3/d85/l14d 0 2024-04-02T04:48:06.883 INFO:tasks.workunit.client.0.smithi070.stdout:6/754: mkdir d0/d20/d5f/de7 0 2024-04-02T04:48:06.884 INFO:tasks.workunit.client.1.smithi179.stdout:1/946: truncate d3/d5/d2f/d4f/d42/d55/fed 159669 0 2024-04-02T04:48:06.886 INFO:tasks.workunit.client.1.smithi179.stdout:8/967: symlink d3/d32/d21/d49/df0/df9/l15c 0 2024-04-02T04:48:06.892 INFO:tasks.workunit.client.1.smithi179.stdout:6/954: rmdir d3/d12/d44/d10a/d10e 39 2024-04-02T04:48:06.893 INFO:tasks.workunit.client.1.smithi179.stdout:6/955: chown d3/d42/ff3 3673 1 2024-04-02T04:48:06.894 INFO:tasks.workunit.client.0.smithi070.stdout:8/818: dread d2/d8/d29/d22/d5a/d77/fc6 [0,4194304] 0 2024-04-02T04:48:06.898 INFO:tasks.workunit.client.1.smithi179.stdout:9/927: creat d0/d19/d27/d7a/d87/daf/dc2/f127 x:0 0 0 2024-04-02T04:48:06.900 INFO:tasks.workunit.client.1.smithi179.stdout:7/959: creat d1/dd/d76/d138/da8/d100/d107/f15a x:0 0 0 2024-04-02T04:48:06.902 INFO:tasks.workunit.client.1.smithi179.stdout:5/907: readlink d8/d1b/ldf 0 2024-04-02T04:48:06.903 INFO:tasks.workunit.client.1.smithi179.stdout:4/964: getdents d1/d93/de6 0 2024-04-02T04:48:06.905 INFO:tasks.workunit.client.1.smithi179.stdout:2/833: mknod dc/d29/d31/d57/dea/c115 0 2024-04-02T04:48:06.907 INFO:tasks.workunit.client.0.smithi070.stdout:5/989: truncate d5/d17/f53 2788297 0 2024-04-02T04:48:06.907 INFO:tasks.workunit.client.0.smithi070.stdout:5/990: chown d5/d9/d10/ff6 349385938 1 2024-04-02T04:48:06.909 INFO:tasks.workunit.client.1.smithi179.stdout:8/968: chown d3/d32/d21/d22/da5/lac 92364 1 2024-04-02T04:48:06.909 INFO:tasks.workunit.client.1.smithi179.stdout:3/971: creat d8/d26/d83/d89/f135 x:0 0 0 2024-04-02T04:48:06.910 INFO:tasks.workunit.client.1.smithi179.stdout:2/834: dwrite dc/d29/f6a [0,4194304] 0 2024-04-02T04:48:06.911 INFO:tasks.workunit.client.1.smithi179.stdout:8/969: read d3/d32/d21/d49/fb9 [660901,129750] 0 2024-04-02T04:48:06.915 INFO:tasks.workunit.client.1.smithi179.stdout:7/960: rmdir d1/d9/d27/d87/d8b 39 2024-04-02T04:48:06.918 INFO:tasks.workunit.client.1.smithi179.stdout:5/908: fsync d8/d1b/d23/d32/dbe/daf/dd4/d105/fd8 0 2024-04-02T04:48:06.928 INFO:tasks.workunit.client.1.smithi179.stdout:3/972: unlink d8/l16 0 2024-04-02T04:48:06.929 INFO:tasks.workunit.client.1.smithi179.stdout:6/956: write d3/d12/d23/d39/d4e/d6c/fd8 [956909,83964] 0 2024-04-02T04:48:06.930 INFO:tasks.workunit.client.1.smithi179.stdout:2/835: truncate dc/f68 3340215 0 2024-04-02T04:48:06.935 INFO:tasks.workunit.client.1.smithi179.stdout:7/961: creat d1/dd/d76/f15b x:0 0 0 2024-04-02T04:48:06.935 INFO:tasks.workunit.client.1.smithi179.stdout:1/947: truncate d3/d5/d2f/d4f/d42/df2/fba 2449252 0 2024-04-02T04:48:06.940 INFO:tasks.workunit.client.1.smithi179.stdout:4/965: mknod d1/d3/dc0/d147/c148 0 2024-04-02T04:48:06.943 INFO:tasks.workunit.client.1.smithi179.stdout:3/973: mkdir d8/dd3/d9a/d136 0 2024-04-02T04:48:06.947 INFO:tasks.workunit.client.1.smithi179.stdout:2/836: mknod dc/d29/d4a/d1e/d8a/c116 0 2024-04-02T04:48:06.950 INFO:tasks.workunit.client.1.smithi179.stdout:9/928: link d0/d1/d10/d6d/d58/cd3 d0/d1/d11/d29/d86/c128 0 2024-04-02T04:48:06.955 INFO:tasks.workunit.client.1.smithi179.stdout:7/962: read d1/dd/d23/f2f [2922966,56528] 0 2024-04-02T04:48:06.961 INFO:tasks.workunit.client.1.smithi179.stdout:5/909: dwrite d8/d1b/d23/d36/d53/d62/da9/fc3 [0,4194304] 0 2024-04-02T04:48:06.962 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:06 smithi070 ceph-mon[29212]: pgmap v22: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 108 MiB/s rd, 266 MiB/s wr, 601 op/s 2024-04-02T04:48:06.970 INFO:tasks.workunit.client.0.smithi070.stdout:3/822: sync 2024-04-02T04:48:06.970 INFO:tasks.workunit.client.0.smithi070.stdout:4/887: sync 2024-04-02T04:48:06.971 INFO:tasks.workunit.client.0.smithi070.stdout:1/778: sync 2024-04-02T04:48:06.971 INFO:tasks.workunit.client.1.smithi179.stdout:9/929: write d0/d1/d11/d77/f74 [2726496,56695] 0 2024-04-02T04:48:06.972 INFO:tasks.workunit.client.0.smithi070.stdout:3/823: stat d10/d17/d51/f76 0 2024-04-02T04:48:06.972 INFO:tasks.workunit.client.0.smithi070.stdout:1/779: chown d0/d7/d1a/db9/l5b 235966 1 2024-04-02T04:48:06.972 INFO:tasks.workunit.client.1.smithi179.stdout:9/930: chown d0/d1/f11e 5150092 1 2024-04-02T04:48:06.975 INFO:tasks.workunit.client.1.smithi179.stdout:8/970: link d3/d32/d21/d22/d6b/db1/d104/d11d/f12c d3/d1b/d2a/f15d 0 2024-04-02T04:48:06.979 INFO:tasks.workunit.client.1.smithi179.stdout:1/948: mkdir d3/d5/d2f/d4f/d42/d55/df3/d14e 0 2024-04-02T04:48:06.986 INFO:tasks.workunit.client.1.smithi179.stdout:4/966: dwrite d1/d3/df/d10/d2f/d60/d71/f85 [0,4194304] 0 2024-04-02T04:48:06.988 INFO:tasks.workunit.client.1.smithi179.stdout:3/974: symlink d8/d6d/l137 0 2024-04-02T04:48:06.989 INFO:tasks.workunit.client.0.smithi070.stdout:0/935: getdents d4/d5/d58/d60/d83 0 2024-04-02T04:48:06.994 INFO:tasks.workunit.client.1.smithi179.stdout:2/837: mknod dc/d29/d31/d57/d97/ddd/c117 0 2024-04-02T04:48:06.995 INFO:tasks.workunit.client.0.smithi070.stdout:9/816: fdatasync d4/d10/d12/d27/d60/d9e/f77 0 2024-04-02T04:48:06.997 INFO:tasks.workunit.client.0.smithi070.stdout:0/936: dwrite d4/d5/d13/d2a/d3a/d45/d112/f11a [0,4194304] 0 2024-04-02T04:48:07.008 INFO:tasks.workunit.client.1.smithi179.stdout:8/971: rmdir d3/d32/d21/d50 39 2024-04-02T04:48:07.008 INFO:tasks.workunit.client.0.smithi070.stdout:6/755: dwrite d0/d3/d19/d31/dc1/fc8 [0,4194304] 0 2024-04-02T04:48:07.010 INFO:tasks.workunit.client.1.smithi179.stdout:7/963: truncate d1/d9/dd9/f123 1004264 0 2024-04-02T04:48:07.011 INFO:tasks.workunit.client.0.smithi070.stdout:8/819: chown d2/d3/d52/d62/f95 1 1 2024-04-02T04:48:07.013 INFO:tasks.workunit.client.1.smithi179.stdout:6/957: sync 2024-04-02T04:48:07.013 INFO:tasks.workunit.client.1.smithi179.stdout:5/910: write d8/d1b/d23/d36/d53/f7b [394040,88463] 0 2024-04-02T04:48:07.015 INFO:tasks.workunit.client.1.smithi179.stdout:5/911: write d8/d1b/d23/d32/d4e/d50/f112 [814124,107158] 0 2024-04-02T04:48:07.019 INFO:tasks.workunit.client.1.smithi179.stdout:4/967: mknod d1/d93/d9f/c149 0 2024-04-02T04:48:07.021 INFO:tasks.workunit.client.0.smithi070.stdout:2/913: dwrite d2/d12/f52 [0,4194304] 0 2024-04-02T04:48:07.022 INFO:tasks.workunit.client.0.smithi070.stdout:4/888: unlink d1/d9c/d65/l7e 0 2024-04-02T04:48:07.023 INFO:tasks.workunit.client.0.smithi070.stdout:4/889: stat d1/d3/d85/d8e 0 2024-04-02T04:48:07.024 INFO:tasks.workunit.client.1.smithi179.stdout:2/838: symlink dc/d29/d31/d57/d97/l118 0 2024-04-02T04:48:07.024 INFO:tasks.workunit.client.0.smithi070.stdout:2/914: write d2/d10/d21/d71/f9f [846744,69559] 0 2024-04-02T04:48:07.025 INFO:tasks.workunit.client.1.smithi179.stdout:2/839: dread - dc/d29/d31/d57/d97/dbf/de3/fff zero size 2024-04-02T04:48:07.025 INFO:tasks.workunit.client.0.smithi070.stdout:1/780: fdatasync d0/d9/fc 0 2024-04-02T04:48:07.025 INFO:tasks.workunit.client.1.smithi179.stdout:2/840: write dc/d29/de7/ffe [958685,50074] 0 2024-04-02T04:48:07.027 INFO:tasks.workunit.client.0.smithi070.stdout:2/915: chown d2/d10/d21/d71/f76 6800287 1 2024-04-02T04:48:07.029 INFO:tasks.workunit.client.1.smithi179.stdout:9/931: unlink d0/d1/lcf 0 2024-04-02T04:48:07.029 INFO:tasks.workunit.client.1.smithi179.stdout:9/932: readlink d0/d1/d11/d29/d80/l88 0 2024-04-02T04:48:07.031 INFO:tasks.workunit.client.0.smithi070.stdout:3/824: mkdir d10/d32/d4b/dc5/d114 0 2024-04-02T04:48:07.033 INFO:tasks.workunit.client.0.smithi070.stdout:1/781: dwrite d0/d54/f87 [0,4194304] 0 2024-04-02T04:48:07.036 INFO:tasks.workunit.client.0.smithi070.stdout:8/820: dread d2/d8/d23/d24/d5f/fe2 [0,4194304] 0 2024-04-02T04:48:07.038 INFO:tasks.workunit.client.0.smithi070.stdout:8/821: truncate d2/d8/d23/d24/d5f/feb 705100 0 2024-04-02T04:48:07.039 INFO:tasks.workunit.client.1.smithi179.stdout:6/958: truncate d3/d2d/d38/fa9 611143 0 2024-04-02T04:48:07.041 INFO:tasks.workunit.client.1.smithi179.stdout:5/912: mkdir d8/d1b/d23/d36/d53/d120 0 2024-04-02T04:48:07.045 INFO:tasks.workunit.client.0.smithi070.stdout:7/807: unlink d7/dd/df/d7a/d6c/d77/d75/d80/ff6 0 2024-04-02T04:48:07.045 INFO:tasks.workunit.client.1.smithi179.stdout:1/949: write d3/d5/f144 [698094,61353] 0 2024-04-02T04:48:07.047 INFO:tasks.workunit.client.1.smithi179.stdout:1/950: readlink d3/d5/d2f/d4f/d42/l149 0 2024-04-02T04:48:07.050 INFO:tasks.workunit.client.0.smithi070.stdout:6/756: symlink d0/d43/db3/dc7/le8 0 2024-04-02T04:48:07.050 INFO:tasks.workunit.client.0.smithi070.stdout:6/757: stat d0/d20/fcf 0 2024-04-02T04:48:07.051 INFO:tasks.workunit.client.0.smithi070.stdout:6/758: chown d0/d20/d5f/cb8 3 1 2024-04-02T04:48:07.051 INFO:tasks.workunit.client.1.smithi179.stdout:2/841: creat dc/d29/d31/d57/d97/f119 x:0 0 0 2024-04-02T04:48:07.053 INFO:tasks.workunit.client.1.smithi179.stdout:3/975: write d8/d57/ff4 [636495,119955] 0 2024-04-02T04:48:07.054 INFO:tasks.workunit.client.1.smithi179.stdout:3/976: fsync d8/d26/d36/f12a 0 2024-04-02T04:48:07.058 INFO:tasks.workunit.client.1.smithi179.stdout:7/964: creat d1/d9/d27/d87/d8b/d75/db9/ddd/f15c x:0 0 0 2024-04-02T04:48:07.058 INFO:tasks.workunit.client.0.smithi070.stdout:5/991: dwrite d5/d14/f10d [0,4194304] 0 2024-04-02T04:48:07.059 INFO:tasks.workunit.client.0.smithi070.stdout:6/759: dread d0/d55/d5c/f60 [0,4194304] 0 2024-04-02T04:48:07.062 INFO:tasks.workunit.client.1.smithi179.stdout:8/972: write d3/f61 [79177,24326] 0 2024-04-02T04:48:07.062 INFO:tasks.workunit.client.1.smithi179.stdout:5/913: dread d8/d1b/fe1 [0,4194304] 0 2024-04-02T04:48:07.065 INFO:tasks.workunit.client.0.smithi070.stdout:1/782: dread d0/d9/d1d/d22/d2c/f4a [0,4194304] 0 2024-04-02T04:48:07.068 INFO:tasks.workunit.client.1.smithi179.stdout:3/977: dread d8/d26/f62 [0,4194304] 0 2024-04-02T04:48:07.070 INFO:tasks.workunit.client.1.smithi179.stdout:3/978: write d8/d26/d36/f59 [1908465,102909] 0 2024-04-02T04:48:07.070 INFO:tasks.workunit.client.0.smithi070.stdout:2/916: mkdir d2/d10/d5c/d7a/db0/dc7/ded/d12a 0 2024-04-02T04:48:07.071 INFO:tasks.workunit.client.1.smithi179.stdout:3/979: write d8/d26/d36/f59 [1794543,41944] 0 2024-04-02T04:48:07.072 INFO:tasks.workunit.client.1.smithi179.stdout:3/980: dread - d8/d26/d52/d7e/def/f10c zero size 2024-04-02T04:48:07.076 INFO:tasks.workunit.client.0.smithi070.stdout:2/917: dwrite d2/d10/f102 [0,4194304] 0 2024-04-02T04:48:07.079 INFO:tasks.workunit.client.0.smithi070.stdout:2/918: read d2/d10/d19/f39 [2982993,122508] 0 2024-04-02T04:48:07.080 INFO:tasks.workunit.client.0.smithi070.stdout:2/919: fsync d2/d10/d53/f118 0 2024-04-02T04:48:07.081 INFO:tasks.workunit.client.0.smithi070.stdout:2/920: chown d2/d10/d21/d71/dd4/cdb 1 1 2024-04-02T04:48:07.197 INFO:tasks.workunit.client.1.smithi179.stdout:2/842: rename dc/d29/d31/d57/f10f to dc/dcb/f11a 0 2024-04-02T04:48:07.198 INFO:tasks.workunit.client.1.smithi179.stdout:2/843: write dc/d29/d31/d66/fe2 [560370,46088] 0 2024-04-02T04:48:07.201 INFO:tasks.workunit.client.1.smithi179.stdout:9/933: mkdir d0/d19/d27/d7a/d7e/db9/d108/d129 0 2024-04-02T04:48:07.201 INFO:tasks.workunit.client.0.smithi070.stdout:4/890: write d1/d3/d78/d91/f96 [993493,80128] 0 2024-04-02T04:48:07.204 INFO:tasks.workunit.client.1.smithi179.stdout:7/965: unlink d1/dd/d76/d138/da8/l130 0 2024-04-02T04:48:07.209 INFO:tasks.workunit.client.1.smithi179.stdout:6/959: dwrite d3/d12/d23/d39/f71 [4194304,4194304] 0 2024-04-02T04:48:07.210 INFO:tasks.workunit.client.1.smithi179.stdout:6/960: chown d3/d2c/d85/f12c 65231922 1 2024-04-02T04:48:07.211 INFO:tasks.workunit.client.1.smithi179.stdout:6/961: write d3/d2d/da5/f133 [84362,107440] 0 2024-04-02T04:48:07.216 INFO:tasks.workunit.client.1.smithi179.stdout:9/934: dread d0/d1/d10/d6d/f2d [0,4194304] 0 2024-04-02T04:48:07.216 INFO:tasks.workunit.client.1.smithi179.stdout:4/968: write d1/d3/dc/d27/d62/d100/f13e [1129936,19315] 0 2024-04-02T04:48:07.217 INFO:tasks.workunit.client.1.smithi179.stdout:4/969: write d1/d3/f11b [736155,68488] 0 2024-04-02T04:48:07.221 INFO:tasks.workunit.client.1.smithi179.stdout:5/914: rename d8/d1b/d23/d36/d53/d62/l67 to d8/d1b/d23/d36/d53/d62/l121 0 2024-04-02T04:48:07.223 INFO:tasks.workunit.client.1.smithi179.stdout:5/915: chown d8/d1b/d23/d32/l35 1 1 2024-04-02T04:48:07.223 INFO:tasks.workunit.client.1.smithi179.stdout:4/970: dwrite d1/d3/df/d10/d3a/f11f [0,4194304] 0 2024-04-02T04:48:07.243 INFO:tasks.workunit.client.1.smithi179.stdout:3/981: unlink d8/d26/d36/f38 0 2024-04-02T04:48:07.248 INFO:tasks.workunit.client.1.smithi179.stdout:3/982: dwrite d8/d26/d36/f90 [0,4194304] 0 2024-04-02T04:48:07.254 INFO:tasks.workunit.client.0.smithi070.stdout:2/921: fsync d2/d24/f2d 0 2024-04-02T04:48:07.284 INFO:tasks.workunit.client.0.smithi070.stdout:5/992: mknod d5/d9/d10/db9/dd9/c139 0 2024-04-02T04:48:07.291 INFO:tasks.workunit.client.0.smithi070.stdout:2/922: stat d2/c16 0 2024-04-02T04:48:07.295 INFO:tasks.workunit.client.1.smithi179.stdout:5/916: sync 2024-04-02T04:48:07.295 INFO:tasks.workunit.client.1.smithi179.stdout:5/917: readlink d8/d1b/d23/d32/d46/l5b 0 2024-04-02T04:48:07.304 INFO:tasks.workunit.client.0.smithi070.stdout:6/760: creat d0/fe9 x:0 0 0 2024-04-02T04:48:07.307 INFO:tasks.workunit.client.0.smithi070.stdout:5/993: mknod d5/d9/d23/dc3/c13a 0 2024-04-02T04:48:07.313 INFO:tasks.workunit.client.0.smithi070.stdout:8/822: getdents d2/d8/d29/d22/dc8 0 2024-04-02T04:48:07.314 INFO:tasks.workunit.client.0.smithi070.stdout:8/823: chown d2/d8/f6d 30704 1 2024-04-02T04:48:07.316 INFO:tasks.workunit.client.0.smithi070.stdout:0/937: getdents d4/d5/da/d2d 0 2024-04-02T04:48:07.317 INFO:tasks.workunit.client.0.smithi070.stdout:8/824: dread d2/d8/d29/d22/d5a/d77/fc6 [0,4194304] 0 2024-04-02T04:48:07.319 INFO:tasks.workunit.client.0.smithi070.stdout:5/994: dread d5/d17/d2b/d7b/f92 [0,4194304] 0 2024-04-02T04:48:07.320 INFO:tasks.workunit.client.0.smithi070.stdout:7/808: creat d7/d9/f10a x:0 0 0 2024-04-02T04:48:07.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:06 smithi179 ceph-mon[36603]: pgmap v22: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 108 MiB/s rd, 266 MiB/s wr, 601 op/s 2024-04-02T04:48:07.322 INFO:tasks.workunit.client.0.smithi070.stdout:2/923: creat d2/d10/d5c/d7a/db0/dd8/dd2/f12b x:0 0 0 2024-04-02T04:48:07.325 INFO:tasks.workunit.client.0.smithi070.stdout:3/825: write d10/d17/fcc [668467,83852] 0 2024-04-02T04:48:07.329 INFO:tasks.workunit.client.0.smithi070.stdout:8/825: truncate d2/d8/d23/d24/d5f/fb0 347921 0 2024-04-02T04:48:07.330 INFO:tasks.workunit.client.0.smithi070.stdout:8/826: write d2/d8/d29/db/f13 [7239365,88529] 0 2024-04-02T04:48:07.333 INFO:tasks.workunit.client.0.smithi070.stdout:5/995: symlink d5/d9/d23/d30/dde/l13b 0 2024-04-02T04:48:07.334 INFO:tasks.workunit.client.0.smithi070.stdout:5/996: readlink d5/d17/lc6 0 2024-04-02T04:48:07.340 INFO:tasks.workunit.client.0.smithi070.stdout:2/924: symlink d2/d10/d21/d3d/l12c 0 2024-04-02T04:48:07.356 INFO:tasks.workunit.client.0.smithi070.stdout:2/925: truncate d2/f1c 85085 0 2024-04-02T04:48:07.362 INFO:tasks.workunit.client.0.smithi070.stdout:5/997: mkdir d5/d17/d27/d13c 0 2024-04-02T04:48:07.372 INFO:tasks.workunit.client.1.smithi179.stdout:2/844: creat dc/d29/d31/d57/dea/f11b x:0 0 0 2024-04-02T04:48:07.374 INFO:tasks.workunit.client.0.smithi070.stdout:5/998: mkdir d5/d17/da7/d125/d13d 0 2024-04-02T04:48:07.375 INFO:tasks.workunit.client.0.smithi070.stdout:5/999: stat d5/d9/d23/d30/da2/d75/cf0 0 2024-04-02T04:48:07.377 INFO:tasks.workunit.client.1.smithi179.stdout:6/962: fsync d3/d12/d23/d26/dd3/f100 0 2024-04-02T04:48:07.378 INFO:tasks.workunit.client.1.smithi179.stdout:8/973: mknod d3/d32/d21/d22/d6b/db1/df8/d126/c15e 0 2024-04-02T04:48:07.379 INFO:tasks.workunit.client.1.smithi179.stdout:9/935: fdatasync d0/d1/d11/d77/fd6 0 2024-04-02T04:48:07.381 INFO:tasks.workunit.client.1.smithi179.stdout:4/971: truncate d1/d15/d7c/fc2 170966 0 2024-04-02T04:48:07.385 INFO:tasks.workunit.client.1.smithi179.stdout:3/983: truncate d8/d26/f62 1492790 0 2024-04-02T04:48:07.388 INFO:tasks.workunit.client.1.smithi179.stdout:7/966: link d1/f13a d1/dd/d23/dab/f15d 0 2024-04-02T04:48:07.390 INFO:tasks.workunit.client.1.smithi179.stdout:6/963: mkdir d3/d2d/d38/db5/d154 0 2024-04-02T04:48:07.393 INFO:tasks.workunit.client.1.smithi179.stdout:9/936: symlink d0/d19/d27/d7a/dcb/l12a 0 2024-04-02T04:48:07.394 INFO:tasks.workunit.client.1.smithi179.stdout:9/937: dread - d0/d19/f111 zero size 2024-04-02T04:48:07.394 INFO:tasks.workunit.client.1.smithi179.stdout:8/974: truncate d3/d32/d21/d22/d6b/d91/d110/f118 66045 0 2024-04-02T04:48:07.395 INFO:tasks.workunit.client.1.smithi179.stdout:9/938: read - d0/d19/d27/d7a/d87/daf/dc2/f127 zero size 2024-04-02T04:48:07.396 INFO:tasks.workunit.client.1.smithi179.stdout:4/972: write d1/d3/df/ffc [1168363,61767] 0 2024-04-02T04:48:07.399 INFO:tasks.workunit.client.1.smithi179.stdout:5/918: link d8/d1b/f3b d8/d1b/d23/d32/dbe/daf/db7/dad/d111/f122 0 2024-04-02T04:48:07.400 INFO:tasks.workunit.client.1.smithi179.stdout:7/967: creat d1/dd/d76/d13c/f15e x:0 0 0 2024-04-02T04:48:07.402 INFO:tasks.workunit.client.1.smithi179.stdout:8/975: rmdir d3/d32/d21/d49/df0/df9 39 2024-04-02T04:48:07.404 INFO:tasks.workunit.client.1.smithi179.stdout:6/964: dread d3/d2d/d43/dd7/f107 [0,4194304] 0 2024-04-02T04:48:07.406 INFO:tasks.workunit.client.1.smithi179.stdout:5/919: fdatasync d8/d1b/d23/db1/fed 0 2024-04-02T04:48:07.411 INFO:tasks.workunit.client.1.smithi179.stdout:4/973: symlink d1/d15/d127/l14a 0 2024-04-02T04:48:07.414 INFO:tasks.workunit.client.1.smithi179.stdout:7/968: fsync d1/d9/d27/d87/d8b/d75/db9/d8d/f8f 0 2024-04-02T04:48:07.417 INFO:tasks.workunit.client.1.smithi179.stdout:8/976: unlink d3/d32/d21/d50/d56/dad/dbc/fd0 0 2024-04-02T04:48:07.418 INFO:tasks.workunit.client.1.smithi179.stdout:4/974: stat d1/d3/df/d10/d3a/c6a 0 2024-04-02T04:48:07.419 INFO:tasks.workunit.client.1.smithi179.stdout:3/984: getdents d8/d26/d83/dbc/ded 0 2024-04-02T04:48:07.421 INFO:tasks.workunit.client.1.smithi179.stdout:5/920: dread d8/d1b/d23/d32/dbe/fcf [0,4194304] 0 2024-04-02T04:48:07.421 INFO:tasks.workunit.client.1.smithi179.stdout:7/969: unlink d1/d9/d27/d87/d8b/d75/db9/f92 0 2024-04-02T04:48:07.429 INFO:tasks.workunit.client.1.smithi179.stdout:3/985: dread d8/d57/f9b [0,4194304] 0 2024-04-02T04:48:07.430 INFO:tasks.workunit.client.1.smithi179.stdout:7/970: symlink d1/dd/d76/d138/da8/dda/d14b/d4d/d11b/l15f 0 2024-04-02T04:48:07.432 INFO:tasks.workunit.client.1.smithi179.stdout:5/921: mkdir d8/d1b/d23/d36/dcb/d123 0 2024-04-02T04:48:07.433 INFO:tasks.workunit.client.1.smithi179.stdout:5/922: readlink d8/d1b/d23/db1/l110 0 2024-04-02T04:48:07.441 INFO:tasks.workunit.client.1.smithi179.stdout:5/923: dread - d8/f9e zero size 2024-04-02T04:48:07.445 INFO:tasks.workunit.client.1.smithi179.stdout:5/924: chown d8/d1b/d23/d36/d53/d62/da9/fe2 8188909 1 2024-04-02T04:48:07.447 INFO:tasks.workunit.client.1.smithi179.stdout:5/925: fdatasync d8/d1b/d23/d30/d73/f81 0 2024-04-02T04:48:07.459 INFO:tasks.workunit.client.1.smithi179.stdout:5/926: dread d8/d1b/f1f [0,4194304] 0 2024-04-02T04:48:07.460 INFO:tasks.workunit.client.1.smithi179.stdout:5/927: chown d8/d1b/d23/d36/d53/d62/da9/fc3 0 1 2024-04-02T04:48:07.464 INFO:tasks.workunit.client.1.smithi179.stdout:5/928: dread d8/d1b/d23/d30/f71 [0,4194304] 0 2024-04-02T04:48:07.466 INFO:tasks.workunit.client.1.smithi179.stdout:5/929: chown d8/d1b/d23/d32/d4e/d50/ce9 22120443 1 2024-04-02T04:48:07.468 INFO:tasks.workunit.client.1.smithi179.stdout:5/930: mknod d8/d1b/d23/d36/dcb/c124 0 2024-04-02T04:48:07.478 INFO:tasks.workunit.client.0.smithi070.stdout:4/891: dwrite d1/d3/d9/f81 [4194304,4194304] 0 2024-04-02T04:48:07.486 INFO:tasks.workunit.client.0.smithi070.stdout:9/817: rename d4/d10/d11/d7b/f35 to d4/d10/d12/d27/d60/f10c 0 2024-04-02T04:48:07.487 INFO:tasks.workunit.client.0.smithi070.stdout:0/938: dwrite d4/d5/d21/fec [0,4194304] 0 2024-04-02T04:48:07.491 INFO:tasks.workunit.client.0.smithi070.stdout:0/939: creat d4/d5/d13/d23/f120 x:0 0 0 2024-04-02T04:48:07.493 INFO:tasks.workunit.client.0.smithi070.stdout:9/818: creat d4/d10/d11/d7b/d19/d2f/d39/d57/f10d x:0 0 0 2024-04-02T04:48:07.495 INFO:tasks.workunit.client.0.smithi070.stdout:0/940: mknod d4/d5/d13/d17/d98/dd2/c121 0 2024-04-02T04:48:07.498 INFO:tasks.workunit.client.0.smithi070.stdout:0/941: mknod d4/d5/d58/d60/d83/da5/c122 0 2024-04-02T04:48:07.500 INFO:tasks.workunit.client.1.smithi179.stdout:8/977: rename d3/d32/d21/d22/d6b/db1/df8/d126/d96/le5 to d3/d32/l15f 0 2024-04-02T04:48:07.500 INFO:tasks.workunit.client.0.smithi070.stdout:9/819: mknod d4/d10/d11/d7b/d19/d2f/d4d/c10e 0 2024-04-02T04:48:07.501 INFO:tasks.workunit.client.1.smithi179.stdout:8/978: read d3/f61 [480617,76183] 0 2024-04-02T04:48:07.501 INFO:tasks.workunit.client.1.smithi179.stdout:6/965: link d3/d2d/lc5 d3/l155 0 2024-04-02T04:48:07.506 INFO:tasks.workunit.client.1.smithi179.stdout:4/975: rename d1/d3/df/d10/d2f/d60/da6 to d1/d15/d7c/d98/d14b 0 2024-04-02T04:48:07.508 INFO:tasks.workunit.client.0.smithi070.stdout:8/827: write d2/d8/d29/d22/d5a/d94/da2/fad [168942,2501] 0 2024-04-02T04:48:07.508 INFO:tasks.workunit.client.0.smithi070.stdout:8/828: fsync d2/d8/d29/d22/d5a/d94/da2/fc3 0 2024-04-02T04:48:07.512 INFO:tasks.workunit.client.1.smithi179.stdout:4/976: dwrite d1/d3/dc/d26/d8a/d5d/f129 [0,4194304] 0 2024-04-02T04:48:07.513 INFO:tasks.workunit.client.1.smithi179.stdout:6/966: dwrite d3/d42/ff3 [0,4194304] 0 2024-04-02T04:48:07.516 INFO:tasks.workunit.client.1.smithi179.stdout:6/967: fdatasync d3/d12/d23/d39/d4e/d6c/fd8 0 2024-04-02T04:48:07.532 INFO:tasks.workunit.client.0.smithi070.stdout:0/942: creat d4/d5/da/f123 x:0 0 0 2024-04-02T04:48:07.533 INFO:tasks.workunit.client.1.smithi179.stdout:1/951: rmdir d3/d5/d2f/d65 39 2024-04-02T04:48:07.536 INFO:tasks.workunit.client.0.smithi070.stdout:7/809: write d7/dd/df/d7a/dbc/f102 [137128,90133] 0 2024-04-02T04:48:07.539 INFO:tasks.workunit.client.0.smithi070.stdout:0/943: dwrite d4/d3c/fa2 [0,4194304] 0 2024-04-02T04:48:07.542 INFO:tasks.workunit.client.1.smithi179.stdout:4/977: symlink d1/d3/dc/d26/l14c 0 2024-04-02T04:48:07.545 INFO:tasks.workunit.client.0.smithi070.stdout:7/810: mkdir d7/dd/df/d7a/d6c/d77/d54/de5/d10b 0 2024-04-02T04:48:07.547 INFO:tasks.workunit.client.0.smithi070.stdout:7/811: chown d7/dd/df/d79/d46/d65/d7b/fb8 115230 1 2024-04-02T04:48:07.554 INFO:tasks.workunit.client.0.smithi070.stdout:7/812: dwrite d7/dd/df/d7a/ff2 [0,4194304] 0 2024-04-02T04:48:07.554 INFO:tasks.workunit.client.1.smithi179.stdout:1/952: creat d3/d5/d2f/d65/d12f/f14f x:0 0 0 2024-04-02T04:48:07.555 INFO:tasks.workunit.client.0.smithi070.stdout:2/926: write d2/d10/d21/d71/dd4/f82 [7369708,89444] 0 2024-04-02T04:48:07.558 INFO:tasks.workunit.client.1.smithi179.stdout:4/978: creat d1/d3/df/d10/d2f/f14d x:0 0 0 2024-04-02T04:48:07.558 INFO:tasks.workunit.client.0.smithi070.stdout:7/813: rmdir d7/d4e/d82/d57 39 2024-04-02T04:48:07.560 INFO:tasks.workunit.client.1.smithi179.stdout:1/953: unlink d3/d5/d2f/d65/d12f/ce6 0 2024-04-02T04:48:07.561 INFO:tasks.workunit.client.1.smithi179.stdout:1/954: dread - d3/d5/d2f/d4f/d35/d117/f13a zero size 2024-04-02T04:48:07.562 INFO:tasks.workunit.client.1.smithi179.stdout:1/955: stat d3/d5/d96/le1 0 2024-04-02T04:48:07.564 INFO:tasks.workunit.client.0.smithi070.stdout:7/814: mkdir d7/dd/df/d7a/d6c/d77/d54/d10c 0 2024-04-02T04:48:07.564 INFO:tasks.workunit.client.1.smithi179.stdout:4/979: dread - d1/d3/dc/dc6/f10f zero size 2024-04-02T04:48:07.565 INFO:tasks.workunit.client.1.smithi179.stdout:4/980: chown d1/d3/df/d10/d3a/c7d 1607 1 2024-04-02T04:48:07.567 INFO:tasks.workunit.client.1.smithi179.stdout:1/956: mkdir d3/d5/d2f/d4f/d35/d150 0 2024-04-02T04:48:07.568 INFO:tasks.workunit.client.0.smithi070.stdout:7/815: creat d7/dd/df/d106/f10d x:0 0 0 2024-04-02T04:48:07.573 INFO:tasks.workunit.client.0.smithi070.stdout:7/816: symlink d7/dd/df/d7a/d6c/d77/d75/l10e 0 2024-04-02T04:48:07.575 INFO:tasks.workunit.client.0.smithi070.stdout:7/817: write d7/dd/df/d79/d46/f53 [1324913,21549] 0 2024-04-02T04:48:07.576 INFO:tasks.workunit.client.1.smithi179.stdout:1/957: creat d3/d5/d2f/d4f/d42/df2/db6/f151 x:0 0 0 2024-04-02T04:48:07.579 INFO:tasks.workunit.client.1.smithi179.stdout:1/958: chown d3/d5/d2f/d4f/d42/df2/fba 911909 1 2024-04-02T04:48:07.580 INFO:tasks.workunit.client.1.smithi179.stdout:1/959: write d3/d5/f144 [832746,63924] 0 2024-04-02T04:48:07.580 INFO:tasks.workunit.client.0.smithi070.stdout:7/818: symlink d7/d4e/d82/d57/ddc/l10f 0 2024-04-02T04:48:07.581 INFO:tasks.workunit.client.0.smithi070.stdout:0/944: symlink d4/d5/d21/l124 0 2024-04-02T04:48:07.584 INFO:tasks.workunit.client.1.smithi179.stdout:1/960: creat d3/d5/d2f/d4f/d35/d150/f152 x:0 0 0 2024-04-02T04:48:07.585 INFO:tasks.workunit.client.0.smithi070.stdout:0/945: mkdir d4/d5/d58/d125 0 2024-04-02T04:48:07.585 INFO:tasks.workunit.client.1.smithi179.stdout:1/961: stat d3/d5/d2f/d4f/d42/df2/db6/dd8/ce0 0 2024-04-02T04:48:07.586 INFO:tasks.workunit.client.0.smithi070.stdout:7/819: creat d7/d4e/d82/d57/ddb/f110 x:0 0 0 2024-04-02T04:48:07.588 INFO:tasks.workunit.client.1.smithi179.stdout:1/962: rename d3/d5/d2f/d4f/d42/d53/d72/fbc to d3/d5/d2f/d4f/d35/d13f/f153 0 2024-04-02T04:48:07.589 INFO:tasks.workunit.client.0.smithi070.stdout:7/820: dread d7/dd/df/d7a/ff2 [0,4194304] 0 2024-04-02T04:48:07.593 INFO:tasks.workunit.client.1.smithi179.stdout:1/963: rename d3/da8 to d3/d5/d2f/d4f/d42/d55/df3/d154 0 2024-04-02T04:48:07.596 INFO:tasks.workunit.client.0.smithi070.stdout:7/821: rename d7/d4e/d82/d57/ddc/dfa to d7/dd/df/d79/de6/d111 0 2024-04-02T04:48:07.597 INFO:tasks.workunit.client.1.smithi179.stdout:9/939: dwrite d0/d1/d10/d6d/f25 [4194304,4194304] 0 2024-04-02T04:48:07.599 INFO:tasks.workunit.client.1.smithi179.stdout:4/981: getdents d1/d15/d127 0 2024-04-02T04:48:07.603 INFO:tasks.workunit.client.0.smithi070.stdout:7/822: dwrite d7/dd/df/d79/d46/f53 [0,4194304] 0 2024-04-02T04:48:07.605 INFO:tasks.workunit.client.0.smithi070.stdout:7/823: fsync d7/dd/df/d79/fbb 0 2024-04-02T04:48:07.605 INFO:tasks.workunit.client.1.smithi179.stdout:2/845: mknod dc/d29/d31/d57/d97/c11c 0 2024-04-02T04:48:07.612 INFO:tasks.workunit.client.0.smithi070.stdout:7/824: read d7/dd/f8d [1138341,23404] 0 2024-04-02T04:48:07.612 INFO:tasks.workunit.client.1.smithi179.stdout:6/968: mkdir d3/d12/d23/d156 0 2024-04-02T04:48:07.614 INFO:tasks.workunit.client.1.smithi179.stdout:9/940: write d0/d19/d27/d48/f10d [1126398,121547] 0 2024-04-02T04:48:07.617 INFO:tasks.workunit.client.1.smithi179.stdout:7/971: write d1/dd/d76/d138/da8/dda/d14b/fbc [753882,41019] 0 2024-04-02T04:48:07.618 INFO:tasks.workunit.client.0.smithi070.stdout:7/825: dwrite d7/dd/df/d34/f2a [0,4194304] 0 2024-04-02T04:48:07.622 INFO:tasks.workunit.client.1.smithi179.stdout:1/964: link d3/d5/d2f/d4f/d42/d53/d77/f97 d3/d5/d2f/d4f/d42/d55/f155 0 2024-04-02T04:48:07.622 INFO:tasks.workunit.client.1.smithi179.stdout:1/965: stat d3/d5/d2f/l50 0 2024-04-02T04:48:07.626 INFO:tasks.workunit.client.1.smithi179.stdout:2/846: link dc/d29/d4a/d4b/l92 dc/d29/d4a/d4b/l11d 0 2024-04-02T04:48:07.628 INFO:tasks.workunit.client.1.smithi179.stdout:9/941: link d0/d19/d27/d7a/d7e/dd4/f114 d0/d1/d10/de3/dff/f12b 0 2024-04-02T04:48:07.630 INFO:tasks.workunit.client.0.smithi070.stdout:1/783: creat d0/d9/d1d/d22/ff3 x:0 0 0 2024-04-02T04:48:07.631 INFO:tasks.workunit.client.1.smithi179.stdout:2/847: symlink dc/d29/d31/d57/d97/ddd/l11e 0 2024-04-02T04:48:07.632 INFO:tasks.workunit.client.1.smithi179.stdout:2/848: write dc/d29/d31/d66/fe2 [1440568,94226] 0 2024-04-02T04:48:07.632 INFO:tasks.workunit.client.0.smithi070.stdout:7/826: getdents d7/dd/df/d79 0 2024-04-02T04:48:07.634 INFO:tasks.workunit.client.1.smithi179.stdout:9/942: creat d0/d19/d27/d7a/d7e/db9/f12c x:0 0 0 2024-04-02T04:48:07.634 INFO:tasks.workunit.client.0.smithi070.stdout:7/827: chown d7/d4e/d82/d57/ddc/l10f 8072 1 2024-04-02T04:48:07.635 INFO:tasks.workunit.client.0.smithi070.stdout:1/784: rename d0/d9/f14 to d0/db4/ff4 0 2024-04-02T04:48:07.637 INFO:tasks.workunit.client.1.smithi179.stdout:2/849: symlink dc/d29/d31/d83/d8d/l11f 0 2024-04-02T04:48:07.637 INFO:tasks.workunit.client.1.smithi179.stdout:5/931: dwrite d8/d1b/d23/d30/f5c [0,4194304] 0 2024-04-02T04:48:07.637 INFO:tasks.workunit.client.0.smithi070.stdout:4/892: truncate d1/d3/d85/ddb/f104 464694 0 2024-04-02T04:48:07.638 INFO:tasks.workunit.client.1.smithi179.stdout:2/850: write dc/fe [9085,58747] 0 2024-04-02T04:48:07.640 INFO:tasks.workunit.client.0.smithi070.stdout:1/785: rmdir d0/d7/d1a/d95 39 2024-04-02T04:48:07.643 INFO:tasks.workunit.client.1.smithi179.stdout:9/943: creat d0/d19/d27/d7a/d7e/f12d x:0 0 0 2024-04-02T04:48:07.643 INFO:tasks.workunit.client.1.smithi179.stdout:9/944: readlink d0/d1/d10/ld2 0 2024-04-02T04:48:07.646 INFO:tasks.workunit.client.1.smithi179.stdout:5/932: dwrite d8/d1b/d23/d32/d4e/f93 [0,4194304] 0 2024-04-02T04:48:07.648 INFO:tasks.workunit.client.1.smithi179.stdout:9/945: dread d0/d1/d10/d6d/f2d [0,4194304] 0 2024-04-02T04:48:07.649 INFO:tasks.workunit.client.1.smithi179.stdout:9/946: chown d0/d19/d1c/c4f 1057874 1 2024-04-02T04:48:07.650 INFO:tasks.workunit.client.0.smithi070.stdout:4/893: creat d1/f12e x:0 0 0 2024-04-02T04:48:07.650 INFO:tasks.workunit.client.1.smithi179.stdout:9/947: write d0/d19/d27/d7a/d7e/f12d [1045640,112668] 0 2024-04-02T04:48:07.653 INFO:tasks.workunit.client.1.smithi179.stdout:5/933: unlink d8/d1b/d23/d32/f3d 0 2024-04-02T04:48:07.654 INFO:tasks.workunit.client.0.smithi070.stdout:9/820: write d4/d10/d11/d7b/d19/d2f/d4d/d92/fd1 [1031776,92092] 0 2024-04-02T04:48:07.657 INFO:tasks.workunit.client.1.smithi179.stdout:8/979: write d3/d32/d21/d51/fd3 [745154,13639] 0 2024-04-02T04:48:07.658 INFO:tasks.workunit.client.1.smithi179.stdout:9/948: dwrite d0/d1/d11/d29/d86/db0/f11c [0,4194304] 0 2024-04-02T04:48:07.660 INFO:tasks.workunit.client.0.smithi070.stdout:7/828: dread d7/dd/df/d79/f3b [0,4194304] 0 2024-04-02T04:48:07.663 INFO:tasks.workunit.client.0.smithi070.stdout:4/894: fsync d1/d9c/f6e 0 2024-04-02T04:48:07.663 INFO:tasks.workunit.client.1.smithi179.stdout:5/934: dwrite d8/d1b/d23/d36/d53/d62/da9/fc3 [4194304,4194304] 0 2024-04-02T04:48:07.664 INFO:tasks.workunit.client.0.smithi070.stdout:4/895: write d1/de7/f12a [129418,56720] 0 2024-04-02T04:48:07.665 INFO:tasks.workunit.client.0.smithi070.stdout:8/829: write d2/d8/fe5 [1034383,60534] 0 2024-04-02T04:48:07.669 INFO:tasks.workunit.client.0.smithi070.stdout:7/829: creat d7/dd/df/d7a/d6c/d77/d54/f112 x:0 0 0 2024-04-02T04:48:07.674 INFO:tasks.workunit.client.1.smithi179.stdout:9/949: rmdir d0/d1/d10/de3/dff 39 2024-04-02T04:48:07.676 INFO:tasks.workunit.client.0.smithi070.stdout:4/896: mkdir d1/d3/d78/dad/d12f 0 2024-04-02T04:48:07.677 INFO:tasks.workunit.client.1.smithi179.stdout:2/851: getdents dc/d29/de7 0 2024-04-02T04:48:07.679 INFO:tasks.workunit.client.0.smithi070.stdout:8/830: dread d2/d8/f6d [0,4194304] 0 2024-04-02T04:48:07.681 INFO:tasks.workunit.client.0.smithi070.stdout:7/830: creat d7/dd/df/d7a/d6c/d77/d75/d80/f113 x:0 0 0 2024-04-02T04:48:07.682 INFO:tasks.workunit.client.1.smithi179.stdout:3/986: dwrite d8/d7a/fcf [0,4194304] 0 2024-04-02T04:48:07.686 INFO:tasks.workunit.client.0.smithi070.stdout:8/831: truncate d2/d8/d29/d22/d5a/d9e/fce 550041 0 2024-04-02T04:48:07.687 INFO:tasks.workunit.client.0.smithi070.stdout:2/927: dwrite d2/d10/d5c/d7a/fda [0,4194304] 0 2024-04-02T04:48:07.688 INFO:tasks.workunit.client.0.smithi070.stdout:2/928: rename d2/d12 to d2/d12/d3e/d12d 22 2024-04-02T04:48:07.691 INFO:tasks.workunit.client.1.smithi179.stdout:5/935: mkdir d8/d1b/d23/d36/d125 0 2024-04-02T04:48:07.695 INFO:tasks.workunit.client.1.smithi179.stdout:2/852: rename dc/d29/d4a/d1e/l7a to dc/d29/de7/l120 0 2024-04-02T04:48:07.696 INFO:tasks.workunit.client.0.smithi070.stdout:8/832: mknod d2/d8/d29/d22/d5a/d94/c10c 0 2024-04-02T04:48:07.701 INFO:tasks.workunit.client.1.smithi179.stdout:3/987: truncate d8/d26/d83/dbc/ded/ff7 857939 0 2024-04-02T04:48:07.701 INFO:tasks.workunit.client.1.smithi179.stdout:3/988: chown d8/dd3/f97 6841 1 2024-04-02T04:48:07.703 INFO:tasks.workunit.client.1.smithi179.stdout:1/966: sync 2024-04-02T04:48:07.704 INFO:tasks.workunit.client.0.smithi070.stdout:8/833: unlink d2/d8/d23/d24/d5f/cca 0 2024-04-02T04:48:07.708 INFO:tasks.workunit.client.1.smithi179.stdout:5/936: fsync d8/f18 0 2024-04-02T04:48:07.710 INFO:tasks.workunit.client.1.smithi179.stdout:2/853: mkdir dc/d29/d4a/d1e/d6c/dd2/d121 0 2024-04-02T04:48:07.712 INFO:tasks.workunit.client.0.smithi070.stdout:8/834: fsync d2/d8/d29/fba 0 2024-04-02T04:48:07.715 INFO:tasks.workunit.client.1.smithi179.stdout:3/989: unlink d8/d26/d52/d7e/def/f129 0 2024-04-02T04:48:07.717 INFO:tasks.workunit.client.1.smithi179.stdout:1/967: rename d3/d5/d2f/db7 to d3/d5/d2f/d36/d156 0 2024-04-02T04:48:07.719 INFO:tasks.workunit.client.1.smithi179.stdout:4/982: truncate d1/d93/de6/f10e 1113287 0 2024-04-02T04:48:07.720 INFO:tasks.workunit.client.1.smithi179.stdout:3/990: dwrite d8/d26/f56 [4194304,4194304] 0 2024-04-02T04:48:07.721 INFO:tasks.workunit.client.0.smithi070.stdout:8/835: rename d2/fea to d2/d3/d35/d8e/f10d 0 2024-04-02T04:48:07.722 INFO:tasks.workunit.client.1.smithi179.stdout:2/854: fdatasync dc/d29/d4a/f45 0 2024-04-02T04:48:07.723 INFO:tasks.workunit.client.1.smithi179.stdout:6/969: write d3/d12/d151/fd2 [1811565,47511] 0 2024-04-02T04:48:07.728 INFO:tasks.workunit.client.1.smithi179.stdout:1/968: symlink d3/d85/d112/l157 0 2024-04-02T04:48:07.730 INFO:tasks.workunit.client.1.smithi179.stdout:3/991: dwrite d8/d26/d36/f12a [0,4194304] 0 2024-04-02T04:48:07.733 INFO:tasks.workunit.client.1.smithi179.stdout:7/972: dwrite d1/d9/d55/fd7 [0,4194304] 0 2024-04-02T04:48:07.735 INFO:tasks.workunit.client.0.smithi070.stdout:1/786: dread - d0/d9/d1d/d22/d2c/fef zero size 2024-04-02T04:48:07.736 INFO:tasks.workunit.client.0.smithi070.stdout:1/787: chown d0/d54/d64/la8 24 1 2024-04-02T04:48:07.738 INFO:tasks.workunit.client.1.smithi179.stdout:2/855: truncate dc/d29/d31/d57/f71 1368011 0 2024-04-02T04:48:07.740 INFO:tasks.workunit.client.1.smithi179.stdout:4/983: symlink d1/d15/d7c/d140/l14e 0 2024-04-02T04:48:07.742 INFO:tasks.workunit.client.1.smithi179.stdout:4/984: truncate d1/d3/dc0/f13f 774428 0 2024-04-02T04:48:07.744 INFO:tasks.workunit.client.1.smithi179.stdout:2/856: dwrite dc/d29/d31/d83/dae/db0/dbc/f65 [0,4194304] 0 2024-04-02T04:48:07.746 INFO:tasks.workunit.client.1.smithi179.stdout:1/969: mknod d3/d5/d2f/d4f/d35/d117/c158 0 2024-04-02T04:48:07.754 INFO:tasks.workunit.client.1.smithi179.stdout:2/857: dwrite dc/fe [4194304,4194304] 0 2024-04-02T04:48:07.754 INFO:tasks.workunit.client.0.smithi070.stdout:9/821: write d4/d10/d11/d7b/d19/d82/f104 [947260,20446] 0 2024-04-02T04:48:07.754 INFO:tasks.workunit.client.1.smithi179.stdout:5/937: rename d8/d1b/d23/d32/dbe/daf/dd4/l8b to d8/d1b/d23/d32/dbe/daf/db7/l126 0 2024-04-02T04:48:07.757 INFO:tasks.workunit.client.1.smithi179.stdout:5/938: stat d8/l48 0 2024-04-02T04:48:07.758 INFO:tasks.workunit.client.0.smithi070.stdout:9/822: symlink d4/d10/d12/l10f 0 2024-04-02T04:48:07.758 INFO:tasks.workunit.client.0.smithi070.stdout:1/788: dread d0/d7/d1a/f24 [0,4194304] 0 2024-04-02T04:48:07.759 INFO:tasks.workunit.client.0.smithi070.stdout:1/789: chown d0/d7/f81 726934060 1 2024-04-02T04:48:07.761 INFO:tasks.workunit.client.1.smithi179.stdout:8/980: write d3/d31/f69 [4540619,72881] 0 2024-04-02T04:48:07.766 INFO:tasks.workunit.client.1.smithi179.stdout:3/992: rename d8/dd3/dbb to d8/d26/d52/d7e/db8/d138 0 2024-04-02T04:48:07.770 INFO:tasks.workunit.client.1.smithi179.stdout:4/985: fdatasync d1/d3/dc/fd 0 2024-04-02T04:48:07.772 INFO:tasks.workunit.client.0.smithi070.stdout:4/897: write d1/d9c/f48 [2838172,65196] 0 2024-04-02T04:48:07.776 INFO:tasks.workunit.client.0.smithi070.stdout:4/898: dread - d1/d3/d9/fb7 zero size 2024-04-02T04:48:07.779 INFO:tasks.workunit.client.0.smithi070.stdout:7/831: write d7/d36/fae [1630136,79298] 0 2024-04-02T04:48:07.780 INFO:tasks.workunit.client.1.smithi179.stdout:9/950: dwrite d0/d1/d11/dab/ded/d101/ff0 [0,4194304] 0 2024-04-02T04:48:07.781 INFO:tasks.workunit.client.1.smithi179.stdout:9/951: truncate d0/d19/d27/f10f 242298 0 2024-04-02T04:48:07.781 INFO:tasks.workunit.client.0.smithi070.stdout:7/832: write d7/dd/df/d7a/dbc/ff9 [660614,27714] 0 2024-04-02T04:48:07.783 INFO:tasks.workunit.client.0.smithi070.stdout:7/833: fdatasync d7/dd/df/d7a/d6c/d77/d75/d80/fee 0 2024-04-02T04:48:07.786 INFO:tasks.workunit.client.0.smithi070.stdout:4/899: fdatasync d1/d3/d9/f5f 0 2024-04-02T04:48:07.788 INFO:tasks.workunit.client.0.smithi070.stdout:4/900: chown d1/d3/d85/d8e/cde 115383 1 2024-04-02T04:48:07.792 INFO:tasks.workunit.client.1.smithi179.stdout:1/970: link d3/d5/d2f/d4f/d42/d55/df3/l128 d3/d5/d2f/d4f/dab/l159 0 2024-04-02T04:48:07.792 INFO:tasks.workunit.client.1.smithi179.stdout:4/986: read d1/d3/dc/d27/d62/f76 [912774,119250] 0 2024-04-02T04:48:07.793 INFO:tasks.workunit.client.1.smithi179.stdout:4/987: read - d1/d3/d19/f130 zero size 2024-04-02T04:48:07.793 INFO:tasks.workunit.client.1.smithi179.stdout:9/952: dread d0/d19/d1c/f31 [0,4194304] 0 2024-04-02T04:48:07.794 INFO:tasks.workunit.client.1.smithi179.stdout:4/988: write d1/f42 [1389914,13161] 0 2024-04-02T04:48:07.795 INFO:tasks.workunit.client.1.smithi179.stdout:9/953: stat d0/d1/d11/dab/df1/f109 0 2024-04-02T04:48:07.801 INFO:tasks.workunit.client.1.smithi179.stdout:5/939: rename d8/d1b/d23/d30/d73/lcd to d8/d1b/d23/d32/l127 0 2024-04-02T04:48:07.802 INFO:tasks.workunit.client.1.smithi179.stdout:3/993: mknod d8/d57/d8f/d107/c139 0 2024-04-02T04:48:07.809 INFO:tasks.workunit.client.1.smithi179.stdout:3/994: rmdir d8/d26/d52 39 2024-04-02T04:48:07.811 INFO:tasks.workunit.client.1.smithi179.stdout:7/973: write d1/f4e [2021258,98952] 0 2024-04-02T04:48:07.811 INFO:tasks.workunit.client.1.smithi179.stdout:1/971: symlink d3/d5/de/l15a 0 2024-04-02T04:48:07.814 INFO:tasks.workunit.client.1.smithi179.stdout:9/954: creat d0/d19/d27/d7a/d7e/db9/d108/d129/f12e x:0 0 0 2024-04-02T04:48:07.817 INFO:tasks.workunit.client.1.smithi179.stdout:1/972: mknod d3/d5/d2f/d4f/d42/d53/d72/c15b 0 2024-04-02T04:48:07.818 INFO:tasks.workunit.client.1.smithi179.stdout:6/970: dwrite d3/d12/d44/d10a/d10e/d112/f14a [0,4194304] 0 2024-04-02T04:48:07.819 INFO:tasks.workunit.client.1.smithi179.stdout:9/955: write d0/d19/d27/d7a/f126 [4619695,125389] 0 2024-04-02T04:48:07.823 INFO:tasks.workunit.client.1.smithi179.stdout:2/858: truncate dc/d29/d31/f3a 409562 0 2024-04-02T04:48:07.824 INFO:tasks.workunit.client.1.smithi179.stdout:9/956: creat d0/d1/d11/d3a/f12f x:0 0 0 2024-04-02T04:48:07.825 INFO:tasks.workunit.client.1.smithi179.stdout:1/973: creat d3/d5/d2f/d4f/d42/d55/df3/d14e/f15c x:0 0 0 2024-04-02T04:48:07.826 INFO:tasks.workunit.client.1.smithi179.stdout:8/981: write d3/d32/d21/d50/d65/faa [1124524,66491] 0 2024-04-02T04:48:07.827 INFO:tasks.workunit.client.1.smithi179.stdout:6/971: dwrite d3/f10b [0,4194304] 0 2024-04-02T04:48:07.829 INFO:tasks.workunit.client.1.smithi179.stdout:9/957: dread - d0/d19/d1c/fd1 zero size 2024-04-02T04:48:07.833 INFO:tasks.workunit.client.1.smithi179.stdout:6/972: unlink d3/d12/d23/d26/d57/fc3 0 2024-04-02T04:48:07.834 INFO:tasks.workunit.client.1.smithi179.stdout:2/859: mknod dc/d29/d4a/d1e/d6c/dd2/d121/c122 0 2024-04-02T04:48:07.834 INFO:tasks.workunit.client.1.smithi179.stdout:5/940: write d8/f18 [3131670,22614] 0 2024-04-02T04:48:07.836 INFO:tasks.workunit.client.1.smithi179.stdout:4/989: dwrite d1/d3/dc/d26/d8a/f45 [4194304,4194304] 0 2024-04-02T04:48:07.842 INFO:tasks.workunit.client.1.smithi179.stdout:5/941: dwrite d8/d1b/d23/d32/dbe/daf/dd4/d105/fd8 [0,4194304] 0 2024-04-02T04:48:07.913 INFO:tasks.workunit.client.1.smithi179.stdout:7/974: sync 2024-04-02T04:48:07.939 INFO:tasks.workunit.client.1.smithi179.stdout:8/982: getdents d3/d32/d21/d22/d6b/db1/df8/d126/d12e 0 2024-04-02T04:48:07.945 INFO:tasks.workunit.client.0.smithi070.stdout:2/929: dwrite d2/f9c [0,4194304] 0 2024-04-02T04:48:07.948 INFO:tasks.workunit.client.0.smithi070.stdout:9/823: dread d4/d10/d11/d7b/d19/d53/f59 [0,4194304] 0 2024-04-02T04:48:07.949 INFO:tasks.workunit.client.0.smithi070.stdout:2/930: readlink d2/d24/l91 0 2024-04-02T04:48:07.954 INFO:tasks.workunit.client.0.smithi070.stdout:1/790: dwrite d0/d9/d1d/d22/d2c/d4f/d77/d8c/faa [0,4194304] 0 2024-04-02T04:48:07.955 INFO:tasks.workunit.client.1.smithi179.stdout:3/995: dwrite d8/dd3/fd9 [4194304,4194304] 0 2024-04-02T04:48:07.957 INFO:tasks.workunit.client.1.smithi179.stdout:4/990: mkdir d1/d15/d7c/d145/de5/d14f 0 2024-04-02T04:48:07.958 INFO:tasks.workunit.client.1.smithi179.stdout:4/991: fdatasync d1/d3/dc/d26/d8a/d5d/f129 0 2024-04-02T04:48:07.974 INFO:tasks.workunit.client.1.smithi179.stdout:6/973: dread d3/d42/f7e [0,4194304] 0 2024-04-02T04:48:07.975 INFO:tasks.workunit.client.1.smithi179.stdout:6/974: dread - d3/d2d/d38/db5/d150/f124 zero size 2024-04-02T04:48:07.983 INFO:tasks.workunit.client.1.smithi179.stdout:5/942: dread d8/d1b/d23/d32/d4e/d51/f66 [0,4194304] 0 2024-04-02T04:48:07.985 INFO:tasks.workunit.client.1.smithi179.stdout:8/983: truncate d3/d1b/d2a/d41/d148/f14e 894248 0 2024-04-02T04:48:07.986 INFO:tasks.workunit.client.1.smithi179.stdout:1/974: link d3/d5/l2b d3/d5/d2f/d4f/d42/d55/df3/d14e/l15d 0 2024-04-02T04:48:07.989 INFO:tasks.workunit.client.1.smithi179.stdout:2/860: rename dc/d29/d4a/d1e/c20 to dc/d29/d4a/d1e/c123 0 2024-04-02T04:48:07.996 INFO:tasks.workunit.client.1.smithi179.stdout:9/958: getdents d0/d1/d11/dab 0 2024-04-02T04:48:08.001 INFO:tasks.workunit.client.1.smithi179.stdout:2/861: creat dc/d29/d4a/d1e/d6c/dd2/f124 x:0 0 0 2024-04-02T04:48:08.001 INFO:tasks.workunit.client.1.smithi179.stdout:2/862: stat dc/d29/d31/d57/dea/f11b 0 2024-04-02T04:48:08.002 INFO:tasks.workunit.client.1.smithi179.stdout:2/863: chown dc/d29/d31/d57/d97/dbf 241028317 1 2024-04-02T04:48:08.006 INFO:tasks.workunit.client.1.smithi179.stdout:7/975: getdents d1/d9/d27/d87/d8b/d75/db9/ddd 0 2024-04-02T04:48:08.007 INFO:tasks.workunit.client.1.smithi179.stdout:2/864: dwrite dc/d29/d31/d66/fe2 [0,4194304] 0 2024-04-02T04:48:08.014 INFO:tasks.workunit.client.1.smithi179.stdout:6/975: rename d3/d12/d23/d39/d80/d93/c13e to d3/d12/d44/c157 0 2024-04-02T04:48:08.019 INFO:tasks.workunit.client.1.smithi179.stdout:7/976: write d1/d9/d27/d87/f116 [2353837,21698] 0 2024-04-02T04:48:08.020 INFO:tasks.workunit.client.1.smithi179.stdout:7/977: chown d1/dd/d23/f57 25 1 2024-04-02T04:48:08.022 INFO:tasks.workunit.client.1.smithi179.stdout:9/959: dread d0/d1/d11/d29/d40/f67 [0,4194304] 0 2024-04-02T04:48:08.023 INFO:tasks.workunit.client.1.smithi179.stdout:6/976: chown d3/d2d/lc5 1 1 2024-04-02T04:48:08.025 INFO:tasks.workunit.client.1.smithi179.stdout:5/943: rename d8/d1b/d23/d36/d53/d62/da9/fe2 to d8/d1b/d23/d11f/f128 0 2024-04-02T04:48:08.028 INFO:tasks.workunit.client.1.smithi179.stdout:7/978: creat d1/d9/dd9/f160 x:0 0 0 2024-04-02T04:48:08.030 INFO:tasks.workunit.client.1.smithi179.stdout:1/975: truncate d3/d5/d2f/fb0 2896765 0 2024-04-02T04:48:08.031 INFO:tasks.workunit.client.1.smithi179.stdout:9/960: rmdir d0/d1/d11/dab 39 2024-04-02T04:48:08.031 INFO:tasks.workunit.client.1.smithi179.stdout:9/961: stat d0/d1/d11/d3a/lbd 0 2024-04-02T04:48:08.032 INFO:tasks.workunit.client.1.smithi179.stdout:6/977: mknod d3/d12/d44/d10a/d10e/c158 0 2024-04-02T04:48:08.033 INFO:tasks.workunit.client.1.smithi179.stdout:9/962: write d0/d19/d27/d7a/d7e/fb8 [6120281,34401] 0 2024-04-02T04:48:08.033 INFO:tasks.workunit.client.1.smithi179.stdout:9/963: fdatasync d0/d19/d1c/f6b 0 2024-04-02T04:48:08.038 INFO:tasks.workunit.client.0.smithi070.stdout:7/834: link d7/dd/db1/lfd d7/d9/l114 0 2024-04-02T04:48:08.040 INFO:tasks.workunit.client.0.smithi070.stdout:1/791: dread d0/d9/d1d/d22/d2c/f76 [0,4194304] 0 2024-04-02T04:48:08.040 INFO:tasks.workunit.client.1.smithi179.stdout:8/984: rename d3/d32/d21/d50/d56/d94/f60 to d3/d32/d21/d22/d6b/db1/df8/f160 0 2024-04-02T04:48:08.041 INFO:tasks.workunit.client.1.smithi179.stdout:8/985: chown d3/d32/d21/d50/d56 5307 1 2024-04-02T04:48:08.041 INFO:tasks.workunit.client.0.smithi070.stdout:4/901: mknod d1/d3/d85/d8e/d99/c130 0 2024-04-02T04:48:08.042 INFO:tasks.workunit.client.1.smithi179.stdout:2/865: dwrite dc/d29/d31/f77 [0,4194304] 0 2024-04-02T04:48:08.042 INFO:tasks.workunit.client.1.smithi179.stdout:5/944: read d8/d1b/d23/d32/f99 [4835416,117653] 0 2024-04-02T04:48:08.044 INFO:tasks.workunit.client.1.smithi179.stdout:5/945: dread - d8/d1b/d23/f104 zero size 2024-04-02T04:48:08.045 INFO:tasks.workunit.client.0.smithi070.stdout:7/835: rmdir d7/dd/db5 39 2024-04-02T04:48:08.046 INFO:tasks.workunit.client.1.smithi179.stdout:4/992: sync 2024-04-02T04:48:08.046 INFO:tasks.workunit.client.1.smithi179.stdout:3/996: getdents d8/d26/d83/d89 0 2024-04-02T04:48:08.047 INFO:tasks.workunit.client.0.smithi070.stdout:4/902: dwrite d1/dc3/dc4/f11a [0,4194304] 0 2024-04-02T04:48:08.048 INFO:tasks.workunit.client.1.smithi179.stdout:3/997: stat d8/f6f 0 2024-04-02T04:48:08.051 INFO:tasks.workunit.client.1.smithi179.stdout:8/986: dwrite d3/d32/d21/d50/d56/dad/dbc/f15b [0,4194304] 0 2024-04-02T04:48:08.059 INFO:tasks.workunit.client.1.smithi179.stdout:6/978: symlink d3/d2d/d43/dd7/l159 0 2024-04-02T04:48:08.061 INFO:tasks.workunit.client.0.smithi070.stdout:7/836: mkdir d7/d4e/d82/d57/ddc/d115 0 2024-04-02T04:48:08.064 INFO:tasks.workunit.client.1.smithi179.stdout:2/866: dread dc/d29/d31/d57/f62 [0,4194304] 0 2024-04-02T04:48:08.066 INFO:tasks.workunit.client.0.smithi070.stdout:4/903: write d1/d9c/d65/fca [1861126,25584] 0 2024-04-02T04:48:08.068 INFO:tasks.workunit.client.0.smithi070.stdout:2/931: link d2/d10/d21/d71/dd4/fb6 d2/d10/d5c/d7a/db0/dc7/ded/d12a/f12e 0 2024-04-02T04:48:08.069 INFO:tasks.workunit.client.1.smithi179.stdout:1/976: write d3/d5/d2f/d4f/d42/d55/df3/d154/fb3 [123325,108767] 0 2024-04-02T04:48:08.070 INFO:tasks.workunit.client.0.smithi070.stdout:7/837: symlink d7/dd/df/d7a/d6c/d77/d54/de5/l116 0 2024-04-02T04:48:08.071 INFO:tasks.workunit.client.0.smithi070.stdout:7/838: chown d7/dd/df/d34/fba 227 1 2024-04-02T04:48:08.075 INFO:tasks.workunit.client.0.smithi070.stdout:4/904: chown d1/de7/ce8 25 1 2024-04-02T04:48:08.075 INFO:tasks.workunit.client.0.smithi070.stdout:4/905: chown d1/d3/d78/d91 118293590 1 2024-04-02T04:48:08.078 INFO:tasks.workunit.client.1.smithi179.stdout:9/964: write d0/d1/d11/d77/fd6 [3389887,106668] 0 2024-04-02T04:48:08.081 INFO:tasks.workunit.client.1.smithi179.stdout:8/987: mkdir d3/d32/d21/d22/d6b/d91/da8/d11a/d161 0 2024-04-02T04:48:08.082 INFO:tasks.workunit.client.0.smithi070.stdout:7/839: write d7/dd/df/d7a/d6c/f9b [2577459,108577] 0 2024-04-02T04:48:08.083 INFO:tasks.workunit.client.0.smithi070.stdout:7/840: readlink d7/dd/df/d7a/d6c/d77/l73 0 2024-04-02T04:48:08.085 INFO:tasks.workunit.client.0.smithi070.stdout:2/932: mkdir d2/d12f 0 2024-04-02T04:48:08.086 INFO:tasks.workunit.client.0.smithi070.stdout:4/906: write d1/d3/d85/d8e/fda [637458,25192] 0 2024-04-02T04:48:08.089 INFO:tasks.workunit.client.0.smithi070.stdout:1/792: unlink d0/d9/d1d/d22/d56/d79/d89/c97 0 2024-04-02T04:48:08.092 INFO:tasks.workunit.client.1.smithi179.stdout:7/979: rmdir d1/d9/dd9/d121/d145/d14c 0 2024-04-02T04:48:08.092 INFO:tasks.workunit.client.1.smithi179.stdout:1/977: rename d3/d5/d2f/d4f/d42/d55/l56 to d3/d5/d2f/d4f/d35/d117/l15e 0 2024-04-02T04:48:08.093 INFO:tasks.workunit.client.1.smithi179.stdout:5/946: mkdir d8/d1b/d23/d32/d46/d108/d129 0 2024-04-02T04:48:08.093 INFO:tasks.workunit.client.1.smithi179.stdout:3/998: mkdir d8/d26/d52/d7e/ddf/d13a 0 2024-04-02T04:48:08.094 INFO:tasks.workunit.client.1.smithi179.stdout:7/980: chown d1/d9/d27/d87/d8b/d75/db9/d8d 2 1 2024-04-02T04:48:08.095 INFO:tasks.workunit.client.1.smithi179.stdout:8/988: creat d3/d32/d21/d50/d56/d94/d84/f162 x:0 0 0 2024-04-02T04:48:08.101 INFO:tasks.workunit.client.1.smithi179.stdout:8/989: dwrite d3/d32/d21/d50/d56/dad/dbc/f15b [0,4194304] 0 2024-04-02T04:48:08.104 INFO:tasks.workunit.client.1.smithi179.stdout:9/965: write d0/d19/d1c/f47 [1512658,84700] 0 2024-04-02T04:48:08.118 INFO:tasks.workunit.client.1.smithi179.stdout:3/999: fsync d8/d26/d36/f59 0 2024-04-02T04:48:08.118 INFO:tasks.workunit.client.1.smithi179.stdout:7/981: stat d1/d9/ffe 0 2024-04-02T04:48:08.120 INFO:tasks.workunit.client.1.smithi179.stdout:5/947: rmdir d8/d1b/d23/d32/dbe/daf/db7/dad/df1 39 2024-04-02T04:48:08.120 INFO:tasks.workunit.client.1.smithi179.stdout:7/982: write d1/d9/d27/d87/d8b/d75/db9/ddd/f15c [147151,60797] 0 2024-04-02T04:48:08.121 INFO:tasks.workunit.client.1.smithi179.stdout:7/983: chown d1/d9/dd9/d121/d6b/f105 552963 1 2024-04-02T04:48:08.121 INFO:tasks.workunit.client.1.smithi179.stdout:2/867: creat dc/d29/d4a/f125 x:0 0 0 2024-04-02T04:48:08.122 INFO:tasks.workunit.client.1.smithi179.stdout:2/868: chown dc/d29/d4a/d1e/fec 20 1 2024-04-02T04:48:08.124 INFO:tasks.workunit.client.1.smithi179.stdout:4/993: getdents d1/d3/df/d10/d2f/d60 0 2024-04-02T04:48:08.125 INFO:tasks.workunit.client.1.smithi179.stdout:4/994: stat d1/d3/df/d10/d2f/d60 0 2024-04-02T04:48:08.132 INFO:tasks.workunit.client.1.smithi179.stdout:2/869: mkdir dc/d29/d4a/d4b/d126 0 2024-04-02T04:48:08.132 INFO:tasks.workunit.client.0.smithi070.stdout:4/907: dwrite d1/d9c/f48 [0,4194304] 0 2024-04-02T04:48:08.133 INFO:tasks.workunit.client.0.smithi070.stdout:7/841: creat d7/d8e/dec/f117 x:0 0 0 2024-04-02T04:48:08.135 INFO:tasks.workunit.client.0.smithi070.stdout:2/933: symlink d2/d10/d21/d71/dd4/l130 0 2024-04-02T04:48:08.137 INFO:tasks.workunit.client.0.smithi070.stdout:1/793: read - d0/d4c/da2/fdf zero size 2024-04-02T04:48:08.140 INFO:tasks.workunit.client.1.smithi179.stdout:5/948: fsync d8/d1b/d23/d32/dbe/daf/db7/dad/df1/fc1 0 2024-04-02T04:48:08.141 INFO:tasks.workunit.client.0.smithi070.stdout:7/842: fdatasync d7/dd/f38 0 2024-04-02T04:48:08.146 INFO:tasks.workunit.client.1.smithi179.stdout:8/990: rename d3/d32/d21/d45/f10f to d3/d31/f163 0 2024-04-02T04:48:08.146 INFO:tasks.workunit.client.1.smithi179.stdout:6/979: sync 2024-04-02T04:48:08.148 INFO:tasks.workunit.client.0.smithi070.stdout:4/908: dread d1/dce/fd1 [0,4194304] 0 2024-04-02T04:48:08.148 INFO:tasks.workunit.client.0.smithi070.stdout:4/909: stat d1/d3/c3a 0 2024-04-02T04:48:08.150 INFO:tasks.workunit.client.1.smithi179.stdout:4/995: link d1/d3/df/d10/d3a/f11f d1/d93/d101/f150 0 2024-04-02T04:48:08.150 INFO:tasks.workunit.client.0.smithi070.stdout:7/843: dread - d7/d4e/d82/d57/fc2 zero size 2024-04-02T04:48:08.154 INFO:tasks.workunit.client.1.smithi179.stdout:5/949: symlink d8/d1b/d23/d32/d4e/ddd/de5/l12a 0 2024-04-02T04:48:08.156 INFO:tasks.workunit.client.0.smithi070.stdout:1/794: creat d0/d9/d94/dec/ff5 x:0 0 0 2024-04-02T04:48:08.156 INFO:tasks.workunit.client.0.smithi070.stdout:7/844: dwrite d7/dd/df/d34/fba [0,4194304] 0 2024-04-02T04:48:08.164 INFO:tasks.workunit.client.1.smithi179.stdout:2/870: mknod dc/d29/d4a/c127 0 2024-04-02T04:48:08.164 INFO:tasks.workunit.client.0.smithi070.stdout:7/845: dwrite d7/dd/df/d7a/f64 [4194304,4194304] 0 2024-04-02T04:48:08.169 INFO:tasks.workunit.client.1.smithi179.stdout:9/966: truncate d0/d1/d10/d6d/f25 3537414 0 2024-04-02T04:48:08.170 INFO:tasks.workunit.client.1.smithi179.stdout:5/950: mknod d8/d1b/d23/d32/d46/d108/c12b 0 2024-04-02T04:48:08.172 INFO:tasks.workunit.client.0.smithi070.stdout:7/846: rmdir d7/dd/df/d7a/d6c/d77/d54/de5 39 2024-04-02T04:48:08.175 INFO:tasks.workunit.client.1.smithi179.stdout:5/951: read d8/d1b/d23/d32/d4e/fb8 [1966791,86385] 0 2024-04-02T04:48:08.175 INFO:tasks.workunit.client.1.smithi179.stdout:7/984: getdents d1/d9/d27/d87/d8b/d75/db9 0 2024-04-02T04:48:08.176 INFO:tasks.workunit.client.1.smithi179.stdout:1/978: dwrite d3/d5/d2f/d4f/d35/d13f/f153 [0,4194304] 0 2024-04-02T04:48:08.176 INFO:tasks.workunit.client.0.smithi070.stdout:4/910: link d1/c9a d1/d9c/c131 0 2024-04-02T04:48:08.180 INFO:tasks.workunit.client.0.smithi070.stdout:1/795: dread d0/d9/d1d/f38 [0,4194304] 0 2024-04-02T04:48:08.182 INFO:tasks.workunit.client.1.smithi179.stdout:2/871: write dc/dcb/f11a [265726,27073] 0 2024-04-02T04:48:08.182 INFO:tasks.workunit.client.1.smithi179.stdout:2/872: readlink dc/d29/d31/d57/dea/led 0 2024-04-02T04:48:08.183 INFO:tasks.workunit.client.1.smithi179.stdout:6/980: dread d3/d2d/fb6 [0,4194304] 0 2024-04-02T04:48:08.184 INFO:tasks.workunit.client.1.smithi179.stdout:2/873: readlink dc/d29/d4a/d1e/d6c/dd2/l10d 0 2024-04-02T04:48:08.185 INFO:tasks.workunit.client.0.smithi070.stdout:2/934: write d2/d10/d21/d71/dd4/fb6 [566499,62404] 0 2024-04-02T04:48:08.189 INFO:tasks.workunit.client.1.smithi179.stdout:6/981: dwrite d3/d12/d44/d10a/d10e/f12a [0,4194304] 0 2024-04-02T04:48:08.192 INFO:tasks.workunit.client.1.smithi179.stdout:9/967: rename d0/d19/d27/d7a/lc1 to d0/d1/d11/d29/d86/db0/l130 0 2024-04-02T04:48:08.193 INFO:tasks.workunit.client.0.smithi070.stdout:2/935: mknod d2/d10/d53/c131 0 2024-04-02T04:48:08.199 INFO:tasks.workunit.client.0.smithi070.stdout:1/796: creat d0/d7/d1a/d95/ff6 x:0 0 0 2024-04-02T04:48:08.201 INFO:tasks.workunit.client.0.smithi070.stdout:4/911: write d1/d3/d9/d5e/f87 [209795,76701] 0 2024-04-02T04:48:08.204 INFO:tasks.workunit.client.0.smithi070.stdout:1/797: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/lf7 0 2024-04-02T04:48:08.206 INFO:tasks.workunit.client.0.smithi070.stdout:7/847: dwrite d7/d4e/d82/d57/fc2 [0,4194304] 0 2024-04-02T04:48:08.206 INFO:tasks.workunit.client.1.smithi179.stdout:8/991: creat d3/f164 x:0 0 0 2024-04-02T04:48:08.207 INFO:tasks.workunit.client.0.smithi070.stdout:7/848: dread - d7/d36/f105 zero size 2024-04-02T04:48:08.210 INFO:tasks.workunit.client.1.smithi179.stdout:7/985: write d1/d9/d55/f9e [3777995,13141] 0 2024-04-02T04:48:08.212 INFO:tasks.workunit.client.1.smithi179.stdout:4/996: creat d1/d3/dc/f151 x:0 0 0 2024-04-02T04:48:08.212 INFO:tasks.workunit.client.0.smithi070.stdout:4/912: read d1/d3/d9/f9e [2830761,58823] 0 2024-04-02T04:48:08.213 INFO:tasks.workunit.client.1.smithi179.stdout:7/986: write d1/dd/d76/d13c/f15e [242506,130201] 0 2024-04-02T04:48:08.213 INFO:tasks.workunit.client.1.smithi179.stdout:8/992: dwrite d3/d32/d21/d22/d6b/d91/da8/dfe/f100 [0,4194304] 0 2024-04-02T04:48:08.216 INFO:tasks.workunit.client.1.smithi179.stdout:2/874: symlink dc/dcb/l128 0 2024-04-02T04:48:08.218 INFO:tasks.workunit.client.0.smithi070.stdout:2/936: truncate d2/d10/d21/d71/dd4/fb6 729689 0 2024-04-02T04:48:08.220 INFO:tasks.workunit.client.0.smithi070.stdout:7/849: symlink d7/dd/df/dca/l118 0 2024-04-02T04:48:08.222 INFO:tasks.workunit.client.1.smithi179.stdout:9/968: rmdir d0/d19/d27/d7a/d7e/db9 39 2024-04-02T04:48:08.223 INFO:tasks.workunit.client.1.smithi179.stdout:5/952: dwrite d8/d1b/d23/d32/f7a [0,4194304] 0 2024-04-02T04:48:08.224 INFO:tasks.workunit.client.0.smithi070.stdout:2/937: dwrite d2/d10/d5c/f81 [0,4194304] 0 2024-04-02T04:48:08.226 INFO:tasks.workunit.client.0.smithi070.stdout:4/913: rename d1/dc3/ff6 to d1/d9c/f132 0 2024-04-02T04:48:08.226 INFO:tasks.workunit.client.0.smithi070.stdout:4/914: readlink d1/l6c 0 2024-04-02T04:48:08.227 INFO:tasks.workunit.client.1.smithi179.stdout:2/875: dwrite dc/d29/fa3 [0,4194304] 0 2024-04-02T04:48:08.228 INFO:tasks.workunit.client.0.smithi070.stdout:1/798: truncate d0/d7/d1a/fda 981081 0 2024-04-02T04:48:08.236 INFO:tasks.workunit.client.1.smithi179.stdout:1/979: fdatasync d3/d5/d2f/d36/f67 0 2024-04-02T04:48:08.236 INFO:tasks.workunit.client.1.smithi179.stdout:2/876: dwrite f2 [0,4194304] 0 2024-04-02T04:48:08.239 INFO:tasks.workunit.client.1.smithi179.stdout:2/877: write dc/d29/d31/d57/d97/fc2 [6327945,119799] 0 2024-04-02T04:48:08.240 INFO:tasks.workunit.client.1.smithi179.stdout:2/878: fdatasync dc/d29/d31/d57/dea/f11b 0 2024-04-02T04:48:08.241 INFO:tasks.workunit.client.1.smithi179.stdout:2/879: chown dc/d29/d4a/d1e/f7b 0 1 2024-04-02T04:48:08.241 INFO:tasks.workunit.client.1.smithi179.stdout:4/997: read d1/d3/fa [1700204,67081] 0 2024-04-02T04:48:08.242 INFO:tasks.workunit.client.0.smithi070.stdout:1/799: creat d0/d7/d1a/db9/ff8 x:0 0 0 2024-04-02T04:48:08.243 INFO:tasks.workunit.client.0.smithi070.stdout:1/800: chown d0/d9/d1d/c36 0 1 2024-04-02T04:48:08.243 INFO:tasks.workunit.client.1.smithi179.stdout:7/987: fsync d1/d9/d27/d87/d8b/d75/db9/f147 0 2024-04-02T04:48:08.244 INFO:tasks.workunit.client.0.smithi070.stdout:1/801: fsync d0/d54/d64/d6d/fe5 0 2024-04-02T04:48:08.245 INFO:tasks.workunit.client.0.smithi070.stdout:2/938: creat d2/d10/d5c/d7a/db0/dc7/ded/d12a/f132 x:0 0 0 2024-04-02T04:48:08.246 INFO:tasks.workunit.client.0.smithi070.stdout:7/850: write d7/dd/df/d79/f48 [4831904,33941] 0 2024-04-02T04:48:08.248 INFO:tasks.workunit.client.1.smithi179.stdout:6/982: truncate d3/d12/d6a/f75 625654 0 2024-04-02T04:48:08.249 INFO:tasks.workunit.client.0.smithi070.stdout:7/851: write d7/dd/df/d7a/d6c/d77/d75/d80/f113 [255417,91338] 0 2024-04-02T04:48:08.250 INFO:tasks.workunit.client.0.smithi070.stdout:4/915: unlink d1/c72 0 2024-04-02T04:48:08.251 INFO:tasks.workunit.client.1.smithi179.stdout:9/969: fdatasync d0/d1/f44 0 2024-04-02T04:48:08.252 INFO:tasks.workunit.client.1.smithi179.stdout:9/970: chown d0/d1/d10/d6d/d58/f75 193467 1 2024-04-02T04:48:08.253 INFO:tasks.workunit.client.1.smithi179.stdout:5/953: rename d8/d1b/d23/d36/d125 to d8/d1b/d23/d32/dbe/daf/d12c 0 2024-04-02T04:48:08.254 INFO:tasks.workunit.client.1.smithi179.stdout:5/954: dread d8/d1b/d23/d32/f99 [4194304,4194304] 0 2024-04-02T04:48:08.261 INFO:tasks.workunit.client.1.smithi179.stdout:1/980: truncate d3/d5/d96/f11e 315939 0 2024-04-02T04:48:08.267 INFO:tasks.workunit.client.1.smithi179.stdout:2/880: fsync dc/d29/d31/d57/fa0 0 2024-04-02T04:48:08.269 INFO:tasks.workunit.client.1.smithi179.stdout:4/998: mkdir d1/d15/d7c/d145/d152 0 2024-04-02T04:48:08.273 INFO:tasks.workunit.client.0.smithi070.stdout:1/802: dwrite d0/d9/d1d/f13 [0,4194304] 0 2024-04-02T04:48:08.273 INFO:tasks.workunit.client.0.smithi070.stdout:7/852: rename d7/d9 to d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119 0 2024-04-02T04:48:08.276 INFO:tasks.workunit.client.0.smithi070.stdout:4/916: write d1/f2d [471445,18572] 0 2024-04-02T04:48:08.276 INFO:tasks.workunit.client.1.smithi179.stdout:6/983: truncate d3/d12/d23/d26/d57/f8a 5558068 0 2024-04-02T04:48:08.277 INFO:tasks.workunit.client.1.smithi179.stdout:8/993: write d3/d32/d21/d49/fda [1210624,72648] 0 2024-04-02T04:48:08.277 INFO:tasks.workunit.client.0.smithi070.stdout:4/917: write d1/d3/d85/deb/f116 [2753210,29174] 0 2024-04-02T04:48:08.278 INFO:tasks.workunit.client.1.smithi179.stdout:8/994: read - d3/d32/d21/d49/f153 zero size 2024-04-02T04:48:08.279 INFO:tasks.workunit.client.0.smithi070.stdout:4/918: chown d1/d3/d85/ddb/l11e 840776223 1 2024-04-02T04:48:08.279 INFO:tasks.workunit.client.1.smithi179.stdout:9/971: stat d0/d1/d10/d6d/l30 0 2024-04-02T04:48:08.281 INFO:tasks.workunit.client.0.smithi070.stdout:1/803: mkdir d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/df9 0 2024-04-02T04:48:08.282 INFO:tasks.workunit.client.0.smithi070.stdout:2/939: getdents d2/d10/d5c/d7a 0 2024-04-02T04:48:08.283 INFO:tasks.workunit.client.1.smithi179.stdout:2/881: dread dc/d29/d4a/d1e/f7b [0,4194304] 0 2024-04-02T04:48:08.284 INFO:tasks.workunit.client.0.smithi070.stdout:7/853: symlink d7/dd/df/d7a/d6c/d77/l11a 0 2024-04-02T04:48:08.286 INFO:tasks.workunit.client.1.smithi179.stdout:4/999: write d1/d15/d7c/d98/d14b/d125/fc8 [242099,44953] 0 2024-04-02T04:48:08.287 INFO:tasks.workunit.client.0.smithi070.stdout:2/940: mknod d2/d10/d19/dd7/c133 0 2024-04-02T04:48:08.294 INFO:tasks.workunit.client.0.smithi070.stdout:2/941: creat d2/d10/d5c/d7a/db0/dd8/dd2/d90/f134 x:0 0 0 2024-04-02T04:48:08.296 INFO:tasks.workunit.client.1.smithi179.stdout:5/955: symlink d8/d1b/d23/l12d 0 2024-04-02T04:48:08.298 INFO:tasks.workunit.client.0.smithi070.stdout:7/854: write d7/dd/db1/fcd [744001,78913] 0 2024-04-02T04:48:08.300 INFO:tasks.workunit.client.0.smithi070.stdout:1/804: getdents d0/d7/d1a/d95 0 2024-04-02T04:48:08.302 INFO:tasks.workunit.client.1.smithi179.stdout:2/882: creat dc/d29/d4a/d1e/d43/f129 x:0 0 0 2024-04-02T04:48:08.303 INFO:tasks.workunit.client.0.smithi070.stdout:2/942: dwrite d2/d10/d21/d3d/f9d [0,4194304] 0 2024-04-02T04:48:08.303 INFO:tasks.workunit.client.0.smithi070.stdout:4/919: dread d1/d9c/d65/d7a/fa9 [0,4194304] 0 2024-04-02T04:48:08.304 INFO:tasks.workunit.client.0.smithi070.stdout:1/805: chown d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/lcd 22432936 1 2024-04-02T04:48:08.304 INFO:tasks.workunit.client.1.smithi179.stdout:7/988: creat d1/dd/f161 x:0 0 0 2024-04-02T04:48:08.306 INFO:tasks.workunit.client.1.smithi179.stdout:6/984: creat d3/d12/d44/df1/f15a x:0 0 0 2024-04-02T04:48:08.307 INFO:tasks.workunit.client.0.smithi070.stdout:1/806: write d0/d7/d28/f74 [290133,23126] 0 2024-04-02T04:48:08.307 INFO:tasks.workunit.client.1.smithi179.stdout:2/883: dwrite dc/f52 [0,4194304] 0 2024-04-02T04:48:08.308 INFO:tasks.workunit.client.1.smithi179.stdout:6/985: chown d3/f18 1956033 1 2024-04-02T04:48:08.311 INFO:tasks.workunit.client.1.smithi179.stdout:8/995: symlink d3/d31/dc3/d11b/d141/l165 0 2024-04-02T04:48:08.313 INFO:tasks.workunit.client.1.smithi179.stdout:1/981: creat d3/d5/d2f/d4f/d42/d53/f15f x:0 0 0 2024-04-02T04:48:08.313 INFO:tasks.workunit.client.0.smithi070.stdout:7/855: creat d7/dd/df/d7a/d6c/d77/d54/f11b x:0 0 0 2024-04-02T04:48:08.314 INFO:tasks.workunit.client.0.smithi070.stdout:7/856: write d7/dd/df/dca/f100 [1859376,58374] 0 2024-04-02T04:48:08.315 INFO:tasks.workunit.client.0.smithi070.stdout:3/826: sync 2024-04-02T04:48:08.315 INFO:tasks.workunit.client.0.smithi070.stdout:0/946: sync 2024-04-02T04:48:08.315 INFO:tasks.workunit.client.0.smithi070.stdout:9/824: sync 2024-04-02T04:48:08.315 INFO:tasks.workunit.client.0.smithi070.stdout:6/761: sync 2024-04-02T04:48:08.315 INFO:tasks.workunit.client.0.smithi070.stdout:8/836: sync 2024-04-02T04:48:08.318 INFO:tasks.workunit.client.0.smithi070.stdout:6/762: chown d0/d55/d8b/fe0 674198 1 2024-04-02T04:48:08.319 INFO:tasks.workunit.client.1.smithi179.stdout:7/989: stat d1/dd/d76/d138/da8/dda/d14b/d4d/la6 0 2024-04-02T04:48:08.320 INFO:tasks.workunit.client.0.smithi070.stdout:0/947: write d4/d5/d13/d2a/d3a/f101 [371318,120926] 0 2024-04-02T04:48:08.322 INFO:tasks.workunit.client.0.smithi070.stdout:0/948: readlink d4/d5/d13/d2a/d3a/d80/db6/lf5 0 2024-04-02T04:48:08.322 INFO:tasks.workunit.client.1.smithi179.stdout:9/972: creat d0/d1/d11/f131 x:0 0 0 2024-04-02T04:48:08.325 INFO:tasks.workunit.client.0.smithi070.stdout:2/943: truncate d2/d10/d21/fc2 2232962 0 2024-04-02T04:48:08.327 INFO:tasks.workunit.client.1.smithi179.stdout:2/884: creat dc/d29/d31/d83/dae/f12a x:0 0 0 2024-04-02T04:48:08.327 INFO:tasks.workunit.client.0.smithi070.stdout:2/944: read d2/d10/d53/f118 [595008,39096] 0 2024-04-02T04:48:08.328 INFO:tasks.workunit.client.1.smithi179.stdout:5/956: dwrite d8/f9e [0,4194304] 0 2024-04-02T04:48:08.329 INFO:tasks.workunit.client.0.smithi070.stdout:1/807: creat d0/d9/d1d/d22/d2c/de3/ffa x:0 0 0 2024-04-02T04:48:08.329 INFO:tasks.workunit.client.0.smithi070.stdout:4/920: dwrite d1/d3/d78/dad/dbd/fc9 [0,4194304] 0 2024-04-02T04:48:08.331 INFO:tasks.workunit.client.1.smithi179.stdout:8/996: rmdir d3/d32/d21/d50/d56/d94 39 2024-04-02T04:48:08.334 INFO:tasks.workunit.client.0.smithi070.stdout:9/825: mknod d4/d13/d89/df5/c110 0 2024-04-02T04:48:08.335 INFO:tasks.workunit.client.1.smithi179.stdout:1/982: symlink d3/d5/d2f/d4f/d35/d13f/l160 0 2024-04-02T04:48:08.336 INFO:tasks.workunit.client.0.smithi070.stdout:4/921: dread d1/d9c/f48 [0,4194304] 0 2024-04-02T04:48:08.337 INFO:tasks.workunit.client.0.smithi070.stdout:4/922: fdatasync d1/f2d 0 2024-04-02T04:48:08.340 INFO:tasks.workunit.client.0.smithi070.stdout:6/763: unlink d0/d3/f3a 0 2024-04-02T04:48:08.340 INFO:tasks.workunit.client.0.smithi070.stdout:1/808: dwrite d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/dea/ff0 [0,4194304] 0 2024-04-02T04:48:08.345 INFO:tasks.workunit.client.0.smithi070.stdout:8/837: write d2/d3/d35/fb3 [829236,59979] 0 2024-04-02T04:48:08.345 INFO:tasks.workunit.client.0.smithi070.stdout:7/857: chown d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/c45 15903 1 2024-04-02T04:48:08.346 INFO:tasks.workunit.client.1.smithi179.stdout:6/986: symlink d3/d2d/da5/d14c/l15b 0 2024-04-02T04:48:08.347 INFO:tasks.workunit.client.0.smithi070.stdout:7/858: write d7/dd/df/d7a/dbc/fed [363848,37667] 0 2024-04-02T04:48:08.351 INFO:tasks.workunit.client.1.smithi179.stdout:9/973: write d0/d1/f4 [8748379,72256] 0 2024-04-02T04:48:08.353 INFO:tasks.workunit.client.0.smithi070.stdout:1/809: dread d0/d54/d64/fe4 [0,4194304] 0 2024-04-02T04:48:08.355 INFO:tasks.workunit.client.0.smithi070.stdout:1/810: chown d0/d54/d64/d6d/c71 92 1 2024-04-02T04:48:08.355 INFO:tasks.workunit.client.1.smithi179.stdout:5/957: rename d8/d1b/d23/d42/dde/d10c to d8/d1b/d23/d32/d4b/d12e 0 2024-04-02T04:48:08.357 INFO:tasks.workunit.client.0.smithi070.stdout:2/945: chown d2/l3a 632 1 2024-04-02T04:48:08.358 INFO:tasks.workunit.client.1.smithi179.stdout:8/997: fsync d3/d32/d21/d22/d6b/db1/d104/d11d/f12c 0 2024-04-02T04:48:08.359 INFO:tasks.workunit.client.0.smithi070.stdout:2/946: read d2/d12/d59/f8e [544666,71004] 0 2024-04-02T04:48:08.359 INFO:tasks.workunit.client.0.smithi070.stdout:0/949: write d4/d5/d13/d23/f7f [199711,31547] 0 2024-04-02T04:48:08.364 INFO:tasks.workunit.client.1.smithi179.stdout:7/990: mkdir d1/d9/d162 0 2024-04-02T04:48:08.365 INFO:tasks.workunit.client.0.smithi070.stdout:9/826: mknod d4/c111 0 2024-04-02T04:48:08.365 INFO:tasks.workunit.client.1.smithi179.stdout:7/991: write d1/d9/d27/d87/d8b/d75/db9/dcf/f114 [4520740,121075] 0 2024-04-02T04:48:08.366 INFO:tasks.workunit.client.1.smithi179.stdout:1/983: write d3/d5/d2f/d4f/d42/d55/fcf [647579,114654] 0 2024-04-02T04:48:08.370 INFO:tasks.workunit.client.1.smithi179.stdout:1/984: readlink d3/d5/d2f/d65/l98 0 2024-04-02T04:48:08.370 INFO:tasks.workunit.client.1.smithi179.stdout:6/987: fdatasync d3/d12/d151/f132 0 2024-04-02T04:48:08.370 INFO:tasks.workunit.client.1.smithi179.stdout:6/988: write d3/d12/d23/d2f/f3c [5026275,44128] 0 2024-04-02T04:48:08.370 INFO:tasks.workunit.client.1.smithi179.stdout:9/974: truncate d0/d1/ff5 1031894 0 2024-04-02T04:48:08.371 INFO:tasks.workunit.client.1.smithi179.stdout:9/975: dread d0/d19/d27/f10f [0,4194304] 0 2024-04-02T04:48:08.371 INFO:tasks.workunit.client.0.smithi070.stdout:4/923: mknod d1/dc3/c133 0 2024-04-02T04:48:08.372 INFO:tasks.workunit.client.0.smithi070.stdout:2/947: dread d2/d12/d59/f8e [0,4194304] 0 2024-04-02T04:48:08.372 INFO:tasks.workunit.client.0.smithi070.stdout:2/948: dread - d2/d10/d21/d71/fae zero size 2024-04-02T04:48:08.373 INFO:tasks.workunit.client.0.smithi070.stdout:2/949: stat d2/d10/d21/d3d/d42/c5a 0 2024-04-02T04:48:08.375 INFO:tasks.workunit.client.0.smithi070.stdout:6/764: symlink d0/d3/d19/d31/d46/d6b/lea 0 2024-04-02T04:48:08.380 INFO:tasks.workunit.client.0.smithi070.stdout:7/859: mkdir d7/dd/df/d79/de6/d11c 0 2024-04-02T04:48:08.381 INFO:tasks.workunit.client.1.smithi179.stdout:1/985: dread - d3/d5/f104 zero size 2024-04-02T04:48:08.381 INFO:tasks.workunit.client.1.smithi179.stdout:1/986: chown d3/d5/d2f/d4f/dab/dcd 9937 1 2024-04-02T04:48:08.383 INFO:tasks.workunit.client.1.smithi179.stdout:7/992: rename d1/dd/d76/d13c/f15e to d1/d9/dd9/d121/d6b/f163 0 2024-04-02T04:48:08.383 INFO:tasks.workunit.client.1.smithi179.stdout:8/998: write d3/d1b/f9a [4821920,99358] 0 2024-04-02T04:48:08.387 INFO:tasks.workunit.client.1.smithi179.stdout:6/989: dread - d3/d2c/d85/ffd zero size 2024-04-02T04:48:08.387 INFO:tasks.workunit.client.1.smithi179.stdout:6/990: fsync d3/d9a/fe7 0 2024-04-02T04:48:08.388 INFO:tasks.workunit.client.0.smithi070.stdout:4/924: dread - d1/d9c/d65/d7a/fbe zero size 2024-04-02T04:48:08.391 INFO:tasks.workunit.client.0.smithi070.stdout:6/765: rmdir d0/d55/d79 39 2024-04-02T04:48:08.394 INFO:tasks.workunit.client.0.smithi070.stdout:7/860: creat d7/dd/df/d7a/dbc/f11d x:0 0 0 2024-04-02T04:48:08.395 INFO:tasks.workunit.client.0.smithi070.stdout:7/861: chown d7/dd/df/d7a/d6c/d77/fb0 16586 1 2024-04-02T04:48:08.397 INFO:tasks.workunit.client.0.smithi070.stdout:1/811: getdents d0/d9/d1d/d22/d56/df1 0 2024-04-02T04:48:08.398 INFO:tasks.workunit.client.1.smithi179.stdout:7/993: read - d1/dd/d76/ff4 zero size 2024-04-02T04:48:08.399 INFO:tasks.workunit.client.0.smithi070.stdout:3/827: link d10/d32/d3f/f4d d10/d32/d4b/f115 0 2024-04-02T04:48:08.401 INFO:tasks.workunit.client.0.smithi070.stdout:2/950: dread d2/d10/d19/f1e [0,4194304] 0 2024-04-02T04:48:08.401 INFO:tasks.workunit.client.1.smithi179.stdout:5/958: write d8/d1b/d23/f47 [4547023,73214] 0 2024-04-02T04:48:08.402 INFO:tasks.workunit.client.0.smithi070.stdout:4/925: mknod d1/d3/d85/deb/d110/c134 0 2024-04-02T04:48:08.404 INFO:tasks.workunit.client.1.smithi179.stdout:2/885: getdents dc/d29/d31 0 2024-04-02T04:48:08.405 INFO:tasks.workunit.client.0.smithi070.stdout:6/766: truncate d0/d3/d19/d31/d46/f93 2258576 0 2024-04-02T04:48:08.407 INFO:tasks.workunit.client.0.smithi070.stdout:2/951: dwrite d2/fba [0,4194304] 0 2024-04-02T04:48:08.408 INFO:tasks.workunit.client.0.smithi070.stdout:1/812: creat d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/ffb x:0 0 0 2024-04-02T04:48:08.409 INFO:tasks.workunit.client.0.smithi070.stdout:8/838: dwrite d2/d8/d23/d24/d5f/fb0 [0,4194304] 0 2024-04-02T04:48:08.409 INFO:tasks.workunit.client.0.smithi070.stdout:9/827: rename d4/d10/d11/l75 to d4/d10/d12/d55/l112 0 2024-04-02T04:48:08.411 INFO:tasks.workunit.client.1.smithi179.stdout:7/994: mkdir d1/dd/d76/d138/da8/dda/d14b/d4d/d164 0 2024-04-02T04:48:08.411 INFO:tasks.workunit.client.1.smithi179.stdout:7/995: readlink d1/d9/d55/l66 0 2024-04-02T04:48:08.413 INFO:tasks.workunit.client.0.smithi070.stdout:8/839: write d2/d8/d29/d22/f108 [880002,56991] 0 2024-04-02T04:48:08.414 INFO:tasks.workunit.client.1.smithi179.stdout:8/999: mknod d3/d1b/d2a/c166 0 2024-04-02T04:48:08.415 INFO:tasks.workunit.client.0.smithi070.stdout:8/840: chown d2/d8/d29/d72/le0 164 1 2024-04-02T04:48:08.415 INFO:tasks.workunit.client.1.smithi179.stdout:5/959: symlink d8/d1b/d23/d11f/l12f 0 2024-04-02T04:48:08.416 INFO:tasks.workunit.client.0.smithi070.stdout:7/862: mknod d7/c11e 0 2024-04-02T04:48:08.418 INFO:tasks.workunit.client.0.smithi070.stdout:6/767: rmdir d0/d43/db3 39 2024-04-02T04:48:08.419 INFO:tasks.workunit.client.1.smithi179.stdout:6/991: mkdir d3/d12/d23/d39/d4e/d15c 0 2024-04-02T04:48:08.421 INFO:tasks.workunit.client.1.smithi179.stdout:6/992: chown d3/d12/d44/d10a/f119 92 1 2024-04-02T04:48:08.429 INFO:tasks.workunit.client.1.smithi179.stdout:9/976: creat d0/d1/d10/de3/f132 x:0 0 0 2024-04-02T04:48:08.430 INFO:tasks.workunit.client.0.smithi070.stdout:9/828: symlink d4/d10/d11/d7b/d19/d2f/d39/d57/l113 0 2024-04-02T04:48:08.431 INFO:tasks.workunit.client.1.smithi179.stdout:6/993: symlink d3/d12/d23/d39/d80/d93/dd0/l15d 0 2024-04-02T04:48:08.432 INFO:tasks.workunit.client.1.smithi179.stdout:2/886: write dc/f50 [4144807,50468] 0 2024-04-02T04:48:08.434 INFO:tasks.workunit.client.0.smithi070.stdout:8/841: mkdir d2/d8/d29/dc7/df0/d10e 0 2024-04-02T04:48:08.435 INFO:tasks.workunit.client.1.smithi179.stdout:5/960: mkdir d8/d1b/d23/d36/d130 0 2024-04-02T04:48:08.438 INFO:tasks.workunit.client.0.smithi070.stdout:3/828: truncate d10/d17/d51/d73/d9e/fe9 1792629 0 2024-04-02T04:48:08.440 INFO:tasks.workunit.client.1.smithi179.stdout:6/994: mknod d3/d42/dbe/c15e 0 2024-04-02T04:48:08.441 INFO:tasks.workunit.client.1.smithi179.stdout:6/995: chown d3/d12/d23/d2f/f4a 120117202 1 2024-04-02T04:48:08.442 INFO:tasks.workunit.client.0.smithi070.stdout:2/952: dwrite d2/d10/d21/d71/fae [0,4194304] 0 2024-04-02T04:48:08.442 INFO:tasks.workunit.client.1.smithi179.stdout:2/887: symlink dc/d29/d31/d57/d97/dbf/de3/l12b 0 2024-04-02T04:48:08.443 INFO:tasks.workunit.client.0.smithi070.stdout:9/829: truncate d4/d10/d11/f14 1634197 0 2024-04-02T04:48:08.446 INFO:tasks.workunit.client.0.smithi070.stdout:4/926: creat d1/d3/d85/d8e/d99/db1/f135 x:0 0 0 2024-04-02T04:48:08.447 INFO:tasks.workunit.client.0.smithi070.stdout:6/768: dread d0/d3/d19/f7f [0,4194304] 0 2024-04-02T04:48:08.448 INFO:tasks.workunit.client.1.smithi179.stdout:9/977: symlink d0/l133 0 2024-04-02T04:48:08.449 INFO:tasks.workunit.client.0.smithi070.stdout:8/842: rmdir d2/d8 39 2024-04-02T04:48:08.449 INFO:tasks.workunit.client.1.smithi179.stdout:9/978: dread - d0/d1/f11e zero size 2024-04-02T04:48:08.452 INFO:tasks.workunit.client.1.smithi179.stdout:7/996: getdents d1/dd/d76/d138/da8/dda/d14b 0 2024-04-02T04:48:08.453 INFO:tasks.workunit.client.1.smithi179.stdout:6/996: symlink d3/d12/d23/d39/d80/l15f 0 2024-04-02T04:48:08.453 INFO:tasks.workunit.client.1.smithi179.stdout:6/997: readlink d3/d12/d23/d39/d4e/lb9 0 2024-04-02T04:48:08.455 INFO:tasks.workunit.client.1.smithi179.stdout:2/888: mknod dc/d29/d31/d57/dea/c12c 0 2024-04-02T04:48:08.455 INFO:tasks.workunit.client.1.smithi179.stdout:5/961: dread d8/d1b/d23/d32/d4e/d51/f9c [0,4194304] 0 2024-04-02T04:48:08.458 INFO:tasks.workunit.client.0.smithi070.stdout:3/829: fdatasync d10/f22 0 2024-04-02T04:48:08.460 INFO:tasks.workunit.client.1.smithi179.stdout:6/998: dwrite d3/d12/d6a/f140 [0,4194304] 0 2024-04-02T04:48:08.466 INFO:tasks.workunit.client.0.smithi070.stdout:9/830: unlink d4/d13/l90 0 2024-04-02T04:48:08.468 INFO:tasks.workunit.client.1.smithi179.stdout:9/979: rename d0/d1/d11/d29/f50 to d0/d19/d27/d7a/d7e/db9/dfa/f134 0 2024-04-02T04:48:08.470 INFO:tasks.workunit.client.1.smithi179.stdout:2/889: write dc/d29/d31/faf [570295,128893] 0 2024-04-02T04:48:08.470 INFO:tasks.workunit.client.0.smithi070.stdout:4/927: write d1/d3/d85/ddb/ffc [3825463,53133] 0 2024-04-02T04:48:08.474 INFO:tasks.workunit.client.0.smithi070.stdout:6/769: unlink d0/d20/fcf 0 2024-04-02T04:48:08.480 INFO:tasks.workunit.client.1.smithi179.stdout:1/987: sync 2024-04-02T04:48:08.480 INFO:tasks.workunit.client.1.smithi179.stdout:6/999: rename d3/d12/d23/d26/d57/l153 to d3/d2d/da5/d14c/l160 0 2024-04-02T04:48:08.482 INFO:tasks.workunit.client.1.smithi179.stdout:9/980: mkdir d0/d19/d27/d7a/dcb/d135 0 2024-04-02T04:48:08.482 INFO:tasks.workunit.client.1.smithi179.stdout:7/997: dwrite d1/dd/d76/d138/fe7 [0,4194304] 0 2024-04-02T04:48:08.484 INFO:tasks.workunit.client.1.smithi179.stdout:7/998: stat d1/d9/d27/d87/d8b/l159 0 2024-04-02T04:48:08.484 INFO:tasks.workunit.client.0.smithi070.stdout:1/813: getdents d0 0 2024-04-02T04:48:08.485 INFO:tasks.workunit.client.1.smithi179.stdout:7/999: chown d1/d9/d55/f96 821108215 1 2024-04-02T04:48:08.485 INFO:tasks.workunit.client.1.smithi179.stdout:2/890: creat dc/d79/f12d x:0 0 0 2024-04-02T04:48:08.486 INFO:tasks.workunit.client.0.smithi070.stdout:9/831: chown d4/d10/d12/d55/lb0 62 1 2024-04-02T04:48:08.489 INFO:tasks.workunit.client.1.smithi179.stdout:5/962: link d8/d1b/d23/d32/d4e/d51/f9c d8/d1b/d23/d32/d4e/d50/f131 0 2024-04-02T04:48:08.491 INFO:tasks.workunit.client.0.smithi070.stdout:4/928: chown d1/d9c/d65/d7a/ca6 255 1 2024-04-02T04:48:08.494 INFO:tasks.workunit.client.1.smithi179.stdout:2/891: rmdir dc/d29 39 2024-04-02T04:48:08.495 INFO:tasks.workunit.client.0.smithi070.stdout:8/843: creat d2/d8/d29/db/f10f x:0 0 0 2024-04-02T04:48:08.498 INFO:tasks.workunit.client.0.smithi070.stdout:7/863: link d7/dd/df/d7a/d6c/d77/d75/cf0 d7/dd/df/d79/c11f 0 2024-04-02T04:48:08.498 INFO:tasks.workunit.client.1.smithi179.stdout:1/988: write d3/d85/d112/f122 [223874,62662] 0 2024-04-02T04:48:08.501 INFO:tasks.workunit.client.1.smithi179.stdout:2/892: unlink dc/d79/l102 0 2024-04-02T04:48:08.503 INFO:tasks.workunit.client.0.smithi070.stdout:6/770: truncate d0/d20/f23 3889632 0 2024-04-02T04:48:08.504 INFO:tasks.workunit.client.1.smithi179.stdout:1/989: creat d3/d5/d2f/d4f/dab/dcd/f161 x:0 0 0 2024-04-02T04:48:08.505 INFO:tasks.workunit.client.1.smithi179.stdout:9/981: link d0/d19/d27/fe0 d0/d19/d27/d7a/d7e/db9/d108/d129/f136 0 2024-04-02T04:48:08.506 INFO:tasks.workunit.client.0.smithi070.stdout:8/844: read d2/d8/d29/d22/d5a/d94/fc0 [1768536,80396] 0 2024-04-02T04:48:08.507 INFO:tasks.workunit.client.0.smithi070.stdout:3/830: creat d10/d17/db2/dbf/f116 x:0 0 0 2024-04-02T04:48:08.510 INFO:tasks.workunit.client.1.smithi179.stdout:9/982: dwrite d0/d1/d11/f131 [0,4194304] 0 2024-04-02T04:48:08.511 INFO:tasks.workunit.client.0.smithi070.stdout:7/864: creat d7/d36/f120 x:0 0 0 2024-04-02T04:48:08.511 INFO:tasks.workunit.client.0.smithi070.stdout:9/832: write d4/d10/d11/d7b/d19/d53/f59 [1790849,120828] 0 2024-04-02T04:48:08.512 INFO:tasks.workunit.client.0.smithi070.stdout:2/953: link d2/d10/d21/d71/dc1/c119 d2/d10/d5c/d7a/db0/dd8/c135 0 2024-04-02T04:48:08.515 INFO:tasks.workunit.client.1.smithi179.stdout:5/963: link d8/d1b/d23/d32/dbe/daf/dd4/c91 d8/d1b/d23/d42/d5a/c132 0 2024-04-02T04:48:08.518 INFO:tasks.workunit.client.0.smithi070.stdout:9/833: dwrite d4/d10/d12/d55/f7e [4194304,4194304] 0 2024-04-02T04:48:08.520 INFO:tasks.workunit.client.0.smithi070.stdout:9/834: chown d4/d10/d11/d7b/l6a 16465855 1 2024-04-02T04:48:08.521 INFO:tasks.workunit.client.0.smithi070.stdout:9/835: stat d4/d10/d11/d7b/f2b 0 2024-04-02T04:48:08.521 INFO:tasks.workunit.client.0.smithi070.stdout:8/845: readlink d2/d8/l50 0 2024-04-02T04:48:08.525 INFO:tasks.workunit.client.0.smithi070.stdout:7/865: readlink d7/dd/df/d7a/d6c/d77/d54/de5/l108 0 2024-04-02T04:48:08.526 INFO:tasks.workunit.client.0.smithi070.stdout:7/866: truncate d7/d36/fae 8186494 0 2024-04-02T04:48:08.527 INFO:tasks.workunit.client.0.smithi070.stdout:2/954: mkdir d2/d10/d19/dd7/d136 0 2024-04-02T04:48:08.530 INFO:tasks.workunit.client.0.smithi070.stdout:9/836: dread d4/d10/d12/d56/f5b [0,4194304] 0 2024-04-02T04:48:08.530 INFO:tasks.workunit.client.1.smithi179.stdout:1/990: truncate d3/d85/da2/fea 3512649 0 2024-04-02T04:48:08.532 INFO:tasks.workunit.client.1.smithi179.stdout:5/964: creat d8/d1b/d23/d36/f133 x:0 0 0 2024-04-02T04:48:08.533 INFO:tasks.workunit.client.0.smithi070.stdout:3/831: mknod d10/d32/c117 0 2024-04-02T04:48:08.534 INFO:tasks.workunit.client.0.smithi070.stdout:7/867: dwrite d7/d8e/dec/f117 [0,4194304] 0 2024-04-02T04:48:08.535 INFO:tasks.workunit.client.0.smithi070.stdout:3/832: write d10/d32/d4b/dc5/fd0 [672021,105850] 0 2024-04-02T04:48:08.536 INFO:tasks.workunit.client.1.smithi179.stdout:9/983: dwrite d0/d19/d27/d7a/f8e [0,4194304] 0 2024-04-02T04:48:08.538 INFO:tasks.workunit.client.0.smithi070.stdout:3/833: stat d10/d32/d8a/d8d/caf 0 2024-04-02T04:48:08.538 INFO:tasks.workunit.client.0.smithi070.stdout:2/955: stat d2/d10/d5c/d7a/db0/dc7/ded/d12a/f12e 0 2024-04-02T04:48:08.539 INFO:tasks.workunit.client.0.smithi070.stdout:8/846: read d2/d8/d29/d72/fd0 [427764,86582] 0 2024-04-02T04:48:08.540 INFO:tasks.workunit.client.1.smithi179.stdout:1/991: symlink d3/d5/d2f/d4f/d42/d53/l162 0 2024-04-02T04:48:08.540 INFO:tasks.workunit.client.1.smithi179.stdout:5/965: dwrite d8/d1b/d23/d32/dbe/daf/db7/dad/d111/f122 [0,4194304] 0 2024-04-02T04:48:08.542 INFO:tasks.workunit.client.0.smithi070.stdout:6/771: dwrite d0/d55/d8b/d94/fb9 [0,4194304] 0 2024-04-02T04:48:08.542 INFO:tasks.workunit.client.0.smithi070.stdout:9/837: mkdir d4/d34/db9/d114 0 2024-04-02T04:48:08.544 INFO:tasks.workunit.client.0.smithi070.stdout:6/772: fsync d0/d55/d8b/d94/fbd 0 2024-04-02T04:48:08.544 INFO:tasks.workunit.client.1.smithi179.stdout:9/984: mknod d0/d1/d10/d6d/c137 0 2024-04-02T04:48:08.544 INFO:tasks.workunit.client.1.smithi179.stdout:5/966: chown d8/d1b/d23/d32/d46/d108/c12b 41518 1 2024-04-02T04:48:08.550 INFO:tasks.workunit.client.1.smithi179.stdout:9/985: fdatasync d0/d19/d1c/f52 0 2024-04-02T04:48:08.554 INFO:tasks.workunit.client.0.smithi070.stdout:3/834: fsync d10/d17/d51/d73/d8f/fb6 0 2024-04-02T04:48:08.564 INFO:tasks.workunit.client.1.smithi179.stdout:9/986: creat d0/d19/d27/dad/f138 x:0 0 0 2024-04-02T04:48:08.565 INFO:tasks.workunit.client.0.smithi070.stdout:9/838: dread - d4/d10/d11/d7b/d19/d2f/d39/d57/da6/fd2 zero size 2024-04-02T04:48:08.566 INFO:tasks.workunit.client.1.smithi179.stdout:9/987: write d0/d19/d27/d7a/f126 [1685104,106751] 0 2024-04-02T04:48:08.573 INFO:tasks.workunit.client.0.smithi070.stdout:3/835: write d10/d32/d8a/d8d/f92 [3952777,116782] 0 2024-04-02T04:48:08.574 INFO:tasks.workunit.client.1.smithi179.stdout:1/992: write d3/d5/d2f/d4f/d42/df2/f7e [2383930,85002] 0 2024-04-02T04:48:08.574 INFO:tasks.workunit.client.0.smithi070.stdout:7/868: link d7/dd/df/d7a/d6c/d77/d54/ld0 d7/d4e/l121 0 2024-04-02T04:48:08.574 INFO:tasks.workunit.client.1.smithi179.stdout:1/993: chown d3/d5/d2f/d65/cb9 471536 1 2024-04-02T04:48:08.576 INFO:tasks.workunit.client.0.smithi070.stdout:8/847: write d2/d8/d29/fe4 [922085,76878] 0 2024-04-02T04:48:08.577 INFO:tasks.workunit.client.0.smithi070.stdout:3/836: truncate d10/d17/f6c 978331 0 2024-04-02T04:48:08.578 INFO:tasks.workunit.client.1.smithi179.stdout:1/994: creat d3/d5/d2f/d4f/d42/d53/d72/d86/f163 x:0 0 0 2024-04-02T04:48:08.580 INFO:tasks.workunit.client.0.smithi070.stdout:9/839: link d4/d13/d89/fef d4/d34/db9/d114/f115 0 2024-04-02T04:48:08.580 INFO:tasks.workunit.client.0.smithi070.stdout:9/840: stat d4/d10/d12/fb3 0 2024-04-02T04:48:08.580 INFO:tasks.workunit.client.1.smithi179.stdout:2/893: sync 2024-04-02T04:48:08.581 INFO:tasks.workunit.client.1.smithi179.stdout:5/967: truncate d8/d1b/d23/d32/d4e/d50/f68 305305 0 2024-04-02T04:48:08.581 INFO:tasks.workunit.client.0.smithi070.stdout:7/869: mkdir d7/d4e/d122 0 2024-04-02T04:48:08.582 INFO:tasks.workunit.client.0.smithi070.stdout:7/870: chown d7 393171 1 2024-04-02T04:48:08.583 INFO:tasks.workunit.client.0.smithi070.stdout:2/956: link d2/d10/d5c/d7a/db0/dd8/dd2/fe6 d2/d10/d5c/d10f/f137 0 2024-04-02T04:48:08.584 INFO:tasks.workunit.client.0.smithi070.stdout:6/773: write d0/d20/d5f/f62 [3449020,38086] 0 2024-04-02T04:48:08.586 INFO:tasks.workunit.client.1.smithi179.stdout:9/988: dread d0/d19/d27/dad/fae [0,4194304] 0 2024-04-02T04:48:08.590 INFO:tasks.workunit.client.0.smithi070.stdout:9/841: mkdir d4/d10/d11/d7b/d19/d2f/d39/d116 0 2024-04-02T04:48:08.592 INFO:tasks.workunit.client.1.smithi179.stdout:9/989: dwrite d0/d1/f7b [4194304,4194304] 0 2024-04-02T04:48:08.597 INFO:tasks.workunit.client.0.smithi070.stdout:6/774: creat d0/d20/dd8/feb x:0 0 0 2024-04-02T04:48:08.597 INFO:tasks.workunit.client.1.smithi179.stdout:9/990: dread d0/d19/d1c/f52 [0,4194304] 0 2024-04-02T04:48:08.598 INFO:tasks.workunit.client.0.smithi070.stdout:6/775: chown d0/d55/d8b/d94/fbd 214472850 1 2024-04-02T04:48:08.601 INFO:tasks.workunit.client.0.smithi070.stdout:9/842: truncate d4/d34/f7a 1945213 0 2024-04-02T04:48:08.601 INFO:tasks.workunit.client.1.smithi179.stdout:9/991: dread d0/d1/d10/d6d/f2d [0,4194304] 0 2024-04-02T04:48:08.604 INFO:tasks.workunit.client.1.smithi179.stdout:1/995: read d3/d85/da2/fea [2353969,115570] 0 2024-04-02T04:48:08.604 INFO:tasks.workunit.client.0.smithi070.stdout:2/957: mknod d2/d10/d21/d71/dd4/c138 0 2024-04-02T04:48:08.606 INFO:tasks.workunit.client.0.smithi070.stdout:3/837: dwrite d10/d66/fb9 [0,4194304] 0 2024-04-02T04:48:08.606 INFO:tasks.workunit.client.1.smithi179.stdout:5/968: dwrite d8/d1b/d23/d32/f99 [0,4194304] 0 2024-04-02T04:48:08.607 INFO:tasks.workunit.client.1.smithi179.stdout:2/894: write dc/d29/d4a/f48 [1360217,24915] 0 2024-04-02T04:48:08.609 INFO:tasks.workunit.client.0.smithi070.stdout:7/871: rename d7/dd/df/d79/d46 to d7/d8e/d123 0 2024-04-02T04:48:08.610 INFO:tasks.workunit.client.1.smithi179.stdout:9/992: dread d0/d19/d1c/f31 [0,4194304] 0 2024-04-02T04:48:08.611 INFO:tasks.workunit.client.0.smithi070.stdout:7/872: write d7/dd/df/d34/fba [3205401,119348] 0 2024-04-02T04:48:08.613 INFO:tasks.workunit.client.1.smithi179.stdout:1/996: mknod d3/d5/d2f/d4f/d42/d53/c164 0 2024-04-02T04:48:08.619 INFO:tasks.workunit.client.0.smithi070.stdout:3/838: symlink d10/d17/d51/d73/d9e/d107/l118 0 2024-04-02T04:48:08.622 INFO:tasks.workunit.client.1.smithi179.stdout:9/993: rmdir d0/d1/d11 39 2024-04-02T04:48:08.625 INFO:tasks.workunit.client.0.smithi070.stdout:7/873: mknod d7/d4e/c124 0 2024-04-02T04:48:08.629 INFO:tasks.workunit.client.1.smithi179.stdout:5/969: symlink d8/d1b/d23/d36/d53/d62/l134 0 2024-04-02T04:48:08.629 INFO:tasks.workunit.client.0.smithi070.stdout:7/874: dread d7/d4e/d82/d57/fc2 [0,4194304] 0 2024-04-02T04:48:08.630 INFO:tasks.workunit.client.0.smithi070.stdout:7/875: read - d7/dd/df/dca/fd4 zero size 2024-04-02T04:48:08.631 INFO:tasks.workunit.client.0.smithi070.stdout:9/843: creat d4/d10/d12/d27/d60/f117 x:0 0 0 2024-04-02T04:48:08.631 INFO:tasks.workunit.client.1.smithi179.stdout:2/895: dread dc/f44 [0,4194304] 0 2024-04-02T04:48:08.633 INFO:tasks.workunit.client.1.smithi179.stdout:1/997: symlink d3/d5/dff/l165 0 2024-04-02T04:48:08.634 INFO:tasks.workunit.client.1.smithi179.stdout:1/998: chown d3/d5/d2f/d4f/d6a/da0/lf9 2033570474 1 2024-04-02T04:48:08.634 INFO:tasks.workunit.client.0.smithi070.stdout:9/844: dread d4/d10/d12/d27/d60/dc9/fd5 [0,4194304] 0 2024-04-02T04:48:08.635 INFO:tasks.workunit.client.0.smithi070.stdout:2/958: write d2/d10/d21/f34 [1817376,8835] 0 2024-04-02T04:48:08.638 INFO:tasks.workunit.client.0.smithi070.stdout:6/776: getdents d0/d20/d5f/de6 0 2024-04-02T04:48:08.639 INFO:tasks.workunit.client.1.smithi179.stdout:5/970: mkdir d8/d1b/d23/d42/d135 0 2024-04-02T04:48:08.641 INFO:tasks.workunit.client.1.smithi179.stdout:2/896: dread dc/d29/d31/d57/f71 [0,4194304] 0 2024-04-02T04:48:08.643 INFO:tasks.workunit.client.1.smithi179.stdout:5/971: dread d8/d1b/f3b [0,4194304] 0 2024-04-02T04:48:08.643 INFO:tasks.workunit.client.1.smithi179.stdout:1/999: symlink d3/dbe/l166 0 2024-04-02T04:48:08.644 INFO:tasks.workunit.client.0.smithi070.stdout:3/839: mknod d10/d17/c119 0 2024-04-02T04:48:08.644 INFO:tasks.workunit.client.0.smithi070.stdout:3/840: readlink d10/d17/d4c/l9b 0 2024-04-02T04:48:08.647 INFO:tasks.workunit.client.1.smithi179.stdout:5/972: dwrite d8/d1b/d23/d42/d5a/f6b [0,4194304] 0 2024-04-02T04:48:08.648 INFO:tasks.workunit.client.1.smithi179.stdout:9/994: mknod d0/d1/d11/d29/c139 0 2024-04-02T04:48:08.652 INFO:tasks.workunit.client.1.smithi179.stdout:9/995: dread d0/d19/d1c/f31 [0,4194304] 0 2024-04-02T04:48:08.653 INFO:tasks.workunit.client.0.smithi070.stdout:6/777: dread - d0/d20/d77/fbf zero size 2024-04-02T04:48:08.655 INFO:tasks.workunit.client.0.smithi070.stdout:6/778: dread - d0/d20/d77/d98/fe5 zero size 2024-04-02T04:48:08.657 INFO:tasks.workunit.client.0.smithi070.stdout:6/779: chown d0/d20/d5f/fd9 32 1 2024-04-02T04:48:08.658 INFO:tasks.workunit.client.0.smithi070.stdout:3/841: mkdir d10/d32/d4b/dc5/de0/d11a 0 2024-04-02T04:48:08.659 INFO:tasks.workunit.client.1.smithi179.stdout:5/973: truncate d8/d1b/d23/d32/dbe/daf/ff3 784473 0 2024-04-02T04:48:08.661 INFO:tasks.workunit.client.0.smithi070.stdout:9/845: dwrite d4/d34/db9/d114/f115 [0,4194304] 0 2024-04-02T04:48:08.663 INFO:tasks.workunit.client.0.smithi070.stdout:9/846: chown d4/d10/d12/l10f 954011 1 2024-04-02T04:48:08.664 INFO:tasks.workunit.client.0.smithi070.stdout:2/959: dwrite d2/d24/f96 [0,4194304] 0 2024-04-02T04:48:08.665 INFO:tasks.workunit.client.0.smithi070.stdout:9/847: stat d4/d10/d11/d7b 0 2024-04-02T04:48:08.668 INFO:tasks.workunit.client.0.smithi070.stdout:2/960: write d2/d24/f96 [3858262,82861] 0 2024-04-02T04:48:08.669 INFO:tasks.workunit.client.1.smithi179.stdout:9/996: truncate d0/d1/d11/d29/d86/f9e 650867 0 2024-04-02T04:48:08.670 INFO:tasks.workunit.client.0.smithi070.stdout:3/842: mknod d10/d17/d51/d73/d9e/c11b 0 2024-04-02T04:48:08.671 INFO:tasks.workunit.client.0.smithi070.stdout:3/843: write d10/d32/dd1/fdf [4489219,57184] 0 2024-04-02T04:48:08.672 INFO:tasks.workunit.client.1.smithi179.stdout:2/897: link dc/d29/d4a/d1e/d43/c6f dc/d29/d31/d83/dae/c12e 0 2024-04-02T04:48:08.680 INFO:tasks.workunit.client.0.smithi070.stdout:3/844: mkdir d10/d32/d4b/dc5/d11c 0 2024-04-02T04:48:08.680 INFO:tasks.workunit.client.0.smithi070.stdout:6/780: dread d0/d55/d8b/f9e [0,4194304] 0 2024-04-02T04:48:08.681 INFO:tasks.workunit.client.1.smithi179.stdout:5/974: rmdir d8/d1b/d23/d36/d53/d120 0 2024-04-02T04:48:08.682 INFO:tasks.workunit.client.0.smithi070.stdout:6/781: write d0/d20/d5f/fe2 [638966,50082] 0 2024-04-02T04:48:08.683 INFO:tasks.workunit.client.1.smithi179.stdout:2/898: creat dc/d29/d10b/d110/f12f x:0 0 0 2024-04-02T04:48:08.683 INFO:tasks.workunit.client.0.smithi070.stdout:6/782: write d0/d43/f51 [4274779,17590] 0 2024-04-02T04:48:08.685 INFO:tasks.workunit.client.1.smithi179.stdout:5/975: dread - d8/d1b/d23/d36/d53/dd9/ffe zero size 2024-04-02T04:48:08.685 INFO:tasks.workunit.client.0.smithi070.stdout:9/848: symlink d4/d10/d12/d56/d79/l118 0 2024-04-02T04:48:08.686 INFO:tasks.workunit.client.0.smithi070.stdout:2/961: dread d2/d10/d53/f55 [0,4194304] 0 2024-04-02T04:48:08.692 INFO:tasks.workunit.client.0.smithi070.stdout:9/849: symlink d4/d10/d11/d7b/d19/d82/db7/l119 0 2024-04-02T04:48:08.693 INFO:tasks.workunit.client.0.smithi070.stdout:2/962: dwrite d2/d24/f8f [0,4194304] 0 2024-04-02T04:48:08.694 INFO:tasks.workunit.client.0.smithi070.stdout:6/783: creat d0/d3/fec x:0 0 0 2024-04-02T04:48:08.695 INFO:tasks.workunit.client.1.smithi179.stdout:5/976: link d8/d1b/d23/d32/dbe/daf/db7/dad/df1/ca3 d8/d1b/d23/d32/d4e/ddd/de5/c136 0 2024-04-02T04:48:08.698 INFO:tasks.workunit.client.0.smithi070.stdout:9/850: chown d4/d10/d11/d7b/d19/f24 14 1 2024-04-02T04:48:08.700 INFO:tasks.workunit.client.0.smithi070.stdout:9/851: write d4/d34/db9/fe8 [870117,105506] 0 2024-04-02T04:48:08.703 INFO:tasks.workunit.client.0.smithi070.stdout:6/784: fdatasync d0/d43/db3/dc7/f88 0 2024-04-02T04:48:08.705 INFO:tasks.workunit.client.0.smithi070.stdout:3/845: write d10/d32/d8a/d8d/d91/f94 [1985350,58275] 0 2024-04-02T04:48:08.710 INFO:tasks.workunit.client.0.smithi070.stdout:9/852: creat d4/d10/d11/d7b/f11a x:0 0 0 2024-04-02T04:48:08.711 INFO:tasks.workunit.client.1.smithi179.stdout:2/899: dwrite dc/d29/d4a/d1e/fec [0,4194304] 0 2024-04-02T04:48:08.711 INFO:tasks.workunit.client.0.smithi070.stdout:9/853: chown d4/d34/db9/d114/f115 5917 1 2024-04-02T04:48:08.712 INFO:tasks.workunit.client.0.smithi070.stdout:3/846: unlink d10/d17/l20 0 2024-04-02T04:48:08.712 INFO:tasks.workunit.client.0.smithi070.stdout:9/854: fdatasync d4/d10/d11/d7b/d19/d2f/fda 0 2024-04-02T04:48:08.717 INFO:tasks.workunit.client.1.smithi179.stdout:2/900: mkdir dc/d29/d4a/d4b/d130 0 2024-04-02T04:48:08.718 INFO:tasks.workunit.client.0.smithi070.stdout:6/785: dread d0/d20/d5f/de6/da6/fd3 [0,4194304] 0 2024-04-02T04:48:08.723 INFO:tasks.workunit.client.0.smithi070.stdout:9/855: readlink d4/d10/d12/d56/l6e 0 2024-04-02T04:48:08.726 INFO:tasks.workunit.client.0.smithi070.stdout:9/856: write d4/d13/d89/faf [3190289,72322] 0 2024-04-02T04:48:08.727 INFO:tasks.workunit.client.0.smithi070.stdout:2/963: dwrite d2/d10/d21/d71/dd4/fc9 [0,4194304] 0 2024-04-02T04:48:08.728 INFO:tasks.workunit.client.0.smithi070.stdout:9/857: write d4/f8 [2516243,74485] 0 2024-04-02T04:48:08.730 INFO:tasks.workunit.client.0.smithi070.stdout:2/964: dread - d2/d10/d5c/d7a/db0/dd8/dd2/f11d zero size 2024-04-02T04:48:08.731 INFO:tasks.workunit.client.0.smithi070.stdout:2/965: write d2/fba [2013036,103456] 0 2024-04-02T04:48:08.774 INFO:tasks.workunit.client.1.smithi179.stdout:9/997: sync 2024-04-02T04:48:08.815 INFO:tasks.workunit.client.1.smithi179.stdout:5/977: sync 2024-04-02T04:48:08.817 INFO:tasks.workunit.client.0.smithi070.stdout:3/847: write d10/d32/d4b/fdc [959528,47385] 0 2024-04-02T04:48:08.818 INFO:tasks.workunit.client.1.smithi179.stdout:9/998: unlink d0/d19/d27/d7a/d84/cb5 0 2024-04-02T04:48:08.820 INFO:tasks.workunit.client.1.smithi179.stdout:5/978: rmdir d8/d1b/d23/d42/dde 39 2024-04-02T04:48:08.822 INFO:tasks.workunit.client.1.smithi179.stdout:2/901: dwrite dc/d29/d31/f41 [0,4194304] 0 2024-04-02T04:48:08.824 INFO:tasks.workunit.client.0.smithi070.stdout:6/786: dwrite d0/d43/db3/dc7/f88 [8388608,4194304] 0 2024-04-02T04:48:08.825 INFO:tasks.workunit.client.0.smithi070.stdout:6/787: chown d0/d20/d77/fbf 4 1 2024-04-02T04:48:08.873 INFO:tasks.workunit.client.1.smithi179.stdout:5/979: dread - d8/fc9 zero size 2024-04-02T04:48:08.873 INFO:tasks.workunit.client.0.smithi070.stdout:2/966: dread - d2/d10/d5c/d7a/db0/dd8/dd2/f115 zero size 2024-04-02T04:48:08.875 INFO:tasks.workunit.client.1.smithi179.stdout:2/902: mkdir dc/d29/d31/d83/d131 0 2024-04-02T04:48:08.878 INFO:tasks.workunit.client.1.smithi179.stdout:9/999: symlink d0/d19/d27/d7a/l13a 0 2024-04-02T04:48:08.879 INFO:tasks.workunit.client.0.smithi070.stdout:3/848: creat d10/d66/f11d x:0 0 0 2024-04-02T04:48:08.881 INFO:tasks.workunit.client.0.smithi070.stdout:9/858: symlink d4/d10/d12/d27/d83/l11b 0 2024-04-02T04:48:08.886 INFO:tasks.workunit.client.0.smithi070.stdout:9/859: rmdir d4/d13 39 2024-04-02T04:48:08.889 INFO:tasks.workunit.client.1.smithi179.stdout:2/903: dwrite dc/d29/f34 [4194304,4194304] 0 2024-04-02T04:48:08.889 INFO:tasks.workunit.client.0.smithi070.stdout:6/788: rmdir d0/d20/d5f/de7 0 2024-04-02T04:48:08.891 INFO:tasks.workunit.client.0.smithi070.stdout:3/849: mkdir d10/d17/d4c/d11e 0 2024-04-02T04:48:08.891 INFO:tasks.workunit.client.0.smithi070.stdout:2/967: dread d2/d10/d5c/d7a/fb7 [0,4194304] 0 2024-04-02T04:48:08.892 INFO:tasks.workunit.client.0.smithi070.stdout:9/860: creat d4/d10/d12/d56/d79/dd8/ddf/f11c x:0 0 0 2024-04-02T04:48:08.893 INFO:tasks.workunit.client.0.smithi070.stdout:6/789: mkdir d0/d43/d80/ded 0 2024-04-02T04:48:08.894 INFO:tasks.workunit.client.0.smithi070.stdout:6/790: chown d0/d3/d19/d31/f4e 257847 1 2024-04-02T04:48:08.895 INFO:tasks.workunit.client.0.smithi070.stdout:6/791: fdatasync d0/f78 0 2024-04-02T04:48:08.897 INFO:tasks.workunit.client.1.smithi179.stdout:2/904: link dc/fe dc/d29/d4a/d1e/d6c/dd2/d121/f132 0 2024-04-02T04:48:08.903 INFO:tasks.workunit.client.0.smithi070.stdout:2/968: rename d2/d10/d5c/d7a/db0/dd8/dd2/d90/leb to d2/d10/d5c/d7a/db0/dd8/l139 0 2024-04-02T04:48:08.905 INFO:tasks.workunit.client.0.smithi070.stdout:6/792: dread d0/d3/d19/d31/d46/f9b [0,4194304] 0 2024-04-02T04:48:08.906 INFO:tasks.workunit.client.1.smithi179.stdout:2/905: dwrite dc/d29/d4a/d1e/d43/f6e [0,4194304] 0 2024-04-02T04:48:08.907 INFO:tasks.workunit.client.0.smithi070.stdout:6/793: rename d0/d43/c44 to d0/d20/dd8/cee 0 2024-04-02T04:48:08.908 INFO:tasks.workunit.client.0.smithi070.stdout:3/850: dwrite d10/d32/d4b/dc5/fed [0,4194304] 0 2024-04-02T04:48:08.909 INFO:tasks.workunit.client.1.smithi179.stdout:2/906: stat dc/d29/d4a/f72 0 2024-04-02T04:48:08.911 INFO:tasks.workunit.client.0.smithi070.stdout:6/794: symlink d0/d20/dd8/lef 0 2024-04-02T04:48:08.911 INFO:tasks.workunit.client.0.smithi070.stdout:9/861: dread d4/d10/d11/d7b/f2b [0,4194304] 0 2024-04-02T04:48:08.912 INFO:tasks.workunit.client.0.smithi070.stdout:3/851: rmdir d10 39 2024-04-02T04:48:08.915 INFO:tasks.workunit.client.1.smithi179.stdout:2/907: unlink dc/d29/d31/d57/d97/dbf/de3/l109 0 2024-04-02T04:48:08.917 INFO:tasks.workunit.client.0.smithi070.stdout:9/862: symlink d4/d13/l11d 0 2024-04-02T04:48:08.920 INFO:tasks.workunit.client.0.smithi070.stdout:3/852: write d10/d17/fcc [346487,48094] 0 2024-04-02T04:48:08.920 INFO:tasks.workunit.client.0.smithi070.stdout:6/795: truncate d0/f78 490970 0 2024-04-02T04:48:08.922 INFO:tasks.workunit.client.1.smithi179.stdout:2/908: dwrite dc/d29/d4a/d1e/d6c/f9e [0,4194304] 0 2024-04-02T04:48:08.922 INFO:tasks.workunit.client.0.smithi070.stdout:3/853: write d10/d32/d8a/d8d/d91/f94 [4771034,32890] 0 2024-04-02T04:48:08.926 INFO:tasks.workunit.client.0.smithi070.stdout:0/950: sync 2024-04-02T04:48:08.926 INFO:tasks.workunit.client.0.smithi070.stdout:0/951: readlink d4/d5/d13/d2a/d8d/dfc/l10e 0 2024-04-02T04:48:08.928 INFO:tasks.workunit.client.0.smithi070.stdout:9/863: mknod d4/d10/d11/c11e 0 2024-04-02T04:48:08.929 INFO:tasks.workunit.client.0.smithi070.stdout:9/864: chown d4/d10/d11/d7b/d19/d2f/d39/d57 76290 1 2024-04-02T04:48:08.930 INFO:tasks.workunit.client.0.smithi070.stdout:9/865: fsync d4/d34/db9/de1/fe6 0 2024-04-02T04:48:08.937 INFO:tasks.workunit.client.0.smithi070.stdout:9/866: symlink d4/d10/d11/l11f 0 2024-04-02T04:48:08.939 INFO:tasks.workunit.client.0.smithi070.stdout:3/854: truncate d10/d17/d51/d73/d8f/f9c 547838 0 2024-04-02T04:48:08.942 INFO:tasks.workunit.client.0.smithi070.stdout:6/796: rename d0/da2 to d0/df0 0 2024-04-02T04:48:08.943 INFO:tasks.workunit.client.0.smithi070.stdout:6/797: fdatasync d0/d3/d19/d31/fdd 0 2024-04-02T04:48:08.943 INFO:tasks.workunit.client.0.smithi070.stdout:9/867: rmdir d4/d10/d12/d27/d83 39 2024-04-02T04:48:08.943 INFO:tasks.workunit.client.1.smithi179.stdout:2/909: dread dc/d29/d4a/f75 [0,4194304] 0 2024-04-02T04:48:08.947 INFO:tasks.workunit.client.1.smithi179.stdout:2/910: fsync dc/d29/d31/d57/fa0 0 2024-04-02T04:48:08.948 INFO:tasks.workunit.client.1.smithi179.stdout:5/980: sync 2024-04-02T04:48:08.950 INFO:tasks.workunit.client.0.smithi070.stdout:3/855: creat d10/d66/d103/f11f x:0 0 0 2024-04-02T04:48:08.953 INFO:tasks.workunit.client.0.smithi070.stdout:0/952: dread d4/d5/d13/d43/f49 [0,4194304] 0 2024-04-02T04:48:08.953 INFO:tasks.workunit.client.1.smithi179.stdout:2/911: dwrite dc/d29/d31/d83/dae/f12a [0,4194304] 0 2024-04-02T04:48:08.957 INFO:tasks.workunit.client.1.smithi179.stdout:5/981: rename d8/d1b/d23/d32/dbe/daf/db7/dad to d8/d1b/d23/d36/d53/d62/db2/d137 0 2024-04-02T04:48:08.961 INFO:tasks.workunit.client.1.smithi179.stdout:2/912: creat dc/dcb/f133 x:0 0 0 2024-04-02T04:48:08.961 INFO:tasks.workunit.client.1.smithi179.stdout:2/913: stat dc/d29/d31/d57/d97/c9c 0 2024-04-02T04:48:08.961 INFO:tasks.workunit.client.1.smithi179.stdout:2/914: chown dc/f5d 649 1 2024-04-02T04:48:08.966 INFO:tasks.workunit.client.1.smithi179.stdout:5/982: unlink d8/d1b/d23/d32/dbe/daf/db7/de6/l107 0 2024-04-02T04:48:08.970 INFO:tasks.workunit.client.1.smithi179.stdout:5/983: write d8/d1b/d23/d32/dbe/fcf [468505,61978] 0 2024-04-02T04:48:08.982 INFO:tasks.workunit.client.1.smithi179.stdout:5/984: link d8/d1b/d23/d32/d4e/d51/f66 d8/d1b/d23/d36/dcb/f138 0 2024-04-02T04:48:08.982 INFO:tasks.workunit.client.1.smithi179.stdout:2/915: write dc/d29/d4a/f8c [773973,130930] 0 2024-04-02T04:48:08.982 INFO:tasks.workunit.client.1.smithi179.stdout:5/985: read - d8/d1b/d23/d32/d46/fdc zero size 2024-04-02T04:48:08.982 INFO:tasks.workunit.client.1.smithi179.stdout:2/916: read - dc/d29/d4a/fad zero size 2024-04-02T04:48:08.982 INFO:tasks.workunit.client.1.smithi179.stdout:5/986: stat d8/d1b/d23/d36/d53/d62/db2/d137/cb5 0 2024-04-02T04:48:08.992 INFO:tasks.workunit.client.1.smithi179.stdout:5/987: creat d8/d1b/d23/f139 x:0 0 0 2024-04-02T04:48:09.011 INFO:tasks.workunit.client.1.smithi179.stdout:2/917: dwrite dc/d29/d4a/f22 [0,4194304] 0 2024-04-02T04:48:09.014 INFO:tasks.workunit.client.1.smithi179.stdout:2/918: mknod dc/d29/d10b/c134 0 2024-04-02T04:48:09.018 INFO:tasks.workunit.client.0.smithi070.stdout:6/798: dwrite d0/d3/d19/d31/d46/d6b/f70 [0,4194304] 0 2024-04-02T04:48:09.018 INFO:tasks.workunit.client.0.smithi070.stdout:9/868: rename d4/d10/d11/d7b/d19/d82/f104 to d4/d10/d11/d7b/d19/d2f/d39/d57/dfb/f120 0 2024-04-02T04:48:09.027 INFO:tasks.workunit.client.1.smithi179.stdout:5/988: dwrite d8/d1b/f88 [0,4194304] 0 2024-04-02T04:48:09.027 INFO:tasks.workunit.client.1.smithi179.stdout:2/919: rename dc/d29/de7 to dc/d135 0 2024-04-02T04:48:09.028 INFO:tasks.workunit.client.0.smithi070.stdout:0/953: truncate d4/d5/d13/d2a/d3a/d45/fc8 1143365 0 2024-04-02T04:48:09.032 INFO:tasks.workunit.client.1.smithi179.stdout:5/989: chown d8/d1b/c55 16286514 1 2024-04-02T04:48:09.032 INFO:tasks.workunit.client.0.smithi070.stdout:9/869: fsync d4/d34/f7a 0 2024-04-02T04:48:09.037 INFO:tasks.workunit.client.1.smithi179.stdout:5/990: chown d8/d1b/d23/d30/d73/db3/f10b 178 1 2024-04-02T04:48:09.037 INFO:tasks.workunit.client.0.smithi070.stdout:6/799: creat d0/ff1 x:0 0 0 2024-04-02T04:48:09.040 INFO:tasks.workunit.client.1.smithi179.stdout:5/991: creat d8/d1b/d23/d32/dbe/daf/db7/de6/f13a x:0 0 0 2024-04-02T04:48:09.043 INFO:tasks.workunit.client.0.smithi070.stdout:3/856: dwrite d10/fad [0,4194304] 0 2024-04-02T04:48:09.045 INFO:tasks.workunit.client.1.smithi179.stdout:5/992: dwrite d8/f9e [0,4194304] 0 2024-04-02T04:48:09.052 INFO:tasks.workunit.client.0.smithi070.stdout:6/800: dread d0/f78 [0,4194304] 0 2024-04-02T04:48:09.056 INFO:tasks.workunit.client.1.smithi179.stdout:2/920: dwrite dc/d29/d4a/f3f [0,4194304] 0 2024-04-02T04:48:09.056 INFO:tasks.workunit.client.0.smithi070.stdout:0/954: dwrite d4/d5/d13/d2a/f20 [0,4194304] 0 2024-04-02T04:48:09.057 INFO:tasks.workunit.client.1.smithi179.stdout:2/921: readlink dc/d29/d31/d66/leb 0 2024-04-02T04:48:09.058 INFO:tasks.workunit.client.1.smithi179.stdout:5/993: link d8/d1b/d23/d32/d4e/d50/f131 d8/d1b/d23/db1/f13b 0 2024-04-02T04:48:09.060 INFO:tasks.workunit.client.0.smithi070.stdout:0/955: fsync d4/d5/d13/d23/f74 0 2024-04-02T04:48:09.061 INFO:tasks.workunit.client.1.smithi179.stdout:2/922: rmdir dc/d29/d31/d57/d97/dbf/de3 39 2024-04-02T04:48:09.061 INFO:tasks.workunit.client.0.smithi070.stdout:6/801: link d0/d3/d19/d31/d46/d6b/lea d0/d20/d77/lf2 0 2024-04-02T04:48:09.063 INFO:tasks.workunit.client.1.smithi179.stdout:5/994: mknod d8/d1b/d23/d36/d53/d62/da9/c13c 0 2024-04-02T04:48:09.064 INFO:tasks.workunit.client.1.smithi179.stdout:2/923: rmdir dc/d29/d31/d57/d97/ddd 39 2024-04-02T04:48:09.065 INFO:tasks.workunit.client.0.smithi070.stdout:0/956: rename d4/d5/d13/d2a/d29/f59 to d4/d5/d58/f126 0 2024-04-02T04:48:09.067 INFO:tasks.workunit.client.1.smithi179.stdout:2/924: fdatasync dc/d79/f93 0 2024-04-02T04:48:09.069 INFO:tasks.workunit.client.1.smithi179.stdout:5/995: read d8/d1b/d23/d32/dbe/daf/dd4/d105/fbc [407391,47238] 0 2024-04-02T04:48:09.069 INFO:tasks.workunit.client.1.smithi179.stdout:5/996: dread - d8/fc9 zero size 2024-04-02T04:48:09.071 INFO:tasks.workunit.client.1.smithi179.stdout:2/925: rename dc/d29/d31/d57/d97/f119 to dc/d29/d4a/d4b/d126/f136 0 2024-04-02T04:48:09.072 INFO:tasks.workunit.client.1.smithi179.stdout:2/926: dread - dc/d29/d4a/d1e/d6c/dd2/f124 zero size 2024-04-02T04:48:09.074 INFO:tasks.workunit.client.0.smithi070.stdout:0/957: rename d4/d5/d58/d60/led to d4/d5/d13/d2a/l127 0 2024-04-02T04:48:09.079 INFO:tasks.workunit.client.0.smithi070.stdout:0/958: dwrite d4/d5/da/f123 [0,4194304] 0 2024-04-02T04:48:09.087 INFO:tasks.workunit.client.0.smithi070.stdout:0/959: creat d4/d5/d13/d2a/f128 x:0 0 0 2024-04-02T04:48:09.088 INFO:tasks.workunit.client.0.smithi070.stdout:0/960: chown d4/d5/d13/d17 1061 1 2024-04-02T04:48:09.095 INFO:tasks.workunit.client.0.smithi070.stdout:3/857: dwrite d10/d32/f52 [0,4194304] 0 2024-04-02T04:48:09.105 INFO:tasks.workunit.client.0.smithi070.stdout:6/802: truncate d0/d3/d19/d31/dc1/fc8 1168840 0 2024-04-02T04:48:09.109 INFO:tasks.workunit.client.1.smithi179.stdout:2/927: rename dc/d29/d31 to dc/d29/d4a/d137 0 2024-04-02T04:48:09.109 INFO:tasks.workunit.client.1.smithi179.stdout:2/928: stat dc/d29/d4a/d1e/d6c/c76 0 2024-04-02T04:48:09.110 INFO:tasks.workunit.client.1.smithi179.stdout:2/929: chown dc/d29/d4a/d1e/f7b 86022 1 2024-04-02T04:48:09.113 INFO:tasks.workunit.client.1.smithi179.stdout:5/997: dwrite d8/d1b/d23/d32/f33 [0,4194304] 0 2024-04-02T04:48:09.123 INFO:tasks.workunit.client.0.smithi070.stdout:3/858: dwrite d10/d17/d51/fc9 [0,4194304] 0 2024-04-02T04:48:09.132 INFO:tasks.workunit.client.0.smithi070.stdout:6/803: write d0/f26 [519625,54353] 0 2024-04-02T04:48:09.137 INFO:tasks.workunit.client.0.smithi070.stdout:6/804: mkdir d0/d55/d8b/df3 0 2024-04-02T04:48:09.143 INFO:tasks.workunit.client.0.smithi070.stdout:6/805: dwrite d0/d3/d19/d31/d46/d6b/f70 [0,4194304] 0 2024-04-02T04:48:09.149 INFO:tasks.workunit.client.0.smithi070.stdout:6/806: read d0/d3/d19/d31/d46/f93 [323356,26547] 0 2024-04-02T04:48:09.155 INFO:tasks.workunit.client.0.smithi070.stdout:6/807: creat d0/d20/ff4 x:0 0 0 2024-04-02T04:48:09.156 INFO:tasks.workunit.client.0.smithi070.stdout:6/808: readlink d0/d43/db3/dc7/le8 0 2024-04-02T04:48:09.158 INFO:tasks.workunit.client.0.smithi070.stdout:6/809: fdatasync d0/d43/db3/dc7/f66 0 2024-04-02T04:48:09.163 INFO:tasks.workunit.client.1.smithi179.stdout:2/930: creat dc/d29/d4a/d1e/d43/f138 x:0 0 0 2024-04-02T04:48:09.163 INFO:tasks.workunit.client.1.smithi179.stdout:2/931: stat dc/c2a 0 2024-04-02T04:48:09.163 INFO:tasks.workunit.client.1.smithi179.stdout:5/998: rename d8/d1b/d23/d32/d4b/d12e to d8/d1b/d23/d32/d4e/ddd/d13d 0 2024-04-02T04:48:09.164 INFO:tasks.workunit.client.1.smithi179.stdout:2/932: fdatasync dc/f9f 0 2024-04-02T04:48:09.166 INFO:tasks.workunit.client.1.smithi179.stdout:5/999: rename d8/d1b/d23/d30/lef to d8/d1b/d23/d36/d53/d62/db2/d137/d111/l13e 0 2024-04-02T04:48:09.198 INFO:tasks.workunit.client.0.smithi070.stdout:8/848: sync 2024-04-02T04:48:09.198 INFO:tasks.workunit.client.0.smithi070.stdout:4/929: sync 2024-04-02T04:48:09.198 INFO:tasks.workunit.client.0.smithi070.stdout:7/876: sync 2024-04-02T04:48:09.198 INFO:tasks.workunit.client.0.smithi070.stdout:1/814: sync 2024-04-02T04:48:09.202 INFO:tasks.workunit.client.0.smithi070.stdout:8/849: symlink d2/d8/d29/d22/d5a/d94/l110 0 2024-04-02T04:48:09.203 INFO:tasks.workunit.client.0.smithi070.stdout:8/850: readlink d2/d8/d23/d24/d79/l57 0 2024-04-02T04:48:09.203 INFO:tasks.workunit.client.0.smithi070.stdout:1/815: mkdir d0/d9/d1d/d22/d56/dc1/dfc 0 2024-04-02T04:48:09.204 INFO:tasks.workunit.client.0.smithi070.stdout:7/877: dread d7/d4e/d82/d57/fc2 [0,4194304] 0 2024-04-02T04:48:09.205 INFO:tasks.workunit.client.0.smithi070.stdout:1/816: read d0/d7/f10 [207066,82721] 0 2024-04-02T04:48:09.210 INFO:tasks.workunit.client.0.smithi070.stdout:1/817: dread d0/d4c/da2/fb5 [0,4194304] 0 2024-04-02T04:48:09.211 INFO:tasks.workunit.client.0.smithi070.stdout:6/810: write d0/d3/d19/d31/d46/fa9 [256925,79112] 0 2024-04-02T04:48:09.212 INFO:tasks.workunit.client.0.smithi070.stdout:1/818: dread - d0/db4/f9b zero size 2024-04-02T04:48:09.216 INFO:tasks.workunit.client.0.smithi070.stdout:1/819: symlink d0/d9/d1d/d22/d2c/d4f/d77/lfd 0 2024-04-02T04:48:09.217 INFO:tasks.workunit.client.0.smithi070.stdout:7/878: write d7/d8e/d123/d65/f71 [3401314,92709] 0 2024-04-02T04:48:09.218 INFO:tasks.workunit.client.0.smithi070.stdout:6/811: dwrite d0/d3/d19/d31/f4e [8388608,4194304] 0 2024-04-02T04:48:09.218 INFO:tasks.workunit.client.0.smithi070.stdout:1/820: dread - d0/d9/d1d/d22/ff3 zero size 2024-04-02T04:48:09.221 INFO:tasks.workunit.client.0.smithi070.stdout:6/812: read d0/d55/d8b/d94/f6c [3468864,13100] 0 2024-04-02T04:48:09.221 INFO:tasks.workunit.client.0.smithi070.stdout:7/879: chown d7/d4e/d82/cde 1619307657 1 2024-04-02T04:48:09.223 INFO:tasks.workunit.client.0.smithi070.stdout:8/851: dread d2/d3/f49 [0,4194304] 0 2024-04-02T04:48:09.223 INFO:tasks.workunit.client.0.smithi070.stdout:1/821: mknod d0/d9/d1d/d22/d56/d79/d89/cfe 0 2024-04-02T04:48:09.225 INFO:tasks.workunit.client.0.smithi070.stdout:6/813: mknod d0/d55/d5c/cf5 0 2024-04-02T04:48:09.225 INFO:tasks.workunit.client.0.smithi070.stdout:6/814: chown d0/d20/d77/fbf 1 1 2024-04-02T04:48:09.227 INFO:tasks.workunit.client.0.smithi070.stdout:8/852: read d2/d3/d35/f78 [662398,32090] 0 2024-04-02T04:48:09.228 INFO:tasks.workunit.client.0.smithi070.stdout:7/880: dread d7/dd/df/d7a/d6c/d77/fb0 [0,4194304] 0 2024-04-02T04:48:09.228 INFO:tasks.workunit.client.0.smithi070.stdout:1/822: mkdir d0/d9/dff 0 2024-04-02T04:48:09.230 INFO:tasks.workunit.client.0.smithi070.stdout:8/853: rmdir d2/d3 39 2024-04-02T04:48:09.230 INFO:tasks.workunit.client.0.smithi070.stdout:1/823: dread - d0/d9/d1d/d22/d2c/d4f/f99 zero size 2024-04-02T04:48:09.231 INFO:tasks.workunit.client.0.smithi070.stdout:6/815: dread d0/d20/d5f/de6/da6/fd3 [0,4194304] 0 2024-04-02T04:48:09.232 INFO:tasks.workunit.client.0.smithi070.stdout:8/854: mknod d2/d8/d29/db/c111 0 2024-04-02T04:48:09.232 INFO:tasks.workunit.client.0.smithi070.stdout:6/816: chown d0/d55/d8b/d94/cd4 2338476 1 2024-04-02T04:48:09.233 INFO:tasks.workunit.client.0.smithi070.stdout:8/855: readlink d2/d8/d29/l89 0 2024-04-02T04:48:09.233 INFO:tasks.workunit.client.0.smithi070.stdout:1/824: fdatasync d0/d7/f81 0 2024-04-02T04:48:09.235 INFO:tasks.workunit.client.0.smithi070.stdout:1/825: write d0/d7/d28/d83/fbd [1304004,74872] 0 2024-04-02T04:48:09.237 INFO:tasks.workunit.client.0.smithi070.stdout:6/817: fsync d0/d3/d19/d31/d46/f93 0 2024-04-02T04:48:09.237 INFO:tasks.workunit.client.0.smithi070.stdout:7/881: write d7/dd/df/d7a/d6c/d77/d54/de5/feb [942744,80544] 0 2024-04-02T04:48:09.240 INFO:tasks.workunit.client.0.smithi070.stdout:6/818: stat d0/d3/d19/f7f 0 2024-04-02T04:48:09.242 INFO:tasks.workunit.client.0.smithi070.stdout:7/882: write d7/dd/df/d79/f48 [1162719,2346] 0 2024-04-02T04:48:09.244 INFO:tasks.workunit.client.0.smithi070.stdout:7/883: chown d7/d4e/d82/d57/l95 21 1 2024-04-02T04:48:09.245 INFO:tasks.workunit.client.0.smithi070.stdout:1/826: dwrite d0/d7/d1a/db9/ff8 [0,4194304] 0 2024-04-02T04:48:09.252 INFO:tasks.workunit.client.0.smithi070.stdout:1/827: creat d0/d54/d64/d6d/f100 x:0 0 0 2024-04-02T04:48:09.253 INFO:tasks.workunit.client.0.smithi070.stdout:7/884: truncate d7/dd/db1/fe0 1025107 0 2024-04-02T04:48:09.254 INFO:tasks.workunit.client.0.smithi070.stdout:6/819: dwrite d0/d3/d19/d31/f4e [8388608,4194304] 0 2024-04-02T04:48:09.255 INFO:tasks.workunit.client.0.smithi070.stdout:8/856: dwrite d2/d8/d29/fba [0,4194304] 0 2024-04-02T04:48:09.255 INFO:tasks.workunit.client.0.smithi070.stdout:1/828: chown d0/d9/d1d/d22/d2c/de3 126215334 1 2024-04-02T04:48:09.258 INFO:tasks.workunit.client.0.smithi070.stdout:1/829: chown d0/d9/d1d/d22/d2c/c37 102 1 2024-04-02T04:48:09.259 INFO:tasks.workunit.client.0.smithi070.stdout:6/820: mknod d0/d3/cf6 0 2024-04-02T04:48:09.261 INFO:tasks.workunit.client.0.smithi070.stdout:7/885: symlink d7/dd/df/d7a/d6c/d77/d54/d10c/l125 0 2024-04-02T04:48:09.264 INFO:tasks.workunit.client.1.smithi179.stdout:2/933: sync 2024-04-02T04:48:09.265 INFO:tasks.workunit.client.0.smithi070.stdout:1/830: dread d0/d4c/fa0 [0,4194304] 0 2024-04-02T04:48:09.270 INFO:tasks.workunit.client.1.smithi179.stdout:2/934: creat dc/d29/d4a/d137/d83/dae/f139 x:0 0 0 2024-04-02T04:48:09.270 INFO:tasks.workunit.client.0.smithi070.stdout:7/886: mknod d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/c126 0 2024-04-02T04:48:09.271 INFO:tasks.workunit.client.0.smithi070.stdout:8/857: getdents d2/d8/d23/d24/d66 0 2024-04-02T04:48:09.273 INFO:tasks.workunit.client.0.smithi070.stdout:7/887: creat d7/d8e/f127 x:0 0 0 2024-04-02T04:48:09.275 INFO:tasks.workunit.client.0.smithi070.stdout:7/888: chown d7/d8e/d123/cbe 4278811 1 2024-04-02T04:48:09.275 INFO:tasks.workunit.client.1.smithi179.stdout:2/935: dwrite dc/d29/d4a/d1e/d6c/f9e [0,4194304] 0 2024-04-02T04:48:09.276 INFO:tasks.workunit.client.0.smithi070.stdout:6/821: dwrite d0/d3/d19/d31/f74 [0,4194304] 0 2024-04-02T04:48:09.278 INFO:tasks.workunit.client.1.smithi179.stdout:2/936: read dc/dcb/f11a [1782711,123265] 0 2024-04-02T04:48:09.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:08 smithi070 ceph-mon[29212]: pgmap v23: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 79 MiB/s rd, 193 MiB/s wr, 430 op/s 2024-04-02T04:48:09.280 INFO:tasks.workunit.client.0.smithi070.stdout:7/889: rename d7/dd/df/d34/c3c to d7/d4e/d82/d57/c128 0 2024-04-02T04:48:09.284 INFO:tasks.workunit.client.0.smithi070.stdout:1/831: write d0/d9/d1d/f16 [2785586,24238] 0 2024-04-02T04:48:09.286 INFO:tasks.workunit.client.1.smithi179.stdout:2/937: dwrite dc/d29/d4a/f8c [0,4194304] 0 2024-04-02T04:48:09.287 INFO:tasks.workunit.client.0.smithi070.stdout:8/858: write d2/d8/d23/d24/d66/f99 [1069821,85610] 0 2024-04-02T04:48:09.288 INFO:tasks.workunit.client.0.smithi070.stdout:7/890: dwrite d7/d8e/dec/f117 [0,4194304] 0 2024-04-02T04:48:09.294 INFO:tasks.workunit.client.0.smithi070.stdout:8/859: unlink d2/d8/d29/d22/d5a/d9e/f107 0 2024-04-02T04:48:09.294 INFO:tasks.workunit.client.0.smithi070.stdout:1/832: link d0/l44 d0/d9/d1d/d22/d56/dc1/dfc/l101 0 2024-04-02T04:48:09.294 INFO:tasks.workunit.client.1.smithi179.stdout:2/938: dwrite dc/d29/d4a/d1e/d43/f138 [0,4194304] 0 2024-04-02T04:48:09.298 INFO:tasks.workunit.client.0.smithi070.stdout:1/833: mkdir d0/d54/d102 0 2024-04-02T04:48:09.302 INFO:tasks.workunit.client.0.smithi070.stdout:7/891: dread d7/d36/fc6 [0,4194304] 0 2024-04-02T04:48:09.302 INFO:tasks.workunit.client.0.smithi070.stdout:1/834: creat d0/d7/d28/d5c/d6e/f103 x:0 0 0 2024-04-02T04:48:09.306 INFO:tasks.workunit.client.0.smithi070.stdout:7/892: write d7/dd/df/d7a/dbc/f11d [824154,107104] 0 2024-04-02T04:48:09.309 INFO:tasks.workunit.client.0.smithi070.stdout:6/822: dwrite d0/d3/d19/d31/dc1/fc8 [0,4194304] 0 2024-04-02T04:48:09.311 INFO:tasks.workunit.client.0.smithi070.stdout:6/823: read - d0/d55/d8b/fe0 zero size 2024-04-02T04:48:09.313 INFO:tasks.workunit.client.0.smithi070.stdout:1/835: dwrite d0/d9/d1d/d22/d2c/d4f/f92 [4194304,4194304] 0 2024-04-02T04:48:09.314 INFO:tasks.workunit.client.0.smithi070.stdout:6/824: mkdir d0/d43/d80/ded/df7 0 2024-04-02T04:48:09.315 INFO:tasks.workunit.client.0.smithi070.stdout:6/825: stat d0/ff1 0 2024-04-02T04:48:09.317 INFO:tasks.workunit.client.0.smithi070.stdout:1/836: unlink d0/d9/fe0 0 2024-04-02T04:48:09.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:08 smithi179 ceph-mon[36603]: pgmap v23: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 79 MiB/s rd, 193 MiB/s wr, 430 op/s 2024-04-02T04:48:09.329 INFO:tasks.workunit.client.1.smithi179.stdout:2/939: dwrite dc/d29/d4a/d137/d57/f62 [0,4194304] 0 2024-04-02T04:48:09.330 INFO:tasks.workunit.client.1.smithi179.stdout:2/940: stat dc/dcb/fcf 0 2024-04-02T04:48:09.331 INFO:tasks.workunit.client.1.smithi179.stdout:2/941: readlink dc/d79/lb3 0 2024-04-02T04:48:09.333 INFO:tasks.workunit.client.0.smithi070.stdout:8/860: dread d2/d8/d23/d24/d66/f99 [0,4194304] 0 2024-04-02T04:48:09.335 INFO:tasks.workunit.client.0.smithi070.stdout:6/826: dread d0/d3/d19/d31/d46/f68 [4194304,4194304] 0 2024-04-02T04:48:09.336 INFO:tasks.workunit.client.0.smithi070.stdout:6/827: chown d0/d20/l32 339494869 1 2024-04-02T04:48:09.337 INFO:tasks.workunit.client.1.smithi179.stdout:2/942: rename dc/dcb to dc/d29/d4a/d137/d66/d13a 0 2024-04-02T04:48:09.337 INFO:tasks.workunit.client.0.smithi070.stdout:1/837: dread d0/d7/d1a/f75 [0,4194304] 0 2024-04-02T04:48:09.338 INFO:tasks.workunit.client.0.smithi070.stdout:8/861: mkdir d2/d8/d29/d22/d5a/d77/d112 0 2024-04-02T04:48:09.338 INFO:tasks.workunit.client.0.smithi070.stdout:6/828: dread - d0/d55/d8b/fe0 zero size 2024-04-02T04:48:09.342 INFO:tasks.workunit.client.0.smithi070.stdout:6/829: mknod d0/d3/d19/d31/dc1/dd0/cf8 0 2024-04-02T04:48:09.343 INFO:tasks.workunit.client.0.smithi070.stdout:7/893: dread d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/f78 [0,4194304] 0 2024-04-02T04:48:09.344 INFO:tasks.workunit.client.0.smithi070.stdout:6/830: creat d0/d3/d19/d31/dc1/ff9 x:0 0 0 2024-04-02T04:48:09.350 INFO:tasks.workunit.client.0.smithi070.stdout:6/831: dread d0/d3/d19/d31/d46/d6b/f6d [0,4194304] 0 2024-04-02T04:48:09.388 INFO:tasks.workunit.client.0.smithi070.stdout:2/969: sync 2024-04-02T04:48:09.389 INFO:tasks.workunit.client.1.smithi179.stdout:2/943: write dc/d29/d4a/d137/d57/d97/fb6 [944937,94421] 0 2024-04-02T04:48:09.391 INFO:tasks.workunit.client.1.smithi179.stdout:2/944: mknod dc/d29/d4a/d137/d83/dae/c13b 0 2024-04-02T04:48:09.398 INFO:tasks.workunit.client.0.smithi070.stdout:6/832: fsync d0/d55/d8b/d94/f65 0 2024-04-02T04:48:09.401 INFO:tasks.workunit.client.0.smithi070.stdout:7/894: link d7/dd/df/l3f d7/d8e/d123/d65/d7b/l129 0 2024-04-02T04:48:09.402 INFO:tasks.workunit.client.1.smithi179.stdout:2/945: dread dc/fe [4194304,4194304] 0 2024-04-02T04:48:09.402 INFO:tasks.workunit.client.0.smithi070.stdout:1/838: dwrite d0/d9/d1d/d22/d2c/f76 [4194304,4194304] 0 2024-04-02T04:48:09.404 INFO:tasks.workunit.client.0.smithi070.stdout:2/970: mknod d2/d10/d5c/d7a/db0/dd8/c13a 0 2024-04-02T04:48:09.408 INFO:tasks.workunit.client.0.smithi070.stdout:6/833: rename d0/d43/l99 to d0/d20/dd8/lfa 0 2024-04-02T04:48:09.409 INFO:tasks.workunit.client.1.smithi179.stdout:2/946: mkdir dc/d29/d10b/d13c 0 2024-04-02T04:48:09.410 INFO:tasks.workunit.client.0.smithi070.stdout:7/895: write d7/dd/df/dca/fd4 [930683,42401] 0 2024-04-02T04:48:09.413 INFO:tasks.workunit.client.0.smithi070.stdout:2/971: creat d2/d10/d19/dd7/f13b x:0 0 0 2024-04-02T04:48:09.413 INFO:tasks.workunit.client.0.smithi070.stdout:6/834: dwrite d0/d43/d95/f9a [4194304,4194304] 0 2024-04-02T04:48:09.414 INFO:tasks.workunit.client.1.smithi179.stdout:2/947: dwrite dc/d135/ffe [0,4194304] 0 2024-04-02T04:48:09.414 INFO:tasks.workunit.client.0.smithi070.stdout:1/839: creat d0/d9/d1d/d22/d56/dc1/dfc/f104 x:0 0 0 2024-04-02T04:48:09.415 INFO:tasks.workunit.client.0.smithi070.stdout:7/896: mkdir d7/d4e/d82/d57/ddb/d12a 0 2024-04-02T04:48:09.421 INFO:tasks.workunit.client.0.smithi070.stdout:7/897: creat d7/dd/df/d34/f12b x:0 0 0 2024-04-02T04:48:09.430 INFO:tasks.workunit.client.0.smithi070.stdout:7/898: symlink d7/dd/df/d7a/d6c/d77/d54/l12c 0 2024-04-02T04:48:09.431 INFO:tasks.workunit.client.0.smithi070.stdout:2/972: write d2/d24/f2d [479182,46466] 0 2024-04-02T04:48:09.434 INFO:tasks.workunit.client.0.smithi070.stdout:7/899: symlink d7/dd/df/d79/de6/l12d 0 2024-04-02T04:48:09.436 INFO:tasks.workunit.client.0.smithi070.stdout:6/835: dwrite d0/d20/f34 [0,4194304] 0 2024-04-02T04:48:09.438 INFO:tasks.workunit.client.1.smithi179.stdout:2/948: truncate dc/d29/d4a/d1e/d43/f47 2578162 0 2024-04-02T04:48:09.440 INFO:tasks.workunit.client.1.smithi179.stdout:2/949: write dc/d29/d4a/d1e/d6c/dd2/d121/f132 [5005502,55433] 0 2024-04-02T04:48:09.441 INFO:tasks.workunit.client.0.smithi070.stdout:2/973: write d2/d10/f78 [130886,78427] 0 2024-04-02T04:48:09.444 INFO:tasks.workunit.client.0.smithi070.stdout:6/836: getdents d0/d3 0 2024-04-02T04:48:09.446 INFO:tasks.workunit.client.0.smithi070.stdout:2/974: unlink d2/d10/d21/d71/dc1/l109 0 2024-04-02T04:48:09.447 INFO:tasks.workunit.client.0.smithi070.stdout:1/840: dwrite d0/db4/ff4 [0,4194304] 0 2024-04-02T04:48:09.448 INFO:tasks.workunit.client.0.smithi070.stdout:6/837: mkdir d0/d20/d5f/de6/da6/dfb 0 2024-04-02T04:48:09.449 INFO:tasks.workunit.client.0.smithi070.stdout:1/841: chown d0/db4/l9d 3 1 2024-04-02T04:48:09.449 INFO:tasks.workunit.client.0.smithi070.stdout:7/900: dwrite d7/d8e/d123/d65/f8c [0,4194304] 0 2024-04-02T04:48:09.451 INFO:tasks.workunit.client.0.smithi070.stdout:7/901: fsync d7/dd/df/d7a/d6c/d77/d54/f112 0 2024-04-02T04:48:09.455 INFO:tasks.workunit.client.0.smithi070.stdout:2/975: mkdir d2/d10/d5c/d7a/db0/dd8/dd2/d90/d13c 0 2024-04-02T04:48:09.456 INFO:tasks.workunit.client.0.smithi070.stdout:1/842: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/l105 0 2024-04-02T04:48:09.457 INFO:tasks.workunit.client.0.smithi070.stdout:6/838: dread d0/d3/d19/d31/d46/d6b/f70 [0,4194304] 0 2024-04-02T04:48:09.460 INFO:tasks.workunit.client.0.smithi070.stdout:7/902: getdents d7/dd/df/def 0 2024-04-02T04:48:09.461 INFO:tasks.workunit.client.0.smithi070.stdout:6/839: creat d0/d43/d80/ffc x:0 0 0 2024-04-02T04:48:09.461 INFO:tasks.workunit.client.0.smithi070.stdout:1/843: symlink d0/d9/d1d/d22/d56/dc1/dfc/l106 0 2024-04-02T04:48:09.462 INFO:tasks.workunit.client.0.smithi070.stdout:6/840: readlink d0/d43/l3d 0 2024-04-02T04:48:09.463 INFO:tasks.workunit.client.0.smithi070.stdout:1/844: stat d0/d7/d28/d83/ce8 0 2024-04-02T04:48:09.463 INFO:tasks.workunit.client.0.smithi070.stdout:7/903: creat d7/dd/df/def/f12e x:0 0 0 2024-04-02T04:48:09.465 INFO:tasks.workunit.client.0.smithi070.stdout:6/841: creat d0/d43/d80/ded/ffd x:0 0 0 2024-04-02T04:48:09.467 INFO:tasks.workunit.client.0.smithi070.stdout:6/842: write d0/d55/d8b/d94/f6c [6744842,85252] 0 2024-04-02T04:48:09.474 INFO:tasks.workunit.client.0.smithi070.stdout:2/976: dread d2/d10/d5c/d7a/db0/fdf [0,4194304] 0 2024-04-02T04:48:09.476 INFO:tasks.workunit.client.0.smithi070.stdout:1/845: rename d0/d9/d1d/d22/d56/cde to d0/d54/c107 0 2024-04-02T04:48:09.477 INFO:tasks.workunit.client.0.smithi070.stdout:6/843: rmdir d0/d55/d8b/df3 0 2024-04-02T04:48:09.478 INFO:tasks.workunit.client.0.smithi070.stdout:2/977: symlink d2/d12/d3e/l13d 0 2024-04-02T04:48:09.479 INFO:tasks.workunit.client.0.smithi070.stdout:1/846: truncate d0/d7/f6a 1564555 0 2024-04-02T04:48:09.480 INFO:tasks.workunit.client.0.smithi070.stdout:1/847: chown d0/d9/lc8 1 1 2024-04-02T04:48:09.481 INFO:tasks.workunit.client.0.smithi070.stdout:2/978: fsync d2/d10/d53/f55 0 2024-04-02T04:48:09.482 INFO:tasks.workunit.client.0.smithi070.stdout:7/904: write d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/f27 [2928756,27693] 0 2024-04-02T04:48:09.483 INFO:tasks.workunit.client.0.smithi070.stdout:6/844: mkdir d0/dfe 0 2024-04-02T04:48:09.483 INFO:tasks.workunit.client.0.smithi070.stdout:2/979: read d2/d12/f38 [2178537,105218] 0 2024-04-02T04:48:09.486 INFO:tasks.workunit.client.0.smithi070.stdout:1/848: symlink d0/d54/l108 0 2024-04-02T04:48:09.491 INFO:tasks.workunit.client.0.smithi070.stdout:1/849: creat d0/d9/f109 x:0 0 0 2024-04-02T04:48:09.493 INFO:tasks.workunit.client.0.smithi070.stdout:7/905: symlink d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/l12f 0 2024-04-02T04:48:09.493 INFO:tasks.workunit.client.0.smithi070.stdout:1/850: read d0/d9/d1d/f42 [3822443,115924] 0 2024-04-02T04:48:09.498 INFO:tasks.workunit.client.0.smithi070.stdout:1/851: creat d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/df9/f10a x:0 0 0 2024-04-02T04:48:09.499 INFO:tasks.workunit.client.0.smithi070.stdout:2/980: dwrite d2/d12/fa7 [0,4194304] 0 2024-04-02T04:48:09.501 INFO:tasks.workunit.client.0.smithi070.stdout:7/906: dwrite d7/d8e/d123/d65/f8c [0,4194304] 0 2024-04-02T04:48:09.504 INFO:tasks.workunit.client.0.smithi070.stdout:1/852: creat d0/d54/d64/d6d/ddd/f10b x:0 0 0 2024-04-02T04:48:09.506 INFO:tasks.workunit.client.0.smithi070.stdout:7/907: mkdir d7/d8e/d123/d130 0 2024-04-02T04:48:09.507 INFO:tasks.workunit.client.0.smithi070.stdout:7/908: chown d7/d4e/d82/d57/dce/le9 24662 1 2024-04-02T04:48:09.510 INFO:tasks.workunit.client.0.smithi070.stdout:7/909: creat d7/dd/db1/f131 x:0 0 0 2024-04-02T04:48:09.513 INFO:tasks.workunit.client.0.smithi070.stdout:7/910: symlink d7/dd/df/d7a/d6c/d77/d75/d80/l132 0 2024-04-02T04:48:09.515 INFO:tasks.workunit.client.1.smithi179.stdout:2/950: sync 2024-04-02T04:48:09.516 INFO:tasks.workunit.client.1.smithi179.stdout:2/951: read - dc/d79/ff2 zero size 2024-04-02T04:48:09.519 INFO:tasks.workunit.client.1.smithi179.stdout:2/952: dread dc/d29/d4a/f8c [0,4194304] 0 2024-04-02T04:48:09.521 INFO:tasks.workunit.client.1.smithi179.stdout:2/953: rmdir dc/d29/d10b/d110 39 2024-04-02T04:48:09.523 INFO:tasks.workunit.client.1.smithi179.stdout:2/954: mkdir dc/d29/d4a/d137/d57/d97/dbf/d13d 0 2024-04-02T04:48:09.524 INFO:tasks.workunit.client.0.smithi070.stdout:9/870: sync 2024-04-02T04:48:09.524 INFO:tasks.workunit.client.0.smithi070.stdout:0/961: sync 2024-04-02T04:48:09.524 INFO:tasks.workunit.client.1.smithi179.stdout:2/955: fsync dc/d29/d4a/d1e/d43/f98 0 2024-04-02T04:48:09.525 INFO:tasks.workunit.client.0.smithi070.stdout:9/871: rmdir d4/d13 39 2024-04-02T04:48:09.527 INFO:tasks.workunit.client.0.smithi070.stdout:0/962: mkdir d4/d5/d58/da1/d129 0 2024-04-02T04:48:09.528 INFO:tasks.workunit.client.1.smithi179.stdout:2/956: mknod dc/d29/d4a/d1e/d43/c13e 0 2024-04-02T04:48:09.529 INFO:tasks.workunit.client.0.smithi070.stdout:9/872: mkdir d4/d10/d11/d7b/d19/d53/d121 0 2024-04-02T04:48:09.531 INFO:tasks.workunit.client.1.smithi179.stdout:2/957: mkdir dc/d13f 0 2024-04-02T04:48:09.531 INFO:tasks.workunit.client.0.smithi070.stdout:0/963: creat d4/d5/da/f12a x:0 0 0 2024-04-02T04:48:09.532 INFO:tasks.workunit.client.0.smithi070.stdout:0/964: write d4/d5/da/f123 [3636372,239] 0 2024-04-02T04:48:09.533 INFO:tasks.workunit.client.0.smithi070.stdout:9/873: rename d4/d13/cc5 to d4/d10/d11/d7b/d19/d2f/d4d/d92/c122 0 2024-04-02T04:48:09.535 INFO:tasks.workunit.client.1.smithi179.stdout:2/958: mknod dc/d29/d4a/d4b/d10c/c140 0 2024-04-02T04:48:09.536 INFO:tasks.workunit.client.1.smithi179.stdout:2/959: read dc/d29/d4a/d137/faf [418085,9687] 0 2024-04-02T04:48:09.540 INFO:tasks.workunit.client.0.smithi070.stdout:0/965: dwrite d4/d5/d13/d23/f120 [0,4194304] 0 2024-04-02T04:48:09.542 INFO:tasks.workunit.client.1.smithi179.stdout:2/960: mkdir dc/d29/d4a/d1e/d43/df9/d141 0 2024-04-02T04:48:09.545 INFO:tasks.workunit.client.0.smithi070.stdout:0/966: creat d4/d5/d13/d17/d98/f12b x:0 0 0 2024-04-02T04:48:09.547 INFO:tasks.workunit.client.1.smithi179.stdout:2/961: truncate dc/d29/d4a/fad 670583 0 2024-04-02T04:48:09.552 INFO:tasks.workunit.client.1.smithi179.stdout:2/962: mkdir dc/d29/d4a/d1e/d43/df9/d141/d142 0 2024-04-02T04:48:09.553 INFO:tasks.workunit.client.0.smithi070.stdout:9/874: dread d4/ff7 [0,4194304] 0 2024-04-02T04:48:09.555 INFO:tasks.workunit.client.0.smithi070.stdout:0/967: write d4/d5/d13/d17/d72/f90 [3915770,101554] 0 2024-04-02T04:48:09.556 INFO:tasks.workunit.client.0.smithi070.stdout:9/875: getdents d4/d10/d12/d55 0 2024-04-02T04:48:09.558 INFO:tasks.workunit.client.0.smithi070.stdout:0/968: creat d4/d5/d58/da1/d129/f12c x:0 0 0 2024-04-02T04:48:09.560 INFO:tasks.workunit.client.0.smithi070.stdout:9/876: dread d4/d10/d11/d7b/d19/d2f/d8c/fd0 [0,4194304] 0 2024-04-02T04:48:09.561 INFO:tasks.workunit.client.0.smithi070.stdout:9/877: chown d4/d13/f26 44389 1 2024-04-02T04:48:09.561 INFO:tasks.workunit.client.0.smithi070.stdout:9/878: stat d4/ff7 0 2024-04-02T04:48:09.564 INFO:tasks.workunit.client.0.smithi070.stdout:0/969: truncate d4/d5/d13/fc7 2351810 0 2024-04-02T04:48:09.566 INFO:tasks.workunit.client.0.smithi070.stdout:0/970: mknod d4/d5/d58/d60/c12d 0 2024-04-02T04:48:09.569 INFO:tasks.workunit.client.0.smithi070.stdout:0/971: truncate d4/d5/d58/d60/f81 1501070 0 2024-04-02T04:48:09.569 INFO:tasks.workunit.client.0.smithi070.stdout:0/972: readlink d4/d3c/l11c 0 2024-04-02T04:48:09.571 INFO:tasks.workunit.client.0.smithi070.stdout:0/973: write d4/d5/da/f123 [4939344,108507] 0 2024-04-02T04:48:09.571 INFO:tasks.workunit.client.0.smithi070.stdout:0/974: chown d4/l1d 24453 1 2024-04-02T04:48:09.573 INFO:tasks.workunit.client.0.smithi070.stdout:9/879: dread d4/d10/d12/d27/d83/f8e [0,4194304] 0 2024-04-02T04:48:09.573 INFO:tasks.workunit.client.0.smithi070.stdout:0/975: write d4/d5/d13/d23/ff2 [592285,62598] 0 2024-04-02T04:48:09.574 INFO:tasks.workunit.client.0.smithi070.stdout:9/880: truncate d4/d13/d89/faf 6457166 0 2024-04-02T04:48:09.575 INFO:tasks.workunit.client.0.smithi070.stdout:0/976: mknod d4/d5/d13/d17/d98/dd2/c12e 0 2024-04-02T04:48:09.577 INFO:tasks.workunit.client.0.smithi070.stdout:0/977: fdatasync d4/d5/da/d2d/f66 0 2024-04-02T04:48:09.588 INFO:tasks.workunit.client.0.smithi070.stdout:0/978: dread d4/d3c/fa2 [0,4194304] 0 2024-04-02T04:48:09.590 INFO:tasks.workunit.client.0.smithi070.stdout:0/979: dread d4/d3c/f77 [0,4194304] 0 2024-04-02T04:48:09.593 INFO:tasks.workunit.client.0.smithi070.stdout:9/881: dread d4/d13/f37 [0,4194304] 0 2024-04-02T04:48:09.593 INFO:tasks.workunit.client.0.smithi070.stdout:0/980: mkdir d4/d5/d58/d125/d12f 0 2024-04-02T04:48:09.594 INFO:tasks.workunit.client.0.smithi070.stdout:9/882: write d4/d10/d12/d56/d79/dd8/ddf/f11c [999104,48540] 0 2024-04-02T04:48:09.595 INFO:tasks.workunit.client.0.smithi070.stdout:0/981: dread d4/d5/d13/d2a/d3a/d45/f10a [0,4194304] 0 2024-04-02T04:48:09.596 INFO:tasks.workunit.client.0.smithi070.stdout:0/982: fdatasync d4/d5/d13/d2a/d8d/dfc/f10b 0 2024-04-02T04:48:09.597 INFO:tasks.workunit.client.0.smithi070.stdout:9/883: fsync d4/d13/f26 0 2024-04-02T04:48:09.605 INFO:tasks.workunit.client.0.smithi070.stdout:0/983: getdents d4/d5/d13/d2a/d29/d52 0 2024-04-02T04:48:09.606 INFO:tasks.workunit.client.0.smithi070.stdout:0/984: dread - d4/d5/d13/d2a/ff6 zero size 2024-04-02T04:48:09.607 INFO:tasks.workunit.client.1.smithi179.stdout:2/963: sync 2024-04-02T04:48:09.608 INFO:tasks.workunit.client.0.smithi070.stdout:0/985: truncate d4/d5/d13/d2a/ff6 1041383 0 2024-04-02T04:48:09.614 INFO:tasks.workunit.client.0.smithi070.stdout:9/884: dread d4/d10/d12/fb3 [0,4194304] 0 2024-04-02T04:48:09.617 INFO:tasks.workunit.client.0.smithi070.stdout:9/885: creat d4/d10/d11/d7b/f123 x:0 0 0 2024-04-02T04:48:09.618 INFO:tasks.workunit.client.1.smithi179.stdout:2/964: dread dc/d29/d4a/f53 [4194304,4194304] 0 2024-04-02T04:48:09.619 INFO:tasks.workunit.client.0.smithi070.stdout:0/986: dread d4/d5/d58/d60/f92 [4194304,4194304] 0 2024-04-02T04:48:09.619 INFO:tasks.workunit.client.1.smithi179.stdout:2/965: stat dc/d29/d4a/d137/d57/d97/l118 0 2024-04-02T04:48:09.619 INFO:tasks.workunit.client.1.smithi179.stdout:2/966: stat dc/d29/d4a/f75 0 2024-04-02T04:48:09.620 INFO:tasks.workunit.client.0.smithi070.stdout:9/886: creat d4/d10/d11/d7b/d19/d2f/d39/d57/da6/f124 x:0 0 0 2024-04-02T04:48:09.620 INFO:tasks.workunit.client.1.smithi179.stdout:2/967: truncate dc/d29/d4a/d137/d66/d13a/f133 54389 0 2024-04-02T04:48:09.621 INFO:tasks.workunit.client.0.smithi070.stdout:0/987: read d4/d5/d13/d43/f78 [3100098,110963] 0 2024-04-02T04:48:09.621 INFO:tasks.workunit.client.1.smithi179.stdout:2/968: chown dc/d29/d4a/f45 1956 1 2024-04-02T04:48:09.626 INFO:tasks.workunit.client.0.smithi070.stdout:9/887: dwrite d4/d13/d89/fef [4194304,4194304] 0 2024-04-02T04:48:09.627 INFO:tasks.workunit.client.0.smithi070.stdout:0/988: rename d4/d5/d58/d60/d83/da5 to d4/d5/d13/d2a/d8d/dfc/d130 0 2024-04-02T04:48:09.628 INFO:tasks.workunit.client.1.smithi179.stdout:2/969: write dc/d29/d4a/fad [1084503,99835] 0 2024-04-02T04:48:09.630 INFO:tasks.workunit.client.0.smithi070.stdout:0/989: creat d4/d5/d13/d17/d72/f131 x:0 0 0 2024-04-02T04:48:09.635 INFO:tasks.workunit.client.1.smithi179.stdout:2/970: rename dc/d29/d4a/d1e/d43/f98 to dc/d29/d4a/f143 0 2024-04-02T04:48:09.637 INFO:tasks.workunit.client.0.smithi070.stdout:0/990: truncate d4/d5/d13/d2a/d8d/dfc/f10b 229777 0 2024-04-02T04:48:09.639 INFO:tasks.workunit.client.1.smithi179.stdout:2/971: mknod dc/d29/d4a/d4b/d130/c144 0 2024-04-02T04:48:09.641 INFO:tasks.workunit.client.1.smithi179.stdout:2/972: rmdir dc/d29/d4a/d137/d57/d97/dbf 39 2024-04-02T04:48:09.644 INFO:tasks.workunit.client.0.smithi070.stdout:0/991: fdatasync d4/d5/d13/fc7 0 2024-04-02T04:48:09.649 INFO:tasks.workunit.client.1.smithi179.stdout:2/973: creat dc/d29/d10b/d110/f145 x:0 0 0 2024-04-02T04:48:09.650 INFO:tasks.workunit.client.1.smithi179.stdout:2/974: write dc/d29/d4a/d1e/d43/f138 [452802,24288] 0 2024-04-02T04:48:09.650 INFO:tasks.workunit.client.0.smithi070.stdout:0/992: truncate d4/d5/d21/ffd 601819 0 2024-04-02T04:48:09.650 INFO:tasks.workunit.client.0.smithi070.stdout:9/888: dread d4/d10/d12/f62 [0,4194304] 0 2024-04-02T04:48:09.653 INFO:tasks.workunit.client.0.smithi070.stdout:0/993: fdatasync d4/d5/d13/d2a/d29/d52/f82 0 2024-04-02T04:48:09.655 INFO:tasks.workunit.client.1.smithi179.stdout:2/975: rename dc/d29/d4a/d1e/d43/df9/d141 to dc/d29/d4a/d137/d83/dae/db0/d146 0 2024-04-02T04:48:09.656 INFO:tasks.workunit.client.0.smithi070.stdout:9/889: truncate d4/d13/d89/df5/fb5 3251892 0 2024-04-02T04:48:09.659 INFO:tasks.workunit.client.1.smithi179.stdout:2/976: mkdir dc/d29/d4a/d1e/d6c/d147 0 2024-04-02T04:48:09.662 INFO:tasks.workunit.client.1.smithi179.stdout:2/977: readlink dc/d79/ld7 0 2024-04-02T04:48:09.664 INFO:tasks.workunit.client.1.smithi179.stdout:2/978: dread dc/d29/d4a/d137/d57/fa0 [0,4194304] 0 2024-04-02T04:48:09.664 INFO:tasks.workunit.client.0.smithi070.stdout:0/994: dwrite d4/d5/d13/d2a/d29/d52/f82 [4194304,4194304] 0 2024-04-02T04:48:09.665 INFO:tasks.workunit.client.0.smithi070.stdout:9/890: creat d4/d10/d11/d7b/d19/f125 x:0 0 0 2024-04-02T04:48:09.665 INFO:tasks.workunit.client.1.smithi179.stdout:2/979: readlink dc/d29/d4a/d137/d57/d97/ddd/l11e 0 2024-04-02T04:48:09.668 INFO:tasks.workunit.client.0.smithi070.stdout:9/891: mknod d4/c126 0 2024-04-02T04:48:09.669 INFO:tasks.workunit.client.1.smithi179.stdout:2/980: symlink dc/d29/d10b/d110/l148 0 2024-04-02T04:48:09.670 INFO:tasks.workunit.client.1.smithi179.stdout:2/981: fsync dc/d29/f34 0 2024-04-02T04:48:09.672 INFO:tasks.workunit.client.0.smithi070.stdout:9/892: truncate d4/d10/d11/d7b/d19/d82/fc4 829666 0 2024-04-02T04:48:09.673 INFO:tasks.workunit.client.1.smithi179.stdout:2/982: symlink dc/d29/d4a/d137/d83/d131/l149 0 2024-04-02T04:48:09.674 INFO:tasks.workunit.client.0.smithi070.stdout:9/893: mkdir d4/d10/d11/d7b/d127 0 2024-04-02T04:48:09.675 INFO:tasks.workunit.client.1.smithi179.stdout:2/983: creat dc/d29/d4a/d4b/d10c/f14a x:0 0 0 2024-04-02T04:48:09.677 INFO:tasks.workunit.client.0.smithi070.stdout:0/995: write d4/d5/d13/d2a/d3a/d45/fbd [1656199,20307] 0 2024-04-02T04:48:09.679 INFO:tasks.workunit.client.1.smithi179.stdout:2/984: rename dc/d29/d4a/d137/d83/dae/db0/cef to dc/d29/d4a/d137/d83/dae/ddc/c14b 0 2024-04-02T04:48:09.680 INFO:tasks.workunit.client.0.smithi070.stdout:9/894: dread d4/d10/d12/f67 [0,4194304] 0 2024-04-02T04:48:09.682 INFO:tasks.workunit.client.0.smithi070.stdout:9/895: write d4/d10/d11/d7b/d19/d2f/d39/d57/da6/f124 [879495,94416] 0 2024-04-02T04:48:09.682 INFO:tasks.workunit.client.0.smithi070.stdout:0/996: rename d4/d5/d13/d23/c4a to d4/d5/d13/d17/d6e/c132 0 2024-04-02T04:48:09.683 INFO:tasks.workunit.client.0.smithi070.stdout:9/896: fsync d4/d34/db9/d114/f115 0 2024-04-02T04:48:09.684 INFO:tasks.workunit.client.0.smithi070.stdout:9/897: dread - d4/d10/d11/d7b/d19/fc2 zero size 2024-04-02T04:48:09.685 INFO:tasks.workunit.client.1.smithi179.stdout:2/985: getdents dc/d79 0 2024-04-02T04:48:09.687 INFO:tasks.workunit.client.0.smithi070.stdout:9/898: mkdir d4/d10/d11/d7b/d19/d82/d128 0 2024-04-02T04:48:09.690 INFO:tasks.workunit.client.1.smithi179.stdout:2/986: creat dc/d29/d4a/d1e/d6c/dd2/d121/f14c x:0 0 0 2024-04-02T04:48:09.695 INFO:tasks.workunit.client.0.smithi070.stdout:0/997: write d4/d5/d13/d43/f49 [3889930,98664] 0 2024-04-02T04:48:09.695 INFO:tasks.workunit.client.0.smithi070.stdout:9/899: link d4/d10/d11/d7b/d19/f54 d4/d10/d12/d27/d60/f129 0 2024-04-02T04:48:09.698 INFO:tasks.workunit.client.1.smithi179.stdout:2/987: write dc/d29/d4a/d137/d83/fab [151103,114676] 0 2024-04-02T04:48:09.699 INFO:tasks.workunit.client.0.smithi070.stdout:0/998: creat d4/d5/d13/d2a/d3a/d80/da8/f133 x:0 0 0 2024-04-02T04:48:09.700 INFO:tasks.workunit.client.0.smithi070.stdout:0/999: dread - d4/d5/d13/d2a/f128 zero size 2024-04-02T04:48:09.701 INFO:tasks.workunit.client.0.smithi070.stdout:9/900: creat d4/d10/d12/d27/d60/d9e/d107/f12a x:0 0 0 2024-04-02T04:48:09.703 INFO:tasks.workunit.client.1.smithi179.stdout:2/988: dwrite dc/f50 [0,4194304] 0 2024-04-02T04:48:09.704 INFO:tasks.workunit.client.1.smithi179.stdout:2/989: write dc/d29/d4a/f22 [1494251,68969] 0 2024-04-02T04:48:09.708 INFO:tasks.workunit.client.0.smithi070.stdout:3/859: sync 2024-04-02T04:48:09.715 INFO:tasks.workunit.client.0.smithi070.stdout:9/901: dread d4/d10/d11/f42 [0,4194304] 0 2024-04-02T04:48:09.717 INFO:tasks.workunit.client.0.smithi070.stdout:9/902: fdatasync d4/d10/d12/d56/d79/dd8/ddf/fe4 0 2024-04-02T04:48:09.718 INFO:tasks.workunit.client.1.smithi179.stdout:2/990: dread dc/d29/d4a/d4b/f51 [0,4194304] 0 2024-04-02T04:48:09.719 INFO:tasks.workunit.client.0.smithi070.stdout:9/903: write d4/d34/db9/de1/fe6 [887412,52303] 0 2024-04-02T04:48:09.719 INFO:tasks.workunit.client.1.smithi179.stdout:2/991: write dc/d29/d4a/d137/d57/d97/fc2 [929338,1446] 0 2024-04-02T04:48:09.720 INFO:tasks.workunit.client.0.smithi070.stdout:9/904: write d4/d10/d11/d7b/d19/d2f/d39/d57/da6/f124 [1354157,49670] 0 2024-04-02T04:48:09.724 INFO:tasks.workunit.client.0.smithi070.stdout:3/860: dread d10/d17/f74 [0,4194304] 0 2024-04-02T04:48:09.724 INFO:tasks.workunit.client.1.smithi179.stdout:2/992: dread dc/d29/d4a/d137/f41 [0,4194304] 0 2024-04-02T04:48:09.727 INFO:tasks.workunit.client.0.smithi070.stdout:9/905: dread d4/d13/d89/faf [4194304,4194304] 0 2024-04-02T04:48:09.728 INFO:tasks.workunit.client.0.smithi070.stdout:9/906: chown d4/c126 0 1 2024-04-02T04:48:09.728 INFO:tasks.workunit.client.0.smithi070.stdout:3/861: mkdir d10/d17/db2/dbf/dd6/d120 0 2024-04-02T04:48:09.729 INFO:tasks.workunit.client.0.smithi070.stdout:3/862: chown d10/d17/d51/f64 11431600 1 2024-04-02T04:48:09.731 INFO:tasks.workunit.client.0.smithi070.stdout:9/907: creat d4/d10/d11/d7b/d19/d2f/d39/d116/f12b x:0 0 0 2024-04-02T04:48:09.731 INFO:tasks.workunit.client.0.smithi070.stdout:3/863: mkdir d10/d121 0 2024-04-02T04:48:09.735 INFO:tasks.workunit.client.0.smithi070.stdout:3/864: truncate d10/d17/d51/d73/d8f/f9c 82098 0 2024-04-02T04:48:09.741 INFO:tasks.workunit.client.0.smithi070.stdout:3/865: write d10/f4f [3549762,19093] 0 2024-04-02T04:48:09.741 INFO:tasks.workunit.client.1.smithi179.stdout:2/993: dread dc/d29/d4a/d1e/d6c/dd2/fe5 [0,4194304] 0 2024-04-02T04:48:09.744 INFO:tasks.workunit.client.0.smithi070.stdout:3/866: write d10/d17/d4c/fda [491801,1788] 0 2024-04-02T04:48:09.746 INFO:tasks.workunit.client.1.smithi179.stdout:2/994: link dc/cf dc/d29/d4a/d4b/c14d 0 2024-04-02T04:48:09.747 INFO:tasks.workunit.client.0.smithi070.stdout:3/867: rename d10/d17/db2/dbd to d10/d17/d4c/d11e/d122 0 2024-04-02T04:48:09.747 INFO:tasks.workunit.client.1.smithi179.stdout:2/995: fdatasync dc/d29/d4a/d4b/d126/f136 0 2024-04-02T04:48:09.748 INFO:tasks.workunit.client.0.smithi070.stdout:3/868: chown d10/d32/f42 3 1 2024-04-02T04:48:09.748 INFO:tasks.workunit.client.1.smithi179.stdout:2/996: stat dc/d29/d4a/f45 0 2024-04-02T04:48:09.753 INFO:tasks.workunit.client.1.smithi179.stdout:2/997: truncate dc/d29/d4a/f48 1195540 0 2024-04-02T04:48:09.755 INFO:tasks.workunit.client.1.smithi179.stdout:2/998: creat dc/d29/d10b/d13c/f14e x:0 0 0 2024-04-02T04:48:09.762 INFO:tasks.workunit.client.1.smithi179.stdout:2/999: dwrite dc/f5d [0,4194304] 0 2024-04-02T04:48:09.765 INFO:tasks.workunit.client.1.smithi179.stderr:+ rm -rf -- ./tmp.k9eQVjoRPQ 2024-04-02T04:48:09.789 INFO:tasks.workunit.client.0.smithi070.stdout:3/869: dread d10/d17/d51/d73/d8f/fab [0,4194304] 0 2024-04-02T04:48:09.790 INFO:tasks.workunit.client.0.smithi070.stdout:3/870: truncate d10/d66/f11d 412918 0 2024-04-02T04:48:09.790 INFO:tasks.workunit.client.0.smithi070.stdout:3/871: fdatasync d10/d32/f71 0 2024-04-02T04:48:09.792 INFO:tasks.workunit.client.0.smithi070.stdout:3/872: read d10/d32/f36 [3053973,97430] 0 2024-04-02T04:48:09.798 INFO:tasks.workunit.client.0.smithi070.stdout:3/873: symlink d10/d17/l123 0 2024-04-02T04:48:09.805 INFO:tasks.workunit.client.0.smithi070.stdout:3/874: fsync d10/d17/d4c/f62 0 2024-04-02T04:48:09.807 INFO:tasks.workunit.client.0.smithi070.stdout:3/875: truncate d10/d17/d4c/fda 558617 0 2024-04-02T04:48:09.878 INFO:tasks.workunit.client.0.smithi070.stdout:4/930: sync 2024-04-02T04:48:09.879 INFO:tasks.workunit.client.0.smithi070.stdout:8/862: sync 2024-04-02T04:48:09.892 INFO:tasks.workunit.client.0.smithi070.stdout:3/876: symlink d10/d17/d51/d73/l124 0 2024-04-02T04:48:09.897 INFO:tasks.workunit.client.0.smithi070.stdout:3/877: symlink d10/d17/d4c/d11e/d122/l125 0 2024-04-02T04:48:09.898 INFO:tasks.workunit.client.0.smithi070.stdout:4/931: mkdir d1/d3/d78/dad/dbd/d128/d136 0 2024-04-02T04:48:09.904 INFO:tasks.workunit.client.0.smithi070.stdout:8/863: dwrite d2/d8/f6d [0,4194304] 0 2024-04-02T04:48:09.905 INFO:tasks.workunit.client.0.smithi070.stdout:3/878: write d10/d17/db2/dbf/fef [284872,73378] 0 2024-04-02T04:48:09.906 INFO:tasks.workunit.client.0.smithi070.stdout:4/932: write d1/d3/d9/fb7 [699937,58659] 0 2024-04-02T04:48:09.911 INFO:tasks.workunit.client.0.smithi070.stdout:8/864: rename d2/d3/d35/d8e/f10d to d2/d3/d52/f113 0 2024-04-02T04:48:09.912 INFO:tasks.workunit.client.0.smithi070.stdout:4/933: rmdir d1/d3/d85/ddb 39 2024-04-02T04:48:09.917 INFO:tasks.workunit.client.0.smithi070.stdout:3/879: symlink d10/d17/d51/d73/d9e/d107/d10d/l126 0 2024-04-02T04:48:09.921 INFO:tasks.workunit.client.0.smithi070.stdout:3/880: mkdir d10/d32/d4b/dc5/d114/d127 0 2024-04-02T04:48:09.921 INFO:tasks.workunit.client.0.smithi070.stdout:4/934: dwrite d1/d3/d85/ddb/ffc [0,4194304] 0 2024-04-02T04:48:09.925 INFO:tasks.workunit.client.0.smithi070.stdout:8/865: write d2/d3/d35/d3a/d4f/f87 [3158853,106773] 0 2024-04-02T04:48:09.927 INFO:tasks.workunit.client.0.smithi070.stdout:4/935: dread d1/d3/d85/d8e/fda [0,4194304] 0 2024-04-02T04:48:09.928 INFO:tasks.workunit.client.0.smithi070.stdout:3/881: creat d10/d32/f128 x:0 0 0 2024-04-02T04:48:09.929 INFO:tasks.workunit.client.0.smithi070.stdout:3/882: dread - d10/d17/db2/dbf/f116 zero size 2024-04-02T04:48:09.929 INFO:tasks.workunit.client.0.smithi070.stdout:8/866: unlink d2/d8/d29/d22/d5a/d9e/cbc 0 2024-04-02T04:48:09.930 INFO:tasks.workunit.client.0.smithi070.stdout:3/883: chown d10/d32/d4b/dc5/fd0 61016 1 2024-04-02T04:48:09.932 INFO:tasks.workunit.client.0.smithi070.stdout:4/936: fsync d1/d3/d85/d8e/dc8/fd7 0 2024-04-02T04:48:09.937 INFO:tasks.workunit.client.0.smithi070.stdout:3/884: fsync d10/d17/f5d 0 2024-04-02T04:48:09.939 INFO:tasks.workunit.client.0.smithi070.stdout:8/867: creat d2/d8/d23/d24/f114 x:0 0 0 2024-04-02T04:48:09.941 INFO:tasks.workunit.client.0.smithi070.stdout:4/937: dwrite d1/d3/d9/f29 [0,4194304] 0 2024-04-02T04:48:09.954 INFO:tasks.workunit.client.0.smithi070.stdout:4/938: symlink d1/d3/l137 0 2024-04-02T04:48:09.954 INFO:tasks.workunit.client.0.smithi070.stdout:3/885: rename d10/d17/d51/d73/d9e/fe9 to d10/d32/d4b/dc5/d114/d127/f129 0 2024-04-02T04:48:09.962 INFO:tasks.workunit.client.0.smithi070.stdout:8/868: dwrite d2/d8/d29/db/fc5 [0,4194304] 0 2024-04-02T04:48:09.963 INFO:tasks.workunit.client.0.smithi070.stdout:8/869: chown d2/d3/fd 754064978 1 2024-04-02T04:48:09.976 INFO:tasks.workunit.client.0.smithi070.stdout:6/845: sync 2024-04-02T04:48:09.996 INFO:tasks.workunit.client.0.smithi070.stdout:4/939: mknod d1/d3/d85/d8e/dc8/c138 0 2024-04-02T04:48:09.997 INFO:tasks.workunit.client.0.smithi070.stdout:4/940: chown d1/d3/c5b 34918 1 2024-04-02T04:48:09.997 INFO:tasks.workunit.client.0.smithi070.stdout:3/886: truncate d10/d32/d3f/dc0/f100 130157 0 2024-04-02T04:48:09.999 INFO:tasks.workunit.client.0.smithi070.stdout:8/870: truncate d2/d8/d23/d24/d5f/fbf 119231 0 2024-04-02T04:48:10.006 INFO:tasks.workunit.client.0.smithi070.stdout:3/887: unlink d10/d17/d51/fb0 0 2024-04-02T04:48:10.007 INFO:tasks.workunit.client.0.smithi070.stdout:8/871: creat d2/d3/d52/d62/f115 x:0 0 0 2024-04-02T04:48:10.011 INFO:tasks.workunit.client.0.smithi070.stdout:3/888: symlink d10/d17/d51/d73/d8f/l12a 0 2024-04-02T04:48:10.014 INFO:tasks.workunit.client.0.smithi070.stdout:8/872: creat d2/d8/d29/d22/dc8/f116 x:0 0 0 2024-04-02T04:48:10.015 INFO:tasks.workunit.client.0.smithi070.stdout:4/941: getdents d1/d9c 0 2024-04-02T04:48:10.017 INFO:tasks.workunit.client.0.smithi070.stdout:3/889: symlink d10/d17/d51/d73/d9e/l12b 0 2024-04-02T04:48:10.018 INFO:tasks.workunit.client.0.smithi070.stdout:8/873: mknod d2/d8/d23/d24/d5f/dc2/c117 0 2024-04-02T04:48:10.024 INFO:tasks.workunit.client.0.smithi070.stdout:8/874: dwrite d2/d8/d29/d22/fef [0,4194304] 0 2024-04-02T04:48:10.040 INFO:tasks.workunit.client.0.smithi070.stdout:8/875: link d2/d8/d29/d22/c9f d2/d8/d23/c118 0 2024-04-02T04:48:10.042 INFO:tasks.workunit.client.0.smithi070.stdout:1/853: sync 2024-04-02T04:48:10.043 INFO:tasks.workunit.client.0.smithi070.stdout:9/908: sync 2024-04-02T04:48:10.043 INFO:tasks.workunit.client.0.smithi070.stdout:7/911: sync 2024-04-02T04:48:10.043 INFO:tasks.workunit.client.0.smithi070.stdout:2/981: sync 2024-04-02T04:48:10.046 INFO:tasks.workunit.client.0.smithi070.stdout:1/854: read d0/d9/d1d/d22/d56/fbf [5671781,56879] 0 2024-04-02T04:48:10.047 INFO:tasks.workunit.client.0.smithi070.stdout:1/855: chown d0/d7/d1a/c6b 213105197 1 2024-04-02T04:48:10.047 INFO:tasks.workunit.client.0.smithi070.stdout:7/912: fsync d7/dd/df/ffc 0 2024-04-02T04:48:10.049 INFO:tasks.workunit.client.0.smithi070.stdout:9/909: unlink d4/d10/d12/d56/l6e 0 2024-04-02T04:48:10.053 INFO:tasks.workunit.client.0.smithi070.stdout:1/856: mkdir d0/d7/d28/d5c/d10c 0 2024-04-02T04:48:10.054 INFO:tasks.workunit.client.0.smithi070.stdout:7/913: chown d7/dd/df/d7a/d6c/d77/d54/ld0 38 1 2024-04-02T04:48:10.055 INFO:tasks.workunit.client.0.smithi070.stdout:9/910: dread - d4/d10/d11/d7b/d19/d82/fbb zero size 2024-04-02T04:48:10.058 INFO:tasks.workunit.client.0.smithi070.stdout:2/982: link d2/d10/d5c/d7a/l56 d2/d10/d5c/d7a/db0/dc7/l13e 0 2024-04-02T04:48:10.059 INFO:tasks.workunit.client.0.smithi070.stdout:2/983: stat d2/d10/d5c/d7a/db0/dc7/ded/ff3 0 2024-04-02T04:48:10.060 INFO:tasks.workunit.client.0.smithi070.stdout:7/914: truncate d7/ff1 2801904 0 2024-04-02T04:48:10.062 INFO:tasks.workunit.client.0.smithi070.stdout:2/984: rmdir d2/d10/d5c 39 2024-04-02T04:48:10.065 INFO:tasks.workunit.client.0.smithi070.stdout:9/911: rmdir d4/d10/d12/d27/d60/dc9/d102 0 2024-04-02T04:48:10.071 INFO:tasks.workunit.client.0.smithi070.stdout:2/985: creat d2/d10/d5c/d10f/f13f x:0 0 0 2024-04-02T04:48:10.071 INFO:tasks.workunit.client.0.smithi070.stdout:6/846: dwrite d0/d3/d19/f47 [4194304,4194304] 0 2024-04-02T04:48:10.072 INFO:tasks.workunit.client.0.smithi070.stdout:2/986: dread - d2/d10/d19/dd7/f13b zero size 2024-04-02T04:48:10.073 INFO:tasks.workunit.client.0.smithi070.stdout:6/847: chown d0/d3/fec 566618 1 2024-04-02T04:48:10.075 INFO:tasks.workunit.client.0.smithi070.stdout:7/915: creat d7/f133 x:0 0 0 2024-04-02T04:48:10.075 INFO:tasks.workunit.client.0.smithi070.stdout:6/848: chown d0/d43/l3d 3 1 2024-04-02T04:48:10.079 INFO:tasks.workunit.client.0.smithi070.stdout:3/890: write d10/d17/d4c/f62 [2709898,124974] 0 2024-04-02T04:48:10.082 INFO:tasks.workunit.client.0.smithi070.stdout:4/942: dwrite d1/d3/fb2 [0,4194304] 0 2024-04-02T04:48:10.088 INFO:tasks.workunit.client.0.smithi070.stdout:2/987: chown d2/d10/lde 15759748 1 2024-04-02T04:48:10.095 INFO:tasks.workunit.client.0.smithi070.stdout:9/912: creat d4/d10/d11/d7b/d19/f12c x:0 0 0 2024-04-02T04:48:10.098 INFO:tasks.workunit.client.0.smithi070.stdout:2/988: dread - d2/d24/ff5 zero size 2024-04-02T04:48:10.101 INFO:tasks.workunit.client.0.smithi070.stdout:8/876: dwrite d2/d8/d29/d22/fe1 [0,4194304] 0 2024-04-02T04:48:10.106 INFO:tasks.workunit.client.0.smithi070.stdout:3/891: symlink d10/d17/db2/dbf/dd6/d120/l12c 0 2024-04-02T04:48:10.122 INFO:tasks.workunit.client.0.smithi070.stdout:3/892: rename d10/d66/c75 to d10/d32/dd1/c12d 0 2024-04-02T04:48:10.124 INFO:tasks.workunit.client.0.smithi070.stdout:7/916: dwrite d7/d36/fc6 [0,4194304] 0 2024-04-02T04:48:10.125 INFO:tasks.workunit.client.0.smithi070.stdout:3/893: truncate d10/d32/d3f/f10a 244187 0 2024-04-02T04:48:10.126 INFO:tasks.workunit.client.0.smithi070.stdout:6/849: getdents d0/d20/d5f/de6/da6 0 2024-04-02T04:48:10.128 INFO:tasks.workunit.client.0.smithi070.stdout:4/943: dwrite d1/d3/d9/f9e [4194304,4194304] 0 2024-04-02T04:48:10.129 INFO:tasks.workunit.client.0.smithi070.stdout:8/877: unlink d2/d3/d35/d3a/d4f/d59/f90 0 2024-04-02T04:48:10.129 INFO:tasks.workunit.client.0.smithi070.stdout:4/944: chown d1/d3/f39 6 1 2024-04-02T04:48:10.132 INFO:tasks.workunit.client.0.smithi070.stdout:2/989: write d2/d12/f38 [2202354,126206] 0 2024-04-02T04:48:10.138 INFO:tasks.workunit.client.0.smithi070.stdout:7/917: chown d7/dd/df/l3f 7199467 1 2024-04-02T04:48:10.141 INFO:tasks.workunit.client.0.smithi070.stdout:6/850: dread d0/d43/f9f [0,4194304] 0 2024-04-02T04:48:10.142 INFO:tasks.workunit.client.0.smithi070.stdout:4/945: mkdir d1/d3/d78/dad/dbd/d128/d139 0 2024-04-02T04:48:10.144 INFO:tasks.workunit.client.0.smithi070.stdout:2/990: symlink d2/d10/d5c/d7a/db0/dc7/ded/l140 0 2024-04-02T04:48:10.145 INFO:tasks.workunit.client.0.smithi070.stdout:6/851: dread d0/d43/d95/f9a [4194304,4194304] 0 2024-04-02T04:48:10.145 INFO:tasks.workunit.client.0.smithi070.stdout:6/852: chown d0/d20 16162 1 2024-04-02T04:48:10.149 INFO:tasks.workunit.client.0.smithi070.stdout:7/918: unlink d7/dd/df/d7a/d6c/d77/d54/d10c/l125 0 2024-04-02T04:48:10.154 INFO:tasks.workunit.client.0.smithi070.stdout:4/946: fsync d1/d3/d85/d8e/d99/db1/ffa 0 2024-04-02T04:48:10.157 INFO:tasks.workunit.client.0.smithi070.stdout:3/894: fsync d10/d66/d103/d111/ffa 0 2024-04-02T04:48:10.246 INFO:tasks.workunit.client.0.smithi070.stdout:7/919: symlink d7/d4e/d82/d57/l134 0 2024-04-02T04:48:10.251 INFO:tasks.workunit.client.0.smithi070.stdout:4/947: rename d1/d3/d85/d8e/fc6 to d1/d3/d85/d8e/f13a 0 2024-04-02T04:48:10.256 INFO:tasks.workunit.client.0.smithi070.stdout:2/991: write d2/d10/d21/d71/fdd [535260,92240] 0 2024-04-02T04:48:10.259 INFO:tasks.workunit.client.0.smithi070.stdout:4/948: mknod d1/d3/d85/deb/c13b 0 2024-04-02T04:48:10.261 INFO:tasks.workunit.client.0.smithi070.stdout:6/853: link d0/d20/d5f/cb8 d0/d3/dc2/cff 0 2024-04-02T04:48:10.264 INFO:tasks.workunit.client.0.smithi070.stdout:7/920: mknod d7/dd/df/d34/de3/c135 0 2024-04-02T04:48:10.266 INFO:tasks.workunit.client.0.smithi070.stdout:4/949: unlink d1/d3/d9/f81 0 2024-04-02T04:48:10.267 INFO:tasks.workunit.client.0.smithi070.stdout:4/950: fdatasync d1/d3/fb2 0 2024-04-02T04:48:10.268 INFO:tasks.workunit.client.0.smithi070.stdout:6/854: dwrite d0/d55/d5c/fda [0,4194304] 0 2024-04-02T04:48:10.275 INFO:tasks.workunit.client.0.smithi070.stdout:6/855: dwrite d0/ff1 [0,4194304] 0 2024-04-02T04:48:10.277 INFO:tasks.workunit.client.0.smithi070.stdout:3/895: rename d10/d17/l2d to d10/d17/l12e 0 2024-04-02T04:48:10.284 INFO:tasks.workunit.client.0.smithi070.stdout:2/992: link d2/d10/d19/ca2 d2/d10/d21/d3d/ddc/de8/d120/c141 0 2024-04-02T04:48:10.285 INFO:tasks.workunit.client.0.smithi070.stdout:2/993: chown d2/d10/d21/d71/dc1/cca 286 1 2024-04-02T04:48:10.286 INFO:tasks.workunit.client.0.smithi070.stdout:2/994: chown d2/d10/d21/d71/dd4/l73 1508 1 2024-04-02T04:48:10.290 INFO:tasks.workunit.client.0.smithi070.stdout:6/856: mkdir d0/d20/d100 0 2024-04-02T04:48:10.291 INFO:tasks.workunit.client.0.smithi070.stdout:6/857: stat d0/d3/l18 0 2024-04-02T04:48:10.293 INFO:tasks.workunit.client.0.smithi070.stdout:7/921: dread d7/dd/df/d7a/f64 [4194304,4194304] 0 2024-04-02T04:48:10.293 INFO:tasks.workunit.client.0.smithi070.stdout:3/896: read d10/d32/d8a/d8d/fc1 [1461855,1602] 0 2024-04-02T04:48:10.295 INFO:tasks.workunit.client.0.smithi070.stdout:3/897: dread d10/d66/f11d [0,4194304] 0 2024-04-02T04:48:10.297 INFO:tasks.workunit.client.0.smithi070.stdout:2/995: creat d2/d10/d5c/d10f/f142 x:0 0 0 2024-04-02T04:48:10.300 INFO:tasks.workunit.client.0.smithi070.stdout:7/922: mkdir d7/d4e/d136 0 2024-04-02T04:48:10.309 INFO:tasks.workunit.client.0.smithi070.stdout:6/858: symlink d0/d55/d79/l101 0 2024-04-02T04:48:10.314 INFO:tasks.workunit.client.0.smithi070.stdout:7/923: rename d7/dd/df/d7a/d6c/d77/fb0 to d7/dd/df/d7a/d6c/f137 0 2024-04-02T04:48:10.314 INFO:tasks.workunit.client.0.smithi070.stdout:7/924: chown d7/d4e/l121 1763 1 2024-04-02T04:48:10.319 INFO:tasks.workunit.client.0.smithi070.stdout:4/951: getdents d1/d3/d9/d5e 0 2024-04-02T04:48:10.324 INFO:tasks.workunit.client.0.smithi070.stdout:4/952: read d1/d3/d9/f2a [777288,118833] 0 2024-04-02T04:48:10.330 INFO:tasks.workunit.client.0.smithi070.stdout:2/996: link d2/d24/l79 d2/d10/d5c/d7a/db0/dc7/ded/l143 0 2024-04-02T04:48:10.330 INFO:tasks.workunit.client.0.smithi070.stdout:4/953: dwrite d1/d9c/f88 [0,4194304] 0 2024-04-02T04:48:10.331 INFO:tasks.workunit.client.0.smithi070.stdout:2/997: stat d2/d10/d5c/d7a/db0/c105 0 2024-04-02T04:48:10.336 INFO:tasks.workunit.client.0.smithi070.stdout:4/954: rename d1/d3/d85/d8e/cde to d1/dc3/dc4/c13c 0 2024-04-02T04:48:10.338 INFO:tasks.workunit.client.0.smithi070.stdout:3/898: write d10/d17/db2/fff [500358,16236] 0 2024-04-02T04:48:10.341 INFO:tasks.workunit.client.0.smithi070.stdout:4/955: write d1/d3/d85/d8e/d99/db1/dbb/fff [1023393,119321] 0 2024-04-02T04:48:10.342 INFO:tasks.workunit.client.0.smithi070.stdout:7/925: getdents d7/d4e/d82/d57/dce 0 2024-04-02T04:48:10.343 INFO:tasks.workunit.client.0.smithi070.stdout:4/956: read d1/d9c/f48 [3589674,69728] 0 2024-04-02T04:48:10.345 INFO:tasks.workunit.client.0.smithi070.stdout:2/998: link d2/d24/l91 d2/d10/d5c/d7a/db0/de4/l144 0 2024-04-02T04:48:10.346 INFO:tasks.workunit.client.0.smithi070.stdout:7/926: creat d7/d4e/d82/d57/dce/f138 x:0 0 0 2024-04-02T04:48:10.350 INFO:tasks.workunit.client.0.smithi070.stdout:3/899: unlink f0 0 2024-04-02T04:48:10.351 INFO:tasks.workunit.client.0.smithi070.stdout:2/999: dwrite d2/d10/d21/d71/dd4/f82 [0,4194304] 0 2024-04-02T04:48:10.355 INFO:tasks.workunit.client.0.smithi070.stdout:6/859: truncate d0/d3/d19/f47 2813057 0 2024-04-02T04:48:10.356 INFO:tasks.workunit.client.0.smithi070.stdout:3/900: mknod d10/d17/d51/d73/d9e/d107/d10d/c12f 0 2024-04-02T04:48:10.361 INFO:tasks.workunit.client.0.smithi070.stdout:6/860: mknod d0/d20/d5f/c102 0 2024-04-02T04:48:10.362 INFO:tasks.workunit.client.0.smithi070.stdout:4/957: getdents d1/d3/d78 0 2024-04-02T04:48:10.364 INFO:tasks.workunit.client.0.smithi070.stdout:3/901: mkdir d10/d32/d4b/dc5/d130 0 2024-04-02T04:48:10.366 INFO:tasks.workunit.client.0.smithi070.stdout:6/861: dread d0/d20/f34 [0,4194304] 0 2024-04-02T04:48:10.367 INFO:tasks.workunit.client.0.smithi070.stdout:6/862: dread - d0/d43/d80/ded/ffd zero size 2024-04-02T04:48:10.367 INFO:tasks.workunit.client.0.smithi070.stdout:4/958: readlink d1/l9d 0 2024-04-02T04:48:10.372 INFO:tasks.workunit.client.0.smithi070.stdout:7/927: getdents d7/d8e/d123 0 2024-04-02T04:48:10.373 INFO:tasks.workunit.client.0.smithi070.stdout:3/902: dread d10/d32/d8a/d8d/d91/f9f [0,4194304] 0 2024-04-02T04:48:10.374 INFO:tasks.workunit.client.0.smithi070.stdout:1/857: sync 2024-04-02T04:48:10.381 INFO:tasks.workunit.client.0.smithi070.stdout:7/928: mknod d7/d8e/c139 0 2024-04-02T04:48:10.382 INFO:tasks.workunit.client.0.smithi070.stdout:7/929: write d7/d36/f120 [478671,76891] 0 2024-04-02T04:48:10.392 INFO:tasks.workunit.client.0.smithi070.stdout:1/858: creat d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/f10d x:0 0 0 2024-04-02T04:48:10.396 INFO:tasks.workunit.client.0.smithi070.stdout:6/863: symlink d0/dfe/l103 0 2024-04-02T04:48:10.397 INFO:tasks.workunit.client.0.smithi070.stdout:6/864: chown d0/d43/db3/dc7/le8 56048555 1 2024-04-02T04:48:10.397 INFO:tasks.workunit.client.0.smithi070.stdout:3/903: dwrite d10/d17/db2/ffb [0,4194304] 0 2024-04-02T04:48:10.399 INFO:tasks.workunit.client.0.smithi070.stdout:6/865: dread - d0/d3/d19/d31/fdd zero size 2024-04-02T04:48:10.399 INFO:tasks.workunit.client.0.smithi070.stdout:6/866: chown d0/d20/c56 159177 1 2024-04-02T04:48:10.402 INFO:tasks.workunit.client.0.smithi070.stdout:4/959: creat d1/d9c/d118/f13d x:0 0 0 2024-04-02T04:48:10.403 INFO:tasks.workunit.client.0.smithi070.stdout:3/904: dwrite d10/d32/dd1/fdf [0,4194304] 0 2024-04-02T04:48:10.404 INFO:tasks.workunit.client.0.smithi070.stdout:3/905: readlink d10/d32/l58 0 2024-04-02T04:48:10.405 INFO:tasks.workunit.client.0.smithi070.stdout:7/930: creat d7/dd/df/dca/f13a x:0 0 0 2024-04-02T04:48:10.407 INFO:tasks.workunit.client.0.smithi070.stdout:7/931: fsync d7/dd/df/d7a/d6c/d77/d54/de5/feb 0 2024-04-02T04:48:10.416 INFO:tasks.workunit.client.0.smithi070.stdout:6/867: write d0/d3/d19/d31/d46/d6b/f70 [3250252,18649] 0 2024-04-02T04:48:10.418 INFO:tasks.workunit.client.0.smithi070.stdout:6/868: write d0/d43/db3/dc7/f88 [9591241,107544] 0 2024-04-02T04:48:10.418 INFO:tasks.workunit.client.0.smithi070.stdout:4/960: mknod d1/d3/d85/d8e/d99/db1/de9/c13e 0 2024-04-02T04:48:10.422 INFO:tasks.workunit.client.0.smithi070.stdout:3/906: symlink d10/d32/d4b/dc5/d114/l131 0 2024-04-02T04:48:10.424 INFO:tasks.workunit.client.0.smithi070.stdout:4/961: dwrite d1/d3/d85/d8e/fd9 [0,4194304] 0 2024-04-02T04:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout:7/932: mkdir d7/d36/d13b 0 2024-04-02T04:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout:4/962: fsync d1/d9c/f88 0 2024-04-02T04:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout:8/878: sync 2024-04-02T04:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout:9/913: sync 2024-04-02T04:48:10.431 INFO:tasks.workunit.client.0.smithi070.stdout:1/859: dwrite d0/d7/f10 [4194304,4194304] 0 2024-04-02T04:48:10.436 INFO:tasks.workunit.client.0.smithi070.stdout:3/907: rmdir d10/d32/d4b/dc5 39 2024-04-02T04:48:10.441 INFO:tasks.workunit.client.0.smithi070.stdout:9/914: dread d4/d10/d12/d27/d60/f10c [4194304,4194304] 0 2024-04-02T04:48:10.444 INFO:tasks.workunit.client.0.smithi070.stdout:8/879: symlink d2/d8/d29/d22/d5a/d94/de6/l119 0 2024-04-02T04:48:10.450 INFO:tasks.workunit.client.0.smithi070.stdout:6/869: mkdir d0/d20/d77/d98/dd5/d104 0 2024-04-02T04:48:10.453 INFO:tasks.workunit.client.0.smithi070.stdout:3/908: fsync d10/d32/d4b/f115 0 2024-04-02T04:48:10.454 INFO:tasks.workunit.client.0.smithi070.stdout:3/909: stat d10/d32/d4b/l101 0 2024-04-02T04:48:10.455 INFO:tasks.workunit.client.0.smithi070.stdout:1/860: write d0/db4/fd6 [400989,33923] 0 2024-04-02T04:48:10.456 INFO:tasks.workunit.client.0.smithi070.stdout:1/861: chown d0/d7/d28/d5c 82154196 1 2024-04-02T04:48:10.457 INFO:tasks.workunit.client.0.smithi070.stdout:1/862: fsync d0/db4/fd6 0 2024-04-02T04:48:10.458 INFO:tasks.workunit.client.0.smithi070.stdout:1/863: stat d0/d54/d64/cca 0 2024-04-02T04:48:10.459 INFO:tasks.workunit.client.0.smithi070.stdout:1/864: chown d0/d54/d64/d6d/db6/fe7 7 1 2024-04-02T04:48:10.460 INFO:tasks.workunit.client.0.smithi070.stdout:1/865: fdatasync d0/d9/d1d/d22/d2c/d4f/f92 0 2024-04-02T04:48:10.462 INFO:tasks.workunit.client.0.smithi070.stdout:8/880: mkdir d2/d3/d52/d62/d7c/d11a 0 2024-04-02T04:48:10.463 INFO:tasks.workunit.client.0.smithi070.stdout:9/915: write d4/d10/d12/d55/d66/f6d [274109,77480] 0 2024-04-02T04:48:10.469 INFO:tasks.workunit.client.0.smithi070.stdout:4/963: creat d1/d3/d9/f13f x:0 0 0 2024-04-02T04:48:10.473 INFO:tasks.workunit.client.0.smithi070.stdout:8/881: mkdir d2/d3/d35/d8e/d11b 0 2024-04-02T04:48:10.474 INFO:tasks.workunit.client.0.smithi070.stdout:8/882: stat d2/d8/d23/d24/d5f/dc2/fd7 0 2024-04-02T04:48:10.477 INFO:tasks.workunit.client.0.smithi070.stdout:3/910: mknod d10/c132 0 2024-04-02T04:48:10.477 INFO:tasks.workunit.client.0.smithi070.stdout:7/933: link d7/dd/cc3 d7/d8e/d123/d130/c13c 0 2024-04-02T04:48:10.479 INFO:tasks.workunit.client.0.smithi070.stdout:7/934: write d7/dd/df/d34/f12b [884984,126] 0 2024-04-02T04:48:10.480 INFO:tasks.workunit.client.0.smithi070.stdout:7/935: fsync d7/d8e/d123/d65/d7b/fb8 0 2024-04-02T04:48:10.481 INFO:tasks.workunit.client.0.smithi070.stdout:1/866: mkdir d0/d9/d1d/d22/d56/d79/d10e 0 2024-04-02T04:48:10.481 INFO:tasks.workunit.client.0.smithi070.stdout:7/936: truncate d7/d36/f120 1531937 0 2024-04-02T04:48:10.485 INFO:tasks.workunit.client.0.smithi070.stdout:8/883: fsync d2/d8/d23/d24/d79/f4e 0 2024-04-02T04:48:10.488 INFO:tasks.workunit.client.0.smithi070.stdout:3/911: fdatasync d10/d17/d51/d73/d9e/dd5/fd8 0 2024-04-02T04:48:10.491 INFO:tasks.workunit.client.0.smithi070.stdout:9/916: dwrite d4/d10/f2c [0,4194304] 0 2024-04-02T04:48:10.504 INFO:tasks.workunit.client.0.smithi070.stdout:8/884: rmdir d2/d8/d29/d22/d5a/d94/da2/dde 39 2024-04-02T04:48:10.508 INFO:tasks.workunit.client.0.smithi070.stdout:9/917: fsync d4/d10/d11/f58 0 2024-04-02T04:48:10.509 INFO:tasks.workunit.client.0.smithi070.stdout:9/918: stat d4/d10/d12/d27/d60/d9e/f77 0 2024-04-02T04:48:10.511 INFO:tasks.workunit.client.0.smithi070.stdout:9/919: write d4/d34/db9/fe8 [706916,19657] 0 2024-04-02T04:48:10.512 INFO:tasks.workunit.client.0.smithi070.stdout:1/867: symlink d0/db4/d9f/l10f 0 2024-04-02T04:48:10.518 INFO:tasks.workunit.client.0.smithi070.stdout:8/885: rename d2/d8/d29/d22/f46 to d2/d8/d23/d24/d66/f11c 0 2024-04-02T04:48:10.528 INFO:tasks.workunit.client.0.smithi070.stdout:9/920: symlink d4/d10/d12/d55/d66/l12d 0 2024-04-02T04:48:10.531 INFO:tasks.workunit.client.0.smithi070.stdout:1/868: mknod d0/d9/d1d/d22/d2c/d4f/d77/d8c/c110 0 2024-04-02T04:48:10.534 INFO:tasks.workunit.client.0.smithi070.stdout:3/912: creat d10/f133 x:0 0 0 2024-04-02T04:48:10.536 INFO:tasks.workunit.client.0.smithi070.stdout:7/937: link d7/l14 d7/d4e/d82/d57/ddb/l13d 0 2024-04-02T04:48:10.536 INFO:tasks.workunit.client.0.smithi070.stdout:7/938: dread - d7/dd/df/d7a/fc4 zero size 2024-04-02T04:48:10.539 INFO:tasks.workunit.client.0.smithi070.stdout:8/886: rmdir d2/d8/d29/d22/d5a/d94/da2/dde 39 2024-04-02T04:48:10.551 INFO:tasks.workunit.client.0.smithi070.stdout:8/887: symlink d2/d8/d23/l11d 0 2024-04-02T04:48:10.552 INFO:tasks.workunit.client.0.smithi070.stdout:7/939: truncate d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/fa 963816 0 2024-04-02T04:48:10.554 INFO:tasks.workunit.client.0.smithi070.stdout:7/940: chown d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/de4 166666 1 2024-04-02T04:48:10.556 INFO:tasks.workunit.client.0.smithi070.stdout:3/913: creat d10/d32/d4b/dc5/d11c/f134 x:0 0 0 2024-04-02T04:48:10.557 INFO:tasks.workunit.client.0.smithi070.stdout:1/869: getdents d0/d9/d1d/d22/d2c/de3 0 2024-04-02T04:48:10.558 INFO:tasks.workunit.client.0.smithi070.stdout:8/888: rmdir d2 39 2024-04-02T04:48:10.561 INFO:tasks.workunit.client.0.smithi070.stdout:7/941: creat d7/d8e/d123/d65/f13e x:0 0 0 2024-04-02T04:48:10.566 INFO:tasks.workunit.client.0.smithi070.stdout:9/921: truncate d4/d10/d11/d7b/d19/d53/f59 5712481 0 2024-04-02T04:48:10.568 INFO:tasks.workunit.client.0.smithi070.stdout:9/922: fsync d4/d10/d11/d7b/f11a 0 2024-04-02T04:48:10.570 INFO:tasks.workunit.client.0.smithi070.stdout:3/914: dread d10/d32/f71 [0,4194304] 0 2024-04-02T04:48:10.571 INFO:tasks.workunit.client.0.smithi070.stdout:7/942: mknod d7/d4e/d82/d57/ddc/d115/c13f 0 2024-04-02T04:48:10.571 INFO:tasks.workunit.client.0.smithi070.stdout:8/889: dwrite d2/d8/d29/d22/fdd [0,4194304] 0 2024-04-02T04:48:10.579 INFO:tasks.workunit.client.0.smithi070.stdout:3/915: mknod d10/d32/dd1/c135 0 2024-04-02T04:48:10.584 INFO:tasks.workunit.client.0.smithi070.stdout:8/890: unlink d2/d3/ca1 0 2024-04-02T04:48:10.585 INFO:tasks.workunit.client.0.smithi070.stdout:1/870: dwrite d0/d7/d1a/d66/f88 [0,4194304] 0 2024-04-02T04:48:10.587 INFO:tasks.workunit.client.0.smithi070.stdout:7/943: creat d7/d4e/d82/d57/ddc/d115/f140 x:0 0 0 2024-04-02T04:48:10.596 INFO:tasks.workunit.client.0.smithi070.stdout:7/944: dwrite d7/dd/df/d7a/d6c/f9b [0,4194304] 0 2024-04-02T04:48:10.597 INFO:tasks.workunit.client.0.smithi070.stdout:9/923: dwrite d4/d10/d12/f62 [4194304,4194304] 0 2024-04-02T04:48:10.601 INFO:tasks.workunit.client.0.smithi070.stdout:8/891: mkdir d2/d8/d23/d24/d11e 0 2024-04-02T04:48:10.602 INFO:tasks.workunit.client.0.smithi070.stdout:8/892: chown d2/d3/d35/d8e/d11b 600128 1 2024-04-02T04:48:10.602 INFO:tasks.workunit.client.0.smithi070.stdout:8/893: readlink d2/d8/d23/l11d 0 2024-04-02T04:48:10.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.606+0000 7f87f06dc700 1 -- 172.21.15.70:0/1391848988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e8101ea0 msgr2=0x7f87e8104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:10.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.606+0000 7f87f06dc700 1 --2- 172.21.15.70:0/1391848988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e8101ea0 0x7f87e8104330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009910 tx=0x7f87d8009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:10.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.606+0000 7f87f06dc700 1 -- 172.21.15.70:0/1391848988 shutdown_connections 2024-04-02T04:48:10.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.606+0000 7f87f06dc700 1 --2- 172.21.15.70:0/1391848988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e8101ea0 0x7f87e8104330 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:10.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.606+0000 7f87f06dc700 1 --2- 172.21.15.70:0/1391848988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80ff530 0x7f87e8101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:10.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.606+0000 7f87f06dc700 1 -- 172.21.15.70:0/1391848988 >> 172.21.15.70:0/1391848988 conn(0x7f87e80fb150 msgr2=0x7f87e80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:10.610 INFO:tasks.workunit.client.0.smithi070.stdout:7/945: symlink d7/dd/df/d106/l141 0 2024-04-02T04:48:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.607+0000 7f87f06dc700 1 -- 172.21.15.70:0/1391848988 shutdown_connections 2024-04-02T04:48:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.607+0000 7f87f06dc700 1 -- 172.21.15.70:0/1391848988 wait complete. 2024-04-02T04:48:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87f06dc700 1 Processor -- start 2024-04-02T04:48:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87f06dc700 1 -- start start 2024-04-02T04:48:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87f06dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80ff530 0x7f87e819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87f06dc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e8101ea0 0x7f87e819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87f06dc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e819d600 con 0x7f87e8101ea0 2024-04-02T04:48:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87f06dc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e819d740 con 0x7f87e80ff530 2024-04-02T04:48:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87ee478700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80ff530 0x7f87e819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87ee478700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80ff530 0x7f87e819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:51186/0 (socket says 172.21.15.70:51186) 2024-04-02T04:48:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87ee478700 1 -- 172.21.15.70:0/957984781 learned_addr learned my addr 172.21.15.70:0/957984781 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.608+0000 7f87edc77700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e8101ea0 0x7f87e819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.609+0000 7f87ee478700 1 -- 172.21.15.70:0/957984781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e8101ea0 msgr2=0x7f87e819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.609+0000 7f87ee478700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e8101ea0 0x7f87e819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.609+0000 7f87ee478700 1 -- 172.21.15.70:0/957984781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80095f0 con 0x7f87e80ff530 2024-04-02T04:48:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.609+0000 7f87ee478700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80ff530 0x7f87e819caa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87e400d900 tx=0x7f87e400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.609+0000 7f87df7fe700 1 -- 172.21.15.70:0/957984781 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e4004a00 con 0x7f87e80ff530 2024-04-02T04:48:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.609+0000 7f87df7fe700 1 -- 172.21.15.70:0/957984781 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f87e40054d0 con 0x7f87e80ff530 2024-04-02T04:48:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.609+0000 7f87df7fe700 1 -- 172.21.15.70:0/957984781 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e4009d00 con 0x7f87e80ff530 2024-04-02T04:48:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.610+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e81a21f0 con 0x7f87e80ff530 2024-04-02T04:48:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.610+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e81a26f0 con 0x7f87e80ff530 2024-04-02T04:48:10.613 INFO:tasks.workunit.client.0.smithi070.stdout:3/916: rename d10/d32/d8a/d8d/cae to d10/d32/d8a/c136 0 2024-04-02T04:48:10.614 INFO:tasks.workunit.client.0.smithi070.stdout:8/894: truncate d2/d3/f49 1989446 0 2024-04-02T04:48:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.612+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e8196c30 con 0x7f87e80ff530 2024-04-02T04:48:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.612+0000 7f87df7fe700 1 -- 172.21.15.70:0/957984781 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f87e4005020 con 0x7f87e80ff530 2024-04-02T04:48:10.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.613+0000 7f87df7fe700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f87d406d0a0 0x7f87d406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:10.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.613+0000 7f87df7fe700 1 -- 172.21.15.70:0/957984781 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f87e408c880 con 0x7f87e80ff530 2024-04-02T04:48:10.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.613+0000 7f87edc77700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f87d406d0a0 0x7f87d406f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:10.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.614+0000 7f87edc77700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f87d406d0a0 0x7f87d406f560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f87d8006010 tx=0x7f87d800b540 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:10.617 INFO:tasks.workunit.client.0.smithi070.stdout:9/924: fdatasync d4/d10/d11/d7b/d19/f54 0 2024-04-02T04:48:10.617 INFO:tasks.workunit.client.0.smithi070.stdout:7/946: read d7/dd/df/d79/f5e [1511559,27962] 0 2024-04-02T04:48:10.619 INFO:tasks.workunit.client.0.smithi070.stdout:8/895: fdatasync d2/d8/d29/d22/d5a/d94/fc0 0 2024-04-02T04:48:10.620 INFO:tasks.workunit.client.0.smithi070.stdout:8/896: chown d2/d3/d35/d3a/d4f/de8 547778 1 2024-04-02T04:48:10.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:10.618+0000 7f87df7fe700 1 -- 172.21.15.70:0/957984781 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f87e4059db0 con 0x7f87e80ff530 2024-04-02T04:48:10.629 INFO:tasks.workunit.client.0.smithi070.stdout:1/871: dwrite d0/d7/f6a [0,4194304] 0 2024-04-02T04:48:10.632 INFO:tasks.workunit.client.0.smithi070.stdout:8/897: creat d2/d8/d29/d22/d5a/d94/f11f x:0 0 0 2024-04-02T04:48:10.632 INFO:tasks.workunit.client.0.smithi070.stdout:3/917: creat d10/d32/d8a/d8d/f137 x:0 0 0 2024-04-02T04:48:10.638 INFO:tasks.workunit.client.0.smithi070.stdout:9/925: dread d4/d10/d11/d7b/d19/d2f/d39/f44 [0,4194304] 0 2024-04-02T04:48:10.638 INFO:tasks.workunit.client.0.smithi070.stdout:9/926: readlink d4/d13/l3e 0 2024-04-02T04:48:10.640 INFO:tasks.workunit.client.0.smithi070.stdout:1/872: mknod d0/d9/d1d/d22/d2c/d4f/d77/c111 0 2024-04-02T04:48:10.641 INFO:tasks.workunit.client.0.smithi070.stdout:3/918: truncate d10/d32/d3f/f38 4650072 0 2024-04-02T04:48:10.643 INFO:tasks.workunit.client.0.smithi070.stdout:8/898: rename d2/d8/f44 to d2/d3/d52/d62/d7c/f120 0 2024-04-02T04:48:10.649 INFO:tasks.workunit.client.0.smithi070.stdout:1/873: dwrite d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/dea/ff0 [4194304,4194304] 0 2024-04-02T04:48:10.652 INFO:tasks.workunit.client.0.smithi070.stdout:8/899: unlink d2/d8/d29/d22/d5a/d9e/cdc 0 2024-04-02T04:48:10.655 INFO:tasks.workunit.client.0.smithi070.stdout:3/919: fdatasync d10/d66/d103/f11f 0 2024-04-02T04:48:10.655 INFO:tasks.workunit.client.0.smithi070.stdout:7/947: dwrite d7/dd/df/d79/fbf [0,4194304] 0 2024-04-02T04:48:10.655 INFO:tasks.workunit.client.0.smithi070.stdout:8/900: rename d2/d8 to d2/d8/d29/d72/d121 22 2024-04-02T04:48:10.658 INFO:tasks.workunit.client.0.smithi070.stdout:8/901: stat d2/d3/d35/d3a/d4f/d59/c6c 0 2024-04-02T04:48:10.664 INFO:tasks.workunit.client.0.smithi070.stdout:7/948: mkdir d7/d4e/d82/d57/ddc/d115/d142 0 2024-04-02T04:48:10.667 INFO:tasks.workunit.client.0.smithi070.stdout:8/902: dwrite d2/d8/d23/d24/d5f/dc2/fd7 [0,4194304] 0 2024-04-02T04:48:10.670 INFO:tasks.workunit.client.0.smithi070.stdout:9/927: dwrite d4/d10/d11/d7b/d19/d82/fc4 [0,4194304] 0 2024-04-02T04:48:10.670 INFO:tasks.workunit.client.0.smithi070.stdout:7/949: fdatasync d7/d8e/fa6 0 2024-04-02T04:48:10.673 INFO:tasks.workunit.client.0.smithi070.stdout:7/950: dread - d7/dd/df/d7a/d6c/d77/d54/f11b zero size 2024-04-02T04:48:10.673 INFO:tasks.workunit.client.0.smithi070.stdout:9/928: chown d4/d10/d11/d7b/d19/d53/l76 3583 1 2024-04-02T04:48:10.676 INFO:tasks.workunit.client.0.smithi070.stdout:7/951: readlink d7/dd/df/d7a/d6c/d77/l7f 0 2024-04-02T04:48:10.684 INFO:tasks.workunit.client.0.smithi070.stdout:1/874: dwrite d0/d7/d28/f7a [4194304,4194304] 0 2024-04-02T04:48:10.684 INFO:tasks.workunit.client.0.smithi070.stdout:3/920: dread d10/d32/d3f/f65 [0,4194304] 0 2024-04-02T04:48:10.686 INFO:tasks.workunit.client.0.smithi070.stdout:7/952: symlink d7/dd/df/dca/l143 0 2024-04-02T04:48:10.686 INFO:tasks.workunit.client.0.smithi070.stdout:8/903: mknod d2/d8/d29/dc7/df0/d10e/c122 0 2024-04-02T04:48:10.690 INFO:tasks.workunit.client.0.smithi070.stdout:9/929: creat d4/d10/d11/d7b/d19/d2f/d39/f12e x:0 0 0 2024-04-02T04:48:10.690 INFO:tasks.workunit.client.0.smithi070.stdout:9/930: fdatasync d4/d10/d11/d7b/f123 0 2024-04-02T04:48:10.699 INFO:tasks.workunit.client.0.smithi070.stdout:8/904: rmdir d2/d3/d35 39 2024-04-02T04:48:10.701 INFO:tasks.workunit.client.0.smithi070.stdout:1/875: rename d0/d54/d64/cca to d0/d7/d28/c112 0 2024-04-02T04:48:10.702 INFO:tasks.workunit.client.0.smithi070.stdout:1/876: fdatasync d0/d7/d1a/db9/ff8 0 2024-04-02T04:48:10.704 INFO:tasks.workunit.client.0.smithi070.stdout:9/931: mkdir d4/d10/d11/d7b/d19/d2f/d39/d116/d12f 0 2024-04-02T04:48:10.706 INFO:tasks.workunit.client.0.smithi070.stdout:3/921: write d10/d32/d3f/ff9 [346858,46579] 0 2024-04-02T04:48:10.708 INFO:tasks.workunit.client.0.smithi070.stdout:7/953: write d7/dd/df/d34/d68/f70 [2646700,12463] 0 2024-04-02T04:48:10.712 INFO:tasks.workunit.client.0.smithi070.stdout:3/922: dread d10/d17/db2/ffb [0,4194304] 0 2024-04-02T04:48:10.715 INFO:tasks.workunit.client.0.smithi070.stdout:9/932: rmdir d4/d10/d12/d27/d68 39 2024-04-02T04:48:10.716 INFO:tasks.workunit.client.0.smithi070.stdout:9/933: stat d4/d10/d11/d7b/d19/d53/cfa 0 2024-04-02T04:48:10.717 INFO:tasks.workunit.client.0.smithi070.stdout:8/905: write d2/d8/d23/d24/d79/f4e [4424145,34252] 0 2024-04-02T04:48:10.718 INFO:tasks.workunit.client.0.smithi070.stdout:7/954: mknod d7/d4e/d82/d57/dce/c144 0 2024-04-02T04:48:10.720 INFO:tasks.workunit.client.0.smithi070.stdout:3/923: write d10/d66/fb9 [292208,27082] 0 2024-04-02T04:48:10.725 INFO:tasks.workunit.client.0.smithi070.stdout:8/906: chown d2/d3/d35/d3a/d4f/d59/c6c 1 1 2024-04-02T04:48:10.726 INFO:tasks.workunit.client.0.smithi070.stdout:7/955: dwrite d7/dd/df/d7a/dbc/ff9 [0,4194304] 0 2024-04-02T04:48:10.728 INFO:tasks.workunit.client.0.smithi070.stdout:1/877: rename d0/d9/d1d/d22/d2c/f76 to d0/d54/f113 0 2024-04-02T04:48:10.729 INFO:tasks.workunit.client.0.smithi070.stdout:8/907: truncate d2/d8/d29/d22/d5a/d94/f11f 298479 0 2024-04-02T04:48:10.733 INFO:tasks.workunit.client.0.smithi070.stdout:7/956: chown d7/c9a 0 1 2024-04-02T04:48:10.735 INFO:tasks.workunit.client.0.smithi070.stdout:1/878: fdatasync d0/d7/d1a/f75 0 2024-04-02T04:48:10.735 INFO:tasks.workunit.client.0.smithi070.stdout:9/934: dread d4/d10/d12/d55/d66/f7c [0,4194304] 0 2024-04-02T04:48:10.736 INFO:tasks.workunit.client.0.smithi070.stdout:3/924: mkdir d10/d32/d4b/dc5/d130/d138 0 2024-04-02T04:48:10.740 INFO:tasks.workunit.client.0.smithi070.stdout:7/957: mkdir d7/dd/df/d7a/d6c/d145 0 2024-04-02T04:48:10.741 INFO:tasks.workunit.client.0.smithi070.stdout:1/879: creat d0/d4c/da2/f114 x:0 0 0 2024-04-02T04:48:10.743 INFO:tasks.workunit.client.0.smithi070.stdout:3/925: dread d10/d17/d4c/d11e/d122/fdd [0,4194304] 0 2024-04-02T04:48:10.745 INFO:tasks.workunit.client.0.smithi070.stdout:8/908: link d2/d3/d35/d3a/d4f/f87 d2/d3/d52/d62/f123 0 2024-04-02T04:48:10.746 INFO:tasks.workunit.client.0.smithi070.stdout:9/935: rmdir d4/d10/d12/d27/d83 39 2024-04-02T04:48:10.748 INFO:tasks.workunit.client.0.smithi070.stdout:7/958: creat d7/dd/df/d79/de6/f146 x:0 0 0 2024-04-02T04:48:10.750 INFO:tasks.workunit.client.0.smithi070.stdout:8/909: truncate d2/d8/d29/db/f8b 431466 0 2024-04-02T04:48:10.751 INFO:tasks.workunit.client.0.smithi070.stdout:8/910: stat d2/cb1 0 2024-04-02T04:48:10.753 INFO:tasks.workunit.client.0.smithi070.stdout:1/880: mknod d0/d7/d28/d5c/d10c/c115 0 2024-04-02T04:48:10.758 INFO:tasks.workunit.client.0.smithi070.stdout:3/926: creat d10/d32/d4b/dc5/d130/d138/f139 x:0 0 0 2024-04-02T04:48:10.759 INFO:tasks.workunit.client.0.smithi070.stdout:3/927: fsync d10/d17/d4c/f62 0 2024-04-02T04:48:10.763 INFO:tasks.workunit.client.0.smithi070.stdout:7/959: truncate d7/dd/db1/fe0 832264 0 2024-04-02T04:48:10.764 INFO:tasks.workunit.client.0.smithi070.stdout:6/870: sync 2024-04-02T04:48:10.764 INFO:tasks.workunit.client.0.smithi070.stdout:4/964: sync 2024-04-02T04:48:10.765 INFO:tasks.workunit.client.0.smithi070.stdout:9/936: write d4/f106 [867524,116316] 0 2024-04-02T04:48:10.765 INFO:tasks.workunit.client.0.smithi070.stdout:7/960: read - d7/d4e/d82/d57/ddb/f110 zero size 2024-04-02T04:48:10.769 INFO:tasks.workunit.client.0.smithi070.stdout:1/881: link d0/d54/d64/d6d/ddd/f10b d0/d54/d64/d6d/ddd/f116 0 2024-04-02T04:48:10.773 INFO:tasks.workunit.client.0.smithi070.stdout:9/937: dwrite d4/d10/d11/d7b/d19/f125 [0,4194304] 0 2024-04-02T04:48:10.775 INFO:tasks.workunit.client.0.smithi070.stdout:7/961: symlink d7/d8e/dec/l147 0 2024-04-02T04:48:10.775 INFO:tasks.workunit.client.0.smithi070.stdout:7/962: fdatasync d7/dd/df/d7a/dbc/f11d 0 2024-04-02T04:48:10.776 INFO:tasks.workunit.client.0.smithi070.stdout:7/963: chown d7/d4e/l90 702 1 2024-04-02T04:48:10.778 INFO:tasks.workunit.client.0.smithi070.stdout:7/964: fdatasync d7/dd/df/d7a/d6c/d77/d54/f112 0 2024-04-02T04:48:10.781 INFO:tasks.workunit.client.0.smithi070.stdout:6/871: rename d0/d20/d77/d98/dd5/d104 to d0/d20/d5f/d105 0 2024-04-02T04:48:10.781 INFO:tasks.workunit.client.0.smithi070.stdout:3/928: write d10/d17/db2/ffb [474459,81813] 0 2024-04-02T04:48:10.782 INFO:tasks.workunit.client.0.smithi070.stdout:6/872: readlink d0/d55/d8b/d94/l63 0 2024-04-02T04:48:10.784 INFO:tasks.workunit.client.0.smithi070.stdout:3/929: read - d10/d32/d4b/dc5/d130/d138/f139 zero size 2024-04-02T04:48:10.786 INFO:tasks.workunit.client.0.smithi070.stdout:8/911: getdents d2/d3/d35 0 2024-04-02T04:48:10.788 INFO:tasks.workunit.client.0.smithi070.stdout:1/882: mknod d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/c117 0 2024-04-02T04:48:10.791 INFO:tasks.workunit.client.0.smithi070.stdout:9/938: fsync d4/d10/d11/d7b/d19/d2f/d8c/fd0 0 2024-04-02T04:48:10.796 INFO:tasks.workunit.client.0.smithi070.stdout:4/965: write d1/d3/d9/f53 [1646885,84563] 0 2024-04-02T04:48:10.799 INFO:tasks.workunit.client.0.smithi070.stdout:8/912: truncate d2/d3/d52/d62/f95 2064756 0 2024-04-02T04:48:10.800 INFO:tasks.workunit.client.0.smithi070.stdout:1/883: rmdir d0/d9/d1d/d22/d2c/d4f 39 2024-04-02T04:48:10.801 INFO:tasks.workunit.client.0.smithi070.stdout:8/913: read d2/d3/d35/d3a/d4f/f87 [1941926,88004] 0 2024-04-02T04:48:10.805 INFO:tasks.workunit.client.0.smithi070.stdout:3/930: unlink d10/d66/cc6 0 2024-04-02T04:48:10.807 INFO:tasks.workunit.client.0.smithi070.stdout:8/914: dwrite d2/d8/d29/d22/d5a/f7e [0,4194304] 0 2024-04-02T04:48:10.808 INFO:tasks.workunit.client.0.smithi070.stdout:3/931: truncate d10/d32/d8a/d8d/f92 5223680 0 2024-04-02T04:48:10.810 INFO:tasks.workunit.client.0.smithi070.stdout:8/915: chown d2/d3/d52/d62/d7c/f120 1253598794 1 2024-04-02T04:48:10.813 INFO:tasks.workunit.client.0.smithi070.stdout:4/966: rename d1/d9c/d65/c12c to d1/d3/d78/dad/d12f/c140 0 2024-04-02T04:48:10.815 INFO:tasks.workunit.client.0.smithi070.stdout:3/932: dwrite ff [0,4194304] 0 2024-04-02T04:48:10.816 INFO:tasks.workunit.client.0.smithi070.stdout:3/933: fsync d10/d32/d3f/f10a 0 2024-04-02T04:48:10.820 INFO:tasks.workunit.client.0.smithi070.stdout:7/965: truncate d7/d36/f120 135085 0 2024-04-02T04:48:10.823 INFO:tasks.workunit.client.0.smithi070.stdout:6/873: truncate d0/d55/d8b/f9e 7284048 0 2024-04-02T04:48:10.828 INFO:tasks.workunit.client.0.smithi070.stdout:8/916: unlink d2/d8/d23/d24/d5f/l3d 0 2024-04-02T04:48:10.829 INFO:tasks.workunit.client.0.smithi070.stdout:8/917: fsync d2/d3/d52/d62/f115 0 2024-04-02T04:48:10.831 INFO:tasks.workunit.client.0.smithi070.stdout:1/884: rename d0/d54/f63 to d0/d9/dff/f118 0 2024-04-02T04:48:10.835 INFO:tasks.workunit.client.0.smithi070.stdout:7/966: rmdir d7/d8e 39 2024-04-02T04:48:10.838 INFO:tasks.workunit.client.0.smithi070.stdout:9/939: creat d4/d10/d11/d7b/f130 x:0 0 0 2024-04-02T04:48:10.840 INFO:tasks.workunit.client.0.smithi070.stdout:9/940: chown d4/d10/d12/d27/d60/f117 9195986 1 2024-04-02T04:48:10.849 INFO:tasks.workunit.client.0.smithi070.stdout:3/934: truncate d10/d32/d4b/dc5/d114/d127/f129 968805 0 2024-04-02T04:48:10.849 INFO:tasks.workunit.client.0.smithi070.stdout:4/967: dwrite d1/d3/fd2 [0,4194304] 0 2024-04-02T04:48:10.854 INFO:tasks.workunit.client.0.smithi070.stdout:9/941: symlink d4/d10/d11/d7b/d19/d82/db7/l131 0 2024-04-02T04:48:10.858 INFO:tasks.workunit.client.0.smithi070.stdout:1/885: rename d0/d9/d1d/d22/c32 to d0/d9/d1d/d22/d56/df1/c119 0 2024-04-02T04:48:10.860 INFO:tasks.workunit.client.0.smithi070.stdout:8/918: dwrite d2/d8/d23/d24/d79/fa0 [0,4194304] 0 2024-04-02T04:48:10.871 INFO:tasks.workunit.client.0.smithi070.stdout:7/967: symlink d7/d8e/dec/l148 0 2024-04-02T04:48:10.873 INFO:tasks.workunit.client.0.smithi070.stdout:6/874: creat d0/d3/d19/f106 x:0 0 0 2024-04-02T04:48:10.874 INFO:tasks.workunit.client.0.smithi070.stdout:9/942: write d4/d10/d12/fb3 [3042853,32602] 0 2024-04-02T04:48:10.875 INFO:tasks.workunit.client.0.smithi070.stdout:9/943: write d4/f106 [175350,127924] 0 2024-04-02T04:48:10.877 INFO:tasks.workunit.client.0.smithi070.stdout:8/919: rename d2/d8/d29/db/c2d to d2/d8/d23/d24/d5f/c124 0 2024-04-02T04:48:10.878 INFO:tasks.workunit.client.0.smithi070.stdout:8/920: chown d2/d8/d29/d72/le0 7 1 2024-04-02T04:48:10.881 INFO:tasks.workunit.client.0.smithi070.stdout:3/935: dread d10/d32/d8a/d8d/f92 [0,4194304] 0 2024-04-02T04:48:10.887 INFO:tasks.workunit.client.0.smithi070.stdout:8/921: symlink d2/d8/d29/d72/df1/l125 0 2024-04-02T04:48:10.888 INFO:tasks.workunit.client.0.smithi070.stdout:8/922: chown d2/d3/d52/d62/f123 3050 1 2024-04-02T04:48:10.888 INFO:tasks.workunit.client.0.smithi070.stdout:4/968: rename d1/d3/d9/f22 to d1/d3/d9/f141 0 2024-04-02T04:48:10.890 INFO:tasks.workunit.client.0.smithi070.stdout:6/875: mkdir d0/d3/d19/d107 0 2024-04-02T04:48:10.891 INFO:tasks.workunit.client.0.smithi070.stdout:6/876: fdatasync d0/d3/d19/d31/fdd 0 2024-04-02T04:48:10.891 INFO:tasks.workunit.client.0.smithi070.stdout:7/968: truncate d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/fa 820742 0 2024-04-02T04:48:10.892 INFO:tasks.workunit.client.0.smithi070.stdout:3/936: rename d10/d17/d4c/d11e/d122/ddb to d10/d17/d51/d73/d9e/d107/d13a 0 2024-04-02T04:48:10.897 INFO:tasks.workunit.client.0.smithi070.stdout:7/969: dread d7/dd/df/d7a/dbc/ff9 [0,4194304] 0 2024-04-02T04:48:10.897 INFO:tasks.workunit.client.0.smithi070.stdout:8/923: mknod d2/d3/d35/d3a/d4f/de8/c126 0 2024-04-02T04:48:10.899 INFO:tasks.workunit.client.0.smithi070.stdout:3/937: dwrite d10/d32/f42 [0,4194304] 0 2024-04-02T04:48:10.904 INFO:tasks.workunit.client.0.smithi070.stdout:1/886: getdents d0/d9 0 2024-04-02T04:48:10.915 INFO:tasks.workunit.client.0.smithi070.stdout:8/924: chown d2/d8/d29/d22/d5a/d77/cf3 14849 1 2024-04-02T04:48:10.916 INFO:tasks.workunit.client.0.smithi070.stdout:8/925: chown d2/d8/d23/d24/d79/f4e 339812677 1 2024-04-02T04:48:10.918 INFO:tasks.workunit.client.0.smithi070.stdout:6/877: mknod d0/d3/d19/d31/d46/c108 0 2024-04-02T04:48:10.919 INFO:tasks.workunit.client.0.smithi070.stdout:6/878: stat d0/d43/db3/dc7/fc3 0 2024-04-02T04:48:10.923 INFO:tasks.workunit.client.0.smithi070.stdout:9/944: getdents d4/d10/d12/d55/d66 0 2024-04-02T04:48:10.925 INFO:tasks.workunit.client.0.smithi070.stdout:4/969: link d1/d9c/d65/d7a/l100 d1/d3/d9/d5e/l142 0 2024-04-02T04:48:10.927 INFO:tasks.workunit.client.0.smithi070.stdout:4/970: write d1/d9c/d65/f12d [12857,46734] 0 2024-04-02T04:48:10.928 INFO:tasks.workunit.client.0.smithi070.stdout:7/970: dwrite d7/dd/df/d7a/fc4 [0,4194304] 0 2024-04-02T04:48:10.931 INFO:tasks.workunit.client.0.smithi070.stdout:8/926: truncate d2/d8/d29/d22/d5a/d94/fc0 3498449 0 2024-04-02T04:48:10.932 INFO:tasks.workunit.client.0.smithi070.stdout:6/879: symlink d0/d20/d77/l109 0 2024-04-02T04:48:10.937 INFO:tasks.workunit.client.0.smithi070.stdout:3/938: creat d10/d32/d4b/f13b x:0 0 0 2024-04-02T04:48:10.940 INFO:tasks.workunit.client.0.smithi070.stdout:7/971: mkdir d7/dd/df/d34/d68/d149 0 2024-04-02T04:48:10.941 INFO:tasks.workunit.client.0.smithi070.stdout:8/927: mkdir d2/d3/d35/d3a/d4f/d127 0 2024-04-02T04:48:10.943 INFO:tasks.workunit.client.0.smithi070.stdout:7/972: read d7/dd/df/d7a/f5a [3652529,56010] 0 2024-04-02T04:48:10.946 INFO:tasks.workunit.client.0.smithi070.stdout:6/880: fsync d0/d3/d19/d31/faf 0 2024-04-02T04:48:10.947 INFO:tasks.workunit.client.0.smithi070.stdout:8/928: dread d2/d8/d23/d24/d79/fa0 [0,4194304] 0 2024-04-02T04:48:10.948 INFO:tasks.workunit.client.0.smithi070.stdout:6/881: write d0/fe9 [500329,24358] 0 2024-04-02T04:48:10.950 INFO:tasks.workunit.client.0.smithi070.stdout:9/945: symlink d4/d10/d11/d7b/d19/d2f/d4d/l132 0 2024-04-02T04:48:10.953 INFO:tasks.workunit.client.0.smithi070.stdout:7/973: dwrite d7/d8e/d123/d65/f71 [0,4194304] 0 2024-04-02T04:48:10.963 INFO:tasks.workunit.client.0.smithi070.stdout:3/939: dread d10/d32/d8a/d8d/d91/f94 [0,4194304] 0 2024-04-02T04:48:10.968 INFO:tasks.workunit.client.0.smithi070.stdout:1/887: symlink d0/d9/d1d/d22/d2c/d4f/d77/l11a 0 2024-04-02T04:48:10.971 INFO:tasks.workunit.client.0.smithi070.stdout:8/929: mkdir d2/d8/d29/d72/d128 0 2024-04-02T04:48:10.973 INFO:tasks.workunit.client.0.smithi070.stdout:4/971: dwrite d1/d9c/fa5 [4194304,4194304] 0 2024-04-02T04:48:10.973 INFO:tasks.workunit.client.0.smithi070.stdout:3/940: dwrite d10/d32/d4b/dc5/fed [0,4194304] 0 2024-04-02T04:48:10.976 INFO:tasks.workunit.client.0.smithi070.stdout:6/882: unlink d0/d3/d19/d31/d46/fa9 0 2024-04-02T04:48:10.980 INFO:tasks.workunit.client.0.smithi070.stdout:9/946: creat d4/d10/d11/d7b/d19/d2f/d4d/d92/f133 x:0 0 0 2024-04-02T04:48:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.012+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 --> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87e80611d0 con 0x7f87d406d0a0 2024-04-02T04:48:11.015 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:10 smithi070 ceph-mon[29212]: pgmap v24: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 114 MiB/s rd, 276 MiB/s wr, 622 op/s 2024-04-02T04:48:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.013+0000 7f87df7fe700 1 -- 172.21.15.70:0/957984781 <== mgr.24359 v2:172.21.15.179:6828/1862820759 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f87e80611d0 con 0x7f87d406d0a0 2024-04-02T04:48:11.017 INFO:tasks.workunit.client.0.smithi070.stdout:1/888: symlink d0/d7/d1a/d95/l11b 0 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f87d406d0a0 msgr2=0x7f87d406f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f87d406d0a0 0x7f87d406f560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f87d8006010 tx=0x7f87d800b540 comp rx=0 tx=0).stop 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80ff530 msgr2=0x7f87e819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80ff530 0x7f87e819caa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87e400d900 tx=0x7f87e400dc10 comp rx=0 tx=0).stop 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 shutdown_connections 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f87d406d0a0 0x7f87d406f560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80ff530 0x7f87e819caa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 --2- 172.21.15.70:0/957984781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e8101ea0 0x7f87e819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 >> 172.21.15.70:0/957984781 conn(0x7f87e80fb150 msgr2=0x7f87e80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.015+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 shutdown_connections 2024-04-02T04:48:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.016+0000 7f87f06dc700 1 -- 172.21.15.70:0/957984781 wait complete. 2024-04-02T04:48:11.020 INFO:tasks.workunit.client.0.smithi070.stdout:8/930: rmdir d2/d8/d29/db 39 2024-04-02T04:48:11.020 INFO:tasks.workunit.client.0.smithi070.stdout:4/972: rmdir d1/d3/d9/d5e 39 2024-04-02T04:48:11.020 INFO:tasks.workunit.client.0.smithi070.stdout:3/941: fdatasync d10/d32/d3f/f35 0 2024-04-02T04:48:11.021 INFO:tasks.workunit.client.0.smithi070.stdout:3/942: chown d10/d17/db2/dbf/lc3 0 1 2024-04-02T04:48:11.022 INFO:tasks.workunit.client.0.smithi070.stdout:9/947: creat d4/d10/d12/d27/d60/d9e/f134 x:0 0 0 2024-04-02T04:48:11.023 INFO:tasks.workunit.client.0.smithi070.stdout:3/943: dread - d10/d17/d51/d73/d8f/fb6 zero size 2024-04-02T04:48:11.033 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:48:11.079 INFO:tasks.workunit.client.0.smithi070.stdout:9/948: stat d4/d10/d11/d7b/d19/d2f/d39/f12e 0 2024-04-02T04:48:11.079 INFO:tasks.workunit.client.0.smithi070.stdout:7/974: truncate d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/f21 2622276 0 2024-04-02T04:48:11.084 INFO:tasks.workunit.client.0.smithi070.stdout:9/949: creat d4/d10/d11/d7b/d19/d53/f135 x:0 0 0 2024-04-02T04:48:11.085 INFO:tasks.workunit.client.0.smithi070.stdout:9/950: dread - d4/d13/df0/ff9 zero size 2024-04-02T04:48:11.086 INFO:tasks.workunit.client.0.smithi070.stdout:7/975: symlink d7/d8e/d123/d65/d7b/l14a 0 2024-04-02T04:48:11.090 INFO:tasks.workunit.client.0.smithi070.stdout:8/931: link d2/d3/c10 d2/d8/d23/d24/d66/c129 0 2024-04-02T04:48:11.092 INFO:tasks.workunit.client.0.smithi070.stdout:7/976: rmdir d7/dd/df/d7a/d6c/d77/d75/d80 39 2024-04-02T04:48:11.094 INFO:tasks.workunit.client.0.smithi070.stdout:9/951: symlink d4/d10/d11/d7b/d19/d2f/d39/d57/dfb/l136 0 2024-04-02T04:48:11.095 INFO:tasks.workunit.client.0.smithi070.stdout:3/944: dread d10/ff1 [0,4194304] 0 2024-04-02T04:48:11.099 INFO:tasks.workunit.client.0.smithi070.stdout:8/932: fdatasync d2/d3/f61 0 2024-04-02T04:48:11.101 INFO:tasks.workunit.client.0.smithi070.stdout:7/977: truncate d7/d36/f120 486830 0 2024-04-02T04:48:11.101 INFO:tasks.workunit.client.0.smithi070.stdout:9/952: dread d4/d10/f2c [0,4194304] 0 2024-04-02T04:48:11.102 INFO:tasks.workunit.client.0.smithi070.stdout:8/933: chown d2/d8/cb4 11352466 1 2024-04-02T04:48:11.103 INFO:tasks.workunit.client.0.smithi070.stdout:3/945: dwrite d10/f133 [0,4194304] 0 2024-04-02T04:48:11.104 INFO:tasks.workunit.client.0.smithi070.stdout:9/953: stat d4/d10/d12/d56/d79 0 2024-04-02T04:48:11.106 INFO:tasks.workunit.client.0.smithi070.stdout:9/954: dread - d4/d10/d11/d7b/d19/d2f/d39/d57/f10d zero size 2024-04-02T04:48:11.109 INFO:tasks.workunit.client.0.smithi070.stdout:7/978: creat d7/d8e/f14b x:0 0 0 2024-04-02T04:48:11.112 INFO:tasks.workunit.client.0.smithi070.stdout:8/934: unlink d2/d8/d29/d72/cd6 0 2024-04-02T04:48:11.114 INFO:tasks.workunit.client.0.smithi070.stdout:9/955: dwrite d4/d10/d11/d7b/d19/d82/fc4 [0,4194304] 0 2024-04-02T04:48:11.121 INFO:tasks.workunit.client.0.smithi070.stdout:7/979: write d7/dd/df/d7a/d6c/d77/d75/d80/f113 [653381,37473] 0 2024-04-02T04:48:11.133 INFO:tasks.workunit.client.0.smithi070.stdout:7/980: rename d7/d4e/l90 to d7/dd/df/d34/d68/l14c 0 2024-04-02T04:48:11.140 INFO:tasks.workunit.client.0.smithi070.stdout:7/981: dwrite d7/dd/df/dca/fd4 [0,4194304] 0 2024-04-02T04:48:11.152 INFO:tasks.workunit.client.0.smithi070.stdout:7/982: link d7/dd/df/d7a/d6c/d77/d54/ld0 d7/d4e/d82/l14d 0 2024-04-02T04:48:11.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.150+0000 7f7636284700 1 -- 172.21.15.70:0/1012403276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301017a0 msgr2=0x7f7630103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:11.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.150+0000 7f7636284700 1 --2- 172.21.15.70:0/1012403276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301017a0 0x7f7630103b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7620009910 tx=0x7f7620009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:11.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.150+0000 7f7636284700 1 -- 172.21.15.70:0/1012403276 shutdown_connections 2024-04-02T04:48:11.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.150+0000 7f7636284700 1 --2- 172.21.15.70:0/1012403276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76301040d0 0x7f76301064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.150+0000 7f7636284700 1 --2- 172.21.15.70:0/1012403276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301017a0 0x7f7630103b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.150+0000 7f7636284700 1 -- 172.21.15.70:0/1012403276 >> 172.21.15.70:0/1012403276 conn(0x7f76300fb150 msgr2=0x7f76300fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:11.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.150+0000 7f7636284700 1 -- 172.21.15.70:0/1012403276 shutdown_connections 2024-04-02T04:48:11.154 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.150+0000 7f7636284700 1 -- 172.21.15.70:0/1012403276 wait complete. 2024-04-02T04:48:11.154 INFO:tasks.workunit.client.0.smithi070.stdout:7/983: chown d7/d36/la1 239142405 1 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.151+0000 7f7636284700 1 Processor -- start 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.151+0000 7f7636284700 1 -- start start 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.151+0000 7f7636284700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76301017a0 0x7f76301964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.151+0000 7f7636284700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301040d0 0x7f76301969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.151+0000 7f7636284700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630197000 con 0x7f76301040d0 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.151+0000 7f7636284700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630197140 con 0x7f76301017a0 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.151+0000 7f762ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76301017a0 0x7f76301964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76301017a0 0x7f76301964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:51210/0 (socket says 172.21.15.70:51210) 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762ffff700 1 -- 172.21.15.70:0/4289027207 learned_addr learned my addr 172.21.15.70:0/4289027207 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762f7fe700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301040d0 0x7f76301969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762ffff700 1 -- 172.21.15.70:0/4289027207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301040d0 msgr2=0x7f76301969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:11.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762ffff700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301040d0 0x7f76301969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762ffff700 1 -- 172.21.15.70:0/4289027207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76200095f0 con 0x7f76301017a0 2024-04-02T04:48:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762f7fe700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301040d0 0x7f76301969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:48:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762ffff700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76301017a0 0x7f76301964a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7620006010 tx=0x7f762000be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762d7fa700 1 -- 172.21.15.70:0/4289027207 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762001d070 con 0x7f76301017a0 2024-04-02T04:48:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763019bb90 con 0x7f76301017a0 2024-04-02T04:48:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763019c080 con 0x7f76301017a0 2024-04-02T04:48:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762d7fa700 1 -- 172.21.15.70:0/4289027207 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f762000f460 con 0x7f76301017a0 2024-04-02T04:48:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.152+0000 7f762d7fa700 1 -- 172.21.15.70:0/4289027207 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7620017630 con 0x7f76301017a0 2024-04-02T04:48:11.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.154+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c005320 con 0x7f76301017a0 2024-04-02T04:48:11.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.154+0000 7f762d7fa700 1 -- 172.21.15.70:0/4289027207 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f7620017790 con 0x7f76301017a0 2024-04-02T04:48:11.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.155+0000 7f762d7fa700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f761806d0a0 0x7f761806f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:11.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.155+0000 7f762d7fa700 1 -- 172.21.15.70:0/4289027207 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f762008dc00 con 0x7f76301017a0 2024-04-02T04:48:11.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.155+0000 7f762f7fe700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f761806d0a0 0x7f761806f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:11.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.156+0000 7f762f7fe700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f761806d0a0 0x7f761806f560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7624005950 tx=0x7f76240058e0 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:11.159 INFO:tasks.workunit.client.0.smithi070.stdout:7/984: dwrite d7/dd/df/d7a/fc4 [0,4194304] 0 2024-04-02T04:48:11.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.160+0000 7f762d7fa700 1 -- 172.21.15.70:0/4289027207 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f7620093020 con 0x7f76301017a0 2024-04-02T04:48:11.166 INFO:tasks.workunit.client.0.smithi070.stdout:1/889: write d0/d54/f58 [490181,129844] 0 2024-04-02T04:48:11.167 INFO:tasks.workunit.client.0.smithi070.stdout:7/985: fdatasync d7/dd/df/d79/f31 0 2024-04-02T04:48:11.169 INFO:tasks.workunit.client.0.smithi070.stdout:4/973: truncate d1/d9c/f102 3071303 0 2024-04-02T04:48:11.179 INFO:tasks.workunit.client.0.smithi070.stdout:7/986: mknod d7/dd/df/d7a/d6c/d77/c14e 0 2024-04-02T04:48:11.185 INFO:tasks.workunit.client.0.smithi070.stdout:4/974: dread d1/d3/d9/f11 [0,4194304] 0 2024-04-02T04:48:11.189 INFO:tasks.workunit.client.0.smithi070.stdout:7/987: creat d7/dd/df/d79/de6/d11c/f14f x:0 0 0 2024-04-02T04:48:11.189 INFO:tasks.workunit.client.0.smithi070.stdout:3/946: dwrite d10/d32/d8a/d8d/fc1 [0,4194304] 0 2024-04-02T04:48:11.191 INFO:tasks.workunit.client.0.smithi070.stdout:7/988: write d7/dd/df/dca/f13a [455970,53076] 0 2024-04-02T04:48:11.192 INFO:tasks.workunit.client.0.smithi070.stdout:7/989: chown d7/dd/df/d7a/dbc/fed 227696 1 2024-04-02T04:48:11.198 INFO:tasks.workunit.client.0.smithi070.stdout:1/890: rename d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/dea to d0/d9/d1d/d22/d56/d11c 0 2024-04-02T04:48:11.200 INFO:tasks.workunit.client.0.smithi070.stdout:9/956: dwrite d4/d10/d12/d56/d79/f85 [0,4194304] 0 2024-04-02T04:48:11.205 INFO:tasks.workunit.client.0.smithi070.stdout:3/947: read d10/d17/f2a [2273518,88429] 0 2024-04-02T04:48:11.210 INFO:tasks.workunit.client.0.smithi070.stdout:8/935: dwrite d2/d3/d35/d3a/d4f/d59/f6e [0,4194304] 0 2024-04-02T04:48:11.212 INFO:tasks.workunit.client.0.smithi070.stdout:8/936: chown d2/d3/d35/d3a/d4f/d59/l9d 461372988 1 2024-04-02T04:48:11.213 INFO:tasks.workunit.client.0.smithi070.stdout:9/957: dread - d4/d13/d89/df5/fed zero size 2024-04-02T04:48:11.218 INFO:tasks.workunit.client.0.smithi070.stdout:1/891: dread d0/d54/d64/fe4 [0,4194304] 0 2024-04-02T04:48:11.226 INFO:tasks.workunit.client.0.smithi070.stdout:3/948: mkdir d10/d32/d13c 0 2024-04-02T04:48:11.227 INFO:tasks.workunit.client.0.smithi070.stdout:3/949: write d10/d32/f42 [1558775,73421] 0 2024-04-02T04:48:11.229 INFO:tasks.workunit.client.0.smithi070.stdout:3/950: read d10/d32/d3f/f10a [214771,79493] 0 2024-04-02T04:48:11.235 INFO:tasks.workunit.client.0.smithi070.stdout:9/958: symlink d4/d10b/l137 0 2024-04-02T04:48:11.237 INFO:tasks.workunit.client.0.smithi070.stdout:7/990: getdents d7/dd/df/d7a/d6c 0 2024-04-02T04:48:11.241 INFO:tasks.workunit.client.0.smithi070.stdout:3/951: read - d10/d17/db2/ff8 zero size 2024-04-02T04:48:11.246 INFO:tasks.workunit.client.0.smithi070.stdout:3/952: creat d10/d17/d51/d73/d9e/d107/d13a/f13d x:0 0 0 2024-04-02T04:48:11.250 INFO:tasks.workunit.client.0.smithi070.stdout:4/975: dwrite d1/d3/d85/d8e/dc8/fd7 [0,4194304] 0 2024-04-02T04:48:11.259 INFO:tasks.workunit.client.0.smithi070.stdout:4/976: truncate d1/dc3/dc4/ff8 1389317 0 2024-04-02T04:48:11.259 INFO:tasks.workunit.client.0.smithi070.stdout:4/977: readlink d1/d9c/l5c 0 2024-04-02T04:48:11.259 INFO:tasks.workunit.client.0.smithi070.stdout:7/991: link d7/dd/l52 d7/dd/df/d34/d68/d149/l150 0 2024-04-02T04:48:11.266 INFO:tasks.workunit.client.0.smithi070.stdout:7/992: mknod d7/dd/db5/c151 0 2024-04-02T04:48:11.267 INFO:tasks.workunit.client.0.smithi070.stdout:4/978: dwrite d1/d3/d85/d8e/d99/db1/dbb/fff [0,4194304] 0 2024-04-02T04:48:11.278 INFO:tasks.workunit.client.0.smithi070.stdout:8/937: dwrite d2/d8/d29/fb8 [0,4194304] 0 2024-04-02T04:48:11.279 INFO:tasks.workunit.client.0.smithi070.stdout:1/892: dwrite d0/f1c [4194304,4194304] 0 2024-04-02T04:48:11.282 INFO:tasks.workunit.client.0.smithi070.stdout:3/953: dread d10/d17/f48 [0,4194304] 0 2024-04-02T04:48:11.285 INFO:tasks.workunit.client.0.smithi070.stdout:4/979: rename d1/d3/d85/ddb/cdd to d1/d3/d9/d5e/d12b/c143 0 2024-04-02T04:48:11.286 INFO:tasks.workunit.client.0.smithi070.stdout:9/959: write d4/d10/d12/d27/f5c [1645369,108791] 0 2024-04-02T04:48:11.287 INFO:tasks.workunit.client.0.smithi070.stdout:4/980: read - d1/d3/d85/d8e/d99/db1/f135 zero size 2024-04-02T04:48:11.289 INFO:tasks.workunit.client.0.smithi070.stdout:1/893: rmdir d0/d7/d1a/d66 39 2024-04-02T04:48:11.293 INFO:tasks.workunit.client.0.smithi070.stdout:9/960: readlink d4/l6f 0 2024-04-02T04:48:11.295 INFO:tasks.workunit.client.0.smithi070.stdout:8/938: unlink d2/d8/d23/d24/d66/c129 0 2024-04-02T04:48:11.300 INFO:tasks.workunit.client.0.smithi070.stdout:3/954: symlink d10/l13e 0 2024-04-02T04:48:11.302 INFO:tasks.workunit.client.0.smithi070.stdout:3/955: chown d10/d17/d4c/l9b 2 1 2024-04-02T04:48:11.302 INFO:tasks.workunit.client.0.smithi070.stdout:7/993: truncate d7/dd/df/d34/d68/ff3 3946402 0 2024-04-02T04:48:11.304 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:10 smithi179 ceph-mon[36603]: pgmap v24: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 114 MiB/s rd, 276 MiB/s wr, 622 op/s 2024-04-02T04:48:11.304 INFO:tasks.workunit.client.0.smithi070.stdout:4/981: symlink d1/d3/d78/dad/dbd/d128/d139/l144 0 2024-04-02T04:48:11.309 INFO:tasks.workunit.client.0.smithi070.stdout:3/956: symlink d10/d17/d51/d73/d8f/l13f 0 2024-04-02T04:48:11.310 INFO:tasks.workunit.client.0.smithi070.stdout:3/957: write d10/f22 [4474012,44393] 0 2024-04-02T04:48:11.311 INFO:tasks.workunit.client.0.smithi070.stdout:3/958: write d10/d32/d4b/fdc [325636,74125] 0 2024-04-02T04:48:11.314 INFO:tasks.workunit.client.0.smithi070.stdout:8/939: mkdir d2/d8/d12a 0 2024-04-02T04:48:11.316 INFO:tasks.workunit.client.0.smithi070.stdout:7/994: dread d7/dd/df/d7a/d6c/d77/d54/de5/d10b/d119/f27 [0,4194304] 0 2024-04-02T04:48:11.319 INFO:tasks.workunit.client.0.smithi070.stdout:9/961: rename d4/d10/d11/d7b/d19/la4 to d4/d10/d12/d27/l138 0 2024-04-02T04:48:11.321 INFO:tasks.workunit.client.0.smithi070.stdout:1/894: dwrite d0/d7/f6c [0,4194304] 0 2024-04-02T04:48:11.333 INFO:tasks.workunit.client.0.smithi070.stdout:8/940: mkdir d2/d3/d35/d3a/d4f/d12b 0 2024-04-02T04:48:11.339 INFO:tasks.workunit.client.0.smithi070.stdout:7/995: rename d7/d4e/d82/d57/c5f to d7/dd/df/d7a/d6c/d77/d54/de5/d10b/c152 0 2024-04-02T04:48:11.340 INFO:tasks.workunit.client.0.smithi070.stdout:1/895: stat d0/d7/d1a/l49 0 2024-04-02T04:48:11.342 INFO:tasks.workunit.client.0.smithi070.stdout:4/982: dwrite d1/fd [4194304,4194304] 0 2024-04-02T04:48:11.344 INFO:tasks.workunit.client.0.smithi070.stdout:9/962: dread d4/d13/d89/faf [4194304,4194304] 0 2024-04-02T04:48:11.344 INFO:tasks.workunit.client.0.smithi070.stdout:1/896: truncate d0/d9/d1d/d22/d2c/fef 924540 0 2024-04-02T04:48:11.345 INFO:tasks.workunit.client.0.smithi070.stdout:8/941: creat d2/d8/d29/d22/d5a/d77/f12c x:0 0 0 2024-04-02T04:48:11.350 INFO:tasks.workunit.client.0.smithi070.stdout:7/996: dwrite d7/d8e/dec/f117 [4194304,4194304] 0 2024-04-02T04:48:11.358 INFO:tasks.workunit.client.0.smithi070.stdout:7/997: dread d7/dd/db1/fcd [0,4194304] 0 2024-04-02T04:48:11.360 INFO:tasks.workunit.client.0.smithi070.stdout:9/963: rmdir d4/d10/d12/d56/d79/dd8 39 2024-04-02T04:48:11.364 INFO:tasks.workunit.client.0.smithi070.stdout:7/998: dwrite d7/dd/df/ffc [0,4194304] 0 2024-04-02T04:48:11.366 INFO:tasks.workunit.client.0.smithi070.stdout:3/959: link d10/d32/d8a/d8d/d91/cc8 d10/d32/d4b/de1/c140 0 2024-04-02T04:48:11.369 INFO:tasks.workunit.client.0.smithi070.stdout:8/942: getdents d2/d8/d29/d72/d128 0 2024-04-02T04:48:11.371 INFO:tasks.workunit.client.0.smithi070.stdout:7/999: symlink d7/dd/df/d7a/l153 0 2024-04-02T04:48:11.371 INFO:tasks.workunit.client.0.smithi070.stdout:9/964: read d4/d13/f99 [330800,36143] 0 2024-04-02T04:48:11.374 INFO:tasks.workunit.client.0.smithi070.stdout:3/960: rename d10/d32/d3f/c39 to d10/d32/d4b/dc5/d130/d138/c141 0 2024-04-02T04:48:11.377 INFO:tasks.workunit.client.0.smithi070.stdout:8/943: creat d2/d8/d29/dc7/df0/f12d x:0 0 0 2024-04-02T04:48:11.379 INFO:tasks.workunit.client.0.smithi070.stdout:8/944: write d2/d8/d23/d24/d5f/dc2/fd7 [2694045,52239] 0 2024-04-02T04:48:11.380 INFO:tasks.workunit.client.0.smithi070.stdout:9/965: rename d4/d13/l2a to d4/d34/db9/l139 0 2024-04-02T04:48:11.381 INFO:tasks.workunit.client.0.smithi070.stdout:3/961: truncate d10/d17/d51/d73/d8f/fb6 827523 0 2024-04-02T04:48:11.383 INFO:tasks.workunit.client.0.smithi070.stdout:1/897: getdents d0/d9/d1d 0 2024-04-02T04:48:11.385 INFO:tasks.workunit.client.0.smithi070.stdout:8/945: creat d2/d8/d23/d24/d106/f12e x:0 0 0 2024-04-02T04:48:11.387 INFO:tasks.workunit.client.0.smithi070.stdout:1/898: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/l11d 0 2024-04-02T04:48:11.390 INFO:tasks.workunit.client.0.smithi070.stdout:9/966: dwrite d4/d10/d11/d7b/d19/d2f/d39/d116/f12b [0,4194304] 0 2024-04-02T04:48:11.391 INFO:tasks.workunit.client.0.smithi070.stdout:8/946: mkdir d2/d8/d29/d72/df1/d12f 0 2024-04-02T04:48:11.392 INFO:tasks.workunit.client.0.smithi070.stdout:3/962: truncate d10/d17/d51/d73/d8f/f9c 348788 0 2024-04-02T04:48:11.395 INFO:tasks.workunit.client.0.smithi070.stdout:1/899: creat d0/d7/d28/f11e x:0 0 0 2024-04-02T04:48:11.396 INFO:tasks.workunit.client.0.smithi070.stdout:6/883: sync 2024-04-02T04:48:11.404 INFO:tasks.workunit.client.0.smithi070.stdout:3/963: dread - d10/d66/ff6 zero size 2024-04-02T04:48:11.409 INFO:tasks.workunit.client.0.smithi070.stdout:9/967: symlink d4/d10/d12/d27/d60/l13a 0 2024-04-02T04:48:11.409 INFO:tasks.workunit.client.0.smithi070.stdout:9/968: chown d4 0 1 2024-04-02T04:48:11.413 INFO:tasks.workunit.client.0.smithi070.stdout:8/947: write d2/d3/d35/d3a/d4f/de8/fff [508054,95888] 0 2024-04-02T04:48:11.419 INFO:tasks.workunit.client.0.smithi070.stdout:1/900: getdents d0/d9/d1d/d22/d2c/d4f 0 2024-04-02T04:48:11.422 INFO:tasks.workunit.client.0.smithi070.stdout:8/948: mknod d2/d8/d29/d22/d5a/d9e/c130 0 2024-04-02T04:48:11.424 INFO:tasks.workunit.client.0.smithi070.stdout:9/969: creat d4/d34/db9/d109/f13b x:0 0 0 2024-04-02T04:48:11.425 INFO:tasks.workunit.client.0.smithi070.stdout:3/964: dwrite d10/d17/d51/d73/d9e/d107/d13a/ff4 [0,4194304] 0 2024-04-02T04:48:11.428 INFO:tasks.workunit.client.0.smithi070.stdout:6/884: getdents d0/d3 0 2024-04-02T04:48:11.430 INFO:tasks.workunit.client.0.smithi070.stdout:1/901: mkdir d0/d9/d1d/d22/d56/d79/d10e/d11f 0 2024-04-02T04:48:11.433 INFO:tasks.workunit.client.0.smithi070.stdout:6/885: creat d0/d20/f10a x:0 0 0 2024-04-02T04:48:11.435 INFO:tasks.workunit.client.0.smithi070.stdout:1/902: rmdir d0/d7/d1a/d95 39 2024-04-02T04:48:11.436 INFO:tasks.workunit.client.0.smithi070.stdout:9/970: truncate d4/d10/d11/d7b/d19/d2f/d39/d57/da6/fd2 842649 0 2024-04-02T04:48:11.438 INFO:tasks.workunit.client.0.smithi070.stdout:9/971: chown d4/d10/d11/d7b/fa2 1095028 1 2024-04-02T04:48:11.448 INFO:tasks.workunit.client.0.smithi070.stdout:8/949: dwrite d2/d3/d35/d3a/d4f/fab [0,4194304] 0 2024-04-02T04:48:11.449 INFO:tasks.workunit.client.0.smithi070.stdout:1/903: dwrite d0/d7/d28/f74 [0,4194304] 0 2024-04-02T04:48:11.455 INFO:tasks.workunit.client.0.smithi070.stdout:3/965: dwrite d10/d17/db2/dbf/dd6/fbc [0,4194304] 0 2024-04-02T04:48:11.457 INFO:tasks.workunit.client.0.smithi070.stdout:9/972: write d4/d10/d11/d7b/d19/f54 [2168616,37334] 0 2024-04-02T04:48:11.459 INFO:tasks.workunit.client.0.smithi070.stdout:1/904: mknod d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/df9/c120 0 2024-04-02T04:48:11.459 INFO:tasks.workunit.client.0.smithi070.stdout:3/966: readlink d10/d17/d51/d73/d9e/d107/d10d/l113 0 2024-04-02T04:48:11.462 INFO:tasks.workunit.client.0.smithi070.stdout:3/967: write d10/d17/fcc [755231,56110] 0 2024-04-02T04:48:11.463 INFO:tasks.workunit.client.0.smithi070.stdout:1/905: chown d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5 0 1 2024-04-02T04:48:11.464 INFO:tasks.workunit.client.0.smithi070.stdout:1/906: chown d0/d9/d1d/d22/d56/dc1 68042811 1 2024-04-02T04:48:11.465 INFO:tasks.workunit.client.0.smithi070.stdout:9/973: creat d4/d10/d11/d7b/d19/d2f/d39/da5/f13c x:0 0 0 2024-04-02T04:48:11.468 INFO:tasks.workunit.client.0.smithi070.stdout:3/968: creat d10/d17/d51/d73/d9e/f142 x:0 0 0 2024-04-02T04:48:11.471 INFO:tasks.workunit.client.0.smithi070.stdout:8/950: link d2/d8/d23/d24/d79/c8a d2/d8/d29/db/c131 0 2024-04-02T04:48:11.473 INFO:tasks.workunit.client.0.smithi070.stdout:8/951: readlink d2/d8/d29/db/l74 0 2024-04-02T04:48:11.473 INFO:tasks.workunit.client.0.smithi070.stdout:3/969: mkdir d10/d32/d8a/d143 0 2024-04-02T04:48:11.474 INFO:tasks.workunit.client.0.smithi070.stdout:1/907: dwrite d0/d7/d28/d83/fbd [0,4194304] 0 2024-04-02T04:48:11.479 INFO:tasks.workunit.client.0.smithi070.stdout:1/908: mkdir d0/d54/d64/d121 0 2024-04-02T04:48:11.479 INFO:tasks.workunit.client.0.smithi070.stdout:9/974: dwrite d4/d10/d12/d27/d60/f117 [0,4194304] 0 2024-04-02T04:48:11.484 INFO:tasks.workunit.client.0.smithi070.stdout:8/952: rename d2/d3/d52/d62/d7c to d2/d3/d35/d3a/d132 0 2024-04-02T04:48:11.486 INFO:tasks.workunit.client.0.smithi070.stdout:9/975: symlink d4/d13/d89/l13d 0 2024-04-02T04:48:11.489 INFO:tasks.workunit.client.0.smithi070.stdout:3/970: dwrite d10/d17/d4c/fda [0,4194304] 0 2024-04-02T04:48:11.489 INFO:tasks.workunit.client.0.smithi070.stdout:6/886: dread d0/d55/d8b/d94/fb9 [0,4194304] 0 2024-04-02T04:48:11.490 INFO:tasks.workunit.client.0.smithi070.stdout:1/909: symlink d0/d9/d1d/d22/d56/l122 0 2024-04-02T04:48:11.496 INFO:tasks.workunit.client.0.smithi070.stdout:9/976: mkdir d4/d13/df0/dd3/d13e 0 2024-04-02T04:48:11.501 INFO:tasks.workunit.client.0.smithi070.stdout:8/953: link d2/f48 d2/d3/d35/d3a/f133 0 2024-04-02T04:48:11.505 INFO:tasks.workunit.client.0.smithi070.stdout:6/887: creat d0/d43/d95/f10b x:0 0 0 2024-04-02T04:48:11.508 INFO:tasks.workunit.client.0.smithi070.stdout:8/954: creat d2/d8/d29/d22/dc8/f134 x:0 0 0 2024-04-02T04:48:11.516 INFO:tasks.workunit.client.0.smithi070.stdout:1/910: dread d0/d7/f43 [0,4194304] 0 2024-04-02T04:48:11.519 INFO:tasks.workunit.client.0.smithi070.stdout:8/955: creat d2/d8/d29/d22/d5a/d77/d112/f135 x:0 0 0 2024-04-02T04:48:11.519 INFO:tasks.workunit.client.0.smithi070.stdout:6/888: write d0/d20/dd8/fdc [11168,129400] 0 2024-04-02T04:48:11.519 INFO:tasks.workunit.client.0.smithi070.stdout:9/977: getdents d4/d10/d11/d7b/d19/d2f/d39/d116 0 2024-04-02T04:48:11.519 INFO:tasks.workunit.client.0.smithi070.stdout:3/971: dwrite d10/d32/d3f/f65 [4194304,4194304] 0 2024-04-02T04:48:11.523 INFO:tasks.workunit.client.0.smithi070.stdout:8/956: read d2/d8/d29/d22/d5a/d94/da2/fad [72787,80210] 0 2024-04-02T04:48:11.527 INFO:tasks.workunit.client.0.smithi070.stdout:1/911: mknod d0/d9/d1d/d22/d2c/c123 0 2024-04-02T04:48:11.528 INFO:tasks.workunit.client.0.smithi070.stdout:9/978: write d4/d13/f99 [1006914,49070] 0 2024-04-02T04:48:11.529 INFO:tasks.workunit.client.0.smithi070.stdout:6/889: write d0/d43/db3/fd7 [487796,110897] 0 2024-04-02T04:48:11.531 INFO:tasks.workunit.client.0.smithi070.stdout:6/890: chown d0/d3/d19/f3f 5 1 2024-04-02T04:48:11.532 INFO:tasks.workunit.client.0.smithi070.stdout:3/972: mkdir d10/d121/d144 0 2024-04-02T04:48:11.536 INFO:tasks.workunit.client.0.smithi070.stdout:9/979: mkdir d4/d10/d11/d7b/d19/d82/db7/d13f 0 2024-04-02T04:48:11.538 INFO:tasks.workunit.client.0.smithi070.stdout:1/912: truncate d0/d9/d1d/d22/d56/d79/d89/fdb 3069321 0 2024-04-02T04:48:11.538 INFO:tasks.workunit.client.0.smithi070.stdout:9/980: chown d4/d10/d12/d56 112197659 1 2024-04-02T04:48:11.540 INFO:tasks.workunit.client.0.smithi070.stdout:8/957: write d2/d8/d23/d24/d5f/fe2 [4557417,68304] 0 2024-04-02T04:48:11.540 INFO:tasks.workunit.client.0.smithi070.stdout:3/973: dwrite d10/d66/d103/f11f [0,4194304] 0 2024-04-02T04:48:11.543 INFO:tasks.workunit.client.0.smithi070.stdout:3/974: chown d10/d66/d103/d111 11 1 2024-04-02T04:48:11.546 INFO:tasks.workunit.client.0.smithi070.stdout:9/981: rmdir d4/d13/df0 39 2024-04-02T04:48:11.546 INFO:tasks.workunit.client.0.smithi070.stdout:3/975: dread - d10/d17/d51/d73/d9e/f142 zero size 2024-04-02T04:48:11.549 INFO:tasks.workunit.client.0.smithi070.stdout:9/982: chown d4/d10/d12/d55/lb0 51465 1 2024-04-02T04:48:11.551 INFO:tasks.workunit.client.0.smithi070.stdout:6/891: creat d0/d3/d19/d31/d46/f10c x:0 0 0 2024-04-02T04:48:11.552 INFO:tasks.workunit.client.0.smithi070.stdout:3/976: mkdir d10/d17/db2/dbf/dd6/d145 0 2024-04-02T04:48:11.555 INFO:tasks.workunit.client.0.smithi070.stdout:6/892: rmdir d0/d20/dd8 39 2024-04-02T04:48:11.556 INFO:tasks.workunit.client.0.smithi070.stdout:6/893: stat d0/d55/d8b/d94 0 2024-04-02T04:48:11.557 INFO:tasks.workunit.client.0.smithi070.stdout:6/894: stat d0/d43/d80/ded 0 2024-04-02T04:48:11.557 INFO:tasks.workunit.client.0.smithi070.stdout:3/977: creat d10/d32/d3f/dc0/f146 x:0 0 0 2024-04-02T04:48:11.558 INFO:tasks.workunit.client.0.smithi070.stdout:9/983: link d4/f8 d4/d13/d89/df5/f140 0 2024-04-02T04:48:11.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.559+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 --> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f761c000bf0 con 0x7f761806d0a0 2024-04-02T04:48:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.560+0000 7f762d7fa700 1 -- 172.21.15.70:0/4289027207 <== mgr.24359 v2:172.21.15.179:6828/1862820759 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f761c000bf0 con 0x7f761806d0a0 2024-04-02T04:48:11.565 INFO:tasks.workunit.client.0.smithi070.stdout:6/895: dwrite d0/fe9 [0,4194304] 0 2024-04-02T04:48:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f761806d0a0 msgr2=0x7f761806f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f761806d0a0 0x7f761806f560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7624005950 tx=0x7f76240058e0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76301017a0 msgr2=0x7f76301964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76301017a0 0x7f76301964a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7620006010 tx=0x7f762000be20 comp rx=0 tx=0).stop 2024-04-02T04:48:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 shutdown_connections 2024-04-02T04:48:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f761806d0a0 0x7f761806f560 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76301017a0 0x7f76301964a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 --2- 172.21.15.70:0/4289027207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76301040d0 0x7f76301969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 >> 172.21.15.70:0/4289027207 conn(0x7f76300fb150 msgr2=0x7f76300fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 shutdown_connections 2024-04-02T04:48:11.567 INFO:tasks.workunit.client.0.smithi070.stdout:3/978: link d10/d32/f128 d10/d32/d4b/dc5/f147 0 2024-04-02T04:48:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.563+0000 7f7636284700 1 -- 172.21.15.70:0/4289027207 wait complete. 2024-04-02T04:48:11.568 INFO:tasks.workunit.client.0.smithi070.stdout:9/984: dwrite d4/f106 [0,4194304] 0 2024-04-02T04:48:11.568 INFO:tasks.workunit.client.0.smithi070.stdout:8/958: write d2/d3/f9 [3899833,46546] 0 2024-04-02T04:48:11.568 INFO:tasks.workunit.client.0.smithi070.stdout:1/913: write d0/d9/d1d/fd8 [184394,99573] 0 2024-04-02T04:48:11.571 INFO:tasks.workunit.client.0.smithi070.stdout:3/979: chown d10/d32/d4b/l68 118 1 2024-04-02T04:48:11.572 INFO:tasks.workunit.client.0.smithi070.stdout:6/896: unlink d0/d3/dc2/fe3 0 2024-04-02T04:48:11.572 INFO:tasks.workunit.client.0.smithi070.stdout:8/959: write d2/d8/d23/d24/d106/f12e [364610,61676] 0 2024-04-02T04:48:11.581 INFO:tasks.workunit.client.0.smithi070.stdout:8/960: dwrite d2/d8/d29/d22/dc8/f116 [0,4194304] 0 2024-04-02T04:48:11.581 INFO:tasks.workunit.client.0.smithi070.stdout:6/897: creat d0/df0/f10d x:0 0 0 2024-04-02T04:48:11.583 INFO:tasks.workunit.client.0.smithi070.stdout:1/914: mknod d0/d54/d64/dee/c124 0 2024-04-02T04:48:11.585 INFO:tasks.workunit.client.0.smithi070.stdout:3/980: mknod d10/d66/c148 0 2024-04-02T04:48:11.586 INFO:tasks.workunit.client.0.smithi070.stdout:3/981: chown d10/d17/d51/l112 4764 1 2024-04-02T04:48:11.590 INFO:tasks.workunit.client.0.smithi070.stdout:8/961: write d2/d3/d35/d3a/f47 [1069745,92581] 0 2024-04-02T04:48:11.591 INFO:tasks.workunit.client.0.smithi070.stdout:8/962: write d2/d8/d23/d24/d5f/dc2/fd7 [4568062,88335] 0 2024-04-02T04:48:11.593 INFO:tasks.workunit.client.0.smithi070.stdout:1/915: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/l125 0 2024-04-02T04:48:11.596 INFO:tasks.workunit.client.0.smithi070.stdout:8/963: rmdir d2/d8/d29/d22/d5a/d94/da2 39 2024-04-02T04:48:11.597 INFO:tasks.workunit.client.0.smithi070.stdout:6/898: creat d0/d3/d19/d31/d46/f10e x:0 0 0 2024-04-02T04:48:11.599 INFO:tasks.workunit.client.0.smithi070.stdout:1/916: symlink d0/d9/d1d/d22/d56/l126 0 2024-04-02T04:48:11.601 INFO:tasks.workunit.client.0.smithi070.stdout:1/917: write d0/d7/d28/d5c/d6e/f6f [4674902,26543] 0 2024-04-02T04:48:11.601 INFO:tasks.workunit.client.0.smithi070.stdout:8/964: creat d2/d8/d23/d24/f136 x:0 0 0 2024-04-02T04:48:11.603 INFO:tasks.workunit.client.0.smithi070.stdout:6/899: link d0/fe9 d0/d43/db3/f10f 0 2024-04-02T04:48:11.605 INFO:tasks.workunit.client.0.smithi070.stdout:1/918: mkdir d0/d9/d1d/d22/d56/dc1/d127 0 2024-04-02T04:48:11.606 INFO:tasks.workunit.client.0.smithi070.stdout:1/919: read - d0/d7/fd4 zero size 2024-04-02T04:48:11.606 INFO:tasks.workunit.client.0.smithi070.stdout:8/965: dread d2/f7f [0,4194304] 0 2024-04-02T04:48:11.608 INFO:tasks.workunit.client.0.smithi070.stdout:8/966: write d2/d8/d29/db/fc5 [5197127,22254] 0 2024-04-02T04:48:11.610 INFO:tasks.workunit.client.0.smithi070.stdout:6/900: creat d0/f110 x:0 0 0 2024-04-02T04:48:11.611 INFO:tasks.workunit.client.0.smithi070.stdout:1/920: rmdir d0/d9/d1d/d22/d56/df1 39 2024-04-02T04:48:11.617 INFO:tasks.workunit.client.0.smithi070.stdout:1/921: rename d0/d9/d1d/d22/d56/fbf to d0/d9/d1d/d22/d56/d79/d89/f128 0 2024-04-02T04:48:11.620 INFO:tasks.workunit.client.0.smithi070.stdout:8/967: link d2/d8/d23/c118 d2/d3/d35/d3a/c137 0 2024-04-02T04:48:11.622 INFO:tasks.workunit.client.0.smithi070.stdout:8/968: write d2/d3/d35/d3a/fdf [177884,104539] 0 2024-04-02T04:48:11.623 INFO:tasks.workunit.client.0.smithi070.stdout:8/969: write d2/d8/d29/d22/d5a/d77/f12c [90872,68478] 0 2024-04-02T04:48:11.627 INFO:tasks.workunit.client.0.smithi070.stdout:3/982: write d10/d17/d4c/d11e/d122/fcf [222449,79047] 0 2024-04-02T04:48:11.628 INFO:tasks.workunit.client.0.smithi070.stdout:8/970: dread d2/d3/d52/f55 [0,4194304] 0 2024-04-02T04:48:11.636 INFO:tasks.workunit.client.0.smithi070.stdout:1/922: dread d0/d9/d1d/d22/d2c/fb3 [0,4194304] 0 2024-04-02T04:48:11.639 INFO:tasks.workunit.client.0.smithi070.stdout:8/971: dread d2/d8/d29/d22/d5a/d94/da2/fc3 [0,4194304] 0 2024-04-02T04:48:11.640 INFO:tasks.workunit.client.0.smithi070.stdout:1/923: truncate d0/d9/d1d/d22/d2c/fb3 4335765 0 2024-04-02T04:48:11.640 INFO:tasks.workunit.client.0.smithi070.stdout:6/901: write d0/d20/f27 [4085811,113891] 0 2024-04-02T04:48:11.643 INFO:tasks.workunit.client.0.smithi070.stdout:8/972: symlink d2/d3/d35/d3a/d4f/l138 0 2024-04-02T04:48:11.645 INFO:tasks.workunit.client.0.smithi070.stdout:1/924: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/da5/l129 0 2024-04-02T04:48:11.647 INFO:tasks.workunit.client.0.smithi070.stdout:3/983: write d10/d32/d3f/f3b [1752844,94592] 0 2024-04-02T04:48:11.649 INFO:tasks.workunit.client.0.smithi070.stdout:1/925: creat d0/f12a x:0 0 0 2024-04-02T04:48:11.652 INFO:tasks.workunit.client.0.smithi070.stdout:8/973: symlink d2/d8/d29/d22/d5a/d94/de6/dec/l139 0 2024-04-02T04:48:11.654 INFO:tasks.workunit.client.0.smithi070.stdout:3/984: mkdir d10/d32/d3f/d149 0 2024-04-02T04:48:11.655 INFO:tasks.workunit.client.0.smithi070.stdout:3/985: chown d10/d32/d8a/d8d/d91/cc8 43 1 2024-04-02T04:48:11.661 INFO:tasks.workunit.client.0.smithi070.stdout:6/902: rename d0/d20/d5f/de6/cc0 to d0/d43/c111 0 2024-04-02T04:48:11.662 INFO:tasks.workunit.client.0.smithi070.stdout:6/903: read - d0/d43/d80/ffc zero size 2024-04-02T04:48:11.663 INFO:tasks.workunit.client.0.smithi070.stdout:6/904: readlink d0/d3/d19/d31/d46/ld6 0 2024-04-02T04:48:11.664 INFO:tasks.workunit.client.0.smithi070.stdout:8/974: dread d2/d8/d29/db/f13 [4194304,4194304] 0 2024-04-02T04:48:11.664 INFO:tasks.workunit.client.0.smithi070.stdout:6/905: fdatasync d0/d43/db3/dc7/f88 0 2024-04-02T04:48:11.666 INFO:tasks.workunit.client.0.smithi070.stdout:1/926: creat d0/d9/d1d/d22/d56/dc1/d127/f12b x:0 0 0 2024-04-02T04:48:11.672 INFO:tasks.workunit.client.0.smithi070.stdout:3/986: symlink d10/d32/l14a 0 2024-04-02T04:48:11.676 INFO:tasks.workunit.client.0.smithi070.stdout:8/975: truncate d2/d8/d23/d24/d66/f6a 666584 0 2024-04-02T04:48:11.677 INFO:tasks.workunit.client.0.smithi070.stdout:3/987: dwrite d10/fad [4194304,4194304] 0 2024-04-02T04:48:11.679 INFO:tasks.workunit.client.0.smithi070.stdout:1/927: truncate d0/db4/d9f/ff2 4639370 0 2024-04-02T04:48:11.684 INFO:tasks.workunit.client.0.smithi070.stdout:6/906: unlink d0/d3/d19/f47 0 2024-04-02T04:48:11.684 INFO:tasks.workunit.client.0.smithi070.stdout:8/976: creat d2/d8/d29/d22/d5a/d77/d112/f13a x:0 0 0 2024-04-02T04:48:11.689 INFO:tasks.workunit.client.0.smithi070.stdout:3/988: rename d10/d17/db2/dbf/f116 to d10/d66/f14b 0 2024-04-02T04:48:11.690 INFO:tasks.workunit.client.0.smithi070.stdout:6/907: symlink d0/d20/dd8/l112 0 2024-04-02T04:48:11.691 INFO:tasks.workunit.client.0.smithi070.stdout:1/928: creat d0/d7/d1a/f12c x:0 0 0 2024-04-02T04:48:11.692 INFO:tasks.workunit.client.0.smithi070.stdout:6/908: chown d0/d55/d79/l101 0 1 2024-04-02T04:48:11.693 INFO:tasks.workunit.client.0.smithi070.stdout:8/977: dwrite d2/d8/d23/d24/d5f/fe2 [0,4194304] 0 2024-04-02T04:48:11.694 INFO:tasks.workunit.client.0.smithi070.stdout:3/989: rmdir d10/d17/db2/dbf/dd6/d120 39 2024-04-02T04:48:11.694 INFO:tasks.workunit.client.0.smithi070.stdout:6/909: dread - d0/d20/ff4 zero size 2024-04-02T04:48:11.694 INFO:tasks.workunit.client.0.smithi070.stdout:3/990: chown d10/d17/db2/dbf/dd2 6934236 1 2024-04-02T04:48:11.695 INFO:tasks.workunit.client.0.smithi070.stdout:6/910: dread - d0/d20/d77/d98/fe5 zero size 2024-04-02T04:48:11.697 INFO:tasks.workunit.client.0.smithi070.stdout:8/978: mknod d2/d8/d23/d24/d5f/dc2/c13b 0 2024-04-02T04:48:11.709 INFO:tasks.workunit.client.0.smithi070.stdout:8/979: rename d2/d8/d29/d22/d5a/d94/da2/c6b to d2/d8/d29/d22/d5a/d94/da2/dde/c13c 0 2024-04-02T04:48:11.719 INFO:tasks.workunit.client.0.smithi070.stdout:1/929: write d0/d9/d1d/d22/fab [784563,36831] 0 2024-04-02T04:48:11.724 INFO:tasks.workunit.client.0.smithi070.stdout:1/930: mknod d0/d7/d28/c12d 0 2024-04-02T04:48:11.724 INFO:tasks.workunit.client.0.smithi070.stdout:3/991: dwrite d10/f29 [0,4194304] 0 2024-04-02T04:48:11.726 INFO:tasks.workunit.client.0.smithi070.stdout:6/911: dwrite d0/d43/fae [0,4194304] 0 2024-04-02T04:48:11.733 INFO:tasks.workunit.client.0.smithi070.stdout:1/931: mkdir d0/d7/d28/d5c/d6e/d12e 0 2024-04-02T04:48:11.735 INFO:tasks.workunit.client.0.smithi070.stdout:3/992: truncate d10/d32/d3f/f43 367803 0 2024-04-02T04:48:11.737 INFO:tasks.workunit.client.0.smithi070.stdout:8/980: dwrite d2/d3/d35/d3a/d4f/d59/f8f [0,4194304] 0 2024-04-02T04:48:11.742 INFO:tasks.workunit.client.0.smithi070.stdout:6/912: unlink d0/d55/d5c/c9d 0 2024-04-02T04:48:11.747 INFO:tasks.workunit.client.0.smithi070.stdout:8/981: mknod d2/d3/d35/d3a/d4f/de8/c13d 0 2024-04-02T04:48:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.744+0000 7f18bf573700 1 -- 172.21.15.70:0/3597636689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18b810a700 msgr2=0x7f18b810cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.744+0000 7f18bf573700 1 --2- 172.21.15.70:0/3597636689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18b810a700 0x7f18b810cb90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f18b000b0a0 tx=0x7f18b000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.745+0000 7f18bf573700 1 -- 172.21.15.70:0/3597636689 shutdown_connections 2024-04-02T04:48:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.745+0000 7f18bf573700 1 --2- 172.21.15.70:0/3597636689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18b810a700 0x7f18b810cb90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.745+0000 7f18bf573700 1 --2- 172.21.15.70:0/3597636689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18b8107d90 0x7f18b810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.745+0000 7f18bf573700 1 -- 172.21.15.70:0/3597636689 >> 172.21.15.70:0/3597636689 conn(0x7f18b806da40 msgr2=0x7f18b806fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.746+0000 7f18bf573700 1 -- 172.21.15.70:0/3597636689 shutdown_connections 2024-04-02T04:48:11.749 INFO:tasks.workunit.client.0.smithi070.stdout:8/982: readlink d2/d8/d29/d22/d5a/d94/de6/dec/l139 0 2024-04-02T04:48:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.746+0000 7f18bf573700 1 -- 172.21.15.70:0/3597636689 wait complete. 2024-04-02T04:48:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.747+0000 7f18bf573700 1 Processor -- start 2024-04-02T04:48:11.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.747+0000 7f18bf573700 1 -- start start 2024-04-02T04:48:11.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.747+0000 7f18bf573700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18b8107d90 0x7f18b81a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.747+0000 7f18bf573700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18b810a700 0x7f18b81a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.747+0000 7f18bf573700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b81a5e00 con 0x7f18b8107d90 2024-04-02T04:48:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.747+0000 7f18bf573700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b81a5f40 con 0x7f18b810a700 2024-04-02T04:48:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.748+0000 7f18bd30f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18b8107d90 0x7f18b81a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.748+0000 7f18bd30f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18b8107d90 0x7f18b81a52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47740/0 (socket says 172.21.15.70:47740) 2024-04-02T04:48:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.748+0000 7f18bd30f700 1 -- 172.21.15.70:0/522440464 learned_addr learned my addr 172.21.15.70:0/522440464 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.748+0000 7f18bd30f700 1 -- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18b810a700 msgr2=0x7f18b81a57e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:48:11.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.748+0000 7f18bd30f700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18b810a700 0x7f18b81a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:11.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.748+0000 7f18bd30f700 1 -- 172.21.15.70:0/522440464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b0009d00 con 0x7f18b8107d90 2024-04-02T04:48:11.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.749+0000 7f18bd30f700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18b8107d90 0x7f18b81a52a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f18a800d8d0 tx=0x7f18a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:11.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.749+0000 7f18ae7fc700 1 -- 172.21.15.70:0/522440464 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18a8009940 con 0x7f18b8107d90 2024-04-02T04:48:11.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.749+0000 7f18ae7fc700 1 -- 172.21.15.70:0/522440464 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f18a8010460 con 0x7f18b8107d90 2024-04-02T04:48:11.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.749+0000 7f18ae7fc700 1 -- 172.21.15.70:0/522440464 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18a8009940 con 0x7f18b8107d90 2024-04-02T04:48:11.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.749+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b81aa9f0 con 0x7f18b8107d90 2024-04-02T04:48:11.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.749+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b81aaef0 con 0x7f18b8107d90 2024-04-02T04:48:11.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.751+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b819f400 con 0x7f18b8107d90 2024-04-02T04:48:11.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.751+0000 7f18ae7fc700 1 -- 172.21.15.70:0/522440464 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f18a80105d0 con 0x7f18b8107d90 2024-04-02T04:48:11.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.752+0000 7f18ae7fc700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f18a406d0f0 0x7f18a406f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:11.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.752+0000 7f18ae7fc700 1 -- 172.21.15.70:0/522440464 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f18a808c580 con 0x7f18b8107d90 2024-04-02T04:48:11.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.753+0000 7f18bcb0e700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f18a406d0f0 0x7f18a406f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:11.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.753+0000 7f18bcb0e700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f18a406d0f0 0x7f18a406f5b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009480 tx=0x7f18b0000f40 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:11.757 INFO:tasks.workunit.client.0.smithi070.stdout:8/983: mkdir d2/d8/d29/d22/d5a/d13e 0 2024-04-02T04:48:11.759 INFO:tasks.workunit.client.0.smithi070.stdout:8/984: write d2/d3/d35/d3a/d4f/d59/dcc/f101 [1182492,65549] 0 2024-04-02T04:48:11.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:11.759+0000 7f18ae7fc700 1 -- 172.21.15.70:0/522440464 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f18a8059ab0 con 0x7f18b8107d90 2024-04-02T04:48:11.763 INFO:tasks.workunit.client.0.smithi070.stdout:3/993: write f1 [876770,79336] 0 2024-04-02T04:48:11.765 INFO:tasks.workunit.client.0.smithi070.stdout:6/913: write d0/d55/d5c/f60 [3123475,58651] 0 2024-04-02T04:48:11.767 INFO:tasks.workunit.client.0.smithi070.stdout:1/932: getdents d0/d9/d1d/d22/d56/df1 0 2024-04-02T04:48:11.768 INFO:tasks.workunit.client.0.smithi070.stdout:8/985: dwrite d2/d8/d29/d22/f108 [0,4194304] 0 2024-04-02T04:48:11.772 INFO:tasks.workunit.client.0.smithi070.stdout:1/933: dread d0/d4c/fa0 [0,4194304] 0 2024-04-02T04:48:11.773 INFO:tasks.workunit.client.0.smithi070.stdout:3/994: rmdir d10/d32/dd1 39 2024-04-02T04:48:11.778 INFO:tasks.workunit.client.0.smithi070.stdout:1/934: dwrite d0/d7/f6c [0,4194304] 0 2024-04-02T04:48:11.783 INFO:tasks.workunit.client.0.smithi070.stdout:1/935: stat d0/d9/d1d/d22/d56/d79/d89/cfe 0 2024-04-02T04:48:11.783 INFO:tasks.workunit.client.0.smithi070.stdout:8/986: fsync d2/d3/d35/f78 0 2024-04-02T04:48:11.784 INFO:tasks.workunit.client.0.smithi070.stdout:6/914: mknod d0/d20/d5f/de6/da6/dfb/c113 0 2024-04-02T04:48:11.785 INFO:tasks.workunit.client.0.smithi070.stdout:1/936: creat d0/d9/d1d/d22/d56/dc1/f12f x:0 0 0 2024-04-02T04:48:11.786 INFO:tasks.workunit.client.0.smithi070.stdout:1/937: readlink d0/d9/d1d/lb 0 2024-04-02T04:48:11.790 INFO:tasks.workunit.client.0.smithi070.stdout:3/995: truncate d10/d32/d4b/dc5/d114/d127/f129 999275 0 2024-04-02T04:48:11.791 INFO:tasks.workunit.client.0.smithi070.stdout:6/915: mknod d0/d3/d19/d31/dc1/c114 0 2024-04-02T04:48:11.793 INFO:tasks.workunit.client.0.smithi070.stdout:1/938: unlink d0/d7/d28/f7a 0 2024-04-02T04:48:11.805 INFO:tasks.workunit.client.0.smithi070.stdout:6/916: mkdir d0/d43/db3/dc7/d115 0 2024-04-02T04:48:11.805 INFO:tasks.workunit.client.0.smithi070.stdout:6/917: dread - d0/d3/d19/f106 zero size 2024-04-02T04:48:11.806 INFO:tasks.workunit.client.0.smithi070.stdout:1/939: creat d0/d54/d64/d6d/ddd/f130 x:0 0 0 2024-04-02T04:48:11.809 INFO:tasks.workunit.client.0.smithi070.stdout:4/983: sync 2024-04-02T04:48:11.811 INFO:tasks.workunit.client.0.smithi070.stdout:6/918: fdatasync d0/d3/d19/d31/d46/d6b/f72 0 2024-04-02T04:48:11.812 INFO:tasks.workunit.client.0.smithi070.stdout:8/987: write d2/d3/d35/fd5 [822668,67546] 0 2024-04-02T04:48:11.815 INFO:tasks.workunit.client.0.smithi070.stdout:6/919: read - d0/d20/d5f/fd9 zero size 2024-04-02T04:48:11.817 INFO:tasks.workunit.client.0.smithi070.stdout:6/920: read d0/d43/db3/f10f [3290445,33379] 0 2024-04-02T04:48:11.818 INFO:tasks.workunit.client.0.smithi070.stdout:6/921: chown d0/d20/d77/l81 88687 1 2024-04-02T04:48:11.819 INFO:tasks.workunit.client.0.smithi070.stdout:6/922: write d0/df0/f10d [510689,24908] 0 2024-04-02T04:48:11.823 INFO:tasks.workunit.client.0.smithi070.stdout:8/988: mkdir d2/d3/d35/d3a/d13f 0 2024-04-02T04:48:11.823 INFO:tasks.workunit.client.0.smithi070.stdout:1/940: write d0/db4/d9f/ff2 [3839946,49695] 0 2024-04-02T04:48:11.824 INFO:tasks.workunit.client.0.smithi070.stdout:8/989: fdatasync d2/d8/d29/fba 0 2024-04-02T04:48:11.832 INFO:tasks.workunit.client.0.smithi070.stdout:8/990: dwrite d2/d3/d35/fb3 [0,4194304] 0 2024-04-02T04:48:11.832 INFO:tasks.workunit.client.0.smithi070.stdout:4/984: dwrite d1/d3/f4c [0,4194304] 0 2024-04-02T04:48:11.840 INFO:tasks.workunit.client.0.smithi070.stdout:6/923: write d0/d55/d8b/d94/fb9 [901387,35150] 0 2024-04-02T04:48:11.842 INFO:tasks.workunit.client.0.smithi070.stdout:1/941: write d0/d7/d28/d5c/d6e/fd9 [4083511,61187] 0 2024-04-02T04:48:11.847 INFO:tasks.workunit.client.0.smithi070.stdout:6/924: write d0/d3/d19/d31/d46/d6b/f6d [924550,15434] 0 2024-04-02T04:48:11.848 INFO:tasks.workunit.client.0.smithi070.stdout:4/985: dwrite d1/d3/d9/f13 [0,4194304] 0 2024-04-02T04:48:11.853 INFO:tasks.workunit.client.0.smithi070.stdout:1/942: rename d0/d4c/f70 to d0/d9/d1d/d22/d56/d79/d10e/f131 0 2024-04-02T04:48:11.854 INFO:tasks.workunit.client.0.smithi070.stdout:6/925: mkdir d0/d20/dd8/d116 0 2024-04-02T04:48:11.855 INFO:tasks.workunit.client.0.smithi070.stdout:4/986: truncate d1/f26 3266602 0 2024-04-02T04:48:11.856 INFO:tasks.workunit.client.0.smithi070.stdout:8/991: link d2/d8/d23/d24/f4b d2/d3/d52/f140 0 2024-04-02T04:48:11.858 INFO:tasks.workunit.client.0.smithi070.stdout:6/926: unlink d0/d43/d80/ca1 0 2024-04-02T04:48:11.863 INFO:tasks.workunit.client.0.smithi070.stdout:4/987: unlink d1/d3/d78/dad/dbd/d128/d139/l144 0 2024-04-02T04:48:11.875 INFO:tasks.workunit.client.0.smithi070.stdout:8/992: write d2/d8/d23/d24/f5b [591160,117227] 0 2024-04-02T04:48:11.876 INFO:tasks.workunit.client.0.smithi070.stdout:8/993: dread - d2/d8/d23/d24/f114 zero size 2024-04-02T04:48:11.877 INFO:tasks.workunit.client.0.smithi070.stdout:8/994: chown d2/d3/f53 853231706 1 2024-04-02T04:48:11.883 INFO:tasks.workunit.client.0.smithi070.stdout:8/995: rename d2/d8 to d2/d3/d35/d3a/d141 0 2024-04-02T04:48:11.891 INFO:tasks.workunit.client.0.smithi070.stdout:8/996: dwrite d2/d3/d35/d3a/d141/d23/d24/f136 [0,4194304] 0 2024-04-02T04:48:11.893 INFO:tasks.workunit.client.0.smithi070.stdout:8/997: mknod d2/d3/d35/d3a/c142 0 2024-04-02T04:48:11.894 INFO:tasks.workunit.client.0.smithi070.stdout:8/998: readlink d2/d3/d35/d3a/d141/d29/dc7/lf8 0 2024-04-02T04:48:11.898 INFO:tasks.workunit.client.0.smithi070.stdout:9/985: sync 2024-04-02T04:48:11.901 INFO:tasks.workunit.client.0.smithi070.stdout:8/999: creat d2/d3/d35/d3a/d141/d23/d24/f143 x:0 0 0 2024-04-02T04:48:11.901 INFO:tasks.workunit.client.0.smithi070.stdout:4/988: dread d1/d3/f39 [4194304,4194304] 0 2024-04-02T04:48:11.902 INFO:tasks.workunit.client.0.smithi070.stdout:9/986: symlink d4/d10/d11/d7b/d19/d2f/d4d/de2/l141 0 2024-04-02T04:48:11.987 INFO:tasks.workunit.client.0.smithi070.stdout:4/989: dwrite d1/d3/d85/deb/d110/f120 [0,4194304] 0 2024-04-02T04:48:12.013 INFO:tasks.workunit.client.0.smithi070.stdout:9/987: link d4/d10/d11/d7b/d19/d2f/fb8 d4/d10/d12/d55/f142 0 2024-04-02T04:48:12.020 INFO:tasks.workunit.client.0.smithi070.stdout:9/988: dwrite d4/d10/d12/d55/d66/f6d [0,4194304] 0 2024-04-02T04:48:12.034 INFO:tasks.workunit.client.0.smithi070.stdout:4/990: dwrite d1/d3/d85/d8e/d99/fd4 [0,4194304] 0 2024-04-02T04:48:12.041 INFO:tasks.workunit.client.0.smithi070.stdout:4/991: getdents d1/d3/d78/d91 0 2024-04-02T04:48:12.047 INFO:tasks.workunit.client.0.smithi070.stdout:4/992: getdents d1/d3/d85/da8 0 2024-04-02T04:48:12.047 INFO:tasks.workunit.client.0.smithi070.stdout:4/993: dread - d1/f12e zero size 2024-04-02T04:48:12.064 INFO:tasks.workunit.client.0.smithi070.stdout:4/994: dread d1/d3/d85/d8e/d99/db1/dbb/fec [0,4194304] 0 2024-04-02T04:48:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.063+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 --> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f18b80611d0 con 0x7f18a406d0f0 2024-04-02T04:48:12.073 INFO:tasks.workunit.client.0.smithi070.stdout:4/995: dwrite d1/d9c/df0/f117 [0,4194304] 0 2024-04-02T04:48:12.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.077+0000 7f18ae7fc700 1 -- 172.21.15.70:0/522440464 <== mgr.24359 v2:172.21.15.179:6828/1862820759 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f18b80611d0 con 0x7f18a406d0f0 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (7m) 11s ago 8m 24.2M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (9m) 11s ago 9m 7662k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (8m) 26s ago 8m 7365k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (9m) 11s ago 9m 6665k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (8m) 26s ago 7m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (7m) 11s ago 8m 88.2M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (3m) 11s ago 3m 182M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (3m) 11s ago 3m 16.2M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (3m) 26s ago 3m 14.7M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:48:12.081 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (3m) 26s ago 3m 16.3M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:9283,8765,8443 running (10m) 11s ago 10m 341M - 18.2.1 5be31c24972a 3299b9e265dd 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (59s) 26s ago 7m 486M - 19.0.0-1979-g6cec793d 78f830190839 a0c5bed955ba 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (10m) 11s ago 10m 48.1M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (7m) 26s ago 7m 44.2M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 11s ago 8m 15.4M - 1.5.0 0da6a335fe13 bc06c0b59df5 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (7m) 26s ago 7m 16.7M - 1.5.0 0da6a335fe13 6fe44da459b3 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (7m) 11s ago 6m 121M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (6m) 11s ago 6m 146M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (6m) 11s ago 6m 124M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (5m) 26s ago 5m 135M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (5m) 26s ago 5m 130M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (4m) 26s ago 4m 116M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:48:12.082 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (7m) 11s ago 8m 60.7M - 2.43.0 a07b618ecd1d b1f1f05601f5 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.081+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f18a406d0f0 msgr2=0x7f18a406f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.081+0000 7f18bf573700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f18a406d0f0 0x7f18a406f5b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009480 tx=0x7f18b0000f40 comp rx=0 tx=0).stop 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.081+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18b8107d90 msgr2=0x7f18b81a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.081+0000 7f18bf573700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18b8107d90 0x7f18b81a52a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f18a800d8d0 tx=0x7f18a800dc90 comp rx=0 tx=0).stop 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.082+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 shutdown_connections 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.082+0000 7f18bf573700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f18a406d0f0 0x7f18a406f5b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.082+0000 7f18bf573700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18b8107d90 0x7f18b81a52a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.082+0000 7f18bf573700 1 --2- 172.21.15.70:0/522440464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18b810a700 0x7f18b81a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.082+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 >> 172.21.15.70:0/522440464 conn(0x7f18b806da40 msgr2=0x7f18b806fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.082+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 shutdown_connections 2024-04-02T04:48:12.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.082+0000 7f18bf573700 1 -- 172.21.15.70:0/522440464 wait complete. 2024-04-02T04:48:12.183 INFO:tasks.workunit.client.0.smithi070.stdout:4/996: mknod d1/d3/d78/dad/dbd/d128/d139/c145 0 2024-04-02T04:48:12.192 INFO:tasks.workunit.client.0.smithi070.stdout:4/997: dread d1/d3/f62 [0,4194304] 0 2024-04-02T04:48:12.198 INFO:tasks.workunit.client.0.smithi070.stdout:4/998: mknod d1/d3/d78/dad/d12f/c146 0 2024-04-02T04:48:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.230+0000 7f748aca9700 1 -- 172.21.15.70:0/1180408677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ff590 msgr2=0x7f74840ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.230+0000 7f748aca9700 1 --2- 172.21.15.70:0/1180408677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ff590 0x7f74840ff9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7478009910 tx=0x7f7478009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.230+0000 7f748aca9700 1 -- 172.21.15.70:0/1180408677 shutdown_connections 2024-04-02T04:48:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.230+0000 7f748aca9700 1 --2- 172.21.15.70:0/1180408677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74840ffef0 0x7f7484100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.230+0000 7f748aca9700 1 --2- 172.21.15.70:0/1180408677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ff590 0x7f74840ff9b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.230+0000 7f748aca9700 1 -- 172.21.15.70:0/1180408677 >> 172.21.15.70:0/1180408677 conn(0x7f74840fb190 msgr2=0x7f74840fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.230+0000 7f748aca9700 1 -- 172.21.15.70:0/1180408677 shutdown_connections 2024-04-02T04:48:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.230+0000 7f748aca9700 1 -- 172.21.15.70:0/1180408677 wait complete. 2024-04-02T04:48:12.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:11 smithi070 ceph-mon[29212]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f748aca9700 1 Processor -- start 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f748aca9700 1 -- start start 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f748aca9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74840ff590 0x7f7484198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f748aca9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ffef0 0x7f7484198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f748aca9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841991d0 con 0x7f74840ffef0 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f748aca9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7484199310 con 0x7f74840ff590 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f7483fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ffef0 0x7f7484198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f7483fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ffef0 0x7f7484198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47752/0 (socket says 172.21.15.70:47752) 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f7483fff700 1 -- 172.21.15.70:0/2518495766 learned_addr learned my addr 172.21.15.70:0/2518495766 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.231+0000 7f7488a45700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74840ff590 0x7f7484198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:12.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f7483fff700 1 -- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74840ff590 msgr2=0x7f7484198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:12.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f7483fff700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74840ff590 0x7f7484198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f7483fff700 1 -- 172.21.15.70:0/2518495766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74780095f0 con 0x7f74840ffef0 2024-04-02T04:48:12.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f7483fff700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ffef0 0x7f7484198bb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f747000e920 tx=0x7f747000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:12.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f7481ffb700 1 -- 172.21.15.70:0/2518495766 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f747000c9d0 con 0x7f74840ffef0 2024-04-02T04:48:12.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f748aca9700 1 -- 172.21.15.70:0/2518495766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748419ddc0 con 0x7f74840ffef0 2024-04-02T04:48:12.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f7481ffb700 1 -- 172.21.15.70:0/2518495766 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7470010430 con 0x7f74840ffef0 2024-04-02T04:48:12.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f7481ffb700 1 -- 172.21.15.70:0/2518495766 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7470018650 con 0x7f74840ffef0 2024-04-02T04:48:12.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.232+0000 7f748aca9700 1 -- 172.21.15.70:0/2518495766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748419e2e0 con 0x7f74840ffef0 2024-04-02T04:48:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.234+0000 7f746f7fe700 1 -- 172.21.15.70:0/2518495766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748410b9f0 con 0x7f74840ffef0 2024-04-02T04:48:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.234+0000 7f7481ffb700 1 -- 172.21.15.70:0/2518495766 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f74700105a0 con 0x7f74840ffef0 2024-04-02T04:48:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.234+0000 7f7481ffb700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f747406d0a0 0x7f747406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.235+0000 7f7481ffb700 1 -- 172.21.15.70:0/2518495766 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f7470014070 con 0x7f74840ffef0 2024-04-02T04:48:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.235+0000 7f7488a45700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f747406d0a0 0x7f747406f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.235+0000 7f7488a45700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f747406d0a0 0x7f747406f560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7478009fd0 tx=0x7f7478005940 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:12.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.241+0000 7f7481ffb700 1 -- 172.21.15.70:0/2518495766 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f7470052fc0 con 0x7f74840ffef0 2024-04-02T04:48:12.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:11 smithi179 ceph-mon[36603]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:12.331 INFO:tasks.workunit.client.0.smithi070.stdout:4/999: dwrite d1/d3/f89 [0,4194304] 0 2024-04-02T04:48:12.335 INFO:tasks.workunit.client.0.smithi070.stdout:3/996: sync 2024-04-02T04:48:12.335 INFO:tasks.workunit.client.0.smithi070.stdout:9/989: sync 2024-04-02T04:48:12.336 INFO:tasks.workunit.client.0.smithi070.stdout:6/927: sync 2024-04-02T04:48:12.336 INFO:tasks.workunit.client.0.smithi070.stdout:1/943: sync 2024-04-02T04:48:12.340 INFO:tasks.workunit.client.0.smithi070.stdout:6/928: mknod d0/d55/d79/c117 0 2024-04-02T04:48:12.343 INFO:tasks.workunit.client.0.smithi070.stdout:9/990: rename d4/d13/d89/df5 to d4/d10/d12/d55/d66/d143 0 2024-04-02T04:48:12.346 INFO:tasks.workunit.client.0.smithi070.stdout:1/944: dwrite d0/d54/f87 [4194304,4194304] 0 2024-04-02T04:48:12.347 INFO:tasks.workunit.client.0.smithi070.stdout:3/997: dwrite d10/d32/d4b/dc5/d11c/f134 [0,4194304] 0 2024-04-02T04:48:12.348 INFO:tasks.workunit.client.0.smithi070.stdout:6/929: creat d0/d55/d5c/f118 x:0 0 0 2024-04-02T04:48:12.353 INFO:tasks.workunit.client.0.smithi070.stdout:6/930: creat d0/d55/d79/f119 x:0 0 0 2024-04-02T04:48:12.355 INFO:tasks.workunit.client.0.smithi070.stdout:9/991: rename d4/d10/d12/d27/d60/d9e/fc7 to d4/d10/d11/d7b/d19/d2f/d4d/de2/f144 0 2024-04-02T04:48:12.357 INFO:tasks.workunit.client.0.smithi070.stdout:3/998: fdatasync d10/d17/db2/dbf/dd2/f10e 0 2024-04-02T04:48:12.367 INFO:tasks.workunit.client.0.smithi070.stdout:1/945: link d0/d9/d1d/d22/d56/c65 d0/d54/d64/d121/c132 0 2024-04-02T04:48:12.367 INFO:tasks.workunit.client.0.smithi070.stdout:9/992: dwrite d4/d10/d12/d27/d60/f10c [0,4194304] 0 2024-04-02T04:48:12.369 INFO:tasks.workunit.client.0.smithi070.stdout:3/999: mknod d10/d32/d8a/d8d/c14c 0 2024-04-02T04:48:12.370 INFO:tasks.workunit.client.0.smithi070.stdout:1/946: readlink d0/d7/d1a/d95/l11b 0 2024-04-02T04:48:12.373 INFO:tasks.workunit.client.0.smithi070.stdout:1/947: creat d0/d54/d64/f133 x:0 0 0 2024-04-02T04:48:12.375 INFO:tasks.workunit.client.0.smithi070.stdout:1/948: chown d0/d9/d94 0 1 2024-04-02T04:48:12.375 INFO:tasks.workunit.client.0.smithi070.stdout:9/993: dwrite d4/d10/d12/d27/d60/f117 [0,4194304] 0 2024-04-02T04:48:12.382 INFO:tasks.workunit.client.0.smithi070.stdout:1/949: dread d0/d4c/da2/fd0 [0,4194304] 0 2024-04-02T04:48:12.410 INFO:tasks.workunit.client.0.smithi070.stdout:1/950: dread d0/d9/d1d/d22/d2c/d4f/f92 [0,4194304] 0 2024-04-02T04:48:12.413 INFO:tasks.workunit.client.0.smithi070.stdout:1/951: creat d0/d9/d1d/d22/d56/d79/d89/f134 x:0 0 0 2024-04-02T04:48:12.414 INFO:tasks.workunit.client.0.smithi070.stdout:1/952: chown d0/lae 3 1 2024-04-02T04:48:12.418 INFO:tasks.workunit.client.0.smithi070.stdout:1/953: creat d0/d7/d1a/d66/f135 x:0 0 0 2024-04-02T04:48:12.423 INFO:tasks.workunit.client.0.smithi070.stdout:6/931: dwrite d0/d55/d8b/f9e [0,4194304] 0 2024-04-02T04:48:12.425 INFO:tasks.workunit.client.0.smithi070.stdout:6/932: stat d0/d43/d80/l8e 0 2024-04-02T04:48:12.427 INFO:tasks.workunit.client.0.smithi070.stdout:9/994: dwrite d4/d10/d11/f58 [0,4194304] 0 2024-04-02T04:48:12.434 INFO:tasks.workunit.client.0.smithi070.stdout:6/933: creat d0/d55/d8b/f11a x:0 0 0 2024-04-02T04:48:12.435 INFO:tasks.workunit.client.0.smithi070.stdout:1/954: write d0/db4/f73 [408625,73575] 0 2024-04-02T04:48:12.436 INFO:tasks.workunit.client.0.smithi070.stdout:1/955: dread - d0/d7/d1a/d95/ff6 zero size 2024-04-02T04:48:12.441 INFO:tasks.workunit.client.0.smithi070.stdout:9/995: link d4/d10/d11/d7b/d19/d2f/d39/f12e d4/d10/d11/d7b/f145 0 2024-04-02T04:48:12.442 INFO:tasks.workunit.client.0.smithi070.stdout:1/956: mknod d0/d9/d1d/d22/d2c/de3/c136 0 2024-04-02T04:48:12.444 INFO:tasks.workunit.client.0.smithi070.stdout:9/996: symlink d4/d34/db9/l146 0 2024-04-02T04:48:12.446 INFO:tasks.workunit.client.0.smithi070.stdout:6/934: write d0/d55/d8b/d94/f65 [1972111,18710] 0 2024-04-02T04:48:12.447 INFO:tasks.workunit.client.0.smithi070.stdout:6/935: write d0/d20/d77/d98/fe5 [888122,108028] 0 2024-04-02T04:48:12.453 INFO:tasks.workunit.client.0.smithi070.stdout:1/957: write d0/f2a [3292289,47576] 0 2024-04-02T04:48:12.455 INFO:tasks.workunit.client.0.smithi070.stdout:6/936: rename d0/d20/d5f/de6/da6/dfb/c113 to d0/d55/c11b 0 2024-04-02T04:48:12.459 INFO:tasks.workunit.client.0.smithi070.stdout:1/958: rename d0/d4c/da2/fd0 to d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/df9/f137 0 2024-04-02T04:48:12.461 INFO:tasks.workunit.client.0.smithi070.stdout:6/937: dwrite d0/f26 [0,4194304] 0 2024-04-02T04:48:12.465 INFO:tasks.workunit.client.0.smithi070.stdout:1/959: creat d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/f138 x:0 0 0 2024-04-02T04:48:12.466 INFO:tasks.workunit.client.0.smithi070.stdout:9/997: dread d4/d10/d11/d7b/d19/d2f/d39/da5/fca [0,4194304] 0 2024-04-02T04:48:12.470 INFO:tasks.workunit.client.0.smithi070.stdout:6/938: dwrite d0/fe9 [0,4194304] 0 2024-04-02T04:48:12.475 INFO:tasks.workunit.client.0.smithi070.stdout:6/939: truncate d0/f78 267565 0 2024-04-02T04:48:12.477 INFO:tasks.workunit.client.0.smithi070.stdout:1/960: dwrite d0/d9/d1d/d22/d56/d79/d89/f128 [0,4194304] 0 2024-04-02T04:48:12.478 INFO:tasks.workunit.client.0.smithi070.stdout:1/961: truncate d0/d54/d64/d6d/f100 277936 0 2024-04-02T04:48:12.481 INFO:tasks.workunit.client.0.smithi070.stdout:6/940: creat d0/d55/d79/f11c x:0 0 0 2024-04-02T04:48:12.482 INFO:tasks.workunit.client.0.smithi070.stdout:6/941: chown d0/d43/l3d 0 1 2024-04-02T04:48:12.487 INFO:tasks.workunit.client.0.smithi070.stdout:1/962: dread d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/df9/f137 [0,4194304] 0 2024-04-02T04:48:12.490 INFO:tasks.workunit.client.0.smithi070.stdout:9/998: dwrite d4/fc [0,4194304] 0 2024-04-02T04:48:12.490 INFO:tasks.workunit.client.0.smithi070.stdout:6/942: dwrite d0/d20/d77/d98/fe5 [0,4194304] 0 2024-04-02T04:48:12.498 INFO:tasks.workunit.client.0.smithi070.stdout:9/999: symlink d4/d10/d11/d7b/d19/d2f/d4d/d92/l147 0 2024-04-02T04:48:12.499 INFO:tasks.workunit.client.0.smithi070.stdout:6/943: dwrite d0/d43/db3/fd7 [0,4194304] 0 2024-04-02T04:48:12.510 INFO:tasks.workunit.client.0.smithi070.stdout:1/963: write d0/d7/f43 [5114481,105810] 0 2024-04-02T04:48:12.525 INFO:tasks.workunit.client.0.smithi070.stdout:6/944: write d0/d20/d5f/f6e [559115,83711] 0 2024-04-02T04:48:12.529 INFO:tasks.workunit.client.0.smithi070.stdout:6/945: creat d0/d43/d95/f11d x:0 0 0 2024-04-02T04:48:12.534 INFO:tasks.workunit.client.0.smithi070.stdout:1/964: dwrite d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/df9/f137 [0,4194304] 0 2024-04-02T04:48:12.543 INFO:tasks.workunit.client.0.smithi070.stdout:1/965: symlink d0/d9/d1d/l139 0 2024-04-02T04:48:12.547 INFO:tasks.workunit.client.0.smithi070.stdout:6/946: dread d0/d3/d19/d31/d46/d6b/f70 [0,4194304] 0 2024-04-02T04:48:12.553 INFO:tasks.workunit.client.0.smithi070.stdout:6/947: rename d0/d55/d5c/f118 to d0/d20/dd8/d116/f11e 0 2024-04-02T04:48:12.558 INFO:tasks.workunit.client.0.smithi070.stdout:1/966: dread d0/d7/f10 [4194304,4194304] 0 2024-04-02T04:48:12.558 INFO:tasks.workunit.client.0.smithi070.stdout:6/948: mkdir d0/d3/d19/d31/d46/d11f 0 2024-04-02T04:48:12.560 INFO:tasks.workunit.client.0.smithi070.stdout:6/949: write d0/d3/d19/d31/d46/f10e [205471,120519] 0 2024-04-02T04:48:12.562 INFO:tasks.workunit.client.0.smithi070.stdout:6/950: chown d0/d43/db3/dc7/f66 1883454 1 2024-04-02T04:48:12.562 INFO:tasks.workunit.client.0.smithi070.stdout:6/951: chown d0/d55/d8b/d94/fbd 59 1 2024-04-02T04:48:12.565 INFO:tasks.workunit.client.0.smithi070.stdout:1/967: rename d0/d54/d64/d6d to d0/d9/d1d/d22/d2c/d4f/d77/d8c/d13a 0 2024-04-02T04:48:12.566 INFO:tasks.workunit.client.0.smithi070.stdout:1/968: chown d0/d7/d1a/db9/ff8 765679088 1 2024-04-02T04:48:12.567 INFO:tasks.workunit.client.0.smithi070.stdout:6/952: dread d0/d55/d8b/f9e [0,4194304] 0 2024-04-02T04:48:12.573 INFO:tasks.workunit.client.0.smithi070.stdout:1/969: link d0/d9/d1d/d22/d2c/l62 d0/d9/dff/l13b 0 2024-04-02T04:48:12.575 INFO:tasks.workunit.client.0.smithi070.stdout:6/953: dwrite d0/d43/d95/f10b [0,4194304] 0 2024-04-02T04:48:12.577 INFO:tasks.workunit.client.0.smithi070.stdout:6/954: write d0/d20/d5f/fd9 [70662,2114] 0 2024-04-02T04:48:12.578 INFO:tasks.workunit.client.0.smithi070.stdout:6/955: readlink d0/d20/d77/l81 0 2024-04-02T04:48:12.582 INFO:tasks.workunit.client.0.smithi070.stdout:6/956: creat d0/d20/d5f/de6/da6/dfb/f120 x:0 0 0 2024-04-02T04:48:12.587 INFO:tasks.workunit.client.0.smithi070.stdout:6/957: rename d0/d43/d95/c96 to d0/d20/dd8/d116/c121 0 2024-04-02T04:48:12.593 INFO:tasks.workunit.client.0.smithi070.stdout:6/958: link d0/d3/d19/d31/d37/c64 d0/d43/d95/c122 0 2024-04-02T04:48:12.596 INFO:tasks.workunit.client.0.smithi070.stdout:6/959: mkdir d0/d3/d19/d31/d123 0 2024-04-02T04:48:12.597 INFO:tasks.workunit.client.0.smithi070.stdout:6/960: stat d0/df0/f10d 0 2024-04-02T04:48:12.597 INFO:tasks.workunit.client.0.smithi070.stdout:6/961: dread - d0/d20/dd8/feb zero size 2024-04-02T04:48:12.609 INFO:tasks.workunit.client.0.smithi070.stdout:6/962: dwrite d0/d55/d5c/fad [0,4194304] 0 2024-04-02T04:48:12.610 INFO:tasks.workunit.client.0.smithi070.stdout:6/963: write d0/d20/d77/d98/fe5 [2472839,111719] 0 2024-04-02T04:48:12.612 INFO:tasks.workunit.client.0.smithi070.stdout:6/964: read d0/d20/d77/d98/fe5 [3552837,108004] 0 2024-04-02T04:48:12.616 INFO:tasks.workunit.client.0.smithi070.stdout:6/965: mknod d0/d43/db3/c124 0 2024-04-02T04:48:12.621 INFO:tasks.workunit.client.0.smithi070.stdout:6/966: dwrite d0/d43/d95/f10b [0,4194304] 0 2024-04-02T04:48:12.625 INFO:tasks.workunit.client.0.smithi070.stdout:6/967: mknod d0/d43/db3/dc7/d115/c125 0 2024-04-02T04:48:12.628 INFO:tasks.workunit.client.0.smithi070.stdout:6/968: mkdir d0/d55/d79/d8d/d126 0 2024-04-02T04:48:12.640 INFO:tasks.workunit.client.0.smithi070.stdout:6/969: dread d0/d3/d19/d31/dc1/fc8 [0,4194304] 0 2024-04-02T04:48:12.644 INFO:tasks.workunit.client.0.smithi070.stdout:6/970: dread d0/f78 [0,4194304] 0 2024-04-02T04:48:12.644 INFO:tasks.workunit.client.0.smithi070.stdout:6/971: chown d0/d20/d5f/f6e 50856129 1 2024-04-02T04:48:12.646 INFO:tasks.workunit.client.0.smithi070.stdout:6/972: rmdir d0/d20/dd8 39 2024-04-02T04:48:12.647 INFO:tasks.workunit.client.0.smithi070.stdout:6/973: symlink d0/d3/l127 0 2024-04-02T04:48:12.653 INFO:tasks.workunit.client.0.smithi070.stdout:6/974: write d0/d3/d19/d31/d37/fcb [178928,100286] 0 2024-04-02T04:48:12.657 INFO:tasks.workunit.client.0.smithi070.stdout:6/975: link d0/d43/d95/f11d d0/d43/d80/ded/df7/f128 0 2024-04-02T04:48:12.657 INFO:tasks.workunit.client.0.smithi070.stdout:6/976: chown d0/d3/dc2 104296 1 2024-04-02T04:48:12.658 INFO:tasks.workunit.client.0.smithi070.stdout:6/977: rmdir d0 39 2024-04-02T04:48:12.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.676+0000 7f746f7fe700 1 -- 172.21.15.70:0/2518495766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7484061dc0 con 0x7f74840ffef0 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.677+0000 7f7481ffb700 1 -- 172.21.15.70:0/2518495766 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f747005a150 con 0x7f74840ffef0 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:48:12.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:48:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:48:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:48:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:48:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:48:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-04-02T04:48:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:48:12.681 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:48:12.681 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:48:12.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 -- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f747406d0a0 msgr2=0x7f747406f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:12.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f747406d0a0 0x7f747406f560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7478009fd0 tx=0x7f7478005940 comp rx=0 tx=0).stop 2024-04-02T04:48:12.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 -- 172.21.15.70:0/2518495766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ffef0 msgr2=0x7f7484198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:12.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ffef0 0x7f7484198bb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f747000e920 tx=0x7f747000ec30 comp rx=0 tx=0).stop 2024-04-02T04:48:12.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 -- 172.21.15.70:0/2518495766 shutdown_connections 2024-04-02T04:48:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7f747406d0a0 0x7f747406f560 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74840ff590 0x7f7484198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 --2- 172.21.15.70:0/2518495766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74840ffef0 0x7f7484198bb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 -- 172.21.15.70:0/2518495766 >> 172.21.15.70:0/2518495766 conn(0x7f74840fb190 msgr2=0x7f7484107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 -- 172.21.15.70:0/2518495766 shutdown_connections 2024-04-02T04:48:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.679+0000 7f746f7fe700 1 -- 172.21.15.70:0/2518495766 wait complete. 2024-04-02T04:48:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.782+0000 7fd1264a4700 1 -- 172.21.15.70:0/2237287367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd120100e10 msgr2=0x7fd120101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.782+0000 7fd1264a4700 1 --2- 172.21.15.70:0/2237287367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd120100e10 0x7fd120101270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd114009910 tx=0x7fd114009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.782+0000 7fd1264a4700 1 -- 172.21.15.70:0/2237287367 shutdown_connections 2024-04-02T04:48:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.782+0000 7fd1264a4700 1 --2- 172.21.15.70:0/2237287367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd120100e10 0x7fd120101270 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.782+0000 7fd1264a4700 1 --2- 172.21.15.70:0/2237287367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1200ffc10 0x7fd120100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.782+0000 7fd1264a4700 1 -- 172.21.15.70:0/2237287367 >> 172.21.15.70:0/2237287367 conn(0x7fd1200fb190 msgr2=0x7fd1200fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.782+0000 7fd1264a4700 1 -- 172.21.15.70:0/2237287367 shutdown_connections 2024-04-02T04:48:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.782+0000 7fd1264a4700 1 -- 172.21.15.70:0/2237287367 wait complete. 2024-04-02T04:48:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.783+0000 7fd1264a4700 1 Processor -- start 2024-04-02T04:48:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.783+0000 7fd1264a4700 1 -- start start 2024-04-02T04:48:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd1264a4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1200ffc10 0x7fd1201953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd1264a4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd120100e10 0x7fd1201958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd1264a4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd120195f00 con 0x7fd1200ffc10 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd1264a4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd120196040 con 0x7fd120100e10 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1200ffc10 0x7fd1201953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1200ffc10 0x7fd1201953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47774/0 (socket says 172.21.15.70:47774) 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11ffff700 1 -- 172.21.15.70:0/3163788899 learned_addr learned my addr 172.21.15.70:0/3163788899 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11f7fe700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd120100e10 0x7fd1201958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11ffff700 1 -- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd120100e10 msgr2=0x7fd1201958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11ffff700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd120100e10 0x7fd1201958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11ffff700 1 -- 172.21.15.70:0/3163788899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140095f0 con 0x7fd1200ffc10 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11ffff700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1200ffc10 0x7fd1201953a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd11000be80 tx=0x7fd11000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11d7fa700 1 -- 172.21.15.70:0/3163788899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1100041f0 con 0x7fd1200ffc10 2024-04-02T04:48:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1200685e0 con 0x7fd1200ffc10 2024-04-02T04:48:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11d7fa700 1 -- 172.21.15.70:0/3163788899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd110010ce0 con 0x7fd1200ffc10 2024-04-02T04:48:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd11d7fa700 1 -- 172.21.15.70:0/3163788899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd110009420 con 0x7fd1200ffc10 2024-04-02T04:48:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.784+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd120068b30 con 0x7fd1200ffc10 2024-04-02T04:48:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.785+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd120108700 con 0x7fd1200ffc10 2024-04-02T04:48:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.786+0000 7fd11d7fa700 1 -- 172.21.15.70:0/3163788899 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 91117+0+0 (secure 0 0 0) 0x7fd110004350 con 0x7fd1200ffc10 2024-04-02T04:48:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.786+0000 7fd11d7fa700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fd10806d0a0 0x7fd10806f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.786+0000 7fd11d7fa700 1 -- 172.21.15.70:0/3163788899 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fd11008e2f0 con 0x7fd1200ffc10 2024-04-02T04:48:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.786+0000 7fd11f7fe700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fd10806d0a0 0x7fd10806f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.787+0000 7fd11f7fe700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fd10806d0a0 0x7fd10806f560 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd114005b40 tx=0x7fd114005a90 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:12.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:12.791+0000 7fd11d7fa700 1 -- 172.21.15.70:0/3163788899 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fd11005b930 con 0x7fd1200ffc10 2024-04-02T04:48:12.881 INFO:tasks.workunit.client.0.smithi070.stdout:6/978: rmdir d0/d20/d77/d98/dd5 0 2024-04-02T04:48:12.881 INFO:tasks.workunit.client.0.smithi070.stdout:6/979: read d0/d3/d19/d31/dc1/fc8 [1005786,55612] 0 2024-04-02T04:48:12.882 INFO:tasks.workunit.client.0.smithi070.stdout:6/980: chown d0/d3/d19/d31/d46/c6a 1887199 1 2024-04-02T04:48:12.884 INFO:tasks.workunit.client.0.smithi070.stdout:6/981: mknod d0/d55/d8b/d94/c129 0 2024-04-02T04:48:12.887 INFO:tasks.workunit.client.0.smithi070.stdout:6/982: mkdir d0/d20/dd8/d116/d12a 0 2024-04-02T04:48:12.889 INFO:tasks.workunit.client.0.smithi070.stdout:6/983: chown d0/d55/d8b/d94/f53 3 1 2024-04-02T04:48:12.890 INFO:tasks.workunit.client.0.smithi070.stdout:6/984: mknod d0/d3/d19/d31/d37/c12b 0 2024-04-02T04:48:12.897 INFO:tasks.workunit.client.0.smithi070.stdout:6/985: creat d0/d43/d80/ded/df7/f12c x:0 0 0 2024-04-02T04:48:12.898 INFO:tasks.workunit.client.0.smithi070.stdout:6/986: rename d0 to d0/d3/d19/d107/d12d 22 2024-04-02T04:48:12.901 INFO:tasks.workunit.client.0.smithi070.stdout:6/987: creat d0/d3/d19/d31/d37/f12e x:0 0 0 2024-04-02T04:48:12.904 INFO:tasks.workunit.client.0.smithi070.stdout:6/988: dread d0/df0/f10d [0,4194304] 0 2024-04-02T04:48:12.907 INFO:tasks.workunit.client.0.smithi070.stdout:6/989: unlink d0/d20/dd8/lfa 0 2024-04-02T04:48:12.909 INFO:tasks.workunit.client.0.smithi070.stdout:6/990: truncate d0/d3/d19/d31/fb1 1378209 0 2024-04-02T04:48:12.911 INFO:tasks.workunit.client.0.smithi070.stdout:6/991: read d0/d55/d8b/d94/fbd [3347270,64183] 0 2024-04-02T04:48:12.912 INFO:tasks.workunit.client.0.smithi070.stdout:6/992: mknod d0/d20/d5f/c12f 0 2024-04-02T04:48:12.914 INFO:tasks.workunit.client.0.smithi070.stdout:6/993: chown d0/d55/c11b 27 1 2024-04-02T04:48:12.915 INFO:tasks.workunit.client.0.smithi070.stdout:6/994: chown d0/d3/d19/d31/dc1/ff9 4992997 1 2024-04-02T04:48:12.918 INFO:tasks.workunit.client.0.smithi070.stdout:6/995: rename d0/d43/f9f to d0/d3/d19/d31/d123/f130 0 2024-04-02T04:48:12.919 INFO:tasks.workunit.client.0.smithi070.stdout:6/996: mkdir d0/dfe/d131 0 2024-04-02T04:48:12.920 INFO:tasks.workunit.client.0.smithi070.stdout:6/997: dread - d0/f110 zero size 2024-04-02T04:48:12.922 INFO:tasks.workunit.client.0.smithi070.stdout:6/998: creat d0/d43/d80/f132 x:0 0 0 2024-04-02T04:48:12.922 INFO:tasks.workunit.client.0.smithi070.stdout:6/999: chown d0/d3/d19/d31/d46/ld6 14 1 2024-04-02T04:48:13.058 INFO:tasks.workunit.client.0.smithi070.stdout:1/970: sync 2024-04-02T04:48:13.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.057+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 --> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd120105760 con 0x7fd10806d0a0 2024-04-02T04:48:13.060 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:12 smithi070 ceph-mon[29212]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:13.060 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:12 smithi070 ceph-mon[29212]: pgmap v25: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 69 MiB/s rd, 175 MiB/s wr, 392 op/s 2024-04-02T04:48:13.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:12 smithi070 ceph-mon[29212]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:13.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:12 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2518495766' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:13.061 INFO:tasks.workunit.client.0.smithi070.stdout:1/971: mknod d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/c13c 0 2024-04-02T04:48:13.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.058+0000 7fd11d7fa700 1 -- 172.21.15.70:0/3163788899 <== mgr.24359 v2:172.21.15.179:6828/1862820759 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fd120105760 con 0x7fd10806d0a0 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "1/2 daemons upgraded", 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:48:13.062 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:48:13.063 INFO:tasks.workunit.client.0.smithi070.stdout:1/972: chown d0/d7/d28/c112 515621 1 2024-04-02T04:48:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.060+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fd10806d0a0 msgr2=0x7fd10806f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.060+0000 7fd1264a4700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fd10806d0a0 0x7fd10806f560 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd114005b40 tx=0x7fd114005a90 comp rx=0 tx=0).stop 2024-04-02T04:48:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1200ffc10 msgr2=0x7fd1201953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:13.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1200ffc10 0x7fd1201953a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd11000be80 tx=0x7fd11000dbe0 comp rx=0 tx=0).stop 2024-04-02T04:48:13.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 shutdown_connections 2024-04-02T04:48:13.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:6828/1862820759,v1:172.21.15.179:6829/1862820759] conn(0x7fd10806d0a0 0x7fd10806f560 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:13.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1200ffc10 0x7fd1201953a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:13.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 --2- 172.21.15.70:0/3163788899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd120100e10 0x7fd1201958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:13.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 >> 172.21.15.70:0/3163788899 conn(0x7fd1200fb190 msgr2=0x7fd120104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:13.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 shutdown_connections 2024-04-02T04:48:13.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:13.061+0000 7fd1264a4700 1 -- 172.21.15.70:0/3163788899 wait complete. 2024-04-02T04:48:13.069 INFO:tasks.workunit.client.0.smithi070.stdout:1/973: link d0/d9/d1d/d22/f2d d0/d9/d1d/d22/d2c/d4f/d77/d8c/d13a/db6/f13d 0 2024-04-02T04:48:13.087 INFO:tasks.workunit.client.0.smithi070.stdout:1/974: dwrite d0/d54/d64/fe4 [0,4194304] 0 2024-04-02T04:48:13.099 INFO:tasks.workunit.client.0.smithi070.stdout:1/975: dwrite d0/d9/d1d/d22/d2c/d4f/d77/d8c/d13a/ddd/f10b [0,4194304] 0 2024-04-02T04:48:13.104 INFO:tasks.workunit.client.0.smithi070.stdout:1/976: rmdir d0/db4 39 2024-04-02T04:48:13.108 INFO:tasks.workunit.client.0.smithi070.stdout:1/977: creat d0/d9/d1d/d22/d2c/d4f/d77/d8c/d13a/f13e x:0 0 0 2024-04-02T04:48:13.112 INFO:tasks.workunit.client.0.smithi070.stdout:1/978: creat d0/d9/f13f x:0 0 0 2024-04-02T04:48:13.116 INFO:tasks.workunit.client.0.smithi070.stdout:1/979: mkdir d0/d9/d1d/dc4/d140 0 2024-04-02T04:48:13.120 INFO:tasks.workunit.client.0.smithi070.stdout:1/980: dwrite d0/f1c [0,4194304] 0 2024-04-02T04:48:13.124 INFO:tasks.workunit.client.0.smithi070.stdout:1/981: symlink d0/d7/l141 0 2024-04-02T04:48:13.125 INFO:tasks.workunit.client.0.smithi070.stdout:1/982: write d0/d7/d1a/fda [4627920,42304] 0 2024-04-02T04:48:13.129 INFO:tasks.workunit.client.0.smithi070.stdout:1/983: mknod d0/d9/d1d/d22/d2c/d4f/d77/d8c/d13a/ddd/c142 0 2024-04-02T04:48:13.135 INFO:tasks.workunit.client.0.smithi070.stdout:1/984: dwrite d0/d7/d1a/d66/f88 [0,4194304] 0 2024-04-02T04:48:13.138 INFO:tasks.workunit.client.0.smithi070.stdout:1/985: dread d0/d7/f6c [0,4194304] 0 2024-04-02T04:48:13.158 INFO:tasks.workunit.client.0.smithi070.stdout:1/986: dread d0/d7/f43 [0,4194304] 0 2024-04-02T04:48:13.160 INFO:tasks.workunit.client.0.smithi070.stdout:1/987: creat d0/d9/d1d/d22/d56/d79/d89/f143 x:0 0 0 2024-04-02T04:48:13.178 INFO:tasks.workunit.client.0.smithi070.stdout:1/988: dwrite d0/d9/d1d/d22/d2c/d4f/d77/d8c/faa [0,4194304] 0 2024-04-02T04:48:13.187 INFO:tasks.workunit.client.0.smithi070.stdout:1/989: rename d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/df9 to d0/d54/d144 0 2024-04-02T04:48:13.192 INFO:tasks.workunit.client.0.smithi070.stdout:1/990: creat d0/d7/f145 x:0 0 0 2024-04-02T04:48:13.193 INFO:tasks.workunit.client.0.smithi070.stdout:1/991: rmdir d0/d7/d28/d83 39 2024-04-02T04:48:13.196 INFO:tasks.workunit.client.0.smithi070.stdout:1/992: mkdir d0/d9/d1d/d22/d2c/d4f/d77/d8c/d13a/d146 0 2024-04-02T04:48:13.197 INFO:tasks.workunit.client.0.smithi070.stdout:1/993: mknod d0/d7/d28/d5c/d10c/c147 0 2024-04-02T04:48:13.199 INFO:tasks.workunit.client.0.smithi070.stdout:1/994: creat d0/d4c/f148 x:0 0 0 2024-04-02T04:48:13.203 INFO:tasks.workunit.client.0.smithi070.stdout:1/995: creat d0/db4/f149 x:0 0 0 2024-04-02T04:48:13.206 INFO:tasks.workunit.client.0.smithi070.stdout:1/996: rename d0/d9/d1d/lb to d0/db4/d9f/l14a 0 2024-04-02T04:48:13.206 INFO:tasks.workunit.client.0.smithi070.stdout:1/997: dread - d0/d7/d28/d5c/d6e/f103 zero size 2024-04-02T04:48:13.208 INFO:tasks.workunit.client.0.smithi070.stdout:1/998: symlink d0/d9/d1d/d22/d2c/d4f/d77/d8c/dbc/l14b 0 2024-04-02T04:48:13.208 INFO:tasks.workunit.client.0.smithi070.stdout:1/999: stat d0/db4 0 2024-04-02T04:48:13.212 INFO:tasks.workunit.client.0.smithi070.stderr:+ rm -rf -- ./tmp.9CbRg7ThXK 2024-04-02T04:48:13.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:12 smithi179 ceph-mon[36603]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:13.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:12 smithi179 ceph-mon[36603]: pgmap v25: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 69 MiB/s rd, 175 MiB/s wr, 392 op/s 2024-04-02T04:48:13.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:12 smithi179 ceph-mon[36603]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:13.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:12 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2518495766' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:13 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:13 smithi070 ceph-mon[29212]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:14.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:13 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:14.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:13 smithi179 ceph-mon[36603]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:15.247 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:14 smithi070 ceph-mon[29212]: pgmap v26: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 74 MiB/s rd, 186 MiB/s wr, 410 op/s 2024-04-02T04:48:15.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:14 smithi179 ceph-mon[36603]: pgmap v26: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 74 MiB/s rd, 186 MiB/s wr, 410 op/s 2024-04-02T04:48:17.677 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:17 smithi070 ceph-mon[29212]: pgmap v27: 65 pgs: 65 active+clean; 4.1 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 77 MiB/s rd, 207 MiB/s wr, 439 op/s 2024-04-02T04:48:17.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:17 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:17.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:17 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:17.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:17 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:48:17.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:17 smithi070 ceph-mon[29212]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:48:17.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:17 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:17.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:17 smithi070 ceph-mon[29212]: Upgrade: Need to upgrade myself (mgr.smithi179.labmvs) 2024-04-02T04:48:17.678 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:17 smithi070 ceph-mon[29212]: Upgrade: Need to upgrade myself (mgr.smithi179.labmvs) 2024-04-02T04:48:17.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:17 smithi179 ceph-mon[36603]: pgmap v27: 65 pgs: 65 active+clean; 4.1 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 77 MiB/s rd, 207 MiB/s wr, 439 op/s 2024-04-02T04:48:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:17 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:17 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:17 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:48:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:17 smithi179 ceph-mon[36603]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:48:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:17 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:17 smithi179 ceph-mon[36603]: Upgrade: Need to upgrade myself (mgr.smithi179.labmvs) 2024-04-02T04:48:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:17 smithi179 ceph-mon[36603]: Upgrade: Need to upgrade myself (mgr.smithi179.labmvs) 2024-04-02T04:48:19.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:19 smithi070 ceph-mon[29212]: pgmap v28: 65 pgs: 65 active+clean; 4.1 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 53 MiB/s rd, 134 MiB/s wr, 318 op/s 2024-04-02T04:48:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:19 smithi070 ceph-mon[29212]: Upgrade: Updating mgr.smithi070.igidvr 2024-04-02T04:48:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:19 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:19 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.igidvr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:48:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:19 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.igidvr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:48:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:19 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:48:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:19 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:19 smithi070 ceph-mon[29212]: Deploying daemon mgr.smithi070.igidvr on smithi070 2024-04-02T04:48:19.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:19 smithi179 ceph-mon[36603]: pgmap v28: 65 pgs: 65 active+clean; 4.1 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 53 MiB/s rd, 134 MiB/s wr, 318 op/s 2024-04-02T04:48:19.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:19 smithi179 ceph-mon[36603]: Upgrade: Updating mgr.smithi070.igidvr 2024-04-02T04:48:19.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:19 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:19 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.igidvr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:48:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:19 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.igidvr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:48:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:19 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:48:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:19 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:19 smithi179 ceph-mon[36603]: Deploying daemon mgr.smithi070.igidvr on smithi070 2024-04-02T04:48:21.464 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:21 smithi070 ceph-mon[29212]: pgmap v29: 65 pgs: 65 active+clean; 3.2 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 53 MiB/s rd, 134 MiB/s wr, 395 op/s 2024-04-02T04:48:21.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:21 smithi179 ceph-mon[36603]: pgmap v29: 65 pgs: 65 active+clean; 3.2 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 53 MiB/s rd, 134 MiB/s wr, 395 op/s 2024-04-02T04:48:23.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:23 smithi070 ceph-mon[29212]: pgmap v30: 65 pgs: 65 active+clean; 3.2 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 18 MiB/s rd, 51 MiB/s wr, 203 op/s 2024-04-02T04:48:23.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:23 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:23.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:23 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:23 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:23 smithi179 ceph-mon[36603]: pgmap v30: 65 pgs: 65 active+clean; 3.2 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 18 MiB/s rd, 51 MiB/s wr, 203 op/s 2024-04-02T04:48:23.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:23 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:23 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:23 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:25.430 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:25 smithi070 ceph-mon[29212]: pgmap v31: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 18 MiB/s rd, 52 MiB/s wr, 223 op/s 2024-04-02T04:48:25.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:25 smithi179 ceph-mon[36603]: pgmap v31: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 18 MiB/s rd, 52 MiB/s wr, 223 op/s 2024-04-02T04:48:27.160 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:26 smithi070 ceph-mon[29212]: pgmap v32: 65 pgs: 65 active+clean; 2.1 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 13 MiB/s rd, 41 MiB/s wr, 257 op/s 2024-04-02T04:48:27.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:26 smithi179 ceph-mon[36603]: pgmap v32: 65 pgs: 65 active+clean; 2.1 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 13 MiB/s rd, 41 MiB/s wr, 257 op/s 2024-04-02T04:48:29.254 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-02T04:48:29.254 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-02T04:48:29.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:28 smithi070 ceph-mon[29212]: pgmap v33: 65 pgs: 65 active+clean; 2.1 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-04-02T04:48:29.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:28 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:29.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:28 smithi179 ceph-mon[36603]: pgmap v33: 65 pgs: 65 active+clean; 2.1 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-04-02T04:48:29.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:28 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:29.969 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-02T04:48:29.969 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-02T04:48:30.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:30 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:30.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:30 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:30.333 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:30 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:30.333 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:30 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:31.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:31 smithi070 ceph-mon[29212]: pgmap v34: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 33 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-02T04:48:31.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:31 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:31.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:31 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:31.486 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:31 smithi179 ceph-mon[36603]: pgmap v34: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 33 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-02T04:48:31.486 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:31 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:31.486 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:31 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:32.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:32 smithi070 ceph-mon[29212]: Standby manager daemon smithi070.igidvr restarted 2024-04-02T04:48:32.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:32 smithi070 ceph-mon[29212]: Standby manager daemon smithi070.igidvr started 2024-04-02T04:48:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:32 smithi179 ceph-mon[36603]: Standby manager daemon smithi070.igidvr restarted 2024-04-02T04:48:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:32 smithi179 ceph-mon[36603]: Standby manager daemon smithi070.igidvr started 2024-04-02T04:48:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:33 smithi070 ceph-mon[29212]: pgmap v35: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 23 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-04-02T04:48:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:33 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.igidvr/crt"}]: dispatch 2024-04-02T04:48:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:33 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:48:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:33 smithi070 ceph-mon[29212]: mgrmap e25: smithi179.labmvs(active, since 64s), standbys: smithi070.igidvr 2024-04-02T04:48:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:33 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.igidvr/key"}]: dispatch 2024-04-02T04:48:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:33 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:48:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:33 smithi179 ceph-mon[36603]: pgmap v35: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 23 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-04-02T04:48:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:33 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.igidvr/crt"}]: dispatch 2024-04-02T04:48:33.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:33 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:48:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:33 smithi179 ceph-mon[36603]: mgrmap e25: smithi179.labmvs(active, since 64s), standbys: smithi070.igidvr 2024-04-02T04:48:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:33 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.igidvr/key"}]: dispatch 2024-04-02T04:48:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:33 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:48:35.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:35 smithi070 ceph-mon[29212]: pgmap v36: 65 pgs: 65 active+clean; 773 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-04-02T04:48:35.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:35 smithi179 ceph-mon[36603]: pgmap v36: 65 pgs: 65 active+clean; 773 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-04-02T04:48:37.168 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:36 smithi070 ceph-mon[29212]: pgmap v37: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-04-02T04:48:37.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:36 smithi179 ceph-mon[36603]: pgmap v37: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-04-02T04:48:39.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:38 smithi070 ceph-mon[29212]: pgmap v38: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-04-02T04:48:39.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:38 smithi179 ceph-mon[36603]: pgmap v38: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-04-02T04:48:41.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:40 smithi070 ceph-mon[29212]: pgmap v39: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.6 MiB/s wr, 327 op/s 2024-04-02T04:48:41.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:40 smithi179 ceph-mon[36603]: pgmap v39: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.6 MiB/s wr, 327 op/s 2024-04-02T04:48:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:48:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr fail", "who": "smithi179.labmvs"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr fail", "who": "smithi179.labmvs"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: osdmap e48: 6 total, 6 up, 6 in 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd='[{"prefix": "mgr fail", "who": "smithi179.labmvs"}]': finished 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: mgrmap e26: smithi070.igidvr(active, starting, since 0.0056717s) 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:48:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: Manager daemon smithi070.igidvr is now available 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:48:42.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/trash_purge_schedule"}]: dispatch 2024-04-02T04:48:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:48:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' 2024-04-02T04:48:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 172.21.15.179:0/1666354215' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr fail", "who": "smithi179.labmvs"}]: dispatch 2024-04-02T04:48:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "mgr fail", "who": "smithi179.labmvs"}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: osdmap e48: 6 total, 6 up, 6 in 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24359 ' entity='mgr.smithi179.labmvs' cmd='[{"prefix": "mgr fail", "who": "smithi179.labmvs"}]': finished 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: mgrmap e26: smithi070.igidvr(active, starting, since 0.0056717s) 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:48:42.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: Manager daemon smithi070.igidvr is now available 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:48:42.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/trash_purge_schedule"}]: dispatch 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.207+0000 7fd00ba67700 1 -- 172.21.15.70:0/200850430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 msgr2=0x7fd004100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.207+0000 7fd00ba67700 1 --2- 172.21.15.70:0/200850430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 0x7fd004100270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009910 tx=0x7fcff4009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.207+0000 7fd00ba67700 1 -- 172.21.15.70:0/200850430 shutdown_connections 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.207+0000 7fd00ba67700 1 --2- 172.21.15.70:0/200850430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 0x7fd004100270 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.207+0000 7fd00ba67700 1 --2- 172.21.15.70:0/200850430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd004101220 0x7fd004101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.207+0000 7fd00ba67700 1 -- 172.21.15.70:0/200850430 >> 172.21.15.70:0/200850430 conn(0x7fd0040fa8c0 msgr2=0x7fd0040fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.208+0000 7fd00ba67700 1 -- 172.21.15.70:0/200850430 shutdown_connections 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.208+0000 7fd00ba67700 1 -- 172.21.15.70:0/200850430 wait complete. 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.208+0000 7fd00ba67700 1 Processor -- start 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.208+0000 7fd00ba67700 1 -- start start 2024-04-02T04:48:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd00ba67700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 0x7fd00419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd00ba67700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd004101220 0x7fd00419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd00ba67700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00419d630 con 0x7fd0040ffdf0 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd00ba67700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00419d770 con 0x7fd004101220 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd009803700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 0x7fd00419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd009803700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 0x7fd00419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50808/0 (socket says 172.21.15.70:50808) 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd009803700 1 -- 172.21.15.70:0/632834354 learned_addr learned my addr 172.21.15.70:0/632834354 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd009002700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd004101220 0x7fd00419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd009002700 1 -- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 msgr2=0x7fd00419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd009002700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 0x7fd00419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd009002700 1 -- 172.21.15.70:0/632834354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40095f0 con 0x7fd004101220 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.209+0000 7fd009002700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd004101220 0x7fd00419d010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005b40 tx=0x7fcff400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.210+0000 7fcffaffd700 1 -- 172.21.15.70:0/632834354 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff401d070 con 0x7fd004101220 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.210+0000 7fcffaffd700 1 -- 172.21.15.70:0/632834354 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcff4005dc0 con 0x7fd004101220 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.210+0000 7fcffaffd700 1 -- 172.21.15.70:0/632834354 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff4022470 con 0x7fd004101220 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.210+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0041a21c0 con 0x7fd004101220 2024-04-02T04:48:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.210+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0041a2660 con 0x7fd004101220 2024-04-02T04:48:43.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.211+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0041076b0 con 0x7fd004101220 2024-04-02T04:48:43.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.212+0000 7fcffaffd700 1 -- 172.21.15.70:0/632834354 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 27) v1 ==== 46199+0+0 (secure 0 0 0) 0x7fcff4005160 con 0x7fd004101220 2024-04-02T04:48:43.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.212+0000 7fcffaffd700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fcff0039090 0x7fcff003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:43.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.212+0000 7fcffaffd700 1 -- 172.21.15.70:0/632834354 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fcff404e820 con 0x7fd004101220 2024-04-02T04:48:43.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.212+0000 7fd009803700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fcff0039090 0x7fcff003b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:43.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.213+0000 7fd009803700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fcff0039090 0x7fcff003b550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd00000f660 tx=0x7fd000011040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.219+0000 7fcffaffd700 1 -- 172.21.15.70:0/632834354 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fcff40163b0 con 0x7fd004101220 2024-04-02T04:48:43.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.630+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0041a2db0 con 0x7fcff0039090 2024-04-02T04:48:43.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.633+0000 7fcffaffd700 1 -- 172.21.15.70:0/632834354 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fd0041a2db0 con 0x7fcff0039090 2024-04-02T04:48:43.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.638+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fcff0039090 msgr2=0x7fcff003b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:43.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.638+0000 7fd00ba67700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fcff0039090 0x7fcff003b550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd00000f660 tx=0x7fd000011040 comp rx=0 tx=0).stop 2024-04-02T04:48:43.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.638+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd004101220 msgr2=0x7fd00419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:43.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.638+0000 7fd00ba67700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd004101220 0x7fd00419d010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005b40 tx=0x7fcff400fa00 comp rx=0 tx=0).stop 2024-04-02T04:48:43.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.640+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 shutdown_connections 2024-04-02T04:48:43.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.640+0000 7fd00ba67700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fcff0039090 0x7fcff003b550 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.640+0000 7fd00ba67700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040ffdf0 0x7fd00419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.640+0000 7fd00ba67700 1 --2- 172.21.15.70:0/632834354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd004101220 0x7fd00419d010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.640+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 >> 172.21.15.70:0/632834354 conn(0x7fd0040fa8c0 msgr2=0x7fd004104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:43.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.640+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 shutdown_connections 2024-04-02T04:48:43.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.640+0000 7fd00ba67700 1 -- 172.21.15.70:0/632834354 wait complete. 2024-04-02T04:48:43.663 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:48:43.706 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:43 smithi179 ceph-mon[36603]: [02/Apr/2024:04:48:42] ENGINE Bus STARTING 2024-04-02T04:48:43.706 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:43 smithi179 ceph-mon[36603]: mgrmap e27: smithi070.igidvr(active, since 1.01653s) 2024-04-02T04:48:43.707 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:43 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:43.707 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:43 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:43.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:43 smithi070 ceph-mon[29212]: [02/Apr/2024:04:48:42] ENGINE Bus STARTING 2024-04-02T04:48:43.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:43 smithi070 ceph-mon[29212]: mgrmap e27: smithi070.igidvr(active, since 1.01653s) 2024-04-02T04:48:43.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:43 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:43.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:43 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.786+0000 7f1eca532700 1 -- 172.21.15.70:0/554462152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 msgr2=0x7f1ec4106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.786+0000 7f1eca532700 1 --2- 172.21.15.70:0/554462152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 0x7f1ec4106560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009910 tx=0x7f1eb8009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.786+0000 7f1eca532700 1 -- 172.21.15.70:0/554462152 shutdown_connections 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.786+0000 7f1eca532700 1 --2- 172.21.15.70:0/554462152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 0x7f1ec4106560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.786+0000 7f1eca532700 1 --2- 172.21.15.70:0/554462152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4101840 0x7f1ec4103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.786+0000 7f1eca532700 1 -- 172.21.15.70:0/554462152 >> 172.21.15.70:0/554462152 conn(0x7f1ec40fb190 msgr2=0x7f1ec40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.786+0000 7f1eca532700 1 -- 172.21.15.70:0/554462152 shutdown_connections 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.786+0000 7f1eca532700 1 -- 172.21.15.70:0/554462152 wait complete. 2024-04-02T04:48:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.787+0000 7f1eca532700 1 Processor -- start 2024-04-02T04:48:43.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.787+0000 7f1eca532700 1 -- start start 2024-04-02T04:48:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1eca532700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4101840 0x7f1ec4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1eca532700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 0x7f1ec4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1eca532700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec41991e0 con 0x7f1ec4104170 2024-04-02T04:48:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1eca532700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec4199320 con 0x7f1ec4101840 2024-04-02T04:48:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 0x7f1ec4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4101840 0x7f1ec4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 0x7f1ec4198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50828/0 (socket says 172.21.15.70:50828) 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec37fe700 1 -- 172.21.15.70:0/3211993107 learned_addr learned my addr 172.21.15.70:0/3211993107 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec3fff700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4101840 0x7f1ec4198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:47092/0 (socket says 172.21.15.70:47092) 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec37fe700 1 -- 172.21.15.70:0/3211993107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4101840 msgr2=0x7f1ec4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec37fe700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4101840 0x7f1ec4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec37fe700 1 -- 172.21.15.70:0/3211993107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb80095f0 con 0x7f1ec4104170 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.788+0000 7f1ec37fe700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 0x7f1ec4198bc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8000c00 tx=0x7f1eb800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.789+0000 7f1ec17fa700 1 -- 172.21.15.70:0/3211993107 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1eb801d070 con 0x7f1ec4104170 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.789+0000 7f1ec17fa700 1 -- 172.21.15.70:0/3211993107 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1eb8005640 con 0x7f1ec4104170 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.789+0000 7f1ec17fa700 1 -- 172.21.15.70:0/3211993107 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1eb8022470 con 0x7f1ec4104170 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.789+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec419dd70 con 0x7f1ec4104170 2024-04-02T04:48:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.789+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec419e260 con 0x7f1ec4104170 2024-04-02T04:48:43.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.790+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec40fcd80 con 0x7f1ec4104170 2024-04-02T04:48:43.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.790+0000 7f1ec17fa700 1 -- 172.21.15.70:0/3211993107 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 27) v1 ==== 46199+0+0 (secure 0 0 0) 0x7f1eb800fa70 con 0x7f1ec4104170 2024-04-02T04:48:43.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.791+0000 7f1ec17fa700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f1eac0390e0 0x7f1eac03b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:43.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.791+0000 7f1ec17fa700 1 -- 172.21.15.70:0/3211993107 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f1eb804e770 con 0x7f1ec4104170 2024-04-02T04:48:43.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.791+0000 7f1ec3fff700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f1eac0390e0 0x7f1eac03b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:43.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.791+0000 7f1ec3fff700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f1eac0390e0 0x7f1eac03b5a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009c60 tx=0x7f1eb4009260 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:43.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:43.797+0000 7f1ec17fa700 1 -- 172.21.15.70:0/3211993107 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f1eb80052a0 con 0x7f1ec4104170 2024-04-02T04:48:44.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.098+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ec40611d0 con 0x7f1eac0390e0 2024-04-02T04:48:44.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.099+0000 7f1ec17fa700 1 -- 172.21.15.70:0/3211993107 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f1ec40611d0 con 0x7f1eac0390e0 2024-04-02T04:48:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f1eac0390e0 msgr2=0x7f1eac03b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f1eac0390e0 0x7f1eac03b5a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009c60 tx=0x7f1eb4009260 comp rx=0 tx=0).stop 2024-04-02T04:48:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 msgr2=0x7f1ec4198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 0x7f1ec4198bc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8000c00 tx=0x7f1eb800f690 comp rx=0 tx=0).stop 2024-04-02T04:48:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 shutdown_connections 2024-04-02T04:48:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f1eac0390e0 0x7f1eac03b5a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4101840 0x7f1ec4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 --2- 172.21.15.70:0/3211993107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ec4104170 0x7f1ec4198bc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 >> 172.21.15.70:0/3211993107 conn(0x7f1ec40fb190 msgr2=0x7f1ec4104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:44.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 shutdown_connections 2024-04-02T04:48:44.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.103+0000 7f1eca532700 1 -- 172.21.15.70:0/3211993107 wait complete. 2024-04-02T04:48:44.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.243+0000 7f3728edb700 1 -- 172.21.15.70:0/3005510071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37240ffbb0 msgr2=0x7f37240fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:44.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.243+0000 7f3728edb700 1 --2- 172.21.15.70:0/3005510071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37240ffbb0 0x7f37240fffd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3714009910 tx=0x7f3714009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:44.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.243+0000 7f3728edb700 1 -- 172.21.15.70:0/3005510071 shutdown_connections 2024-04-02T04:48:44.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.243+0000 7f3728edb700 1 --2- 172.21.15.70:0/3005510071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3724100db0 0x7f3724101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.243+0000 7f3728edb700 1 --2- 172.21.15.70:0/3005510071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37240ffbb0 0x7f37240fffd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.243+0000 7f3728edb700 1 -- 172.21.15.70:0/3005510071 >> 172.21.15.70:0/3005510071 conn(0x7f37240fb150 msgr2=0x7f37240fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:44.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.243+0000 7f3728edb700 1 -- 172.21.15.70:0/3005510071 shutdown_connections 2024-04-02T04:48:44.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.243+0000 7f3728edb700 1 -- 172.21.15.70:0/3005510071 wait complete. 2024-04-02T04:48:44.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.244+0000 7f3728edb700 1 Processor -- start 2024-04-02T04:48:44.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.244+0000 7f3728edb700 1 -- start start 2024-04-02T04:48:44.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.244+0000 7f3728edb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37240ffbb0 0x7f37241953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.244+0000 7f3728edb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3724100db0 0x7f3724195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.244+0000 7f3728edb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724195f40 con 0x7f3724100db0 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.244+0000 7f3728edb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724196080 con 0x7f37240ffbb0 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f3721d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3724100db0 0x7f3724195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f372259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37240ffbb0 0x7f37241953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f3721d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3724100db0 0x7f3724195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50858/0 (socket says 172.21.15.70:50858) 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f3721d9b700 1 -- 172.21.15.70:0/3153981406 learned_addr learned my addr 172.21.15.70:0/3153981406 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f372259c700 1 -- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3724100db0 msgr2=0x7f3724195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f372259c700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3724100db0 0x7f3724195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f372259c700 1 -- 172.21.15.70:0/3153981406 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140095f0 con 0x7f37240ffbb0 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f3721d9b700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3724100db0 0x7f3724195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f372259c700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37240ffbb0 0x7f37241953e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3714000c00 tx=0x7f371400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f37137fe700 1 -- 172.21.15.70:0/3153981406 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f371401d070 con 0x7f37240ffbb0 2024-04-02T04:48:44.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f37137fe700 1 -- 172.21.15.70:0/3153981406 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3714005dc0 con 0x7f37240ffbb0 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f37137fe700 1 -- 172.21.15.70:0/3153981406 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3714022470 con 0x7f37240ffbb0 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3724068f50 con 0x7f37240ffbb0 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.245+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3724069410 con 0x7f37240ffbb0 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.246+0000 7f37137fe700 1 -- 172.21.15.70:0/3153981406 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 27) v1 ==== 46199+0+0 (secure 0 0 0) 0x7f37140225d0 con 0x7f37240ffbb0 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.246+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3724108710 con 0x7f37240ffbb0 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.247+0000 7f37137fe700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f370c039090 0x7f370c03b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.247+0000 7f37137fe700 1 -- 172.21.15.70:0/3153981406 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f371404e630 con 0x7f37240ffbb0 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.247+0000 7f3721d9b700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f370c039090 0x7f370c03b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:44.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.247+0000 7f3721d9b700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f370c039090 0x7f370c03b550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3718011550 tx=0x7f3718009260 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:44.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.251+0000 7f37137fe700 1 -- 172.21.15.70:0/3153981406 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f371402a350 con 0x7f37240ffbb0 2024-04-02T04:48:44.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.514+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3724105700 con 0x7f370c039090 2024-04-02T04:48:44.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.530+0000 7f37137fe700 1 -- 172.21.15.70:0/3153981406 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3724105700 con 0x7f370c039090 2024-04-02T04:48:44.534 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:48:44.534 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (7m) 15s ago 9m 24.2M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:48:44.534 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (9m) 15s ago 9m 7730k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:48:44.534 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (8m) 59s ago 8m 7365k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:48:44.534 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (9m) 15s ago 9m 6665k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:48:44.534 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (8m) 59s ago 8m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (7m) 15s ago 9m 87.8M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (3m) 15s ago 3m 223M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (3m) 15s ago 3m 16.5M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (3m) 59s ago 3m 14.7M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (3m) 59s ago 3m 16.3M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (22s) 15s ago 10m 63.7M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (91s) 59s ago 8m 486M - 19.0.0-1979-g6cec793d 78f830190839 a0c5bed955ba 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (10m) 15s ago 10m 48.9M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (8m) 59s ago 8m 44.2M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (9m) 15s ago 9m 15.1M - 1.5.0 0da6a335fe13 bc06c0b59df5 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (8m) 59s ago 8m 16.7M - 1.5.0 0da6a335fe13 6fe44da459b3 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (7m) 15s ago 7m 196M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (7m) 15s ago 7m 220M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (6m) 15s ago 6m 183M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (6m) 59s ago 6m 135M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:48:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (5m) 59s ago 5m 130M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:48:44.536 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (5m) 59s ago 5m 116M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:48:44.536 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (28s) 15s ago 8m 39.6M - 2.43.0 a07b618ecd1d 7426bbefce94 2024-04-02T04:48:44.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f370c039090 msgr2=0x7f370c03b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f370c039090 0x7f370c03b550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3718011550 tx=0x7f3718009260 comp rx=0 tx=0).stop 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37240ffbb0 msgr2=0x7f37241953e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37240ffbb0 0x7f37241953e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3714000c00 tx=0x7f371400fa00 comp rx=0 tx=0).stop 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 shutdown_connections 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f370c039090 0x7f370c03b550 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37240ffbb0 0x7f37241953e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 --2- 172.21.15.70:0/3153981406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3724100db0 0x7f3724195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.532+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 >> 172.21.15.70:0/3153981406 conn(0x7f37240fb150 msgr2=0x7f3724103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.533+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 shutdown_connections 2024-04-02T04:48:44.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.533+0000 7f3728edb700 1 -- 172.21.15.70:0/3153981406 wait complete. 2024-04-02T04:48:44.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:44 smithi179 ceph-mon[36603]: [02/Apr/2024:04:48:42] ENGINE Serving on http://172.21.15.70:8765 2024-04-02T04:48:44.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:44 smithi179 ceph-mon[36603]: pgmap v3: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:44.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:44 smithi179 ceph-mon[36603]: [02/Apr/2024:04:48:42] ENGINE Serving on https://172.21.15.70:7150 2024-04-02T04:48:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:44 smithi179 ceph-mon[36603]: [02/Apr/2024:04:48:42] ENGINE Bus STARTED 2024-04-02T04:48:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:44 smithi179 ceph-mon[36603]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:44.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.662+0000 7fe212718700 1 -- 172.21.15.70:0/3124759480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204093e30 msgr2=0x7fe204094250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:44.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.662+0000 7fe212718700 1 --2- 172.21.15.70:0/3124759480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204093e30 0x7fe204094250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe208009910 tx=0x7fe208009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:44.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.662+0000 7fe212718700 1 -- 172.21.15.70:0/3124759480 shutdown_connections 2024-04-02T04:48:44.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.662+0000 7fe212718700 1 --2- 172.21.15.70:0/3124759480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe204094ba0 0x7fe204095020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.662+0000 7fe212718700 1 --2- 172.21.15.70:0/3124759480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204093e30 0x7fe204094250 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.662+0000 7fe212718700 1 -- 172.21.15.70:0/3124759480 >> 172.21.15.70:0/3124759480 conn(0x7fe20408f770 msgr2=0x7fe204091bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:44.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.662+0000 7fe212718700 1 -- 172.21.15.70:0/3124759480 shutdown_connections 2024-04-02T04:48:44.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.664+0000 7fe212718700 1 -- 172.21.15.70:0/3124759480 wait complete. 2024-04-02T04:48:44.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.665+0000 7fe212718700 1 Processor -- start 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.665+0000 7fe212718700 1 -- start start 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.665+0000 7fe212718700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe204093e30 0x7fe2040a2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.665+0000 7fe212718700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204094ba0 0x7fe2040a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.666+0000 7fe212718700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040a32a0 con 0x7fe204094ba0 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.666+0000 7fe212718700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20413f5f0 con 0x7fe204093e30 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.666+0000 7fe210f15700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204094ba0 0x7fe2040a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.666+0000 7fe210f15700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204094ba0 0x7fe2040a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50878/0 (socket says 172.21.15.70:50878) 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.666+0000 7fe210f15700 1 -- 172.21.15.70:0/1385047924 learned_addr learned my addr 172.21.15.70:0/1385047924 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.667+0000 7fe210f15700 1 -- 172.21.15.70:0/1385047924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe204093e30 msgr2=0x7fe2040a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.667+0000 7fe210f15700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe204093e30 0x7fe2040a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:44.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.667+0000 7fe210f15700 1 -- 172.21.15.70:0/1385047924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080095f0 con 0x7fe204094ba0 2024-04-02T04:48:44.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.667+0000 7fe210f15700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204094ba0 0x7fe2040a2c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00d8d0 tx=0x7fe1fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:44.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.668+0000 7fe2027fc700 1 -- 172.21.15.70:0/1385047924 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1fc009940 con 0x7fe204094ba0 2024-04-02T04:48:44.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.668+0000 7fe2027fc700 1 -- 172.21.15.70:0/1385047924 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe1fc010460 con 0x7fe204094ba0 2024-04-02T04:48:44.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.668+0000 7fe2027fc700 1 -- 172.21.15.70:0/1385047924 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1fc00f5f0 con 0x7fe204094ba0 2024-04-02T04:48:44.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.668+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe204150800 con 0x7fe204094ba0 2024-04-02T04:48:44.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.668+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe204150cc0 con 0x7fe204094ba0 2024-04-02T04:48:44.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.669+0000 7fe2027fc700 1 -- 172.21.15.70:0/1385047924 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 27) v1 ==== 46199+0+0 (secure 0 0 0) 0x7fe1fc00f750 con 0x7fe204094ba0 2024-04-02T04:48:44.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.669+0000 7fe2027fc700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fe1f8039110 0x7fe1f803b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:44.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.669+0000 7fe2027fc700 1 -- 172.21.15.70:0/1385047924 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fe1fc022680 con 0x7fe204094ba0 2024-04-02T04:48:44.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.670+0000 7fe211716700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fe1f8039110 0x7fe1f803b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:44.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.671+0000 7fe211716700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fe1f8039110 0x7fe1f803b5d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe208000c00 tx=0x7fe20800b540 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:44.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.671+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2040a4ce0 con 0x7fe204094ba0 2024-04-02T04:48:44.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.680+0000 7fe2027fc700 1 -- 172.21.15.70:0/1385047924 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fe1fc016020 con 0x7fe204094ba0 2024-04-02T04:48:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:44.697+0000 7fe2027fc700 1 -- 172.21.15.70:0/1385047924 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 28) v1 ==== 46255+0+0 (secure 0 0 0) 0x7fe1fc022300 con 0x7fe204094ba0 2024-04-02T04:48:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:44 smithi070 ceph-mon[29212]: [02/Apr/2024:04:48:42] ENGINE Serving on http://172.21.15.70:8765 2024-04-02T04:48:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:44 smithi070 ceph-mon[29212]: pgmap v3: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:44 smithi070 ceph-mon[29212]: [02/Apr/2024:04:48:42] ENGINE Serving on https://172.21.15.70:7150 2024-04-02T04:48:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:44 smithi070 ceph-mon[29212]: [02/Apr/2024:04:48:42] ENGINE Bus STARTED 2024-04-02T04:48:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:44 smithi070 ceph-mon[29212]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:45.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.084+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe2041513b0 con 0x7fe204094ba0 2024-04-02T04:48:45.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.085+0000 7fe2027fc700 1 -- 172.21.15.70:0/1385047924 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fe1fc00fa00 con 0x7fe204094ba0 2024-04-02T04:48:45.089 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:48:45.089 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:48:45.089 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:48:45.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:48:45.089 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:48:45.089 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:48:45.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:48:45.090 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.087+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fe1f8039110 msgr2=0x7fe1f803b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.087+0000 7fe212718700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fe1f8039110 0x7fe1f803b5d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe208000c00 tx=0x7fe20800b540 comp rx=0 tx=0).stop 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.087+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204094ba0 msgr2=0x7fe2040a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.087+0000 7fe212718700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204094ba0 0x7fe2040a2c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00d8d0 tx=0x7fe1fc00dc90 comp rx=0 tx=0).stop 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.088+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 shutdown_connections 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.088+0000 7fe212718700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fe1f8039110 0x7fe1f803b5d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.088+0000 7fe212718700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe204093e30 0x7fe2040a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.088+0000 7fe212718700 1 --2- 172.21.15.70:0/1385047924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe204094ba0 0x7fe2040a2c80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.088+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 >> 172.21.15.70:0/1385047924 conn(0x7fe20408f770 msgr2=0x7fe204097dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.088+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 shutdown_connections 2024-04-02T04:48:45.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.088+0000 7fe212718700 1 -- 172.21.15.70:0/1385047924 wait complete. 2024-04-02T04:48:45.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.206+0000 7ff85944c700 1 -- 172.21.15.70:0/439145372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 msgr2=0x7ff854103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.206+0000 7ff85944c700 1 --2- 172.21.15.70:0/439145372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 0x7ff854103b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff844009910 tx=0x7ff844009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.206+0000 7ff85944c700 1 -- 172.21.15.70:0/439145372 shutdown_connections 2024-04-02T04:48:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.206+0000 7ff85944c700 1 --2- 172.21.15.70:0/439145372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8541040d0 0x7ff8541064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.206+0000 7ff85944c700 1 --2- 172.21.15.70:0/439145372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 0x7ff854103b90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.206+0000 7ff85944c700 1 -- 172.21.15.70:0/439145372 >> 172.21.15.70:0/439145372 conn(0x7ff8540fb190 msgr2=0x7ff8540fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.206+0000 7ff85944c700 1 -- 172.21.15.70:0/439145372 shutdown_connections 2024-04-02T04:48:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.206+0000 7ff85944c700 1 -- 172.21.15.70:0/439145372 wait complete. 2024-04-02T04:48:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff85944c700 1 Processor -- start 2024-04-02T04:48:45.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff85944c700 1 -- start start 2024-04-02T04:48:45.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff85944c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 0x7ff854194240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff85944c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8541040d0 0x7ff854194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff85944c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff854194da0 con 0x7ff8541017a0 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff85944c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff854194ee0 con 0x7ff8541040d0 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff852ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 0x7ff854194240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff8527fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8541040d0 0x7ff854194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.207+0000 7ff852ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 0x7ff854194240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50904/0 (socket says 172.21.15.70:50904) 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff852ffd700 1 -- 172.21.15.70:0/3794358355 learned_addr learned my addr 172.21.15.70:0/3794358355 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff8527fc700 1 -- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 msgr2=0x7ff854194240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff8527fc700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 0x7ff854194240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff8527fc700 1 -- 172.21.15.70:0/3794358355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440095f0 con 0x7ff8541040d0 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff852ffd700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 0x7ff854194240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff8527fc700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8541040d0 0x7ff854194780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff84800e970 tx=0x7ff84800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:45.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff83bfff700 1 -- 172.21.15.70:0/3794358355 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff84800c9d0 con 0x7ff8541040d0 2024-04-02T04:48:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff854199990 con 0x7ff8541040d0 2024-04-02T04:48:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff83bfff700 1 -- 172.21.15.70:0/3794358355 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff848010430 con 0x7ff8541040d0 2024-04-02T04:48:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff854199ee0 con 0x7ff8541040d0 2024-04-02T04:48:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.208+0000 7ff83bfff700 1 -- 172.21.15.70:0/3794358355 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff848018650 con 0x7ff8541040d0 2024-04-02T04:48:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.210+0000 7ff83bfff700 1 -- 172.21.15.70:0/3794358355 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 28) v1 ==== 46255+0+0 (secure 0 0 0) 0x7ff848018910 con 0x7ff8541040d0 2024-04-02T04:48:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.210+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85418e400 con 0x7ff8541040d0 2024-04-02T04:48:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.210+0000 7ff83bfff700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff83c039110 0x7ff83c03b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.210+0000 7ff83bfff700 1 -- 172.21.15.70:0/3794358355 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7ff848014070 con 0x7ff8541040d0 2024-04-02T04:48:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.210+0000 7ff852ffd700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff83c039110 0x7ff83c03b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.210+0000 7ff852ffd700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff83c039110 0x7ff83c03b5d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff844000c00 tx=0x7ff844009f90 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:45.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.221+0000 7ff83bfff700 1 -- 172.21.15.70:0/3794358355 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7ff84801fc30 con 0x7ff8541040d0 2024-04-02T04:48:45.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.535+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8540611d0 con 0x7ff83c039110 2024-04-02T04:48:45.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.536+0000 7ff83bfff700 1 -- 172.21.15.70:0/3794358355 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff8540611d0 con 0x7ff83c039110 2024-04-02T04:48:45.540 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/2 daemons upgraded", 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:48:45.541 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff83c039110 msgr2=0x7ff83c03b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff83c039110 0x7ff83c03b5d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff844000c00 tx=0x7ff844009f90 comp rx=0 tx=0).stop 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8541040d0 msgr2=0x7ff854194780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8541040d0 0x7ff854194780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff84800e970 tx=0x7ff84800ec80 comp rx=0 tx=0).stop 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 shutdown_connections 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff83c039110 0x7ff83c03b5d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8541017a0 0x7ff854194240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 --2- 172.21.15.70:0/3794358355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8541040d0 0x7ff854194780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 >> 172.21.15.70:0/3794358355 conn(0x7ff8540fb190 msgr2=0x7ff8540fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 shutdown_connections 2024-04-02T04:48:45.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:48:45.539+0000 7ff85944c700 1 -- 172.21.15.70:0/3794358355 wait complete. 2024-04-02T04:48:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:45 smithi070 ceph-mon[29212]: pgmap v4: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:45 smithi070 ceph-mon[29212]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:45 smithi070 ceph-mon[29212]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:45.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:45 smithi070 ceph-mon[29212]: mgrmap e28: smithi070.igidvr(active, since 3s) 2024-04-02T04:48:45.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:45 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1385047924' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:45.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:45 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:45.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:45 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:45 smithi179 ceph-mon[36603]: pgmap v4: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:45 smithi179 ceph-mon[36603]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:45 smithi179 ceph-mon[36603]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:45 smithi179 ceph-mon[36603]: mgrmap e28: smithi070.igidvr(active, since 3s) 2024-04-02T04:48:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:45 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1385047924' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:45 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:45 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:46 smithi070 ceph-mon[29212]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:46.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:46 smithi179 ceph-mon[36603]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:47.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:47 smithi179 ceph-mon[36603]: pgmap v5: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:47.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:47 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:47.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:47 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:48.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:47 smithi070 ceph-mon[29212]: pgmap v5: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:48.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:47 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:48.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:47 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:49 smithi179 ceph-mon[36603]: pgmap v6: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:49 smithi179 ceph-mon[36603]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:48:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:49 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1346161201' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:48:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:49 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1346161201' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:48:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:49 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1346161201' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:48:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:49 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1346161201' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:48:50.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:49 smithi070 ceph-mon[29212]: pgmap v6: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:50.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:49 smithi070 ceph-mon[29212]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:48:50.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:49 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1346161201' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:48:50.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:49 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1346161201' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:48:50.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:49 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1346161201' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:48:50.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:49 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1346161201' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:48:50.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:50 smithi179 ceph-mon[36603]: mgrmap e29: smithi070.igidvr(active, since 7s), standbys: smithi179.labmvs 2024-04-02T04:48:50.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:50 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi179.labmvs", "id": "smithi179.labmvs"}]: dispatch 2024-04-02T04:48:51.013 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:50 smithi070 ceph-mon[29212]: mgrmap e29: smithi070.igidvr(active, since 7s), standbys: smithi179.labmvs 2024-04-02T04:48:51.014 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:50 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi179.labmvs", "id": "smithi179.labmvs"}]: dispatch 2024-04-02T04:48:52.025 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:51 smithi070 ceph-mon[29212]: pgmap v7: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.0 MiB/s wr, 233 op/s 2024-04-02T04:48:52.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:51 smithi179 ceph-mon[36603]: pgmap v7: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.0 MiB/s wr, 233 op/s 2024-04-02T04:48:54.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:53 smithi070 ceph-mon[29212]: pgmap v8: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 814 KiB/s wr, 181 op/s 2024-04-02T04:48:54.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:53 smithi179 ceph-mon[36603]: pgmap v8: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 814 KiB/s wr, 181 op/s 2024-04-02T04:48:55.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:55 smithi070 ceph-mon[29212]: pgmap v9: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 1.2 MiB/s wr, 329 op/s 2024-04-02T04:48:56.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:55 smithi179 ceph-mon[36603]: pgmap v9: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 1.2 MiB/s wr, 329 op/s 2024-04-02T04:48:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:57 smithi070 ceph-mon[29212]: pgmap v10: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-04-02T04:48:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:58.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:57 smithi179 ceph-mon[36603]: pgmap v10: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-04-02T04:48:58.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:58.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:58.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:59.900 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:59 smithi070 ceph-mon[29212]: pgmap v11: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-04-02T04:48:59.900 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:48:59.900 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:48:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:00.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:59 smithi179 ceph-mon[36603]: pgmap v11: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-04-02T04:49:00.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:00.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:48:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:01.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:01 smithi070 ceph-mon[29212]: pgmap v12: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s rd, 1.8 MiB/s wr, 449 op/s 2024-04-02T04:49:02.047 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:01 smithi179 ceph-mon[36603]: pgmap v12: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s rd, 1.8 MiB/s wr, 449 op/s 2024-04-02T04:49:03.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:03 smithi179 ceph-mon[36603]: pgmap v13: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-04-02T04:49:03.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:03 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:03.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:03 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:03.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:03 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:03.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:03 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:04.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:03 smithi070 ceph-mon[29212]: pgmap v13: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-04-02T04:49:04.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:03 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:04.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:03 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:04.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:03 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:04.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:03 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:05.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:49:05.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:49:05.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:49:05.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:49:05.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:04 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:49:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:49:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:49:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:49:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:04 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:05.770 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:05 smithi070 ceph-mon[29212]: pgmap v14: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 786 op/s 2024-04-02T04:49:05.770 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:05 smithi070 ceph-mon[29212]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:49:05.770 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:05 smithi070 ceph-mon[29212]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:49:05.771 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:05 smithi070 ceph-mon[29212]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:49:05.771 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:05 smithi070 ceph-mon[29212]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:49:05.771 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:05 smithi070 ceph-mon[29212]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-04-02T04:49:06.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:05 smithi179 ceph-mon[36603]: pgmap v14: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 786 op/s 2024-04-02T04:49:06.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:05 smithi179 ceph-mon[36603]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:49:06.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:05 smithi179 ceph-mon[36603]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:49:06.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:05 smithi179 ceph-mon[36603]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:49:06.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:05 smithi179 ceph-mon[36603]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:49:06.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:05 smithi179 ceph-mon[36603]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-04-02T04:49:06.902 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:06 smithi070 ceph-mon[29212]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-04-02T04:49:07.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:06 smithi179 ceph-mon[36603]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-04-02T04:49:07.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:07 smithi070 ceph-mon[29212]: pgmap v15: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 620 op/s 2024-04-02T04:49:08.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:07 smithi179 ceph-mon[36603]: pgmap v15: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 620 op/s 2024-04-02T04:49:09.948 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:09 smithi070 ceph-mon[29212]: pgmap v16: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 620 op/s 2024-04-02T04:49:10.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:09 smithi179 ceph-mon[36603]: pgmap v16: 65 pgs: 65 active+clean; 274 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 620 op/s 2024-04-02T04:49:11.875 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:11 smithi179 ceph-mon[36603]: pgmap v17: 65 pgs: 65 active+clean; 264 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.2 MiB/s wr, 1.11k op/s 2024-04-02T04:49:11.876 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:11 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:11.876 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:11 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:11.876 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:11 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:49:11.876 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:11 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:11.939 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:11 smithi070 ceph-mon[29212]: pgmap v17: 65 pgs: 65 active+clean; 264 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.2 MiB/s wr, 1.11k op/s 2024-04-02T04:49:11.939 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:11 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:11.939 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:11 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:11.939 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:11 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:49:11.939 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:11 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:12.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:12 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:49:12.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:12 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:12.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:12 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:49:12.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:12 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:13.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:13 smithi179 ceph-mon[36603]: pgmap v18: 65 pgs: 65 active+clean; 264 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.5 MiB/s wr, 960 op/s 2024-04-02T04:49:13.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:13 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:49:13.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:13 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:49:13.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:13 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:13 smithi070 ceph-mon[29212]: pgmap v18: 65 pgs: 65 active+clean; 264 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.5 MiB/s wr, 960 op/s 2024-04-02T04:49:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:13 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:49:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:13 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:49:14.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:13 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:14.798 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:14 smithi179 ceph-mon[36603]: Upgrade: Updating mgr.smithi179.labmvs 2024-04-02T04:49:14.798 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:14 smithi179 ceph-mon[36603]: Deploying daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:49:15.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:14 smithi070 ceph-mon[29212]: Upgrade: Updating mgr.smithi179.labmvs 2024-04-02T04:49:15.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:14 smithi070 ceph-mon[29212]: Deploying daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:49:15.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 -- 172.21.15.70:0/2024216685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 msgr2=0x7f4a64103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:15.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 --2- 172.21.15.70:0/2024216685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 0x7f4a64103bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009910 tx=0x7f4a60009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:15.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 -- 172.21.15.70:0/2024216685 shutdown_connections 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 --2- 172.21.15.70:0/2024216685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a64104130 0x7f4a64106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 --2- 172.21.15.70:0/2024216685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 0x7f4a64103bf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 -- 172.21.15.70:0/2024216685 >> 172.21.15.70:0/2024216685 conn(0x7f4a640fb190 msgr2=0x7f4a640fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 -- 172.21.15.70:0/2024216685 shutdown_connections 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 -- 172.21.15.70:0/2024216685 wait complete. 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 Processor -- start 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.687+0000 7f4a6b312700 1 -- start start 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a6b312700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 0x7f4a64196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a6b312700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a64104130 0x7f4a641969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a6b312700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a64196fe0 con 0x7f4a64101800 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a6b312700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a64197120 con 0x7f4a64104130 2024-04-02T04:49:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a690ae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 0x7f4a64196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a690ae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 0x7f4a64196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45294/0 (socket says 172.21.15.70:45294) 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a690ae700 1 -- 172.21.15.70:0/4092961801 learned_addr learned my addr 172.21.15.70:0/4092961801 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a688ad700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a64104130 0x7f4a641969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a690ae700 1 -- 172.21.15.70:0/4092961801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a64104130 msgr2=0x7f4a641969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a690ae700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a64104130 0x7f4a641969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a690ae700 1 -- 172.21.15.70:0/4092961801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a600095f0 con 0x7f4a64101800 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.688+0000 7f4a688ad700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a64104130 0x7f4a641969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.689+0000 7f4a690ae700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 0x7f4a64196480 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4a60000c00 tx=0x7f4a6000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.689+0000 7f4a5a7fc700 1 -- 172.21.15.70:0/4092961801 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a6001d070 con 0x7f4a64101800 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.689+0000 7f4a5a7fc700 1 -- 172.21.15.70:0/4092961801 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a60005dc0 con 0x7f4a64101800 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.689+0000 7f4a5a7fc700 1 -- 172.21.15.70:0/4092961801 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a600226a0 con 0x7f4a64101800 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.689+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6419bb70 con 0x7f4a64101800 2024-04-02T04:49:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.689+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6419c010 con 0x7f4a64101800 2024-04-02T04:49:15.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.690+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a64190610 con 0x7f4a64101800 2024-04-02T04:49:15.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.691+0000 7f4a5a7fc700 1 -- 172.21.15.70:0/4092961801 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 29) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f4a60003680 con 0x7f4a64101800 2024-04-02T04:49:15.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.691+0000 7f4a5a7fc700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4a5006db20 0x7f4a5006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:15.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.691+0000 7f4a5a7fc700 1 -- 172.21.15.70:0/4092961801 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f4a6008ed70 con 0x7f4a64101800 2024-04-02T04:49:15.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.691+0000 7f4a688ad700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4a5006db20 0x7f4a5006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:15.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.692+0000 7f4a688ad700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4a5006db20 0x7f4a5006ffe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009be0 tx=0x7f4a54009260 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:15.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.696+0000 7f4a5a7fc700 1 -- 172.21.15.70:0/4092961801 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f4a6005c140 con 0x7f4a64101800 2024-04-02T04:49:15.788 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:15 smithi179 ceph-mon[36603]: pgmap v19: 65 pgs: 65 active+clean; 261 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51 KiB/s rd, 2.0 MiB/s wr, 1.16k op/s 2024-04-02T04:49:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.994+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a640611d0 con 0x7f4a5006db20 2024-04-02T04:49:15.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:15 smithi070 ceph-mon[29212]: pgmap v19: 65 pgs: 65 active+clean; 261 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51 KiB/s rd, 2.0 MiB/s wr, 1.16k op/s 2024-04-02T04:49:15.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.995+0000 7f4a5a7fc700 1 -- 172.21.15.70:0/4092961801 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f4a640611d0 con 0x7f4a5006db20 2024-04-02T04:49:16.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4a5006db20 msgr2=0x7f4a5006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4a5006db20 0x7f4a5006ffe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009be0 tx=0x7f4a54009260 comp rx=0 tx=0).stop 2024-04-02T04:49:16.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 msgr2=0x7f4a64196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 0x7f4a64196480 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4a60000c00 tx=0x7f4a6000f740 comp rx=0 tx=0).stop 2024-04-02T04:49:16.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 shutdown_connections 2024-04-02T04:49:16.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4a5006db20 0x7f4a5006ffe0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64101800 0x7f4a64196480 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 --2- 172.21.15.70:0/4092961801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a64104130 0x7f4a641969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 >> 172.21.15.70:0/4092961801 conn(0x7f4a640fb190 msgr2=0x7f4a640fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:16.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 shutdown_connections 2024-04-02T04:49:16.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:15.997+0000 7f4a6b312700 1 -- 172.21.15.70:0/4092961801 wait complete. 2024-04-02T04:49:16.012 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:49:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.122+0000 7fea27c3f700 1 -- 172.21.15.70:0/3580134849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea200ffe80 msgr2=0x7fea201002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.122+0000 7fea27c3f700 1 --2- 172.21.15.70:0/3580134849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea200ffe80 0x7fea201002e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fea10009910 tx=0x7fea10009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.122+0000 7fea27c3f700 1 -- 172.21.15.70:0/3580134849 shutdown_connections 2024-04-02T04:49:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.122+0000 7fea27c3f700 1 --2- 172.21.15.70:0/3580134849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea200ffe80 0x7fea201002e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.122+0000 7fea27c3f700 1 --2- 172.21.15.70:0/3580134849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea20101140 0x7fea20101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.122+0000 7fea27c3f700 1 -- 172.21.15.70:0/3580134849 >> 172.21.15.70:0/3580134849 conn(0x7fea200fa8b0 msgr2=0x7fea200fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:16.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.122+0000 7fea27c3f700 1 -- 172.21.15.70:0/3580134849 shutdown_connections 2024-04-02T04:49:16.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.122+0000 7fea27c3f700 1 -- 172.21.15.70:0/3580134849 wait complete. 2024-04-02T04:49:16.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea27c3f700 1 Processor -- start 2024-04-02T04:49:16.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea27c3f700 1 -- start start 2024-04-02T04:49:16.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea27c3f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea200ffe80 0x7fea201953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:16.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea27c3f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20101140 0x7fea20195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea27c3f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea20195f50 con 0x7fea20101140 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea27c3f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea20196090 con 0x7fea200ffe80 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea251da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20101140 0x7fea20195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea251da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20101140 0x7fea20195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45308/0 (socket says 172.21.15.70:45308) 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea251da700 1 -- 172.21.15.70:0/1746968651 learned_addr learned my addr 172.21.15.70:0/1746968651 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea259db700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea200ffe80 0x7fea201953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea251da700 1 -- 172.21.15.70:0/1746968651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea200ffe80 msgr2=0x7fea201953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea251da700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea200ffe80 0x7fea201953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.123+0000 7fea251da700 1 -- 172.21.15.70:0/1746968651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea100095f0 con 0x7fea20101140 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.124+0000 7fea251da700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20101140 0x7fea20195930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fea10005b40 tx=0x7fea1000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.124+0000 7fea16ffd700 1 -- 172.21.15.70:0/1746968651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea1001d070 con 0x7fea20101140 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.124+0000 7fea16ffd700 1 -- 172.21.15.70:0/1746968651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fea1000f460 con 0x7fea20101140 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.124+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2019aae0 con 0x7fea20101140 2024-04-02T04:49:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.124+0000 7fea16ffd700 1 -- 172.21.15.70:0/1746968651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea10021620 con 0x7fea20101140 2024-04-02T04:49:16.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.124+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2019afd0 con 0x7fea20101140 2024-04-02T04:49:16.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.126+0000 7fea16ffd700 1 -- 172.21.15.70:0/1746968651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 29) v1 ==== 91816+0+0 (secure 0 0 0) 0x7fea1000fa40 con 0x7fea20101140 2024-04-02T04:49:16.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.126+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea20108720 con 0x7fea20101140 2024-04-02T04:49:16.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.126+0000 7fea16ffd700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fea0c06db10 0x7fea0c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:16.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.126+0000 7fea16ffd700 1 -- 172.21.15.70:0/1746968651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fea1008ea40 con 0x7fea20101140 2024-04-02T04:49:16.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.126+0000 7fea259db700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fea0c06db10 0x7fea0c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:16.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.127+0000 7fea259db700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fea0c06db10 0x7fea0c06ffd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea20068bb0 tx=0x7fea1c00b410 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.131+0000 7fea16ffd700 1 -- 172.21.15.70:0/1746968651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fea1005be10 con 0x7fea20101140 2024-04-02T04:49:16.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.461+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea201057e0 con 0x7fea0c06db10 2024-04-02T04:49:16.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.463+0000 7fea16ffd700 1 -- 172.21.15.70:0/1746968651 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fea201057e0 con 0x7fea0c06db10 2024-04-02T04:49:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fea0c06db10 msgr2=0x7fea0c06ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fea0c06db10 0x7fea0c06ffd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea20068bb0 tx=0x7fea1c00b410 comp rx=0 tx=0).stop 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20101140 msgr2=0x7fea20195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20101140 0x7fea20195930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fea10005b40 tx=0x7fea1000bd70 comp rx=0 tx=0).stop 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 shutdown_connections 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fea0c06db10 0x7fea0c06ffd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea200ffe80 0x7fea201953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 --2- 172.21.15.70:0/1746968651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20101140 0x7fea20195930 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 >> 172.21.15.70:0/1746968651 conn(0x7fea200fa8b0 msgr2=0x7fea201040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 shutdown_connections 2024-04-02T04:49:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.465+0000 7fea27c3f700 1 -- 172.21.15.70:0/1746968651 wait complete. 2024-04-02T04:49:16.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.584+0000 7f4c47441700 1 -- 172.21.15.70:0/2968226875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 msgr2=0x7f4c400fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.584+0000 7f4c47441700 1 --2- 172.21.15.70:0/2968226875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 0x7f4c400fffc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009910 tx=0x7f4c3c009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.584+0000 7f4c47441700 1 -- 172.21.15.70:0/2968226875 shutdown_connections 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.584+0000 7f4c47441700 1 --2- 172.21.15.70:0/2968226875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c40100da0 0x7f4c40101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.584+0000 7f4c47441700 1 --2- 172.21.15.70:0/2968226875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 0x7f4c400fffc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.584+0000 7f4c47441700 1 -- 172.21.15.70:0/2968226875 >> 172.21.15.70:0/2968226875 conn(0x7f4c400fb120 msgr2=0x7f4c400fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.585+0000 7f4c47441700 1 -- 172.21.15.70:0/2968226875 shutdown_connections 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.585+0000 7f4c47441700 1 -- 172.21.15.70:0/2968226875 wait complete. 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.585+0000 7f4c47441700 1 Processor -- start 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.585+0000 7f4c47441700 1 -- start start 2024-04-02T04:49:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.585+0000 7f4c47441700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 0x7f4c401964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.585+0000 7f4c47441700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c40100da0 0x7f4c401969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c47441700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40197010 con 0x7f4c400ffba0 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c47441700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40197150 con 0x7f4c40100da0 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c451dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 0x7f4c401964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c449dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c40100da0 0x7f4c401969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c451dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 0x7f4c401964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45320/0 (socket says 172.21.15.70:45320) 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c451dd700 1 -- 172.21.15.70:0/161570629 learned_addr learned my addr 172.21.15.70:0/161570629 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c449dc700 1 -- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 msgr2=0x7f4c401964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c449dc700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 0x7f4c401964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c449dc700 1 -- 172.21.15.70:0/161570629 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0095f0 con 0x7f4c40100da0 2024-04-02T04:49:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c451dd700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 0x7f4c401964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:49:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c449dc700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c40100da0 0x7f4c401969f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000e970 tx=0x7f4c3000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c367fc700 1 -- 172.21.15.70:0/161570629 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c3000c9d0 con 0x7f4c40100da0 2024-04-02T04:49:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c367fc700 1 -- 172.21.15.70:0/161570629 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c30010430 con 0x7f4c40100da0 2024-04-02T04:49:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c367fc700 1 -- 172.21.15.70:0/161570629 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c30018650 con 0x7f4c40100da0 2024-04-02T04:49:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.586+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4019bc00 con 0x7f4c40100da0 2024-04-02T04:49:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.587+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4019c100 con 0x7f4c40100da0 2024-04-02T04:49:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.588+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c40069560 con 0x7f4c40100da0 2024-04-02T04:49:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.588+0000 7f4c367fc700 1 -- 172.21.15.70:0/161570629 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 29) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f4c3000cb30 con 0x7f4c40100da0 2024-04-02T04:49:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.589+0000 7f4c367fc700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4c2c06db20 0x7f4c2c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.589+0000 7f4c367fc700 1 -- 172.21.15.70:0/161570629 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f4c30014070 con 0x7f4c40100da0 2024-04-02T04:49:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.589+0000 7f4c451dd700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4c2c06db20 0x7f4c2c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.589+0000 7f4c451dd700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4c2c06db20 0x7f4c2c06ffe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009fd0 tx=0x7f4c3c005940 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.594+0000 7f4c367fc700 1 -- 172.21.15.70:0/161570629 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f4c30052fc0 con 0x7f4c40100da0 2024-04-02T04:49:16.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.879+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c401056f0 con 0x7f4c2c06db20 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.892+0000 7f4c367fc700 1 -- 172.21.15.70:0/161570629 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4c401056f0 con 0x7f4c2c06db20 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (8m) 19s ago 9m 24.2M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (10m) 19s ago 10m 7780k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (9m) 31s ago 9m 7457k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (10m) 19s ago 10m 6665k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (9m) 31s ago 9m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (8m) 19s ago 9m 87.8M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (4m) 19s ago 4m 218M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:49:16.897 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (4m) 19s ago 4m 16.8M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (4m) 31s ago 4m 15.4M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (4m) 31s ago 4m 16.9M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (54s) 19s ago 11m 478M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 starting - - - - 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (11m) 19s ago 11m 51.4M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (8m) 31s ago 8m 46.4M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (10m) 19s ago 10m 15.1M - 1.5.0 0da6a335fe13 bc06c0b59df5 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (9m) 31s ago 8m 16.9M - 1.5.0 0da6a335fe13 6fe44da459b3 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (8m) 19s ago 8m 197M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (7m) 19s ago 7m 225M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (7m) 19s ago 7m 177M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (6m) 31s ago 6m 249M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:49:16.898 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (6m) 31s ago 6m 237M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:49:16.899 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (5m) 31s ago 5m 192M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:49:16.899 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 starting - - - - 2024-04-02T04:49:16.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4c2c06db20 msgr2=0x7f4c2c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4c2c06db20 0x7f4c2c06ffe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009fd0 tx=0x7f4c3c005940 comp rx=0 tx=0).stop 2024-04-02T04:49:16.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c40100da0 msgr2=0x7f4c401969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:16.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c40100da0 0x7f4c401969f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000e970 tx=0x7f4c3000ec80 comp rx=0 tx=0).stop 2024-04-02T04:49:16.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 shutdown_connections 2024-04-02T04:49:16.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f4c2c06db20 0x7f4c2c06ffe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c400ffba0 0x7f4c401964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 --2- 172.21.15.70:0/161570629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c40100da0 0x7f4c401969f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:16.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 >> 172.21.15.70:0/161570629 conn(0x7f4c400fb120 msgr2=0x7f4c40103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:16.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 shutdown_connections 2024-04-02T04:49:16.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:16.895+0000 7f4c47441700 1 -- 172.21.15.70:0/161570629 wait complete. 2024-04-02T04:49:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.029+0000 7fa2b877a700 1 -- 172.21.15.70:0/779963077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 msgr2=0x7fa2b0102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.029+0000 7fa2b877a700 1 --2- 172.21.15.70:0/779963077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 0x7fa2b0102300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009910 tx=0x7fa2a0009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.029+0000 7fa2b877a700 1 -- 172.21.15.70:0/779963077 shutdown_connections 2024-04-02T04:49:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.029+0000 7fa2b877a700 1 --2- 172.21.15.70:0/779963077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 0x7fa2b0102300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.029+0000 7fa2b877a700 1 --2- 172.21.15.70:0/779963077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2b0100ca0 0x7fa2b01010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.030+0000 7fa2b877a700 1 -- 172.21.15.70:0/779963077 >> 172.21.15.70:0/779963077 conn(0x7fa2b00fc260 msgr2=0x7fa2b00fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.030+0000 7fa2b877a700 1 -- 172.21.15.70:0/779963077 shutdown_connections 2024-04-02T04:49:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.030+0000 7fa2b877a700 1 -- 172.21.15.70:0/779963077 wait complete. 2024-04-02T04:49:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.030+0000 7fa2b877a700 1 Processor -- start 2024-04-02T04:49:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.030+0000 7fa2b877a700 1 -- start start 2024-04-02T04:49:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b877a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2b0100ca0 0x7fa2b0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b877a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 0x7fa2b0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b877a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00736c0 con 0x7fa2b0101ea0 2024-04-02T04:49:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b877a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0073800 con 0x7fa2b0100ca0 2024-04-02T04:49:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b5d15700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 0x7fa2b0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:17.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b5d15700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 0x7fa2b0073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46724/0 (socket says 172.21.15.70:46724) 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b5d15700 1 -- 172.21.15.70:0/768893001 learned_addr learned my addr 172.21.15.70:0/768893001 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b6516700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2b0100ca0 0x7fa2b0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b5d15700 1 -- 172.21.15.70:0/768893001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2b0100ca0 msgr2=0x7fa2b0074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b5d15700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2b0100ca0 0x7fa2b0074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b5d15700 1 -- 172.21.15.70:0/768893001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00095f0 con 0x7fa2b0101ea0 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.031+0000 7fa2b5d15700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 0x7fa2b0073180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00092e0 tx=0x7fa2a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.032+0000 7fa2a77fe700 1 -- 172.21.15.70:0/768893001 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a001d070 con 0x7fa2b0101ea0 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.032+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b0073a80 con 0x7fa2b0101ea0 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.032+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b0073f70 con 0x7fa2b0101ea0 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.032+0000 7fa2a77fe700 1 -- 172.21.15.70:0/768893001 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa2a0005dc0 con 0x7fa2b0101ea0 2024-04-02T04:49:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.032+0000 7fa2a77fe700 1 -- 172.21.15.70:0/768893001 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a0017630 con 0x7fa2b0101ea0 2024-04-02T04:49:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.033+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa294005320 con 0x7fa2b0101ea0 2024-04-02T04:49:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.034+0000 7fa2a77fe700 1 -- 172.21.15.70:0/768893001 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 29) v1 ==== 91816+0+0 (secure 0 0 0) 0x7fa2a000faf0 con 0x7fa2b0101ea0 2024-04-02T04:49:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.035+0000 7fa2a77fe700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fa29c06db10 0x7fa29c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.035+0000 7fa2a77fe700 1 -- 172.21.15.70:0/768893001 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fa2a008ea60 con 0x7fa2b0101ea0 2024-04-02T04:49:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.035+0000 7fa2b6516700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fa29c06db10 0x7fa29c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.035+0000 7fa2b6516700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fa29c06db10 0x7fa29c06ffd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0101d00 tx=0x7fa2ac009260 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.043+0000 7fa2a77fe700 1 -- 172.21.15.70:0/768893001 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fa2a005beb0 con 0x7fa2b0101ea0 2024-04-02T04:49:17.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.510+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa294005cc0 con 0x7fa2b0101ea0 2024-04-02T04:49:17.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.510+0000 7fa2a77fe700 1 -- 172.21.15.70:0/768893001 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fa2a005ba40 con 0x7fa2b0101ea0 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:49:17.515 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:49:17.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:49:17.516 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:49:17.516 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-02T04:49:17.516 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:17.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:49:17.516 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:49:17.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fa29c06db10 msgr2=0x7fa29c06ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:17.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fa29c06db10 0x7fa29c06ffd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0101d00 tx=0x7fa2ac009260 comp rx=0 tx=0).stop 2024-04-02T04:49:17.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 msgr2=0x7fa2b0073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:17.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 0x7fa2b0073180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00092e0 tx=0x7fa2a000f690 comp rx=0 tx=0).stop 2024-04-02T04:49:17.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 shutdown_connections 2024-04-02T04:49:17.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fa29c06db10 0x7fa29c06ffd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2b0100ca0 0x7fa2b0074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 --2- 172.21.15.70:0/768893001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2b0101ea0 0x7fa2b0073180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 >> 172.21.15.70:0/768893001 conn(0x7fa2b00fc260 msgr2=0x7fa2b01050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:17.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 shutdown_connections 2024-04-02T04:49:17.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.513+0000 7fa2b877a700 1 -- 172.21.15.70:0/768893001 wait complete. 2024-04-02T04:49:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.669+0000 7f1783012700 1 -- 172.21.15.70:0/949686627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 msgr2=0x7f177c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.669+0000 7f1783012700 1 --2- 172.21.15.70:0/949686627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 0x7f177c101200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1768009910 tx=0x7f1768009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.669+0000 7f1783012700 1 -- 172.21.15.70:0/949686627 shutdown_connections 2024-04-02T04:49:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.669+0000 7f1783012700 1 --2- 172.21.15.70:0/949686627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 0x7f177c101200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.669+0000 7f1783012700 1 --2- 172.21.15.70:0/949686627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f177c0ffba0 0x7f177c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.669+0000 7f1783012700 1 -- 172.21.15.70:0/949686627 >> 172.21.15.70:0/949686627 conn(0x7f177c0fb120 msgr2=0x7f177c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.670+0000 7f1783012700 1 -- 172.21.15.70:0/949686627 shutdown_connections 2024-04-02T04:49:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.670+0000 7f1783012700 1 -- 172.21.15.70:0/949686627 wait complete. 2024-04-02T04:49:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.670+0000 7f1783012700 1 Processor -- start 2024-04-02T04:49:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.670+0000 7f1783012700 1 -- start start 2024-04-02T04:49:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f1783012700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f177c0ffba0 0x7f177c107b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f1783012700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 0x7f177c1061a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f1783012700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c1066e0 con 0x7f177c100da0 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f1783012700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177c106820 con 0x7f177c0ffba0 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f177bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 0x7f177c1061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f177bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 0x7f177c1061a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46750/0 (socket says 172.21.15.70:46750) 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f177bfff700 1 -- 172.21.15.70:0/478021540 learned_addr learned my addr 172.21.15.70:0/478021540 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f1780dae700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f177c0ffba0 0x7f177c107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f177bfff700 1 -- 172.21.15.70:0/478021540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f177c0ffba0 msgr2=0x7f177c107b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f177bfff700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f177c0ffba0 0x7f177c107b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f177bfff700 1 -- 172.21.15.70:0/478021540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17680095f0 con 0x7f177c100da0 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f1780dae700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f177c0ffba0 0x7f177c107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:49:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.671+0000 7f177bfff700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 0x7f177c1061a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17680092e0 tx=0x7f176800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.672+0000 7f1779ffb700 1 -- 172.21.15.70:0/478021540 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f176801d070 con 0x7f177c100da0 2024-04-02T04:49:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.672+0000 7f1779ffb700 1 -- 172.21.15.70:0/478021540 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1768005dc0 con 0x7f177c100da0 2024-04-02T04:49:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.672+0000 7f1783012700 1 -- 172.21.15.70:0/478021540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177c106aa0 con 0x7f177c100da0 2024-04-02T04:49:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.672+0000 7f1779ffb700 1 -- 172.21.15.70:0/478021540 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1768017630 con 0x7f177c100da0 2024-04-02T04:49:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.672+0000 7f1783012700 1 -- 172.21.15.70:0/478021540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177c106ed0 con 0x7f177c100da0 2024-04-02T04:49:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.674+0000 7f17677fe700 1 -- 172.21.15.70:0/478021540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c0052f0 con 0x7f177c100da0 2024-04-02T04:49:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.675+0000 7f1779ffb700 1 -- 172.21.15.70:0/478021540 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 29) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f176800fba0 con 0x7f177c100da0 2024-04-02T04:49:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.675+0000 7f1779ffb700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f176c06db20 0x7f176c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.675+0000 7f1779ffb700 1 -- 172.21.15.70:0/478021540 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f176808ecc0 con 0x7f177c100da0 2024-04-02T04:49:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.676+0000 7f1780dae700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f176c06db20 0x7f176c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.676+0000 7f1780dae700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f176c06db20 0x7f176c06ffe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f177c100c00 tx=0x7f1770009260 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.684+0000 7f1779ffb700 1 -- 172.21.15.70:0/478021540 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f1768094020 con 0x7f177c100da0 2024-04-02T04:49:17.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:17 smithi070 ceph-mon[29212]: pgmap v20: 65 pgs: 65 active+clean; 261 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.2 MiB/s wr, 684 op/s 2024-04-02T04:49:17.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:17 smithi070 ceph-mon[29212]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:17.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:17 smithi070 ceph-mon[29212]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:17.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:17.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:17.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:17.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:17 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/768893001' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:17.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:17 smithi179 ceph-mon[36603]: pgmap v20: 65 pgs: 65 active+clean; 261 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.2 MiB/s wr, 684 op/s 2024-04-02T04:49:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:17 smithi179 ceph-mon[36603]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:17 smithi179 ceph-mon[36603]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:17 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/768893001' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.974+0000 7f17677fe700 1 -- 172.21.15.70:0/478021540 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f175c000bc0 con 0x7f176c06db20 2024-04-02T04:49:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.975+0000 7f1779ffb700 1 -- 172.21.15.70:0/478021540 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f175c000bc0 con 0x7f176c06db20 2024-04-02T04:49:17.979 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:49:17.979 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:49:17.979 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:49:17.979 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:49:17.979 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-04-02T04:49:17.979 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "", 2024-04-02T04:49:17.979 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mgr daemons", 2024-04-02T04:49:17.980 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:49:17.980 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:49:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 -- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f176c06db20 msgr2=0x7f176c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f176c06db20 0x7f176c06ffe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f177c100c00 tx=0x7f1770009260 comp rx=0 tx=0).stop 2024-04-02T04:49:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 -- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 msgr2=0x7f177c1061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 0x7f177c1061a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17680092e0 tx=0x7f176800f740 comp rx=0 tx=0).stop 2024-04-02T04:49:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 -- 172.21.15.70:0/478021540 shutdown_connections 2024-04-02T04:49:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f176c06db20 0x7f176c06ffe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f177c0ffba0 0x7f177c107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 --2- 172.21.15.70:0/478021540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f177c100da0 0x7f177c1061a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 -- 172.21.15.70:0/478021540 >> 172.21.15.70:0/478021540 conn(0x7f177c0fb120 msgr2=0x7f177c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 -- 172.21.15.70:0/478021540 shutdown_connections 2024-04-02T04:49:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:17.977+0000 7f17677fe700 1 -- 172.21.15.70:0/478021540 wait complete. 2024-04-02T04:49:18.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:18 smithi179 ceph-mon[36603]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:19.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:18 smithi070 ceph-mon[29212]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:19.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:19 smithi179 ceph-mon[36603]: pgmap v21: 65 pgs: 65 active+clean; 261 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.2 MiB/s wr, 684 op/s 2024-04-02T04:49:19.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:19 smithi179 ceph-mon[36603]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:19.966 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:19 smithi070 ceph-mon[29212]: pgmap v21: 65 pgs: 65 active+clean; 261 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.2 MiB/s wr, 684 op/s 2024-04-02T04:49:19.966 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:19 smithi070 ceph-mon[29212]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:21.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:21 smithi070 ceph-mon[29212]: pgmap v22: 65 pgs: 65 active+clean; 251 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.8 MiB/s wr, 848 op/s 2024-04-02T04:49:21.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:21 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:21.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:21 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:21.787 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:21 smithi179 ceph-mon[36603]: pgmap v22: 65 pgs: 65 active+clean; 251 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.8 MiB/s wr, 848 op/s 2024-04-02T04:49:21.787 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:21 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:21.787 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:21 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:22.917 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:22 smithi179 ceph-mon[36603]: pgmap v23: 65 pgs: 65 active+clean; 251 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 360 op/s 2024-04-02T04:49:22.917 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:22 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:22.917 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:22 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:23.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:22 smithi070 ceph-mon[29212]: pgmap v23: 65 pgs: 65 active+clean; 251 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 360 op/s 2024-04-02T04:49:23.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:22 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:23.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:22 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:25.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:24 smithi070 ceph-mon[29212]: pgmap v24: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.6 MiB/s wr, 507 op/s 2024-04-02T04:49:25.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:24 smithi179 ceph-mon[36603]: pgmap v24: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.6 MiB/s wr, 507 op/s 2024-04-02T04:49:26.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:25 smithi070 ceph-mon[29212]: Standby manager daemon smithi179.labmvs restarted 2024-04-02T04:49:26.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:25 smithi070 ceph-mon[29212]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:49:26.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:25 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1658270259' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:49:26.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:25 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1658270259' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:49:26.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:25 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1658270259' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:49:26.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:25 smithi070 ceph-mon[29212]: from='mgr.? 172.21.15.179:0/1658270259' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:49:26.066 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:25 smithi179 ceph-mon[36603]: Standby manager daemon smithi179.labmvs restarted 2024-04-02T04:49:26.066 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:25 smithi179 ceph-mon[36603]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:49:26.066 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:25 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1658270259' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:49:26.066 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:25 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1658270259' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:49:26.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:25 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1658270259' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:49:26.067 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:25 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/1658270259' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:49:27.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:26 smithi070 ceph-mon[29212]: pgmap v25: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 311 op/s 2024-04-02T04:49:27.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:26 smithi070 ceph-mon[29212]: mgrmap e30: smithi070.igidvr(active, since 44s), standbys: smithi179.labmvs 2024-04-02T04:49:27.048 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:26 smithi179 ceph-mon[36603]: pgmap v25: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 311 op/s 2024-04-02T04:49:27.049 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:26 smithi179 ceph-mon[36603]: mgrmap e30: smithi070.igidvr(active, since 44s), standbys: smithi179.labmvs 2024-04-02T04:49:28.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:27 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:28.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:27 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:28.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:27 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:28.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:27 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:27 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:27 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:29.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:28 smithi179 ceph-mon[36603]: pgmap v26: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 311 op/s 2024-04-02T04:49:29.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:28 smithi070 ceph-mon[29212]: pgmap v26: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 311 op/s 2024-04-02T04:49:31.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:30 smithi179 ceph-mon[36603]: pgmap v27: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 1.5 MiB/s wr, 452 op/s 2024-04-02T04:49:31.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:30 smithi070 ceph-mon[29212]: pgmap v27: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 1.5 MiB/s wr, 452 op/s 2024-04-02T04:49:33.115 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:32 smithi070 ceph-mon[29212]: pgmap v28: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1011 KiB/s wr, 288 op/s 2024-04-02T04:49:33.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:32 smithi179 ceph-mon[36603]: pgmap v28: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1011 KiB/s wr, 288 op/s 2024-04-02T04:49:35.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:34 smithi070 ceph-mon[29212]: pgmap v29: 65 pgs: 65 active+clean; 232 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.5 MiB/s wr, 430 op/s 2024-04-02T04:49:35.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:34 smithi179 ceph-mon[36603]: pgmap v29: 65 pgs: 65 active+clean; 232 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.5 MiB/s wr, 430 op/s 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.igidvr"}]: dispatch 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.igidvr"}]': finished 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi179.labmvs"}]: dispatch 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi179.labmvs"}]': finished 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:36.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:36 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.igidvr"}]: dispatch 2024-04-02T04:49:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.igidvr"}]': finished 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi179.labmvs"}]: dispatch 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi179.labmvs"}]': finished 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:36 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:37.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:37 smithi070 ceph-mon[29212]: pgmap v30: 65 pgs: 65 active+clean; 232 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1016 KiB/s wr, 283 op/s 2024-04-02T04:49:37.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:37 smithi070 ceph-mon[29212]: Upgrade: Setting container_image for all mgr 2024-04-02T04:49:37.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:37 smithi070 ceph-mon[29212]: Upgrade: Setting container_image for all rgw 2024-04-02T04:49:37.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:37 smithi070 ceph-mon[29212]: Upgrade: Setting container_image for all rbd-mirror 2024-04-02T04:49:37.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:37 smithi070 ceph-mon[29212]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-02T04:49:37.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:37 smithi070 ceph-mon[29212]: Upgrade: Setting container_image for all iscsi 2024-04-02T04:49:37.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:37 smithi070 ceph-mon[29212]: Upgrade: Setting container_image for all nfs 2024-04-02T04:49:37.890 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:37 smithi070 ceph-mon[29212]: Upgrade: Setting container_image for all nvmeof 2024-04-02T04:49:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:37 smithi179 ceph-mon[36603]: pgmap v30: 65 pgs: 65 active+clean; 232 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1016 KiB/s wr, 283 op/s 2024-04-02T04:49:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:37 smithi179 ceph-mon[36603]: Upgrade: Setting container_image for all mgr 2024-04-02T04:49:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:37 smithi179 ceph-mon[36603]: Upgrade: Setting container_image for all rgw 2024-04-02T04:49:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:37 smithi179 ceph-mon[36603]: Upgrade: Setting container_image for all rbd-mirror 2024-04-02T04:49:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:37 smithi179 ceph-mon[36603]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-02T04:49:38.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:37 smithi179 ceph-mon[36603]: Upgrade: Setting container_image for all iscsi 2024-04-02T04:49:38.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:37 smithi179 ceph-mon[36603]: Upgrade: Setting container_image for all nfs 2024-04-02T04:49:38.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:37 smithi179 ceph-mon[36603]: Upgrade: Setting container_image for all nvmeof 2024-04-02T04:49:39.448 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-02T04:49:39.448 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-02T04:49:39.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:39 smithi070 ceph-mon[29212]: pgmap v31: 65 pgs: 65 active+clean; 232 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1016 KiB/s wr, 283 op/s 2024-04-02T04:49:39.971 DEBUG:teuthology.parallel:result is None 2024-04-02T04:49:40.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:39 smithi179 ceph-mon[36603]: pgmap v31: 65 pgs: 65 active+clean; 232 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1016 KiB/s wr, 283 op/s 2024-04-02T04:49:40.706 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-02T04:49:40.706 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-02T04:49:41.036 DEBUG:teuthology.parallel:result is None 2024-04-02T04:49:41.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T04:49:41.070 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T04:49:41.071 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-02T04:49:41.108 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-02T04:49:41.108 DEBUG:teuthology.parallel:result is None 2024-04-02T04:49:41.959 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:41 smithi070 ceph-mon[29212]: pgmap v32: 65 pgs: 65 active+clean; 218 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.5 MiB/s wr, 432 op/s 2024-04-02T04:49:42.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:41 smithi179 ceph-mon[36603]: pgmap v32: 65 pgs: 65 active+clean; 218 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.5 MiB/s wr, 432 op/s 2024-04-02T04:49:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:42 smithi070 ceph-mon[29212]: Upgrade: Updating node-exporter.smithi070 (1/2) 2024-04-02T04:49:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:42 smithi070 ceph-mon[29212]: Deploying daemon node-exporter.smithi070 on smithi070 2024-04-02T04:49:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:42.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:42 smithi179 ceph-mon[36603]: Upgrade: Updating node-exporter.smithi070 (1/2) 2024-04-02T04:49:42.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:42 smithi179 ceph-mon[36603]: Deploying daemon node-exporter.smithi070 on smithi070 2024-04-02T04:49:42.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:44.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:43 smithi070 ceph-mon[29212]: pgmap v33: 65 pgs: 65 active+clean; 218 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-04-02T04:49:44.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:43 smithi179 ceph-mon[36603]: pgmap v33: 65 pgs: 65 active+clean; 218 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-04-02T04:49:45.749 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:45 smithi070 ceph-mon[29212]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.4 MiB/s wr, 426 op/s 2024-04-02T04:49:45.749 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:45 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:45.749 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:45 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:45.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:45 smithi179 ceph-mon[36603]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.4 MiB/s wr, 426 op/s 2024-04-02T04:49:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:45 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:45.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:45 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:47.494 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:47 smithi179 ceph-mon[36603]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 895 KiB/s wr, 283 op/s 2024-04-02T04:49:47.494 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:47 smithi179 ceph-mon[36603]: Upgrade: Updating node-exporter.smithi179 (2/2) 2024-04-02T04:49:47.495 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:47 smithi179 ceph-mon[36603]: Deploying daemon node-exporter.smithi179 on smithi179 2024-04-02T04:49:47.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:47 smithi070 ceph-mon[29212]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 895 KiB/s wr, 283 op/s 2024-04-02T04:49:47.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:47 smithi070 ceph-mon[29212]: Upgrade: Updating node-exporter.smithi179 (2/2) 2024-04-02T04:49:47.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:47 smithi070 ceph-mon[29212]: Deploying daemon node-exporter.smithi179 on smithi179 2024-04-02T04:49:48.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.100+0000 7f6b5e225700 1 -- 172.21.15.70:0/4156791052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 msgr2=0x7f6b58101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.100+0000 7f6b5e225700 1 --2- 172.21.15.70:0/4156791052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 0x7f6b58101270 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009910 tx=0x7f6b40009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:48.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.100+0000 7f6b5e225700 1 -- 172.21.15.70:0/4156791052 shutdown_connections 2024-04-02T04:49:48.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.100+0000 7f6b5e225700 1 --2- 172.21.15.70:0/4156791052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 0x7f6b58101270 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.100+0000 7f6b5e225700 1 --2- 172.21.15.70:0/4156791052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b580ffc10 0x7f6b58100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.100+0000 7f6b5e225700 1 -- 172.21.15.70:0/4156791052 >> 172.21.15.70:0/4156791052 conn(0x7f6b580fb190 msgr2=0x7f6b580fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:48.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.101+0000 7f6b5e225700 1 -- 172.21.15.70:0/4156791052 shutdown_connections 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.101+0000 7f6b5e225700 1 -- 172.21.15.70:0/4156791052 wait complete. 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.101+0000 7f6b5e225700 1 Processor -- start 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.101+0000 7f6b5e225700 1 -- start start 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.101+0000 7f6b5e225700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b580ffc10 0x7f6b58198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.101+0000 7f6b5e225700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 0x7f6b58198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.101+0000 7f6b5e225700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581991e0 con 0x7f6b58100e10 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.101+0000 7f6b5e225700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58199320 con 0x7f6b580ffc10 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b4effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 0x7f6b58198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b4effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 0x7f6b58198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57364/0 (socket says 172.21.15.70:57364) 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b4effd700 1 -- 172.21.15.70:0/4030988207 learned_addr learned my addr 172.21.15.70:0/4030988207 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b577fe700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b580ffc10 0x7f6b58198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b4effd700 1 -- 172.21.15.70:0/4030988207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b580ffc10 msgr2=0x7f6b58198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b4effd700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b580ffc10 0x7f6b58198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b4effd700 1 -- 172.21.15.70:0/4030988207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400095f0 con 0x7f6b58100e10 2024-04-02T04:49:48.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b4effd700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 0x7f6b58198bc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b40006010 tx=0x7f6b4000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:48.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b557fa700 1 -- 172.21.15.70:0/4030988207 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b4001d070 con 0x7f6b58100e10 2024-04-02T04:49:48.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5819dd70 con 0x7f6b58100e10 2024-04-02T04:49:48.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b557fa700 1 -- 172.21.15.70:0/4030988207 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b4000f460 con 0x7f6b58100e10 2024-04-02T04:49:48.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b557fa700 1 -- 172.21.15.70:0/4030988207 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b40021640 con 0x7f6b58100e10 2024-04-02T04:49:48.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.102+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5819e260 con 0x7f6b58100e10 2024-04-02T04:49:48.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.103+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b58108700 con 0x7f6b58100e10 2024-04-02T04:49:48.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.104+0000 7f6b557fa700 1 -- 172.21.15.70:0/4030988207 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f6b4000fab0 con 0x7f6b58100e10 2024-04-02T04:49:48.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.104+0000 7f6b557fa700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f6b4406db20 0x7f6b4406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.104+0000 7f6b557fa700 1 -- 172.21.15.70:0/4030988207 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f6b4008ea40 con 0x7f6b58100e10 2024-04-02T04:49:48.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.104+0000 7f6b577fe700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f6b4406db20 0x7f6b4406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.105+0000 7f6b577fe700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f6b4406db20 0x7f6b4406ffe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6b58100c70 tx=0x7f6b4800b410 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:48.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.109+0000 7f6b557fa700 1 -- 172.21.15.70:0/4030988207 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f6b4005be10 con 0x7f6b58100e10 2024-04-02T04:49:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.327+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b58068c90 con 0x7f6b4406db20 2024-04-02T04:49:48.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.329+0000 7f6b557fa700 1 -- 172.21.15.70:0/4030988207 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f6b58068c90 con 0x7f6b4406db20 2024-04-02T04:49:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f6b4406db20 msgr2=0x7f6b4406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f6b4406db20 0x7f6b4406ffe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6b58100c70 tx=0x7f6b4800b410 comp rx=0 tx=0).stop 2024-04-02T04:49:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 msgr2=0x7f6b58198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 0x7f6b58198bc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b40006010 tx=0x7f6b4000bd70 comp rx=0 tx=0).stop 2024-04-02T04:49:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 shutdown_connections 2024-04-02T04:49:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f6b4406db20 0x7f6b4406ffe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b580ffc10 0x7f6b58198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 --2- 172.21.15.70:0/4030988207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b58100e10 0x7f6b58198bc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 >> 172.21.15.70:0/4030988207 conn(0x7f6b580fb190 msgr2=0x7f6b58068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 shutdown_connections 2024-04-02T04:49:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.331+0000 7f6b5e225700 1 -- 172.21.15.70:0/4030988207 wait complete. 2024-04-02T04:49:48.345 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:49:48.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.438+0000 7f0fef217700 1 -- 172.21.15.70:0/1661510653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 msgr2=0x7f0fe8101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.438+0000 7f0fef217700 1 --2- 172.21.15.70:0/1661510653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 0x7f0fe8101640 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009910 tx=0x7f0fdc009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:48.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.438+0000 7f0fef217700 1 -- 172.21.15.70:0/1661510653 shutdown_connections 2024-04-02T04:49:48.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.438+0000 7f0fef217700 1 --2- 172.21.15.70:0/1661510653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fe80ffdf0 0x7f0fe8100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.438+0000 7f0fef217700 1 --2- 172.21.15.70:0/1661510653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 0x7f0fe8101640 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.438+0000 7f0fef217700 1 -- 172.21.15.70:0/1661510653 >> 172.21.15.70:0/1661510653 conn(0x7f0fe80fa8c0 msgr2=0x7f0fe80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:48.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.438+0000 7f0fef217700 1 -- 172.21.15.70:0/1661510653 shutdown_connections 2024-04-02T04:49:48.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.438+0000 7f0fef217700 1 -- 172.21.15.70:0/1661510653 wait complete. 2024-04-02T04:49:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fef217700 1 Processor -- start 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fef217700 1 -- start start 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fef217700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fe80ffdf0 0x7f0fe80ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fef217700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 0x7f0fe819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fef217700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe819b7d0 con 0x7f0fe8101220 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fef217700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe819b940 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fe7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 0x7f0fe819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fecfb3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fe80ffdf0 0x7f0fe80ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fe7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 0x7f0fe819b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57384/0 (socket says 172.21.15.70:57384) 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fecfb3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fe80ffdf0 0x7f0fe80ff860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:47540/0 (socket says 172.21.15.70:47540) 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.439+0000 7f0fe7fff700 1 -- 172.21.15.70:0/3724792849 learned_addr learned my addr 172.21.15.70:0/3724792849 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fecfb3700 1 -- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 msgr2=0x7f0fe819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fecfb3700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 0x7f0fe819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fecfb3700 1 -- 172.21.15.70:0/3724792849 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc0095f0 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fecfb3700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fe80ffdf0 0x7f0fe80ff860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009910 tx=0x7f0fdc00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:48.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fe5ffb700 1 -- 172.21.15.70:0/3724792849 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fdc01d070 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fe5ffb700 1 -- 172.21.15.70:0/3724792849 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0fdc00f460 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe819bbc0 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fe5ffb700 1 -- 172.21.15.70:0/3724792849 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fdc021620 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.440+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe819c0b0 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.441+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe81087c0 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.442+0000 7f0fe5ffb700 1 -- 172.21.15.70:0/3724792849 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 30) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f0fdc003a60 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.442+0000 7f0fe5ffb700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f0fd806db20 0x7f0fd806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.443+0000 7f0fe5ffb700 1 -- 172.21.15.70:0/3724792849 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f0fdc08e8a0 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.443+0000 7f0fe7fff700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f0fd806db20 0x7f0fd806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.443+0000 7f0fe7fff700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f0fd806db20 0x7f0fd806ffe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80ffdc0 tx=0x7f0fd400b500 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:48.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.447+0000 7f0fe5ffb700 1 -- 172.21.15.70:0/3724792849 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f0fdc05bcf0 con 0x7f0fe80ffdf0 2024-04-02T04:49:48.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.680+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fe8105750 con 0x7f0fd806db20 2024-04-02T04:49:48.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.681+0000 7f0fe5ffb700 1 -- 172.21.15.70:0/3724792849 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f0fe8105750 con 0x7f0fd806db20 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f0fd806db20 msgr2=0x7f0fd806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f0fd806db20 0x7f0fd806ffe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80ffdc0 tx=0x7f0fd400b500 comp rx=0 tx=0).stop 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fe80ffdf0 msgr2=0x7f0fe80ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fe80ffdf0 0x7f0fe80ff860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009910 tx=0x7f0fdc00bd70 comp rx=0 tx=0).stop 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 shutdown_connections 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f0fd806db20 0x7f0fd806ffe0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fe80ffdf0 0x7f0fe80ff860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 --2- 172.21.15.70:0/3724792849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe8101220 0x7f0fe819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 >> 172.21.15.70:0/3724792849 conn(0x7f0fe80fa8c0 msgr2=0x7f0fe8104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.683+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 shutdown_connections 2024-04-02T04:49:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.684+0000 7f0fef217700 1 -- 172.21.15.70:0/3724792849 wait complete. 2024-04-02T04:49:48.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 -- 172.21.15.70:0/1384116773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c100e10 msgr2=0x7faa5c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 --2- 172.21.15.70:0/1384116773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c100e10 0x7faa5c101270 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa50009910 tx=0x7faa50009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:48.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 -- 172.21.15.70:0/1384116773 shutdown_connections 2024-04-02T04:49:48.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 --2- 172.21.15.70:0/1384116773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c100e10 0x7faa5c101270 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 --2- 172.21.15.70:0/1384116773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa5c0ffc10 0x7faa5c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 -- 172.21.15.70:0/1384116773 >> 172.21.15.70:0/1384116773 conn(0x7faa5c0fb190 msgr2=0x7faa5c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:48.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 -- 172.21.15.70:0/1384116773 shutdown_connections 2024-04-02T04:49:48.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 -- 172.21.15.70:0/1384116773 wait complete. 2024-04-02T04:49:48.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.791+0000 7faa612e8700 1 Processor -- start 2024-04-02T04:49:48.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa612e8700 1 -- start start 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa612e8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c0ffc10 0x7faa5c103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa612e8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa5c100e10 0x7faa5c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa612e8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c102730 con 0x7faa5c0ffc10 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa612e8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c102870 con 0x7faa5c100e10 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c0ffc10 0x7faa5c103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c0ffc10 0x7faa5c103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57396/0 (socket says 172.21.15.70:57396) 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa5c100e10 0x7faa5c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5affd700 1 -- 172.21.15.70:0/1470130240 learned_addr learned my addr 172.21.15.70:0/1470130240 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5affd700 1 -- 172.21.15.70:0/1470130240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa5c100e10 msgr2=0x7faa5c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5affd700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa5c100e10 0x7faa5c1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:48.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5affd700 1 -- 172.21.15.70:0/1470130240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa500095f0 con 0x7faa5c0ffc10 2024-04-02T04:49:48.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5a7fc700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa5c100e10 0x7faa5c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:49:48.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa5affd700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c0ffc10 0x7faa5c103ba0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faa5400efc0 tx=0x7faa5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:48.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.792+0000 7faa43fff700 1 -- 172.21.15.70:0/1470130240 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa54003bd0 con 0x7faa5c0ffc10 2024-04-02T04:49:48.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.793+0000 7faa43fff700 1 -- 172.21.15.70:0/1470130240 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faa5400f040 con 0x7faa5c0ffc10 2024-04-02T04:49:48.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.793+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c102b50 con 0x7faa5c0ffc10 2024-04-02T04:49:48.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.793+0000 7faa43fff700 1 -- 172.21.15.70:0/1470130240 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa540046a0 con 0x7faa5c0ffc10 2024-04-02T04:49:48.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.793+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c19e530 con 0x7faa5c0ffc10 2024-04-02T04:49:48.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.794+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c108720 con 0x7faa5c0ffc10 2024-04-02T04:49:48.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.794+0000 7faa43fff700 1 -- 172.21.15.70:0/1470130240 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 91816+0+0 (secure 0 0 0) 0x7faa5400c930 con 0x7faa5c0ffc10 2024-04-02T04:49:48.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.795+0000 7faa43fff700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faa4406db20 0x7faa4406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:48.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.795+0000 7faa43fff700 1 -- 172.21.15.70:0/1470130240 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7faa5408d920 con 0x7faa5c0ffc10 2024-04-02T04:49:48.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.795+0000 7faa5a7fc700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faa4406db20 0x7faa4406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:48.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.795+0000 7faa5a7fc700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faa4406db20 0x7faa4406ffe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faa50009fd0 tx=0x7faa50005940 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:48.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:48.799+0000 7faa43fff700 1 -- 172.21.15.70:0/1470130240 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7faa5405acf0 con 0x7faa5c0ffc10 2024-04-02T04:49:49.032 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.027+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faa5c19eae0 con 0x7faa4406db20 2024-04-02T04:49:49.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.040+0000 7faa43fff700 1 -- 172.21.15.70:0/1470130240 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7faa5c19eae0 con 0x7faa4406db20 2024-04-02T04:49:49.044 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:49:49.044 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (8m) 13s ago 10m 24.2M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:49:49.044 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (10m) 13s ago 10m 7834k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (9m) 28s ago 9m 7507k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (10m) 13s ago 10m 6669k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (9m) 28s ago 9m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (8m) 13s ago 10m 87.8M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (4m) 13s ago 4m 202M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (4m) 13s ago 4m 17.1M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (4m) 28s ago 4m 15.7M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (4m) 28s ago 4m 17.2M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (86s) 13s ago 11m 493M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (32s) 28s ago 9m 64.9M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (11m) 13s ago 11m 52.5M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (9m) 28s ago 9m 46.9M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 starting - - - - 2024-04-02T04:49:49.045 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 starting - - - - 2024-04-02T04:49:49.046 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (8m) 13s ago 8m 219M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:49:49.046 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (8m) 13s ago 8m 252M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:49:49.046 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (7m) 13s ago 7m 201M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:49:49.046 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (7m) 28s ago 7m 280M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:49:49.046 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (6m) 28s ago 6m 260M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:49:49.046 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (6m) 28s ago 6m 216M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:49:49.046 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (38s) 13s ago 9m 38.3M - 2.43.0 a07b618ecd1d 447e1d662538 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faa4406db20 msgr2=0x7faa4406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faa4406db20 0x7faa4406ffe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faa50009fd0 tx=0x7faa50005940 comp rx=0 tx=0).stop 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c0ffc10 msgr2=0x7faa5c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c0ffc10 0x7faa5c103ba0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faa5400efc0 tx=0x7faa5400c3a0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 shutdown_connections 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faa4406db20 0x7faa4406ffe0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa5c0ffc10 0x7faa5c103ba0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 --2- 172.21.15.70:0/1470130240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa5c100e10 0x7faa5c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.042+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 >> 172.21.15.70:0/1470130240 conn(0x7faa5c0fb190 msgr2=0x7faa5c068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.043+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 shutdown_connections 2024-04-02T04:49:49.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.043+0000 7faa612e8700 1 -- 172.21.15.70:0/1470130240 wait complete. 2024-04-02T04:49:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.148+0000 7f21b3446700 1 -- 172.21.15.70:0/3468883561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 msgr2=0x7f21ac102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.148+0000 7f21b3446700 1 --2- 172.21.15.70:0/3468883561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 0x7f21ac102310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f219c009910 tx=0x7f219c009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 -- 172.21.15.70:0/3468883561 shutdown_connections 2024-04-02T04:49:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 --2- 172.21.15.70:0/3468883561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 0x7f21ac102310 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 --2- 172.21.15.70:0/3468883561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21ac100cb0 0x7f21ac1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 -- 172.21.15.70:0/3468883561 >> 172.21.15.70:0/3468883561 conn(0x7f21ac0fc230 msgr2=0x7f21ac0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 -- 172.21.15.70:0/3468883561 shutdown_connections 2024-04-02T04:49:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 -- 172.21.15.70:0/3468883561 wait complete. 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 Processor -- start 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 -- start start 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21ac100cb0 0x7f21ac1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 0x7f21ac1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac197010 con 0x7f21ac101eb0 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.149+0000 7f21b3446700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac197150 con 0x7f21ac100cb0 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b09e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 0x7f21ac1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b09e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 0x7f21ac1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57422/0 (socket says 172.21.15.70:57422) 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b09e1700 1 -- 172.21.15.70:0/976676976 learned_addr learned my addr 172.21.15.70:0/976676976 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b11e2700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21ac100cb0 0x7f21ac1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b09e1700 1 -- 172.21.15.70:0/976676976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21ac100cb0 msgr2=0x7f21ac1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b09e1700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21ac100cb0 0x7f21ac1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b09e1700 1 -- 172.21.15.70:0/976676976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0095f0 con 0x7f21ac101eb0 2024-04-02T04:49:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b09e1700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 0x7f21ac1969f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f219c006010 tx=0x7f219c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21a27fc700 1 -- 172.21.15.70:0/976676976 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219c01d070 con 0x7f21ac101eb0 2024-04-02T04:49:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21a27fc700 1 -- 172.21.15.70:0/976676976 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f219c00f460 con 0x7f21ac101eb0 2024-04-02T04:49:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac19bba0 con 0x7f21ac101eb0 2024-04-02T04:49:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21a27fc700 1 -- 172.21.15.70:0/976676976 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219c021640 con 0x7f21ac101eb0 2024-04-02T04:49:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.150+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac19c090 con 0x7f21ac101eb0 2024-04-02T04:49:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.151+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac109810 con 0x7f21ac101eb0 2024-04-02T04:49:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.152+0000 7f21a27fc700 1 -- 172.21.15.70:0/976676976 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f219c00fab0 con 0x7f21ac101eb0 2024-04-02T04:49:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.152+0000 7f21a27fc700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f219806db20 0x7f219806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.152+0000 7f21a27fc700 1 -- 172.21.15.70:0/976676976 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f219c08ea10 con 0x7f21ac101eb0 2024-04-02T04:49:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.152+0000 7f21b11e2700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f219806db20 0x7f219806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.153+0000 7f21b11e2700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f219806db20 0x7f219806ffe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f21ac101d10 tx=0x7f21a800b410 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:49.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.157+0000 7f21a27fc700 1 -- 172.21.15.70:0/976676976 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f219c05bdc0 con 0x7f21ac101eb0 2024-04-02T04:49:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.439+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f21ac109a00 con 0x7f21ac101eb0 2024-04-02T04:49:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.439+0000 7f21a27fc700 1 -- 172.21.15.70:0/976676976 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f219c05b950 con 0x7f21ac101eb0 2024-04-02T04:49:49.443 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:49:49.443 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-02T04:49:49.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:49.445 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:49:49.445 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:49:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f219806db20 msgr2=0x7f219806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f219806db20 0x7f219806ffe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f21ac101d10 tx=0x7f21a800b410 comp rx=0 tx=0).stop 2024-04-02T04:49:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 msgr2=0x7f21ac1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 0x7f21ac1969f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f219c006010 tx=0x7f219c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:49:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 shutdown_connections 2024-04-02T04:49:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f219806db20 0x7f219806ffe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21ac100cb0 0x7f21ac1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 --2- 172.21.15.70:0/976676976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21ac101eb0 0x7f21ac1969f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 >> 172.21.15.70:0/976676976 conn(0x7f21ac0fc230 msgr2=0x7f21ac1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 shutdown_connections 2024-04-02T04:49:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.441+0000 7f21b3446700 1 -- 172.21.15.70:0/976676976 wait complete. 2024-04-02T04:49:49.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 -- 172.21.15.70:0/780541965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41017a0 msgr2=0x7f52f4103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 --2- 172.21.15.70:0/780541965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41017a0 0x7f52f4103b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f52f000b0a0 tx=0x7f52f000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 -- 172.21.15.70:0/780541965 shutdown_connections 2024-04-02T04:49:49.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 --2- 172.21.15.70:0/780541965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f41040d0 0x7f52f41064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 --2- 172.21.15.70:0/780541965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41017a0 0x7f52f4103b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 -- 172.21.15.70:0/780541965 >> 172.21.15.70:0/780541965 conn(0x7f52f40fb190 msgr2=0x7f52f40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:49.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 -- 172.21.15.70:0/780541965 shutdown_connections 2024-04-02T04:49:49.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 -- 172.21.15.70:0/780541965 wait complete. 2024-04-02T04:49:49.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.549+0000 7f52fbbcd700 1 Processor -- start 2024-04-02T04:49:49.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52fbbcd700 1 -- start start 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52fbbcd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f41017a0 0x7f52f41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52fbbcd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41040d0 0x7f52f4196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52fbbcd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4196fa0 con 0x7f52f41040d0 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52fbbcd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f41970e0 con 0x7f52f41017a0 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52f9168700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41040d0 0x7f52f4196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52f9168700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41040d0 0x7f52f4196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57442/0 (socket says 172.21.15.70:57442) 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52f9168700 1 -- 172.21.15.70:0/1420199866 learned_addr learned my addr 172.21.15.70:0/1420199866 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52f9969700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f41017a0 0x7f52f41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52f9168700 1 -- 172.21.15.70:0/1420199866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f41017a0 msgr2=0x7f52f41964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52f9168700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f41017a0 0x7f52f41964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52f9168700 1 -- 172.21.15.70:0/1420199866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0009d00 con 0x7f52f41040d0 2024-04-02T04:49:49.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52f9168700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41040d0 0x7f52f4196a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f52e400c930 tx=0x7f52e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:49.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52eaffd700 1 -- 172.21.15.70:0/1420199866 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52e4007ad0 con 0x7f52f41040d0 2024-04-02T04:49:49.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52eaffd700 1 -- 172.21.15.70:0/1420199866 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f52e400ce80 con 0x7f52f41040d0 2024-04-02T04:49:49.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52eaffd700 1 -- 172.21.15.70:0/1420199866 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52e4018630 con 0x7f52f41040d0 2024-04-02T04:49:49.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.550+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f419bba0 con 0x7f52f41040d0 2024-04-02T04:49:49.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.551+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f419c0c0 con 0x7f52f41040d0 2024-04-02T04:49:49.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.551+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f4190610 con 0x7f52f41040d0 2024-04-02T04:49:49.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.552+0000 7f52eaffd700 1 -- 172.21.15.70:0/1420199866 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f52e4007c30 con 0x7f52f41040d0 2024-04-02T04:49:49.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.552+0000 7f52eaffd700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f52e006db10 0x7f52e006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:49.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.552+0000 7f52eaffd700 1 -- 172.21.15.70:0/1420199866 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f52e408d810 con 0x7f52f41040d0 2024-04-02T04:49:49.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.552+0000 7f52f9969700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f52e006db10 0x7f52e006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:49.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.553+0000 7f52f9969700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f52e006db10 0x7f52e006ffd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000f80 tx=0x7f52f0009340 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:49.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.557+0000 7f52eaffd700 1 -- 172.21.15.70:0/1420199866 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f52e405abe0 con 0x7f52f41040d0 2024-04-02T04:49:49.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:49 smithi179 ceph-mon[36603]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 895 KiB/s wr, 283 op/s 2024-04-02T04:49:49.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:49 smithi179 ceph-mon[36603]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:49.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:49 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:49.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:49 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:49.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:49 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:49.633 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:49 smithi070 ceph-mon[29212]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 895 KiB/s wr, 283 op/s 2024-04-02T04:49:49.633 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:49 smithi070 ceph-mon[29212]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:49.633 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:49 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:49.633 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:49 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:49.634 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:49 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:49.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.772+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52f40611d0 con 0x7f52e006db10 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.773+0000 7f52eaffd700 1 -- 172.21.15.70:0/1420199866 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f52f40611d0 con 0x7f52e006db10 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/2 daemons upgraded", 2024-04-02T04:49:49.777 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-02T04:49:49.778 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:49:49.778 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:49:49.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f52e006db10 msgr2=0x7f52e006ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f52e006db10 0x7f52e006ffd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000f80 tx=0x7f52f0009340 comp rx=0 tx=0).stop 2024-04-02T04:49:49.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41040d0 msgr2=0x7f52f4196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:49.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41040d0 0x7f52f4196a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f52e400c930 tx=0x7f52e400cc40 comp rx=0 tx=0).stop 2024-04-02T04:49:49.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 shutdown_connections 2024-04-02T04:49:49.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f52e006db10 0x7f52e006ffd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f41017a0 0x7f52f41964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 --2- 172.21.15.70:0/1420199866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f41040d0 0x7f52f4196a10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:49.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 >> 172.21.15.70:0/1420199866 conn(0x7f52f40fb190 msgr2=0x7f52f40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:49.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 shutdown_connections 2024-04-02T04:49:49.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:49:49.775+0000 7f52fbbcd700 1 -- 172.21.15.70:0/1420199866 wait complete. 2024-04-02T04:49:50.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:50 smithi070 ceph-mon[29212]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:50 smithi070 ceph-mon[29212]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:50 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/976676976' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:50.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:50 smithi179 ceph-mon[36603]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:50.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:50 smithi179 ceph-mon[36603]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:50 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/976676976' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:51.694 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:51 smithi070 ceph-mon[29212]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 913 KiB/s wr, 290 op/s 2024-04-02T04:49:51.695 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:51 smithi070 ceph-mon[29212]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:51.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:51 smithi179 ceph-mon[36603]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 913 KiB/s wr, 290 op/s 2024-04-02T04:49:51.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:51 smithi179 ceph-mon[36603]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:52.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:52 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:52 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:52.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:52 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:52.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:52 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:53.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:53 smithi070 ceph-mon[29212]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 359 KiB/s wr, 142 op/s 2024-04-02T04:49:53.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:53 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:53.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:53 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:53 smithi179 ceph-mon[36603]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 359 KiB/s wr, 142 op/s 2024-04-02T04:49:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:53 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:53 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:55.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:54 smithi070 ceph-mon[29212]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 359 KiB/s wr, 142 op/s 2024-04-02T04:49:55.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:54 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:55.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:54 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:55.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:54 smithi179 ceph-mon[36603]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 359 KiB/s wr, 142 op/s 2024-04-02T04:49:55.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:54 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:55.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:54 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:56.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:55 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:56.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:55 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:56.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:55 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:56.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:55 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:57 smithi070 ceph-mon[29212]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 18 KiB/s wr, 7 op/s 2024-04-02T04:49:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:57 smithi179 ceph-mon[36603]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 18 KiB/s wr, 7 op/s 2024-04-02T04:49:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:59.423 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 18 KiB/s wr, 7 op/s 2024-04-02T04:49:59.423 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:59.423 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.424 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:49:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 18 KiB/s wr, 7 op/s 2024-04-02T04:49:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:49:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:49:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:00.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:00 smithi070 ceph-mon[29212]: overall HEALTH_OK 2024-04-02T04:50:00.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:00 smithi179 ceph-mon[36603]: overall HEALTH_OK 2024-04-02T04:50:01.410 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:01 smithi179 ceph-mon[36603]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 18 KiB/s wr, 7 op/s 2024-04-02T04:50:01.410 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:01 smithi179 ceph-mon[36603]: Upgrade: Updating prometheus.smithi070 2024-04-02T04:50:01.481 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:01 smithi070 ceph-mon[29212]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 18 KiB/s wr, 7 op/s 2024-04-02T04:50:01.481 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:01 smithi070 ceph-mon[29212]: Upgrade: Updating prometheus.smithi070 2024-04-02T04:50:02.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:02 smithi070 ceph-mon[29212]: Deploying daemon prometheus.smithi070 on smithi070 2024-04-02T04:50:02.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:02 smithi179 ceph-mon[36603]: Deploying daemon prometheus.smithi070 on smithi070 2024-04-02T04:50:03.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:03 smithi070 ceph-mon[29212]: pgmap v43: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:03.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:03 smithi179 ceph-mon[36603]: pgmap v43: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:05.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:05 smithi070 ceph-mon[29212]: pgmap v44: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:05.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:05 smithi179 ceph-mon[36603]: pgmap v44: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:07.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:07 smithi070 ceph-mon[29212]: pgmap v45: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:07.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:07 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:07.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:07 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:07.463 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:07 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:07.463 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:07 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:07.463 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:07 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:07 smithi179 ceph-mon[36603]: pgmap v45: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:07 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:07 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:07 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:07 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:07 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:09.698 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:09 smithi070 ceph-mon[29212]: pgmap v46: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:09.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:09 smithi179 ceph-mon[36603]: pgmap v46: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:11.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:11 smithi070 ceph-mon[29212]: pgmap v47: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:11.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:11 smithi179 ceph-mon[36603]: pgmap v47: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:12.805 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:12 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:12.805 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:12 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:12.805 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:12 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:13.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:12 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:13.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:12 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:13.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:12 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:13.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:13 smithi070 ceph-mon[29212]: pgmap v48: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:13.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:13 smithi070 ceph-mon[29212]: mgrmap e31: smithi070.igidvr(active, since 90s), standbys: smithi179.labmvs 2024-04-02T04:50:13.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:13 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:13.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:13 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:14.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:13 smithi179 ceph-mon[36603]: pgmap v48: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:14.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:13 smithi179 ceph-mon[36603]: mgrmap e31: smithi070.igidvr(active, since 90s), standbys: smithi179.labmvs 2024-04-02T04:50:14.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:13 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:14.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:13 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:16.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:15 smithi070 ceph-mon[29212]: pgmap v49: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:16.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:15 smithi179 ceph-mon[36603]: pgmap v49: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: pgmap v50: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:17.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:17.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:17 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: pgmap v50: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:17 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:18.959 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:18 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:50:19.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:18 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:50:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.885+0000 7fc660ff1700 1 -- 172.21.15.70:0/3295935317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 msgr2=0x7fc65c1064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.885+0000 7fc660ff1700 1 --2- 172.21.15.70:0/3295935317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 0x7fc65c1064c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc65400b0a0 tx=0x7fc65400b3b0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.885+0000 7fc660ff1700 1 -- 172.21.15.70:0/3295935317 shutdown_connections 2024-04-02T04:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.885+0000 7fc660ff1700 1 --2- 172.21.15.70:0/3295935317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 0x7fc65c1064c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.885+0000 7fc660ff1700 1 --2- 172.21.15.70:0/3295935317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1017a0 0x7fc65c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.885+0000 7fc660ff1700 1 -- 172.21.15.70:0/3295935317 >> 172.21.15.70:0/3295935317 conn(0x7fc65c0fb190 msgr2=0x7fc65c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.885+0000 7fc660ff1700 1 -- 172.21.15.70:0/3295935317 shutdown_connections 2024-04-02T04:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.885+0000 7fc660ff1700 1 -- 172.21.15.70:0/3295935317 wait complete. 2024-04-02T04:50:19.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:19 smithi070 ceph-mon[29212]: pgmap v51: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:19.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:19 smithi070 ceph-mon[29212]: Upgrade: Updating alertmanager.smithi070 2024-04-02T04:50:19.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:19 smithi070 ceph-mon[29212]: Deploying daemon alertmanager.smithi070 on smithi070 2024-04-02T04:50:19.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc660ff1700 1 Processor -- start 2024-04-02T04:50:19.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc660ff1700 1 -- start start 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc660ff1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1017a0 0x7fc65c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc660ff1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 0x7fc65c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc660ff1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c199340 con 0x7fc65c1040d0 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc660ff1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c199480 con 0x7fc65c1017a0 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc659d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 0x7fc65c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc659d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 0x7fc65c198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48974/0 (socket says 172.21.15.70:48974) 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc659d9b700 1 -- 172.21.15.70:0/243423047 learned_addr learned my addr 172.21.15.70:0/243423047 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc65a59c700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1017a0 0x7fc65c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:19.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc659d9b700 1 -- 172.21.15.70:0/243423047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1017a0 msgr2=0x7fc65c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc659d9b700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1017a0 0x7fc65c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc659d9b700 1 -- 172.21.15.70:0/243423047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654009d00 con 0x7fc65c1040d0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.886+0000 7fc659d9b700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 0x7fc65c198d20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc65400b070 tx=0x7fc654003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.887+0000 7fc64b7fe700 1 -- 172.21.15.70:0/243423047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc65400e040 con 0x7fc65c1040d0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.887+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c19ded0 con 0x7fc65c1040d0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.887+0000 7fc64b7fe700 1 -- 172.21.15.70:0/243423047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc65400ce60 con 0x7fc65c1040d0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.887+0000 7fc64b7fe700 1 -- 172.21.15.70:0/243423047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc65401da50 con 0x7fc65c1040d0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.887+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c19e390 con 0x7fc65c1040d0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.888+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c109880 con 0x7fc65c1040d0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.888+0000 7fc64b7fe700 1 -- 172.21.15.70:0/243423047 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7fc654004250 con 0x7fc65c1040d0 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.889+0000 7fc64b7fe700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc64406d9e0 0x7fc64406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:19.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.889+0000 7fc64b7fe700 1 -- 172.21.15.70:0/243423047 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fc65408efe0 con 0x7fc65c1040d0 2024-04-02T04:50:19.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.889+0000 7fc65a59c700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc64406d9e0 0x7fc64406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:19.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.889+0000 7fc65a59c700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc64406d9e0 0x7fc64406fea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009a10 tx=0x7fc64c009190 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:19.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:19.894+0000 7fc64b7fe700 1 -- 172.21.15.70:0/243423047 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fc65405c650 con 0x7fc65c1040d0 2024-04-02T04:50:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:19 smithi179 ceph-mon[36603]: pgmap v51: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:19 smithi179 ceph-mon[36603]: Upgrade: Updating alertmanager.smithi070 2024-04-02T04:50:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:19 smithi179 ceph-mon[36603]: Deploying daemon alertmanager.smithi070 on smithi070 2024-04-02T04:50:20.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.109+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc65c19eab0 con 0x7fc64406d9e0 2024-04-02T04:50:20.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.111+0000 7fc64b7fe700 1 -- 172.21.15.70:0/243423047 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fc65c19eab0 con 0x7fc64406d9e0 2024-04-02T04:50:20.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc64406d9e0 msgr2=0x7fc64406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc64406d9e0 0x7fc64406fea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009a10 tx=0x7fc64c009190 comp rx=0 tx=0).stop 2024-04-02T04:50:20.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 msgr2=0x7fc65c198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 0x7fc65c198d20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc65400b070 tx=0x7fc654003c30 comp rx=0 tx=0).stop 2024-04-02T04:50:20.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 shutdown_connections 2024-04-02T04:50:20.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc64406d9e0 0x7fc64406fea0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1017a0 0x7fc65c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 --2- 172.21.15.70:0/243423047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65c1040d0 0x7fc65c198d20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 >> 172.21.15.70:0/243423047 conn(0x7fc65c0fb190 msgr2=0x7fc65c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:20.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 shutdown_connections 2024-04-02T04:50:20.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.113+0000 7fc660ff1700 1 -- 172.21.15.70:0/243423047 wait complete. 2024-04-02T04:50:20.127 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:50:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.218+0000 7f1304dca700 1 -- 172.21.15.70:0/589738423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 msgr2=0x7f1300101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.218+0000 7f1304dca700 1 --2- 172.21.15.70:0/589738423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 0x7f1300101220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009910 tx=0x7f12f4009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.218+0000 7f1304dca700 1 -- 172.21.15.70:0/589738423 shutdown_connections 2024-04-02T04:50:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.218+0000 7f1304dca700 1 --2- 172.21.15.70:0/589738423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 0x7f1300101220 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.218+0000 7f1304dca700 1 --2- 172.21.15.70:0/589738423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13000ffbc0 0x7f13000fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.218+0000 7f1304dca700 1 -- 172.21.15.70:0/589738423 >> 172.21.15.70:0/589738423 conn(0x7f13000fb160 msgr2=0x7f13000fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.218+0000 7f1304dca700 1 -- 172.21.15.70:0/589738423 shutdown_connections 2024-04-02T04:50:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.218+0000 7f1304dca700 1 -- 172.21.15.70:0/589738423 wait complete. 2024-04-02T04:50:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f1304dca700 1 Processor -- start 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f1304dca700 1 -- start start 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f1304dca700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13000ffbc0 0x7f13001989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f1304dca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 0x7f1300198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f1304dca700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1300199520 con 0x7f1300100dc0 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f1304dca700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1300199660 con 0x7f13000ffbc0 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f12fdd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 0x7f1300198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f12fdd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 0x7f1300198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48994/0 (socket says 172.21.15.70:48994) 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.219+0000 7f12fdd9b700 1 -- 172.21.15.70:0/3410362661 learned_addr learned my addr 172.21.15.70:0/3410362661 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f12fe59c700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13000ffbc0 0x7f13001989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f12fdd9b700 1 -- 172.21.15.70:0/3410362661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13000ffbc0 msgr2=0x7f13001989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f12fdd9b700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13000ffbc0 0x7f13001989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f12fdd9b700 1 -- 172.21.15.70:0/3410362661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f40095f0 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f12fdd9b700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 0x7f1300198f00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12f400bd70 tx=0x7f12f400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f12ef7fe700 1 -- 172.21.15.70:0/3410362661 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f401d070 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130019e0b0 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f12ef7fe700 1 -- 172.21.15.70:0/3410362661 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f12f4022470 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f12ef7fe700 1 -- 172.21.15.70:0/3410362661 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f400f670 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.220+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130019e5a0 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.222+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1300068af0 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.222+0000 7f12ef7fe700 1 -- 172.21.15.70:0/3410362661 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f12f400f8e0 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.222+0000 7f12ef7fe700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f12e806d9e0 0x7f12e806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.222+0000 7f12ef7fe700 1 -- 172.21.15.70:0/3410362661 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f12f408ead0 con 0x7f1300100dc0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.222+0000 7f12fe59c700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f12e806d9e0 0x7f12e806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.223+0000 7f12fe59c700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f12e806d9e0 0x7f12e806fea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005950 tx=0x7f12f00058e0 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.227+0000 7f12ef7fe700 1 -- 172.21.15.70:0/3410362661 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f12f405d1f0 con 0x7f1300100dc0 2024-04-02T04:50:20.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.443+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1300069560 con 0x7f12e806d9e0 2024-04-02T04:50:20.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.444+0000 7f12ef7fe700 1 -- 172.21.15.70:0/3410362661 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f1300069560 con 0x7f12e806d9e0 2024-04-02T04:50:20.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f12e806d9e0 msgr2=0x7f12e806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f12e806d9e0 0x7f12e806fea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005950 tx=0x7f12f00058e0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 msgr2=0x7f1300198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 0x7f1300198f00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12f400bd70 tx=0x7f12f400be50 comp rx=0 tx=0).stop 2024-04-02T04:50:20.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 shutdown_connections 2024-04-02T04:50:20.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f12e806d9e0 0x7f12e806fea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13000ffbc0 0x7f13001989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 --2- 172.21.15.70:0/3410362661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1300100dc0 0x7f1300198f00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 >> 172.21.15.70:0/3410362661 conn(0x7f13000fb160 msgr2=0x7f1300068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:20.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 shutdown_connections 2024-04-02T04:50:20.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.446+0000 7f1304dca700 1 -- 172.21.15.70:0/3410362661 wait complete. 2024-04-02T04:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.556+0000 7facb29e2700 1 -- 172.21.15.70:0/2522175966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 msgr2=0x7facac102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.556+0000 7facb29e2700 1 --2- 172.21.15.70:0/2522175966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 0x7facac102310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faca0009910 tx=0x7faca0009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.557+0000 7facb29e2700 1 -- 172.21.15.70:0/2522175966 shutdown_connections 2024-04-02T04:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.557+0000 7facb29e2700 1 --2- 172.21.15.70:0/2522175966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 0x7facac102310 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.557+0000 7facb29e2700 1 --2- 172.21.15.70:0/2522175966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac100cb0 0x7facac1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.557+0000 7facb29e2700 1 -- 172.21.15.70:0/2522175966 >> 172.21.15.70:0/2522175966 conn(0x7facac0fc230 msgr2=0x7facac0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.557+0000 7facb29e2700 1 -- 172.21.15.70:0/2522175966 shutdown_connections 2024-04-02T04:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.557+0000 7facb29e2700 1 -- 172.21.15.70:0/2522175966 wait complete. 2024-04-02T04:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.557+0000 7facb29e2700 1 Processor -- start 2024-04-02T04:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.557+0000 7facb29e2700 1 -- start start 2024-04-02T04:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facb29e2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac100cb0 0x7facac1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facb29e2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 0x7facac196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facb29e2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac197040 con 0x7facac101eb0 2024-04-02T04:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facb29e2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac197180 con 0x7facac100cb0 2024-04-02T04:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facab7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 0x7facac196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facab7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 0x7facac196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49006/0 (socket says 172.21.15.70:49006) 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facab7fe700 1 -- 172.21.15.70:0/333253401 learned_addr learned my addr 172.21.15.70:0/333253401 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facabfff700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac100cb0 0x7facac1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facab7fe700 1 -- 172.21.15.70:0/333253401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac100cb0 msgr2=0x7facac1964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facab7fe700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac100cb0 0x7facac1964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facab7fe700 1 -- 172.21.15.70:0/333253401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca00095f0 con 0x7facac101eb0 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facab7fe700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 0x7facac196a20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faca0006010 tx=0x7faca000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7faca97fa700 1 -- 172.21.15.70:0/333253401 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faca001d070 con 0x7facac101eb0 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7faca97fa700 1 -- 172.21.15.70:0/333253401 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faca000f460 con 0x7facac101eb0 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac19bbd0 con 0x7facac101eb0 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7faca97fa700 1 -- 172.21.15.70:0/333253401 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faca0021640 con 0x7facac101eb0 2024-04-02T04:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.558+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac19c0c0 con 0x7facac101eb0 2024-04-02T04:50:20.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.560+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac109810 con 0x7facac101eb0 2024-04-02T04:50:20.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.560+0000 7faca97fa700 1 -- 172.21.15.70:0/333253401 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7faca0003a60 con 0x7facac101eb0 2024-04-02T04:50:20.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.560+0000 7faca97fa700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fac9406d9e0 0x7fac9406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.560+0000 7faca97fa700 1 -- 172.21.15.70:0/333253401 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7faca008e7c0 con 0x7facac101eb0 2024-04-02T04:50:20.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.560+0000 7facabfff700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fac9406d9e0 0x7fac9406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.561+0000 7facabfff700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fac9406d9e0 0x7fac9406fea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facac101d10 tx=0x7fac9c00b410 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:20.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.565+0000 7faca97fa700 1 -- 172.21.15.70:0/333253401 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7faca005bdb0 con 0x7facac101eb0 2024-04-02T04:50:20.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.773+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7facac106800 con 0x7fac9406d9e0 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.784+0000 7faca97fa700 1 -- 172.21.15.70:0/333253401 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7facac106800 con 0x7fac9406d9e0 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (9m) 9s ago 10m 24.2M - 0.25.0 c8568f914cd2 3a7524ca5b25 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (11m) 9s ago 11m 7902k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (10m) 29s ago 10m 7545k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (11m) 9s ago 11m 6669k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (10m) 29s ago 10m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (9m) 9s ago 10m 87.8M - 9.4.7 954c08fa6188 304293c65d93 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (5m) 9s ago 5m 154M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:50:20.788 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (5m) 9s ago 5m 17.5M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (5m) 29s ago 5m 15.9M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (5m) 29s ago 5m 17.5M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (118s) 9s ago 12m 499M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (64s) 29s ago 9m 448M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (12m) 9s ago 12m 53.6M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (9m) 29s ago 9m 48.4M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (36s) 9s ago 11m 7558k - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (32s) 29s ago 10m 3439k - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (9m) 9s ago 9m 227M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (8m) 9s ago 8m 262M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (8m) 9s ago 8m 210M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (7m) 29s ago 7m 296M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:50:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (7m) 29s ago 7m 272M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:50:20.790 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (6m) 29s ago 6m 226M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:50:20.790 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (14s) 9s ago 10m 35.7M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:50:20.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fac9406d9e0 msgr2=0x7fac9406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fac9406d9e0 0x7fac9406fea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facac101d10 tx=0x7fac9c00b410 comp rx=0 tx=0).stop 2024-04-02T04:50:20.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 msgr2=0x7facac196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 0x7facac196a20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faca0006010 tx=0x7faca000bd70 comp rx=0 tx=0).stop 2024-04-02T04:50:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 shutdown_connections 2024-04-02T04:50:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fac9406d9e0 0x7fac9406fea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facac100cb0 0x7facac1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 --2- 172.21.15.70:0/333253401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facac101eb0 0x7facac196a20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 >> 172.21.15.70:0/333253401 conn(0x7facac0fc230 msgr2=0x7facac1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 shutdown_connections 2024-04-02T04:50:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.786+0000 7facb29e2700 1 -- 172.21.15.70:0/333253401 wait complete. 2024-04-02T04:50:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.892+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1177817516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a8100520 msgr2=0x7f48a8100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.892+0000 7f48ad5c7700 1 --2- 172.21.15.70:0/1177817516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a8100520 0x7f48a8100980 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f489c009910 tx=0x7f489c009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.892+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1177817516 shutdown_connections 2024-04-02T04:50:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.892+0000 7f48ad5c7700 1 --2- 172.21.15.70:0/1177817516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a8100520 0x7f48a8100980 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.892+0000 7f48ad5c7700 1 --2- 172.21.15.70:0/1177817516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a80ff320 0x7f48a80ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.892+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1177817516 >> 172.21.15.70:0/1177817516 conn(0x7f48a80fa8c0 msgr2=0x7f48a80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.892+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1177817516 shutdown_connections 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.892+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1177817516 wait complete. 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48ad5c7700 1 Processor -- start 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48ad5c7700 1 -- start start 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48ad5c7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80ff320 0x7f48a81022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48ad5c7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a8100520 0x7f48a819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48ad5c7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a819b7d0 con 0x7f48a80ff320 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48ad5c7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a819b910 con 0x7f48a8100520 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80ff320 0x7f48a81022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80ff320 0x7f48a81022a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49036/0 (socket says 172.21.15.70:49036) 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48a6ffd700 1 -- 172.21.15.70:0/1238414542 learned_addr learned my addr 172.21.15.70:0/1238414542 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48a67fc700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a8100520 0x7f48a819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48a6ffd700 1 -- 172.21.15.70:0/1238414542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a8100520 msgr2=0x7f48a819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48a6ffd700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a8100520 0x7f48a819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48a6ffd700 1 -- 172.21.15.70:0/1238414542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0095f0 con 0x7f48a80ff320 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.893+0000 7f48a67fc700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a8100520 0x7f48a819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.894+0000 7f48a6ffd700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80ff320 0x7f48a81022a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f48a0007e90 tx=0x7f48a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.894+0000 7f488ffff700 1 -- 172.21.15.70:0/1238414542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48a0017070 con 0x7f48a80ff320 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.894+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a819bbf0 con 0x7f48a80ff320 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.894+0000 7f488ffff700 1 -- 172.21.15.70:0/1238414542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f48a000f040 con 0x7f48a80ff320 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.894+0000 7f488ffff700 1 -- 172.21.15.70:0/1238414542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48a0004870 con 0x7f48a80ff320 2024-04-02T04:50:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.894+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a819c140 con 0x7f48a80ff320 2024-04-02T04:50:20.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.895+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a8107e80 con 0x7f48a80ff320 2024-04-02T04:50:20.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.895+0000 7f488ffff700 1 -- 172.21.15.70:0/1238414542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f48a00040b0 con 0x7f48a80ff320 2024-04-02T04:50:20.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.896+0000 7f488ffff700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f489006d990 0x7f489006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:20.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.896+0000 7f48a67fc700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f489006d990 0x7f489006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:20.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.896+0000 7f488ffff700 1 -- 172.21.15.70:0/1238414542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f48a008cee0 con 0x7f48a80ff320 2024-04-02T04:50:20.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.896+0000 7f48a67fc700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f489006d990 0x7f489006fe50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f48a8102820 tx=0x7f489c005a90 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:20.900+0000 7f488ffff700 1 -- 172.21.15.70:0/1238414542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f48a005a4d0 con 0x7f48a80ff320 2024-04-02T04:50:21.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.183+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f48a81080a0 con 0x7f48a80ff320 2024-04-02T04:50:21.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.183+0000 7f488ffff700 1 -- 172.21.15.70:0/1238414542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f48a005a060 con 0x7f48a80ff320 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:50:21.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:50:21.189 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:50:21.189 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:50:21.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:50:21.189 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:50:21.189 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-02T04:50:21.189 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:50:21.189 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:50:21.189 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f489006d990 msgr2=0x7f489006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f489006d990 0x7f489006fe50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f48a8102820 tx=0x7f489c005a90 comp rx=0 tx=0).stop 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80ff320 msgr2=0x7f48a81022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80ff320 0x7f48a81022a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f48a0007e90 tx=0x7f48a000d3b0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 shutdown_connections 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f489006d990 0x7f489006fe50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80ff320 0x7f48a81022a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 --2- 172.21.15.70:0/1238414542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48a8100520 0x7f48a819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.185+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 >> 172.21.15.70:0/1238414542 conn(0x7f48a80fa8c0 msgr2=0x7f48a8104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.186+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 shutdown_connections 2024-04-02T04:50:21.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.186+0000 7f48ad5c7700 1 -- 172.21.15.70:0/1238414542 wait complete. 2024-04-02T04:50:21.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 -- 172.21.15.70:0/2524252496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c100e10 msgr2=0x7f181c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:21.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 --2- 172.21.15.70:0/2524252496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c100e10 0x7f181c101270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1810009910 tx=0x7f1810009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:21.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 -- 172.21.15.70:0/2524252496 shutdown_connections 2024-04-02T04:50:21.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 --2- 172.21.15.70:0/2524252496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c100e10 0x7f181c101270 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 --2- 172.21.15.70:0/2524252496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f181c0ffc10 0x7f181c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 -- 172.21.15.70:0/2524252496 >> 172.21.15.70:0/2524252496 conn(0x7f181c0fb190 msgr2=0x7f181c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:21.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 -- 172.21.15.70:0/2524252496 shutdown_connections 2024-04-02T04:50:21.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 -- 172.21.15.70:0/2524252496 wait complete. 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 Processor -- start 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.291+0000 7f1822528700 1 -- start start 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f1822528700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c0ffc10 0x7f181c103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f1822528700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f181c100e10 0x7f181c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f1822528700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c102730 con 0x7f181c0ffc10 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f1822528700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c102870 con 0x7f181c100e10 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c0ffc10 0x7f181c103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c0ffc10 0x7f181c103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49064/0 (socket says 172.21.15.70:49064) 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181bfff700 1 -- 172.21.15.70:0/445854091 learned_addr learned my addr 172.21.15.70:0/445854091 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181b7fe700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f181c100e10 0x7f181c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:21.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181bfff700 1 -- 172.21.15.70:0/445854091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f181c100e10 msgr2=0x7f181c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181bfff700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f181c100e10 0x7f181c1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181bfff700 1 -- 172.21.15.70:0/445854091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18100095f0 con 0x7f181c0ffc10 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181b7fe700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f181c100e10 0x7f181c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.292+0000 7f181bfff700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c0ffc10 0x7f181c103ba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f180c00d900 tx=0x7f180c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.293+0000 7f18197fa700 1 -- 172.21.15.70:0/445854091 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180c009900 con 0x7f181c0ffc10 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.293+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181c102b50 con 0x7f181c0ffc10 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.293+0000 7f18197fa700 1 -- 172.21.15.70:0/445854091 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f180c010460 con 0x7f181c0ffc10 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.293+0000 7f18197fa700 1 -- 172.21.15.70:0/445854091 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180c00f5b0 con 0x7f181c0ffc10 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.293+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c19e530 con 0x7f181c0ffc10 2024-04-02T04:50:21.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.293+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181c108720 con 0x7f181c0ffc10 2024-04-02T04:50:21.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.294+0000 7f18197fa700 1 -- 172.21.15.70:0/445854091 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f180c0105d0 con 0x7f181c0ffc10 2024-04-02T04:50:21.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.294+0000 7f18197fa700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f180406d9e0 0x7f180406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:21.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.295+0000 7f18197fa700 1 -- 172.21.15.70:0/445854091 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f180c08d1d0 con 0x7f181c0ffc10 2024-04-02T04:50:21.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.295+0000 7f181b7fe700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f180406d9e0 0x7f180406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:21.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.295+0000 7f181b7fe700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f180406d9e0 0x7f180406fea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1810005b40 tx=0x7f1810005a90 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:21.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.299+0000 7f18197fa700 1 -- 172.21.15.70:0/445854091 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f180c05a7c0 con 0x7f181c0ffc10 2024-04-02T04:50:21.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.516+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f181c19eae0 con 0x7f180406d9e0 2024-04-02T04:50:21.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.517+0000 7f18197fa700 1 -- 172.21.15.70:0/445854091 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f181c19eae0 con 0x7f180406d9e0 2024-04-02T04:50:21.522 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:50:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:50:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:50:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:50:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:50:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-04-02T04:50:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:50:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/2 daemons upgraded", 2024-04-02T04:50:21.523 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-02T04:50:21.523 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:50:21.523 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f180406d9e0 msgr2=0x7f180406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f180406d9e0 0x7f180406fea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1810005b40 tx=0x7f1810005a90 comp rx=0 tx=0).stop 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c0ffc10 msgr2=0x7f181c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c0ffc10 0x7f181c103ba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f180c00d900 tx=0x7f180c00dcc0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 shutdown_connections 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f180406d9e0 0x7f180406fea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f181c0ffc10 0x7f181c103ba0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 --2- 172.21.15.70:0/445854091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f181c100e10 0x7f181c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 >> 172.21.15.70:0/445854091 conn(0x7f181c0fb190 msgr2=0x7f181c068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 shutdown_connections 2024-04-02T04:50:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:21.519+0000 7f1822528700 1 -- 172.21.15.70:0/445854091 wait complete. 2024-04-02T04:50:21.935 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:21 smithi070 ceph-mon[29212]: pgmap v52: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:21.935 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:21 smithi070 ceph-mon[29212]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:21.935 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:21 smithi070 ceph-mon[29212]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:21.935 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:21 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1238414542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:21 smithi179 ceph-mon[36603]: pgmap v52: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:21 smithi179 ceph-mon[36603]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:21 smithi179 ceph-mon[36603]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:21 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1238414542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:22.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:22 smithi179 ceph-mon[36603]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:22.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:22 smithi179 ceph-mon[36603]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:22.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:22 smithi070 ceph-mon[29212]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:22.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:22 smithi070 ceph-mon[29212]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:23.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:23 smithi070 ceph-mon[29212]: pgmap v53: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:24.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:23 smithi179 ceph-mon[36603]: pgmap v53: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:25.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:25 smithi070 ceph-mon[29212]: pgmap v54: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:25.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:25 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:25.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:25 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:25.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:25 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:26.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:25 smithi179 ceph-mon[36603]: pgmap v54: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:50:26.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:25 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:26.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:25 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:26.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:25 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:27.914 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:27 smithi070 ceph-mon[29212]: pgmap v55: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:27.914 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:27 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:27 smithi179 ceph-mon[36603]: pgmap v55: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:27 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:30.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:29 smithi070 ceph-mon[29212]: pgmap v56: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:30.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:29 smithi179 ceph-mon[36603]: pgmap v56: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:31.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:30 smithi179 ceph-mon[36603]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:31.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:30 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:31.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:30 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:31.073 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:30 smithi070 ceph-mon[29212]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:31.073 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:30 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:31.073 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:30 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:32.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:31 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:32.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:31 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:32.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:31 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:32.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:31 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:32.988 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:32 smithi179 ceph-mon[36603]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:33.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:32 smithi070 ceph-mon[29212]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:35.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:35.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:35 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:36.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:35 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:37 smithi179 ceph-mon[36603]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:37 smithi179 ceph-mon[36603]: Upgrade: Updating grafana.smithi070 2024-04-02T04:50:37.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:37 smithi179 ceph-mon[36603]: Deploying daemon grafana.smithi070 on smithi070 2024-04-02T04:50:38.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:37 smithi070 ceph-mon[29212]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:38.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:37 smithi070 ceph-mon[29212]: Upgrade: Updating grafana.smithi070 2024-04-02T04:50:38.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:37 smithi070 ceph-mon[29212]: Deploying daemon grafana.smithi070 on smithi070 2024-04-02T04:50:39.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:39 smithi179 ceph-mon[36603]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:40.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:39 smithi070 ceph-mon[29212]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:41.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:41 smithi179 ceph-mon[36603]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:42.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:41 smithi070 ceph-mon[29212]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:42.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:42 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:43.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:42 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:43.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:43 smithi179 ceph-mon[36603]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:44.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:43 smithi070 ceph-mon[29212]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:45.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:45 smithi070 ceph-mon[29212]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:45.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:45 smithi179 ceph-mon[36603]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:47.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:47 smithi179 ceph-mon[36603]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:47.885 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:47 smithi070 ceph-mon[29212]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:49 smithi179 ceph-mon[36603]: pgmap v66: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:49 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:49 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:49.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:49 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:50.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:49 smithi070 ceph-mon[29212]: pgmap v66: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:50.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:49 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:50.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:49 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:50.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:49 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.631+0000 7f1772137700 1 -- 172.21.15.70:0/1000288570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 msgr2=0x7f176c0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.631+0000 7f1772137700 1 --2- 172.21.15.70:0/1000288570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 0x7f176c0fffd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f175c009910 tx=0x7f175c009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.631+0000 7f1772137700 1 -- 172.21.15.70:0/1000288570 shutdown_connections 2024-04-02T04:50:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.631+0000 7f1772137700 1 --2- 172.21.15.70:0/1000288570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f176c100db0 0x7f176c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.631+0000 7f1772137700 1 --2- 172.21.15.70:0/1000288570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 0x7f176c0fffd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.631+0000 7f1772137700 1 -- 172.21.15.70:0/1000288570 >> 172.21.15.70:0/1000288570 conn(0x7f176c0fb150 msgr2=0x7f176c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.631+0000 7f1772137700 1 -- 172.21.15.70:0/1000288570 shutdown_connections 2024-04-02T04:50:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.631+0000 7f1772137700 1 -- 172.21.15.70:0/1000288570 wait complete. 2024-04-02T04:50:51.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.632+0000 7f1772137700 1 Processor -- start 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.633+0000 7f1772137700 1 -- start start 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.633+0000 7f1772137700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 0x7f176c1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.633+0000 7f1772137700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f176c100db0 0x7f176c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.633+0000 7f1772137700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c197040 con 0x7f176c0ffbb0 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.633+0000 7f1772137700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c197180 con 0x7f176c100db0 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.633+0000 7f176b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 0x7f176c1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.633+0000 7f176b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 0x7f176c1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45058/0 (socket says 172.21.15.70:45058) 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.633+0000 7f176b7fe700 1 -- 172.21.15.70:0/3425511152 learned_addr learned my addr 172.21.15.70:0/3425511152 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f176affd700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f176c100db0 0x7f176c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f176b7fe700 1 -- 172.21.15.70:0/3425511152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f176c100db0 msgr2=0x7f176c196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f176b7fe700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f176c100db0 0x7f176c196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f176b7fe700 1 -- 172.21.15.70:0/3425511152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0095f0 con 0x7f176c0ffbb0 2024-04-02T04:50:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f176b7fe700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 0x7f176c1964e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f175c00bda0 tx=0x7f175c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f1768ff9700 1 -- 172.21.15.70:0/3425511152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175c01d070 con 0x7f176c0ffbb0 2024-04-02T04:50:51.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c19bbd0 con 0x7f176c0ffbb0 2024-04-02T04:50:51.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f1768ff9700 1 -- 172.21.15.70:0/3425511152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f175c022470 con 0x7f176c0ffbb0 2024-04-02T04:50:51.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f1768ff9700 1 -- 172.21.15.70:0/3425511152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175c00f650 con 0x7f176c0ffbb0 2024-04-02T04:50:51.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.634+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c19c0c0 con 0x7f176c0ffbb0 2024-04-02T04:50:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.636+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c190530 con 0x7f176c0ffbb0 2024-04-02T04:50:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.636+0000 7f1768ff9700 1 -- 172.21.15.70:0/3425511152 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f175c004af0 con 0x7f176c0ffbb0 2024-04-02T04:50:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.637+0000 7f1768ff9700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f175406da30 0x7f175406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.637+0000 7f1768ff9700 1 -- 172.21.15.70:0/3425511152 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f175c08eb70 con 0x7f176c0ffbb0 2024-04-02T04:50:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.637+0000 7f176affd700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f175406da30 0x7f175406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.637+0000 7f176affd700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f175406da30 0x7f175406fef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1760005950 tx=0x7f176000a300 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.642+0000 7f1768ff9700 1 -- 172.21.15.70:0/3425511152 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f175c05c1f0 con 0x7f176c0ffbb0 2024-04-02T04:50:51.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:51 smithi179 ceph-mon[36603]: pgmap v67: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:51.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.876+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f176c0611d0 con 0x7f175406da30 2024-04-02T04:50:51.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:51 smithi070 ceph-mon[29212]: pgmap v67: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:51.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.877+0000 7f1768ff9700 1 -- 172.21.15.70:0/3425511152 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f176c0611d0 con 0x7f175406da30 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.879+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f175406da30 msgr2=0x7f175406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.879+0000 7f1772137700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f175406da30 0x7f175406fef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1760005950 tx=0x7f176000a300 comp rx=0 tx=0).stop 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.879+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 msgr2=0x7f176c1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.879+0000 7f1772137700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 0x7f176c1964e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f175c00bda0 tx=0x7f175c00bdd0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.880+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 shutdown_connections 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.880+0000 7f1772137700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f175406da30 0x7f175406fef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.880+0000 7f1772137700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f176c0ffbb0 0x7f176c1964e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.880+0000 7f1772137700 1 --2- 172.21.15.70:0/3425511152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f176c100db0 0x7f176c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.880+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 >> 172.21.15.70:0/3425511152 conn(0x7f176c0fb150 msgr2=0x7f176c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.880+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 shutdown_connections 2024-04-02T04:50:51.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.880+0000 7f1772137700 1 -- 172.21.15.70:0/3425511152 wait complete. 2024-04-02T04:50:51.894 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.983+0000 7f583dfc9700 1 -- 172.21.15.70:0/668458067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838101eb0 msgr2=0x7f5838102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.983+0000 7f583dfc9700 1 --2- 172.21.15.70:0/668458067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838101eb0 0x7f5838102310 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f582c009910 tx=0x7f582c009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.983+0000 7f583dfc9700 1 -- 172.21.15.70:0/668458067 shutdown_connections 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.983+0000 7f583dfc9700 1 --2- 172.21.15.70:0/668458067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838101eb0 0x7f5838102310 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.983+0000 7f583dfc9700 1 --2- 172.21.15.70:0/668458067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5838100cb0 0x7f58381010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.983+0000 7f583dfc9700 1 -- 172.21.15.70:0/668458067 >> 172.21.15.70:0/668458067 conn(0x7f58380fc230 msgr2=0x7f58380fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.983+0000 7f583dfc9700 1 -- 172.21.15.70:0/668458067 shutdown_connections 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.983+0000 7f583dfc9700 1 -- 172.21.15.70:0/668458067 wait complete. 2024-04-02T04:50:51.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f583dfc9700 1 Processor -- start 2024-04-02T04:50:51.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f583dfc9700 1 -- start start 2024-04-02T04:50:51.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f583dfc9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838100cb0 0x7f5838196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f583dfc9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5838101eb0 0x7f5838196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f583dfc9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58381970b0 con 0x7f5838100cb0 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f583dfc9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58381971f0 con 0x7f5838101eb0 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f58377fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838100cb0 0x7f5838196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f58377fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838100cb0 0x7f5838196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45074/0 (socket says 172.21.15.70:45074) 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f58377fe700 1 -- 172.21.15.70:0/2768012387 learned_addr learned my addr 172.21.15.70:0/2768012387 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f58377fe700 1 -- 172.21.15.70:0/2768012387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5838101eb0 msgr2=0x7f5838196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f5836ffd700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5838101eb0 0x7f5838196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f58377fe700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5838101eb0 0x7f5838196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.984+0000 7f58377fe700 1 -- 172.21.15.70:0/2768012387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0095f0 con 0x7f5838100cb0 2024-04-02T04:50:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.985+0000 7f58377fe700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838100cb0 0x7f5838196550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f582800b480 tx=0x7f582800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.985+0000 7f5834ff9700 1 -- 172.21.15.70:0/2768012387 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58280118c0 con 0x7f5838100cb0 2024-04-02T04:50:51.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.985+0000 7f5834ff9700 1 -- 172.21.15.70:0/2768012387 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5828010380 con 0x7f5838100cb0 2024-04-02T04:50:51.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.985+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583819bca0 con 0x7f5838100cb0 2024-04-02T04:50:51.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.985+0000 7f5834ff9700 1 -- 172.21.15.70:0/2768012387 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582800f4f0 con 0x7f5838100cb0 2024-04-02T04:50:51.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.985+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583819c1a0 con 0x7f5838100cb0 2024-04-02T04:50:51.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.986+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58381098b0 con 0x7f5838100cb0 2024-04-02T04:50:51.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.986+0000 7f5834ff9700 1 -- 172.21.15.70:0/2768012387 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f58280104f0 con 0x7f5838100cb0 2024-04-02T04:50:51.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.987+0000 7f5834ff9700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f582006d9e0 0x7f582006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.987+0000 7f5834ff9700 1 -- 172.21.15.70:0/2768012387 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f582808d160 con 0x7f5838100cb0 2024-04-02T04:50:51.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.987+0000 7f5836ffd700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f582006d9e0 0x7f582006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.987+0000 7f5836ffd700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f582006d9e0 0x7f582006fea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f582c00b5c0 tx=0x7f582c005960 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:51.991+0000 7f5834ff9700 1 -- 172.21.15.70:0/2768012387 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f582805a7d0 con 0x7f5838100cb0 2024-04-02T04:50:52.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.219+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5838106800 con 0x7f582006d9e0 2024-04-02T04:50:52.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.220+0000 7f5834ff9700 1 -- 172.21.15.70:0/2768012387 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f5838106800 con 0x7f582006d9e0 2024-04-02T04:50:52.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f582006d9e0 msgr2=0x7f582006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f582006d9e0 0x7f582006fea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f582c00b5c0 tx=0x7f582c005960 comp rx=0 tx=0).stop 2024-04-02T04:50:52.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838100cb0 msgr2=0x7f5838196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838100cb0 0x7f5838196550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f582800b480 tx=0x7f582800b840 comp rx=0 tx=0).stop 2024-04-02T04:50:52.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 shutdown_connections 2024-04-02T04:50:52.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f582006d9e0 0x7f582006fea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5838100cb0 0x7f5838196550 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 --2- 172.21.15.70:0/2768012387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5838101eb0 0x7f5838196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 >> 172.21.15.70:0/2768012387 conn(0x7f58380fc230 msgr2=0x7f58381050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:52.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 shutdown_connections 2024-04-02T04:50:52.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.222+0000 7f583dfc9700 1 -- 172.21.15.70:0/2768012387 wait complete. 2024-04-02T04:50:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.327+0000 7f7150cc1700 1 -- 172.21.15.70:0/2792790873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c0ffe30 msgr2=0x7f714c1002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.327+0000 7f7150cc1700 1 --2- 172.21.15.70:0/2792790873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c0ffe30 0x7f714c1002b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7140009910 tx=0x7f7140009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.327+0000 7f7150cc1700 1 -- 172.21.15.70:0/2792790873 shutdown_connections 2024-04-02T04:50:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.327+0000 7f7150cc1700 1 --2- 172.21.15.70:0/2792790873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c0ffe30 0x7f714c1002b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.327+0000 7f7150cc1700 1 --2- 172.21.15.70:0/2792790873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f714c1011d0 0x7f714c1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.327+0000 7f7150cc1700 1 -- 172.21.15.70:0/2792790873 >> 172.21.15.70:0/2792790873 conn(0x7f714c0fa8c0 msgr2=0x7f714c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.327+0000 7f7150cc1700 1 -- 172.21.15.70:0/2792790873 shutdown_connections 2024-04-02T04:50:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.327+0000 7f7150cc1700 1 -- 172.21.15.70:0/2792790873 wait complete. 2024-04-02T04:50:52.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7150cc1700 1 Processor -- start 2024-04-02T04:50:52.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7150cc1700 1 -- start start 2024-04-02T04:50:52.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7150cc1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f714c0ffe30 0x7f714c0ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:52.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7150cc1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c1011d0 0x7f714c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7150cc1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c19b840 con 0x7f714c1011d0 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7150cc1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c19b9b0 con 0x7f714c0ffe30 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7149d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c1011d0 0x7f714c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7149d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c1011d0 0x7f714c19b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45094/0 (socket says 172.21.15.70:45094) 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.328+0000 7f7149d9b700 1 -- 172.21.15.70:0/3336877221 learned_addr learned my addr 172.21.15.70:0/3336877221 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f714a59c700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f714c0ffe30 0x7f714c0ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f7149d9b700 1 -- 172.21.15.70:0/3336877221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f714c0ffe30 msgr2=0x7f714c0ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f7149d9b700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f714c0ffe30 0x7f714c0ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f7149d9b700 1 -- 172.21.15.70:0/3336877221 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71400095f0 con 0x7f714c1011d0 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f7149d9b700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c1011d0 0x7f714c19b270 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7140006010 tx=0x7f714000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f713b7fe700 1 -- 172.21.15.70:0/3336877221 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714001d070 con 0x7f714c1011d0 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f713b7fe700 1 -- 172.21.15.70:0/3336877221 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f714000f460 con 0x7f714c1011d0 2024-04-02T04:50:52.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714c19bc30 con 0x7f714c1011d0 2024-04-02T04:50:52.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f713b7fe700 1 -- 172.21.15.70:0/3336877221 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7140021640 con 0x7f714c1011d0 2024-04-02T04:50:52.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.329+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714c19c120 con 0x7f714c1011d0 2024-04-02T04:50:52.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.330+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714c108730 con 0x7f714c1011d0 2024-04-02T04:50:52.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.331+0000 7f713b7fe700 1 -- 172.21.15.70:0/3336877221 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f714000faf0 con 0x7f714c1011d0 2024-04-02T04:50:52.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.331+0000 7f713b7fe700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f713406d9e0 0x7f713406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:52.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.331+0000 7f713b7fe700 1 -- 172.21.15.70:0/3336877221 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f714008e7e0 con 0x7f714c1011d0 2024-04-02T04:50:52.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.331+0000 7f714a59c700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f713406d9e0 0x7f713406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:52.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.331+0000 7f714a59c700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f713406d9e0 0x7f713406fea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f713c005fd0 tx=0x7f713c005dd0 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:52.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.335+0000 7f713b7fe700 1 -- 172.21.15.70:0/3336877221 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f714005be50 con 0x7f714c1011d0 2024-04-02T04:50:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.555+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f714c105790 con 0x7f713406d9e0 2024-04-02T04:50:52.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.566+0000 7f713b7fe700 1 -- 172.21.15.70:0/3336877221 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f714c105790 con 0x7f713406d9e0 2024-04-02T04:50:52.570 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:50:52.570 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (28s) 22s ago 11m 16.9M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:50:52.570 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (11m) 22s ago 11m 7931k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:50:52.570 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (10m) 61s ago 10m 7545k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:50:52.570 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (11m) 22s ago 11m 6669k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:50:52.570 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (10m) 61s ago 10m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 starting - - - - 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (5m) 22s ago 5m 154M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (5m) 22s ago 5m 17.6M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (5m) 61s ago 5m 15.9M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (5m) 61s ago 5m 17.5M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (2m) 22s ago 12m 499M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (96s) 61s ago 10m 448M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (12m) 22s ago 13m 55.8M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (10m) 61s ago 10m 48.4M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (68s) 22s ago 11m 7856k - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (64s) 61s ago 10m 3439k - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (9m) 22s ago 9m 227M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (9m) 22s ago 9m 263M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:50:52.571 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (8m) 22s ago 8m 210M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:50:52.572 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (8m) 61s ago 8m 296M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:50:52.572 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (7m) 61s ago 7m 272M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:50:52.572 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (7m) 61s ago 7m 226M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:50:52.572 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (46s) 22s ago 11m 49.1M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f713406d9e0 msgr2=0x7f713406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f713406d9e0 0x7f713406fea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f713c005fd0 tx=0x7f713c005dd0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c1011d0 msgr2=0x7f714c19b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c1011d0 0x7f714c19b270 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7140006010 tx=0x7f714000be20 comp rx=0 tx=0).stop 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 shutdown_connections 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f713406d9e0 0x7f713406fea0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f714c0ffe30 0x7f714c0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 --2- 172.21.15.70:0/3336877221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c1011d0 0x7f714c19b270 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 >> 172.21.15.70:0/3336877221 conn(0x7f714c0fa8c0 msgr2=0x7f714c104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 shutdown_connections 2024-04-02T04:50:52.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.568+0000 7f7150cc1700 1 -- 172.21.15.70:0/3336877221 wait complete. 2024-04-02T04:50:52.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.669+0000 7fc2e38a7700 1 -- 172.21.15.70:0/1978260766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc101fe0 msgr2=0x7fc2dc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.669+0000 7fc2e38a7700 1 --2- 172.21.15.70:0/1978260766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc101fe0 0x7fc2dc102440 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009910 tx=0x7fc2cc009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:52.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.669+0000 7fc2e38a7700 1 -- 172.21.15.70:0/1978260766 shutdown_connections 2024-04-02T04:50:52.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.669+0000 7fc2e38a7700 1 --2- 172.21.15.70:0/1978260766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc101fe0 0x7fc2dc102440 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.669+0000 7fc2e38a7700 1 --2- 172.21.15.70:0/1978260766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2dc0fef60 0x7fc2dc0ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.669+0000 7fc2e38a7700 1 -- 172.21.15.70:0/1978260766 >> 172.21.15.70:0/1978260766 conn(0x7fc2dc0fa8c0 msgr2=0x7fc2dc0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:52.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e38a7700 1 -- 172.21.15.70:0/1978260766 shutdown_connections 2024-04-02T04:50:52.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e38a7700 1 -- 172.21.15.70:0/1978260766 wait complete. 2024-04-02T04:50:52.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e38a7700 1 Processor -- start 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e38a7700 1 -- start start 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e38a7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc0fef60 0x7fc2dc101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e38a7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2dc101fe0 0x7fc2dc0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e38a7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc1004d0 con 0x7fc2dc0fef60 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e38a7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc100610 con 0x7fc2dc101fe0 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e1643700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc0fef60 0x7fc2dc101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e1643700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc0fef60 0x7fc2dc101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45112/0 (socket says 172.21.15.70:45112) 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.670+0000 7fc2e1643700 1 -- 172.21.15.70:0/3702176805 learned_addr learned my addr 172.21.15.70:0/3702176805 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2e0e42700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2dc101fe0 0x7fc2dc0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2e1643700 1 -- 172.21.15.70:0/3702176805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2dc101fe0 msgr2=0x7fc2dc0fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2e1643700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2dc101fe0 0x7fc2dc0fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2e1643700 1 -- 172.21.15.70:0/3702176805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc0095f0 con 0x7fc2dc0fef60 2024-04-02T04:50:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2e0e42700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2dc101fe0 0x7fc2dc0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:50:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2e1643700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc0fef60 0x7fc2dc101940 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800d900 tx=0x7fc2d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2d27fc700 1 -- 172.21.15.70:0/3702176805 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2d80041f0 con 0x7fc2dc0fef60 2024-04-02T04:50:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2d27fc700 1 -- 172.21.15.70:0/3702176805 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc2d8004350 con 0x7fc2dc0fef60 2024-04-02T04:50:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc1008f0 con 0x7fc2dc0fef60 2024-04-02T04:50:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2d27fc700 1 -- 172.21.15.70:0/3702176805 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2d8003da0 con 0x7fc2dc0fef60 2024-04-02T04:50:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.671+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc19e530 con 0x7fc2dc0fef60 2024-04-02T04:50:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.672+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc109800 con 0x7fc2dc0fef60 2024-04-02T04:50:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.673+0000 7fc2d27fc700 1 -- 172.21.15.70:0/3702176805 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7fc2d8009730 con 0x7fc2dc0fef60 2024-04-02T04:50:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.673+0000 7fc2d27fc700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc2c806d9e0 0x7fc2c806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.673+0000 7fc2d27fc700 1 -- 172.21.15.70:0/3702176805 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fc2d808d6c0 con 0x7fc2dc0fef60 2024-04-02T04:50:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.673+0000 7fc2e0e42700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc2c806d9e0 0x7fc2c806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.674+0000 7fc2e0e42700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc2c806d9e0 0x7fc2c806fea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc005b40 tx=0x7fc2cc005a90 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:52.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.678+0000 7fc2d27fc700 1 -- 172.21.15.70:0/3702176805 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fc2d805acb0 con 0x7fc2dc0fef60 2024-04-02T04:50:52.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.968+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc2dc19eae0 con 0x7fc2dc0fef60 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.969+0000 7fc2d27fc700 1 -- 172.21.15.70:0/3702176805 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fc2d805a840 con 0x7fc2dc0fef60 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:50:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:50:52.974 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:50:52.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc2c806d9e0 msgr2=0x7fc2c806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc2c806d9e0 0x7fc2c806fea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc005b40 tx=0x7fc2cc005a90 comp rx=0 tx=0).stop 2024-04-02T04:50:52.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc0fef60 msgr2=0x7fc2dc101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc0fef60 0x7fc2dc101940 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800d900 tx=0x7fc2d800dcc0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 shutdown_connections 2024-04-02T04:50:52.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fc2c806d9e0 0x7fc2c806fea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2dc0fef60 0x7fc2dc101940 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 --2- 172.21.15.70:0/3702176805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2dc101fe0 0x7fc2dc0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 >> 172.21.15.70:0/3702176805 conn(0x7fc2dc0fa8c0 msgr2=0x7fc2dc105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:52.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 shutdown_connections 2024-04-02T04:50:52.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:52.971+0000 7fc2e38a7700 1 -- 172.21.15.70:0/3702176805 wait complete. 2024-04-02T04:50:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.075+0000 7f8744967700 1 -- 172.21.15.70:0/2388986528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 msgr2=0x7f87401010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.075+0000 7f8744967700 1 --2- 172.21.15.70:0/2388986528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 0x7f87401010d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8734009910 tx=0x7f8734009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.075+0000 7f8744967700 1 -- 172.21.15.70:0/2388986528 shutdown_connections 2024-04-02T04:50:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.075+0000 7f8744967700 1 --2- 172.21.15.70:0/2388986528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8740101eb0 0x7f8740102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.075+0000 7f8744967700 1 --2- 172.21.15.70:0/2388986528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 0x7f87401010d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.075+0000 7f8744967700 1 -- 172.21.15.70:0/2388986528 >> 172.21.15.70:0/2388986528 conn(0x7f87400fc230 msgr2=0x7f87400fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.075+0000 7f8744967700 1 -- 172.21.15.70:0/2388986528 shutdown_connections 2024-04-02T04:50:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.075+0000 7f8744967700 1 -- 172.21.15.70:0/2388986528 wait complete. 2024-04-02T04:50:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f8744967700 1 Processor -- start 2024-04-02T04:50:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f8744967700 1 -- start start 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f8744967700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 0x7f8740196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f8744967700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8740101eb0 0x7f8740196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f8744967700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401970c0 con 0x7f8740100cb0 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f8744967700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8740197200 con 0x7f8740101eb0 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f873e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 0x7f8740196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f873e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 0x7f8740196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45126/0 (socket says 172.21.15.70:45126) 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.076+0000 7f873e59c700 1 -- 172.21.15.70:0/1402265050 learned_addr learned my addr 172.21.15.70:0/1402265050 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f873dd9b700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8740101eb0 0x7f8740196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f873e59c700 1 -- 172.21.15.70:0/1402265050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8740101eb0 msgr2=0x7f8740196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f873e59c700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8740101eb0 0x7f8740196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f873e59c700 1 -- 172.21.15.70:0/1402265050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87340095f0 con 0x7f8740100cb0 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f873e59c700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 0x7f8740196560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f873400bda0 tx=0x7f873400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f872f7fe700 1 -- 172.21.15.70:0/1402265050 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873401d070 con 0x7f8740100cb0 2024-04-02T04:50:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874019bc50 con 0x7f8740100cb0 2024-04-02T04:50:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f872f7fe700 1 -- 172.21.15.70:0/1402265050 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8734004c50 con 0x7f8740100cb0 2024-04-02T04:50:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f872f7fe700 1 -- 172.21.15.70:0/1402265050 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873400f690 con 0x7f8740100cb0 2024-04-02T04:50:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.077+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874019c140 con 0x7f8740100cb0 2024-04-02T04:50:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.078+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8740109890 con 0x7f8740100cb0 2024-04-02T04:50:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.079+0000 7f872f7fe700 1 -- 172.21.15.70:0/1402265050 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f87340229b0 con 0x7f8740100cb0 2024-04-02T04:50:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.079+0000 7f872f7fe700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f872806d9e0 0x7f872806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.080+0000 7f872f7fe700 1 -- 172.21.15.70:0/1402265050 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f873408eca0 con 0x7f8740100cb0 2024-04-02T04:50:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.080+0000 7f873dd9b700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f872806d9e0 0x7f872806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.080+0000 7f873dd9b700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f872806d9e0 0x7f872806fea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8730006fd0 tx=0x7f8730008040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.084+0000 7f872f7fe700 1 -- 172.21.15.70:0/1402265050 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f8734093070 con 0x7f8740100cb0 2024-04-02T04:50:53.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.306+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8740106800 con 0x7f872806d9e0 2024-04-02T04:50:53.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.307+0000 7f872f7fe700 1 -- 172.21.15.70:0/1402265050 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f8740106800 con 0x7f872806d9e0 2024-04-02T04:50:53.311 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:50:53.311 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/2 daemons upgraded", 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading grafana daemons", 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:50:53.312 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:50:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f872806d9e0 msgr2=0x7f872806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f872806d9e0 0x7f872806fea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8730006fd0 tx=0x7f8730008040 comp rx=0 tx=0).stop 2024-04-02T04:50:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 msgr2=0x7f8740196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 0x7f8740196560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f873400bda0 tx=0x7f873400be80 comp rx=0 tx=0).stop 2024-04-02T04:50:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 shutdown_connections 2024-04-02T04:50:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f872806d9e0 0x7f872806fea0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740100cb0 0x7f8740196560 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 --2- 172.21.15.70:0/1402265050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8740101eb0 0x7f8740196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 >> 172.21.15.70:0/1402265050 conn(0x7f87400fc230 msgr2=0x7f87401050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 shutdown_connections 2024-04-02T04:50:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:50:53.310+0000 7f8744967700 1 -- 172.21.15.70:0/1402265050 wait complete. 2024-04-02T04:50:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:53 smithi179 ceph-mon[36603]: pgmap v68: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:53 smithi179 ceph-mon[36603]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:53 smithi179 ceph-mon[36603]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:53 smithi179 ceph-mon[36603]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:53.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:53 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3702176805' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:54.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:53 smithi070 ceph-mon[29212]: pgmap v68: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:54.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:53 smithi070 ceph-mon[29212]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:54.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:53 smithi070 ceph-mon[29212]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:54.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:53 smithi070 ceph-mon[29212]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:54.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:53 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3702176805' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:54.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:54 smithi070 ceph-mon[29212]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:54.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:54 smithi179 ceph-mon[36603]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:55.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:55 smithi179 ceph-mon[36603]: pgmap v69: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:55.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:55 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:55.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:55 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:56.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:55 smithi070 ceph-mon[29212]: pgmap v69: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:56.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:55 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:56.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:55 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:57 smithi070 ceph-mon[29212]: pgmap v70: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:57 smithi179 ceph-mon[36603]: pgmap v70: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:50:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:59.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:50:59 smithi070 ceph-mon[29212]: pgmap v71: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:59.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:50:59 smithi179 ceph-mon[36603]: pgmap v71: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:00.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:00.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: Upgrade: Finalizing container_image settings 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-02T04:51:00.824 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-02T04:51:00.825 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:00.826 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:00.827 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:00.827 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:00.827 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:00 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:01.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: Upgrade: Finalizing container_image settings 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-02T04:51:01.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-02T04:51:01.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-02T04:51:01.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:01.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:01.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:01.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:01.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:01.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:01.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:01.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:00 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:01.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:01 smithi179 ceph-mon[36603]: Upgrade: Complete! 2024-04-02T04:51:01.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:01 smithi179 ceph-mon[36603]: pgmap v72: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:02.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:01 smithi070 ceph-mon[29212]: Upgrade: Complete! 2024-04-02T04:51:02.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:01 smithi070 ceph-mon[29212]: pgmap v72: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:02 smithi179 ceph-mon[36603]: pgmap v73: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:03.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:02 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:03.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:02 smithi070 ceph-mon[29212]: pgmap v73: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:02 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:04 smithi179 ceph-mon[36603]: pgmap v74: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:05.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:04 smithi070 ceph-mon[29212]: pgmap v74: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:07.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:06 smithi070 ceph-mon[29212]: pgmap v75: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:07.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:06 smithi179 ceph-mon[36603]: pgmap v75: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:09.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:08 smithi070 ceph-mon[29212]: pgmap v76: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:09.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:08 smithi179 ceph-mon[36603]: pgmap v76: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:11.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:10 smithi070 ceph-mon[29212]: pgmap v77: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:11.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:10 smithi179 ceph-mon[36603]: pgmap v77: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:12.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:11 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:12.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:11 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:13.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:12 smithi070 ceph-mon[29212]: pgmap v78: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:13.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:12 smithi179 ceph-mon[36603]: pgmap v78: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:15.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:14 smithi070 ceph-mon[29212]: pgmap v79: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:15.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:14 smithi179 ceph-mon[36603]: pgmap v79: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:17.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:16 smithi070 ceph-mon[29212]: pgmap v80: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:17.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:16 smithi179 ceph-mon[36603]: pgmap v80: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:19.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:18 smithi070 ceph-mon[29212]: pgmap v81: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:19.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:18 smithi179 ceph-mon[36603]: pgmap v81: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:21.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:20 smithi070 ceph-mon[29212]: pgmap v82: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:21.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:20 smithi179 ceph-mon[36603]: pgmap v82: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:23.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:22 smithi070 ceph-mon[29212]: pgmap v83: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:23.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:22 smithi179 ceph-mon[36603]: pgmap v83: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.417+0000 7f8365ab8700 1 -- 172.21.15.70:0/2523696181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360104070 msgr2=0x7f8360106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.417+0000 7f8365ab8700 1 --2- 172.21.15.70:0/2523696181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360104070 0x7f8360106460 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8348009910 tx=0x7f8348009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.417+0000 7f8365ab8700 1 -- 172.21.15.70:0/2523696181 shutdown_connections 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.417+0000 7f8365ab8700 1 --2- 172.21.15.70:0/2523696181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360104070 0x7f8360106460 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.417+0000 7f8365ab8700 1 --2- 172.21.15.70:0/2523696181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8360101740 0x7f8360103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.417+0000 7f8365ab8700 1 -- 172.21.15.70:0/2523696181 >> 172.21.15.70:0/2523696181 conn(0x7f83600fb150 msgr2=0x7f83600fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.417+0000 7f8365ab8700 1 -- 172.21.15.70:0/2523696181 shutdown_connections 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.417+0000 7f8365ab8700 1 -- 172.21.15.70:0/2523696181 wait complete. 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f8365ab8700 1 Processor -- start 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f8365ab8700 1 -- start start 2024-04-02T04:51:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f8365ab8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360101740 0x7f83601964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f8365ab8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8360104070 0x7f83601969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f8365ab8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8360197010 con 0x7f8360101740 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f8365ab8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8360197150 con 0x7f8360104070 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f835f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360101740 0x7f83601964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f835f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360101740 0x7f83601964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58908/0 (socket says 172.21.15.70:58908) 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.418+0000 7f835f7fe700 1 -- 172.21.15.70:0/1480993048 learned_addr learned my addr 172.21.15.70:0/1480993048 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f835f7fe700 1 -- 172.21.15.70:0/1480993048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8360104070 msgr2=0x7f83601969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f835f7fe700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8360104070 0x7f83601969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f835f7fe700 1 -- 172.21.15.70:0/1480993048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83480095f0 con 0x7f8360101740 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f835f7fe700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360101740 0x7f83601964b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f835000d900 tx=0x7f835000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f835d7fa700 1 -- 172.21.15.70:0/1480993048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8350004a00 con 0x7f8360101740 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836019bc00 con 0x7f8360101740 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f835d7fa700 1 -- 172.21.15.70:0/1480993048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83500054e0 con 0x7f8360101740 2024-04-02T04:51:23.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f835d7fa700 1 -- 172.21.15.70:0/1480993048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8350009d60 con 0x7f8360101740 2024-04-02T04:51:23.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.419+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836019c150 con 0x7f8360101740 2024-04-02T04:51:23.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.420+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8360190640 con 0x7f8360101740 2024-04-02T04:51:23.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.420+0000 7f835d7fa700 1 -- 172.21.15.70:0/1480993048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f8350005020 con 0x7f8360101740 2024-04-02T04:51:23.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.421+0000 7f835d7fa700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f834c06d9e0 0x7f834c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.421+0000 7f835d7fa700 1 -- 172.21.15.70:0/1480993048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f835008d780 con 0x7f8360101740 2024-04-02T04:51:23.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.421+0000 7f8357fff700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f834c06d9e0 0x7f834c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:23.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.421+0000 7f8357fff700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f834c06d9e0 0x7f834c06fea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f834800b5c0 tx=0x7f8348005960 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:23.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.426+0000 7f835d7fa700 1 -- 172.21.15.70:0/1480993048 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f835005ad70 con 0x7f8360101740 2024-04-02T04:51:23.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.642+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f83600611d0 con 0x7f834c06d9e0 2024-04-02T04:51:23.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.643+0000 7f835d7fa700 1 -- 172.21.15.70:0/1480993048 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f83600611d0 con 0x7f834c06d9e0 2024-04-02T04:51:23.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f834c06d9e0 msgr2=0x7f834c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f834c06d9e0 0x7f834c06fea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f834800b5c0 tx=0x7f8348005960 comp rx=0 tx=0).stop 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360101740 msgr2=0x7f83601964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360101740 0x7f83601964b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f835000d900 tx=0x7f835000dcc0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 shutdown_connections 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f834c06d9e0 0x7f834c06fea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8360101740 0x7f83601964b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 --2- 172.21.15.70:0/1480993048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8360104070 0x7f83601969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 >> 172.21.15.70:0/1480993048 conn(0x7f83600fb150 msgr2=0x7f83600fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 shutdown_connections 2024-04-02T04:51:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:23.646+0000 7f8365ab8700 1 -- 172.21.15.70:0/1480993048 wait complete. 2024-04-02T04:51:23.948 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:23 smithi070 ceph-mon[29212]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:24.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:23 smithi179 ceph-mon[36603]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:24.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-02T04:51:24.622 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:25.027 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:24 smithi070 ceph-mon[29212]: pgmap v84: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:25.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:24 smithi179 ceph-mon[36603]: pgmap v84: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.104+0000 7f9c58c96700 1 -- 172.21.15.70:0/2088093093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c54105bd0 msgr2=0x7f9c54105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.104+0000 7f9c58c96700 1 --2- 172.21.15.70:0/2088093093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c54105bd0 0x7f9c54105fb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009910 tx=0x7f9c48009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.104+0000 7f9c58c96700 1 -- 172.21.15.70:0/2088093093 shutdown_connections 2024-04-02T04:51:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.104+0000 7f9c58c96700 1 --2- 172.21.15.70:0/2088093093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c540ffc20 0x7f9c54100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.104+0000 7f9c58c96700 1 --2- 172.21.15.70:0/2088093093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c54105bd0 0x7f9c54105fb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.104+0000 7f9c58c96700 1 -- 172.21.15.70:0/2088093093 >> 172.21.15.70:0/2088093093 conn(0x7f9c54075670 msgr2=0x7f9c54075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.104+0000 7f9c58c96700 1 -- 172.21.15.70:0/2088093093 shutdown_connections 2024-04-02T04:51:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.104+0000 7f9c58c96700 1 -- 172.21.15.70:0/2088093093 wait complete. 2024-04-02T04:51:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c58c96700 1 Processor -- start 2024-04-02T04:51:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c58c96700 1 -- start start 2024-04-02T04:51:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c58c96700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540ffc20 0x7f9c54101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c58c96700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c54105bd0 0x7f9c54102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c58c96700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c541026d0 con 0x7f9c540ffc20 2024-04-02T04:51:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c58c96700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54102840 con 0x7f9c54105bd0 2024-04-02T04:51:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c5259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540ffc20 0x7f9c54101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c5259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540ffc20 0x7f9c54101c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58924/0 (socket says 172.21.15.70:58924) 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c5259c700 1 -- 172.21.15.70:0/3331836491 learned_addr learned my addr 172.21.15.70:0/3331836491 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.105+0000 7f9c51d9b700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c54105bd0 0x7f9c54102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c5259c700 1 -- 172.21.15.70:0/3331836491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c54105bd0 msgr2=0x7f9c54102190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c5259c700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c54105bd0 0x7f9c54102190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c5259c700 1 -- 172.21.15.70:0/3331836491 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c480095f0 con 0x7f9c540ffc20 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c51d9b700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c54105bd0 0x7f9c54102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c5259c700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540ffc20 0x7f9c54101c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c48006010 tx=0x7f9c4800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c437fe700 1 -- 172.21.15.70:0/3331836491 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c4801d070 con 0x7f9c540ffc20 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c437fe700 1 -- 172.21.15.70:0/3331836491 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c48004fb0 con 0x7f9c540ffc20 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c54102a60 con 0x7f9c540ffc20 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c437fe700 1 -- 172.21.15.70:0/3331836491 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c48022ea0 con 0x7f9c540ffc20 2024-04-02T04:51:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.106+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c541a2d50 con 0x7f9c540ffc20 2024-04-02T04:51:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.108+0000 7f9c437fe700 1 -- 172.21.15.70:0/3331836491 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f9c48022470 con 0x7f9c540ffc20 2024-04-02T04:51:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.108+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5404f2e0 con 0x7f9c540ffc20 2024-04-02T04:51:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.108+0000 7f9c437fe700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f9c3c06d9e0 0x7f9c3c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.108+0000 7f9c437fe700 1 -- 172.21.15.70:0/3331836491 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f9c4808ea60 con 0x7f9c540ffc20 2024-04-02T04:51:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.108+0000 7f9c51d9b700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f9c3c06d9e0 0x7f9c3c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.109+0000 7f9c51d9b700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f9c3c06d9e0 0x7f9c3c06fea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c44005fd0 tx=0x7f9c44005e20 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:26.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.113+0000 7f9c437fe700 1 -- 172.21.15.70:0/3331836491 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f9c48058650 con 0x7f9c540ffc20 2024-04-02T04:51:26.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.392+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9c54103180 con 0x7f9c540ffc20 2024-04-02T04:51:26.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.392+0000 7f9c437fe700 1 -- 172.21.15.70:0/3331836491 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f9c480272c0 con 0x7f9c540ffc20 2024-04-02T04:51:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f9c3c06d9e0 msgr2=0x7f9c3c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f9c3c06d9e0 0x7f9c3c06fea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c44005fd0 tx=0x7f9c44005e20 comp rx=0 tx=0).stop 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540ffc20 msgr2=0x7f9c54101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540ffc20 0x7f9c54101c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c48006010 tx=0x7f9c4800be20 comp rx=0 tx=0).stop 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 shutdown_connections 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f9c3c06d9e0 0x7f9c3c06fea0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540ffc20 0x7f9c54101c50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 --2- 172.21.15.70:0/3331836491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c54105bd0 0x7f9c54102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 >> 172.21.15.70:0/3331836491 conn(0x7f9c54075670 msgr2=0x7f9c54103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 shutdown_connections 2024-04-02T04:51:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:26.394+0000 7f9c58c96700 1 -- 172.21.15.70:0/3331836491 wait complete. 2024-04-02T04:51:26.409 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:51:27.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:26 smithi070 ceph-mon[29212]: pgmap v85: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:27.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:26 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/3331836491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:27.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:26 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:27.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-02T04:51:27.282 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:27.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:26 smithi179 ceph-mon[36603]: pgmap v85: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:27.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:26 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/3331836491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:27.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:26 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:29.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.077+0000 7f726ae9b700 1 -- 172.21.15.70:0/2502088528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 msgr2=0x7f72641058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:29.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.077+0000 7f726ae9b700 1 --2- 172.21.15.70:0/2502088528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 0x7f72641058a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7250009910 tx=0x7f7250009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:29.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.077+0000 7f726ae9b700 1 -- 172.21.15.70:0/2502088528 shutdown_connections 2024-04-02T04:51:29.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.077+0000 7f726ae9b700 1 --2- 172.21.15.70:0/2502088528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 0x7f72641058a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.077+0000 7f726ae9b700 1 --2- 172.21.15.70:0/2502088528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264100ea0 0x7f7264101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.077+0000 7f726ae9b700 1 -- 172.21.15.70:0/2502088528 >> 172.21.15.70:0/2502088528 conn(0x7f72640fc750 msgr2=0x7f72640feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.077+0000 7f726ae9b700 1 -- 172.21.15.70:0/2502088528 shutdown_connections 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.077+0000 7f726ae9b700 1 -- 172.21.15.70:0/2502088528 wait complete. 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f726ae9b700 1 Processor -- start 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f726ae9b700 1 -- start start 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f726ae9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264100ea0 0x7f726419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f726ae9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 0x7f726419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f726ae9b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726419d350 con 0x7f7264101850 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f726ae9b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7264196800 con 0x7f7264100ea0 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f7263fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 0x7f726419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f7263fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 0x7f726419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40036/0 (socket says 172.21.15.70:40036) 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f7263fff700 1 -- 172.21.15.70:0/2251159590 learned_addr learned my addr 172.21.15.70:0/2251159590 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f7263fff700 1 -- 172.21.15.70:0/2251159590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264100ea0 msgr2=0x7f726419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f7263fff700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264100ea0 0x7f726419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.078+0000 7f7263fff700 1 -- 172.21.15.70:0/2251159590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72500095f0 con 0x7f7264101850 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.079+0000 7f7263fff700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 0x7f726419ccc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f72500092e0 tx=0x7f725000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.079+0000 7f7261ffb700 1 -- 172.21.15.70:0/2251159590 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f725001d070 con 0x7f7264101850 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.079+0000 7f7261ffb700 1 -- 172.21.15.70:0/2251159590 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f725000f460 con 0x7f7264101850 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.079+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7264196a80 con 0x7f7264101850 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.079+0000 7f7261ffb700 1 -- 172.21.15.70:0/2251159590 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72500175f0 con 0x7f7264101850 2024-04-02T04:51:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.079+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7264196f70 con 0x7f7264101850 2024-04-02T04:51:29.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.080+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72641091f0 con 0x7f7264101850 2024-04-02T04:51:29.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.080+0000 7f7261ffb700 1 -- 172.21.15.70:0/2251159590 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f725000f5d0 con 0x7f7264101850 2024-04-02T04:51:29.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.081+0000 7f7261ffb700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f725406d9e0 0x7f725406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:29.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.081+0000 7f7261ffb700 1 -- 172.21.15.70:0/2251159590 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f725008eae0 con 0x7f7264101850 2024-04-02T04:51:29.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.081+0000 7f7268c37700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f725406d9e0 0x7f725406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:29.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.081+0000 7f7268c37700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f725406d9e0 0x7f725406fea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f72580097b0 tx=0x7f7258006d20 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:29.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.085+0000 7f7261ffb700 1 -- 172.21.15.70:0/2251159590 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f725005c0d0 con 0x7f7264101850 2024-04-02T04:51:29.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:28 smithi070 ceph-mon[29212]: pgmap v86: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:29.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:28 smithi179 ceph-mon[36603]: pgmap v86: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:29.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.363+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7264108bd0 con 0x7f7264101850 2024-04-02T04:51:29.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.363+0000 7f7261ffb700 1 -- 172.21.15.70:0/2251159590 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f725005bc60 con 0x7f7264101850 2024-04-02T04:51:29.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f725406d9e0 msgr2=0x7f725406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f725406d9e0 0x7f725406fea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f72580097b0 tx=0x7f7258006d20 comp rx=0 tx=0).stop 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 msgr2=0x7f726419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 0x7f726419ccc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f72500092e0 tx=0x7f725000be20 comp rx=0 tx=0).stop 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 shutdown_connections 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f725406d9e0 0x7f725406fea0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264100ea0 0x7f726419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 --2- 172.21.15.70:0/2251159590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7264101850 0x7f726419ccc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 >> 172.21.15.70:0/2251159590 conn(0x7f72640fc750 msgr2=0x7f72640fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 shutdown_connections 2024-04-02T04:51:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:29.365+0000 7f726ae9b700 1 -- 172.21.15.70:0/2251159590 wait complete. 2024-04-02T04:51:29.380 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)" 2024-04-02T04:51:29.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-02T04:51:30.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:29 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/2251159590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:30.173 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:30.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:29 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/2251159590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:31.222 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:30 smithi179 ceph-mon[36603]: pgmap v87: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:31.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:30 smithi070 ceph-mon[29212]: pgmap v87: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.799+0000 7ff989331700 1 -- 172.21.15.70:0/4119995940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9840ffc20 msgr2=0x7ff984100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.799+0000 7ff989331700 1 --2- 172.21.15.70:0/4119995940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9840ffc20 0x7ff984100080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff978009910 tx=0x7ff978009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.799+0000 7ff989331700 1 -- 172.21.15.70:0/4119995940 shutdown_connections 2024-04-02T04:51:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.799+0000 7ff989331700 1 --2- 172.21.15.70:0/4119995940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9840ffc20 0x7ff984100080 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.799+0000 7ff989331700 1 --2- 172.21.15.70:0/4119995940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff984105bd0 0x7ff984105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.799+0000 7ff989331700 1 -- 172.21.15.70:0/4119995940 >> 172.21.15.70:0/4119995940 conn(0x7ff984075670 msgr2=0x7ff984075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.799+0000 7ff989331700 1 -- 172.21.15.70:0/4119995940 shutdown_connections 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.799+0000 7ff989331700 1 -- 172.21.15.70:0/4119995940 wait complete. 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff989331700 1 Processor -- start 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff989331700 1 -- start start 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff989331700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9840ffc20 0x7ff98419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff989331700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff984105bd0 0x7ff98419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff989331700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff984195620 con 0x7ff984105bd0 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff989331700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff984195790 con 0x7ff9840ffc20 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff9827fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff984105bd0 0x7ff98419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff9827fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff984105bd0 0x7ff98419bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40040/0 (socket says 172.21.15.70:40040) 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.800+0000 7ff9827fc700 1 -- 172.21.15.70:0/1243129960 learned_addr learned my addr 172.21.15.70:0/1243129960 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff9827fc700 1 -- 172.21.15.70:0/1243129960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9840ffc20 msgr2=0x7ff98419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff982ffd700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9840ffc20 0x7ff98419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff9827fc700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9840ffc20 0x7ff98419b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff9827fc700 1 -- 172.21.15.70:0/1243129960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9780095f0 con 0x7ff984105bd0 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff9827fc700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff984105bd0 0x7ff98419bb70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff978006010 tx=0x7ff97800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff96bfff700 1 -- 172.21.15.70:0/1243129960 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97801f070 con 0x7ff984105bd0 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff96bfff700 1 -- 172.21.15.70:0/1243129960 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff978003680 con 0x7ff984105bd0 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff96bfff700 1 -- 172.21.15.70:0/1243129960 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97800f7f0 con 0x7ff984105bd0 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff984195a10 con 0x7ff984105bd0 2024-04-02T04:51:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.801+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff984195f00 con 0x7ff984105bd0 2024-04-02T04:51:31.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.802+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98404f2e0 con 0x7ff984105bd0 2024-04-02T04:51:31.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.802+0000 7ff96bfff700 1 -- 172.21.15.70:0/1243129960 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7ff9780037f0 con 0x7ff984105bd0 2024-04-02T04:51:31.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.803+0000 7ff96bfff700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff96c06d9e0 0x7ff96c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:31.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.803+0000 7ff96bfff700 1 -- 172.21.15.70:0/1243129960 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7ff978090aa0 con 0x7ff984105bd0 2024-04-02T04:51:31.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.803+0000 7ff982ffd700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff96c06d9e0 0x7ff96c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:31.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.803+0000 7ff982ffd700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff96c06d9e0 0x7ff96c06fea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff974005950 tx=0x7ff974005cf0 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:31.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:31.807+0000 7ff96bfff700 1 -- 172.21.15.70:0/1243129960 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7ff97805a560 con 0x7ff984105bd0 2024-04-02T04:51:32.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.082+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff984196120 con 0x7ff984105bd0 2024-04-02T04:51:32.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.082+0000 7ff96bfff700 1 -- 172.21.15.70:0/1243129960 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7ff97805dc20 con 0x7ff984105bd0 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff96c06d9e0 msgr2=0x7ff96c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff96c06d9e0 0x7ff96c06fea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff974005950 tx=0x7ff974005cf0 comp rx=0 tx=0).stop 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff984105bd0 msgr2=0x7ff98419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff984105bd0 0x7ff98419bb70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff978006010 tx=0x7ff97800be20 comp rx=0 tx=0).stop 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 shutdown_connections 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff96c06d9e0 0x7ff96c06fea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9840ffc20 0x7ff98419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 --2- 172.21.15.70:0/1243129960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff984105bd0 0x7ff98419bb70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 >> 172.21.15.70:0/1243129960 conn(0x7ff984075670 msgr2=0x7ff9840fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 shutdown_connections 2024-04-02T04:51:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:32.084+0000 7ff989331700 1 -- 172.21.15.70:0/1243129960 wait complete. 2024-04-02T04:51:32.099 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:51:32.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-02T04:51:32.956 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:33.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:32 smithi070 ceph-mon[29212]: pgmap v88: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:33.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:32 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1243129960' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:33.211 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:32 smithi179 ceph-mon[36603]: pgmap v88: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:33.212 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:32 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1243129960' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.566+0000 7faf9d0a2700 1 -- 172.21.15.70:0/3915984113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98068580 msgr2=0x7faf980689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.566+0000 7faf9d0a2700 1 --2- 172.21.15.70:0/3915984113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98068580 0x7faf980689e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faf80009910 tx=0x7faf80009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.566+0000 7faf9d0a2700 1 -- 172.21.15.70:0/3915984113 shutdown_connections 2024-04-02T04:51:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.566+0000 7faf9d0a2700 1 --2- 172.21.15.70:0/3915984113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98068580 0x7faf980689e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.566+0000 7faf9d0a2700 1 --2- 172.21.15.70:0/3915984113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98106cd0 0x7faf981070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.566+0000 7faf9d0a2700 1 -- 172.21.15.70:0/3915984113 >> 172.21.15.70:0/3915984113 conn(0x7faf98075670 msgr2=0x7faf98075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.566+0000 7faf9d0a2700 1 -- 172.21.15.70:0/3915984113 shutdown_connections 2024-04-02T04:51:34.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.566+0000 7faf9d0a2700 1 -- 172.21.15.70:0/3915984113 wait complete. 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9d0a2700 1 Processor -- start 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9d0a2700 1 -- start start 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9d0a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98068580 0x7faf9819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9d0a2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98106cd0 0x7faf9819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9d0a2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9819d180 con 0x7faf98106cd0 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9d0a2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf981966c0 con 0x7faf98068580 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98106cd0 0x7faf9819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98106cd0 0x7faf9819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40060/0 (socket says 172.21.15.70:40060) 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf9659c700 1 -- 172.21.15.70:0/219768884 learned_addr learned my addr 172.21.15.70:0/219768884 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.567+0000 7faf96d9d700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98068580 0x7faf9819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf9659c700 1 -- 172.21.15.70:0/219768884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98068580 msgr2=0x7faf9819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf9659c700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98068580 0x7faf9819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf9659c700 1 -- 172.21.15.70:0/219768884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf800095f0 con 0x7faf98106cd0 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf9659c700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98106cd0 0x7faf9819cb80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faf80006010 tx=0x7faf8000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf8ffff700 1 -- 172.21.15.70:0/219768884 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf8001d070 con 0x7faf98106cd0 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf98196940 con 0x7faf98106cd0 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf8ffff700 1 -- 172.21.15.70:0/219768884 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf80004fb0 con 0x7faf98106cd0 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf8ffff700 1 -- 172.21.15.70:0/219768884 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf80022ea0 con 0x7faf98106cd0 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.568+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf98196e30 con 0x7faf98106cd0 2024-04-02T04:51:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.569+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf981091d0 con 0x7faf98106cd0 2024-04-02T04:51:34.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.569+0000 7faf8ffff700 1 -- 172.21.15.70:0/219768884 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7faf80022470 con 0x7faf98106cd0 2024-04-02T04:51:34.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.570+0000 7faf8ffff700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faf8406d9e0 0x7faf8406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:34.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.570+0000 7faf8ffff700 1 -- 172.21.15.70:0/219768884 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7faf8008eb10 con 0x7faf98106cd0 2024-04-02T04:51:34.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.570+0000 7faf96d9d700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faf8406d9e0 0x7faf8406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:34.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.570+0000 7faf96d9d700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faf8406d9e0 0x7faf8406fea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faf8800e270 tx=0x7faf88009040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.574+0000 7faf8ffff700 1 -- 172.21.15.70:0/219768884 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7faf8005c1b0 con 0x7faf98106cd0 2024-04-02T04:51:34.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:34.783+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}) v1 -- 0x7faf98197050 con 0x7faf8406d9e0 2024-04-02T04:51:35.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:34 smithi070 ceph-mon[29212]: pgmap v89: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:35.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:34 smithi179 ceph-mon[36603]: pgmap v89: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:35.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:35 smithi070 ceph-mon[29212]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:36.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:35 smithi179 ceph-mon[36603]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:36.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.854+0000 7faf8ffff700 1 -- 172.21.15.70:0/219768884 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7faf98197050 con 0x7faf8406d9e0 2024-04-02T04:51:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faf8406d9e0 msgr2=0x7faf8406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faf8406d9e0 0x7faf8406fea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faf8800e270 tx=0x7faf88009040 comp rx=0 tx=0).stop 2024-04-02T04:51:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98106cd0 msgr2=0x7faf9819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98106cd0 0x7faf9819cb80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faf80006010 tx=0x7faf8000be20 comp rx=0 tx=0).stop 2024-04-02T04:51:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 shutdown_connections 2024-04-02T04:51:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7faf8406d9e0 0x7faf8406fea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98068580 0x7faf9819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 --2- 172.21.15.70:0/219768884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf98106cd0 0x7faf9819cb80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.856+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 >> 172.21.15.70:0/219768884 conn(0x7faf98075670 msgr2=0x7faf980fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.857+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 shutdown_connections 2024-04-02T04:51:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:36.857+0000 7faf9d0a2700 1 -- 172.21.15.70:0/219768884 wait complete. 2024-04-02T04:51:36.871 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:51:37.172 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:36 smithi070 ceph-mon[29212]: pgmap v90: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:37.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:36 smithi179 ceph-mon[36603]: pgmap v90: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:37.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch ps' 2024-04-02T04:51:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:39.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:38 smithi070 ceph-mon[29212]: pgmap v91: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.287+0000 7ff3e503b700 1 -- 172.21.15.70:0/2612808403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 msgr2=0x7ff3e01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.287+0000 7ff3e503b700 1 --2- 172.21.15.70:0/2612808403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 0x7ff3e01047a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009910 tx=0x7ff3d4009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.287+0000 7ff3e503b700 1 -- 172.21.15.70:0/2612808403 shutdown_connections 2024-04-02T04:51:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.287+0000 7ff3e503b700 1 --2- 172.21.15.70:0/2612808403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 0x7ff3e01047a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.287+0000 7ff3e503b700 1 --2- 172.21.15.70:0/2612808403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3e00ffda0 0x7ff3e0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.287+0000 7ff3e503b700 1 -- 172.21.15.70:0/2612808403 >> 172.21.15.70:0/2612808403 conn(0x7ff3e00fb650 msgr2=0x7ff3e00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.287+0000 7ff3e503b700 1 -- 172.21.15.70:0/2612808403 shutdown_connections 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.287+0000 7ff3e503b700 1 -- 172.21.15.70:0/2612808403 wait complete. 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3e503b700 1 Processor -- start 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3e503b700 1 -- start start 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3e503b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3e00ffda0 0x7ff3e0195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3e503b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 0x7ff3e0195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3e503b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e01960c0 con 0x7ff3e0100750 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3e503b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e0196230 con 0x7ff3e00ffda0 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3de59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 0x7ff3e0195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3de59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 0x7ff3e0195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59996/0 (socket says 172.21.15.70:59996) 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3de59c700 1 -- 172.21.15.70:0/1297575576 learned_addr learned my addr 172.21.15.70:0/1297575576 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.288+0000 7ff3ded9d700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3e00ffda0 0x7ff3e0195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3de59c700 1 -- 172.21.15.70:0/1297575576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3e00ffda0 msgr2=0x7ff3e0195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3de59c700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3e00ffda0 0x7ff3e0195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3de59c700 1 -- 172.21.15.70:0/1297575576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40095f0 con 0x7ff3e0100750 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3de59c700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 0x7ff3e0195b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40092e0 tx=0x7ff3d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3c7fff700 1 -- 172.21.15.70:0/1297575576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d401d070 con 0x7ff3e0100750 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3c7fff700 1 -- 172.21.15.70:0/1297575576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3d400f460 con 0x7ff3e0100750 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e01964b0 con 0x7ff3e0100750 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3c7fff700 1 -- 172.21.15.70:0/1297575576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d40175f0 con 0x7ff3e0100750 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.289+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e01a2f30 con 0x7ff3e0100750 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.290+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e01080f0 con 0x7ff3e0100750 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.290+0000 7ff3c7fff700 1 -- 172.21.15.70:0/1297575576 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7ff3d400f5d0 con 0x7ff3e0100750 2024-04-02T04:51:39.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.291+0000 7ff3c7fff700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff3c806d9e0 0x7ff3c806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:39.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.291+0000 7ff3c7fff700 1 -- 172.21.15.70:0/1297575576 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7ff3d408e8e0 con 0x7ff3e0100750 2024-04-02T04:51:39.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.291+0000 7ff3ded9d700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff3c806d9e0 0x7ff3c806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:39.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.291+0000 7ff3ded9d700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff3c806d9e0 0x7ff3c806fea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0006fd0 tx=0x7ff3d0008040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:39.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.295+0000 7ff3c7fff700 1 -- 172.21.15.70:0/1297575576 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7ff3d405bed0 con 0x7ff3e0100750 2024-04-02T04:51:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:38 smithi179 ceph-mon[36603]: pgmap v91: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:39.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.501+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3e01964b0 con 0x7ff3c806d9e0 2024-04-02T04:51:39.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.511+0000 7ff3c7fff700 1 -- 172.21.15.70:0/1297575576 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff3e01964b0 con 0x7ff3c806d9e0 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (75s) 44s ago 12m 16.9M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (12m) 44s ago 12m 7960k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (11m) 108s ago 11m 7545k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (12m) 44s ago 12m 6669k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (11m) 108s ago 11m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (50s) 44s ago 11m 52.3M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (6m) 44s ago 6m 154M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (6m) 44s ago 6m 17.8M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (6m) 108s ago 6m 15.9M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:51:39.516 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (6m) 108s ago 6m 17.5M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (3m) 44s ago 13m 507M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (2m) 108s ago 11m 448M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (13m) 44s ago 13m 56.0M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (11m) 108s ago 11m 48.4M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (115s) 44s ago 12m 13.4M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (111s) 108s ago 11m 3439k - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (10m) 44s ago 10m 228M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (10m) 44s ago 9m 263M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (9m) 44s ago 9m 211M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (9m) 108s ago 9m 296M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (8m) 108s ago 8m 272M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (8m) 108s ago 8m 226M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:51:39.517 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (93s) 44s ago 11m 49.3M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:51:39.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff3c806d9e0 msgr2=0x7ff3c806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:39.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff3c806d9e0 0x7ff3c806fea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0006fd0 tx=0x7ff3d0008040 comp rx=0 tx=0).stop 2024-04-02T04:51:39.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 msgr2=0x7ff3e0195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:39.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 0x7ff3e0195b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40092e0 tx=0x7ff3d400be20 comp rx=0 tx=0).stop 2024-04-02T04:51:39.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 shutdown_connections 2024-04-02T04:51:39.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7ff3c806d9e0 0x7ff3c806fea0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:39.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3e00ffda0 0x7ff3e0195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:39.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 --2- 172.21.15.70:0/1297575576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e0100750 0x7ff3e0195b80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:39.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.513+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 >> 172.21.15.70:0/1297575576 conn(0x7ff3e00fb650 msgr2=0x7ff3e00fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:39.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.514+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 shutdown_connections 2024-04-02T04:51:39.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:39.514+0000 7ff3e503b700 1 -- 172.21.15.70:0/1297575576 wait complete. 2024-04-02T04:51:40.128 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:39 smithi070 ceph-mon[29212]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:40.150 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-02T04:51:40.151 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:51:40.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-02T04:51:40.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:39 smithi179 ceph-mon[36603]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:41.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:40 smithi070 ceph-mon[29212]: pgmap v92: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:40 smithi179 ceph-mon[36603]: pgmap v92: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.998+0000 7f22c7a2a700 1 -- 172.21.15.70:0/1195881168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c00ffb40 msgr2=0x7f22c00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.998+0000 7f22c7a2a700 1 --2- 172.21.15.70:0/1195881168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c00ffb40 0x7f22c00fffc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009910 tx=0x7f22b4009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.998+0000 7f22c7a2a700 1 -- 172.21.15.70:0/1195881168 shutdown_connections 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.998+0000 7f22c7a2a700 1 --2- 172.21.15.70:0/1195881168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c00ffb40 0x7f22c00fffc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.998+0000 7f22c7a2a700 1 --2- 172.21.15.70:0/1195881168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22c0105bf0 0x7f22c0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.998+0000 7f22c7a2a700 1 -- 172.21.15.70:0/1195881168 >> 172.21.15.70:0/1195881168 conn(0x7f22c00fb610 msgr2=0x7f22c00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.998+0000 7f22c7a2a700 1 -- 172.21.15.70:0/1195881168 shutdown_connections 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.998+0000 7f22c7a2a700 1 -- 172.21.15.70:0/1195881168 wait complete. 2024-04-02T04:51:42.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c7a2a700 1 Processor -- start 2024-04-02T04:51:42.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c7a2a700 1 -- start start 2024-04-02T04:51:42.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c7a2a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22c00ffb40 0x7f22c0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:42.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c7a2a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c0105bf0 0x7f22c0197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:42.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c7a2a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c01982a0 con 0x7f22c0105bf0 2024-04-02T04:51:42.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c7a2a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c0198410 con 0x7f22c00ffb40 2024-04-02T04:51:42.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c4fc5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c0105bf0 0x7f22c0197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:42.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c4fc5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c0105bf0 0x7f22c0197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60016/0 (socket says 172.21.15.70:60016) 2024-04-02T04:51:42.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c4fc5700 1 -- 172.21.15.70:0/973163924 learned_addr learned my addr 172.21.15.70:0/973163924 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c57c6700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22c00ffb40 0x7f22c0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c4fc5700 1 -- 172.21.15.70:0/973163924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22c00ffb40 msgr2=0x7f22c0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c4fc5700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22c00ffb40 0x7f22c0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:41.999+0000 7f22c4fc5700 1 -- 172.21.15.70:0/973163924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40095f0 con 0x7f22c0105bf0 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.000+0000 7f22c4fc5700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c0105bf0 0x7f22c0197c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f22b4006010 tx=0x7f22b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.000+0000 7f22be7fc700 1 -- 172.21.15.70:0/973163924 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22b401d070 con 0x7f22c0105bf0 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.000+0000 7f22be7fc700 1 -- 172.21.15.70:0/973163924 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22b4004fb0 con 0x7f22c0105bf0 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.000+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c01a07b0 con 0x7f22c0105bf0 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.000+0000 7f22be7fc700 1 -- 172.21.15.70:0/973163924 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22b4022ea0 con 0x7f22c0105bf0 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.000+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c01a0c20 con 0x7f22c0105bf0 2024-04-02T04:51:42.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.001+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c0108020 con 0x7f22c0105bf0 2024-04-02T04:51:42.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.001+0000 7f22be7fc700 1 -- 172.21.15.70:0/973163924 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f22b4022470 con 0x7f22c0105bf0 2024-04-02T04:51:42.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.002+0000 7f22be7fc700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f22a806d9e0 0x7f22a806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:42.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.002+0000 7f22c57c6700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f22a806d9e0 0x7f22a806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:42.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.002+0000 7f22be7fc700 1 -- 172.21.15.70:0/973163924 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f22b408eaf0 con 0x7f22c0105bf0 2024-04-02T04:51:42.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.002+0000 7f22c57c6700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f22a806d9e0 0x7f22a806fea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009f20 tx=0x7f22b00094f0 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:42.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.007+0000 7f22be7fc700 1 -- 172.21.15.70:0/973163924 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f22b4058780 con 0x7f22c0105bf0 2024-04-02T04:51:42.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:41 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.204+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f22c004f2e0 con 0x7f22c0105bf0 2024-04-02T04:51:42.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.208+0000 7f22be7fc700 1 -- 172.21.15.70:0/973163924 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f22b4027020 con 0x7f22c0105bf0 2024-04-02T04:51:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f22a806d9e0 msgr2=0x7f22a806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f22a806d9e0 0x7f22a806fea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009f20 tx=0x7f22b00094f0 comp rx=0 tx=0).stop 2024-04-02T04:51:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c0105bf0 msgr2=0x7f22c0197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c0105bf0 0x7f22c0197c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f22b4006010 tx=0x7f22b400be20 comp rx=0 tx=0).stop 2024-04-02T04:51:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 shutdown_connections 2024-04-02T04:51:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f22a806d9e0 0x7f22a806fea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22c00ffb40 0x7f22c0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 --2- 172.21.15.70:0/973163924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c0105bf0 0x7f22c0197c80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 >> 172.21.15.70:0/973163924 conn(0x7f22c00fb610 msgr2=0x7f22c00fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.210+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 shutdown_connections 2024-04-02T04:51:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:42.211+0000 7f22c7a2a700 1 -- 172.21.15.70:0/973163924 wait complete. 2024-04-02T04:51:42.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:41 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:42.865 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-02T04:51:42.865 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:51:42.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-02T04:51:43.073 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:43.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:43 smithi070 ceph-mon[29212]: pgmap v93: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:43.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:43 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/973163924' entity='client.admin' 2024-04-02T04:51:43.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:43 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:43.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:43 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:43.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:43 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:43.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:43 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:43 smithi179 ceph-mon[36603]: pgmap v93: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:43 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/973163924' entity='client.admin' 2024-04-02T04:51:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:43 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:43 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:43 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:43 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:44.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.778+0000 7fcab7c8d700 1 -- 172.21.15.70:0/1455170184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 msgr2=0x7fcab01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:44.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.778+0000 7fcab7c8d700 1 --2- 172.21.15.70:0/1455170184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 0x7fcab01070b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00b0a0 tx=0x7fcaac00b3b0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.778+0000 7fcab7c8d700 1 -- 172.21.15.70:0/1455170184 shutdown_connections 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.778+0000 7fcab7c8d700 1 --2- 172.21.15.70:0/1455170184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab0068580 0x7fcab00689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.778+0000 7fcab7c8d700 1 --2- 172.21.15.70:0/1455170184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 0x7fcab01070b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.778+0000 7fcab7c8d700 1 -- 172.21.15.70:0/1455170184 >> 172.21.15.70:0/1455170184 conn(0x7fcab0075670 msgr2=0x7fcab0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.778+0000 7fcab7c8d700 1 -- 172.21.15.70:0/1455170184 shutdown_connections 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.778+0000 7fcab7c8d700 1 -- 172.21.15.70:0/1455170184 wait complete. 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.779+0000 7fcab7c8d700 1 Processor -- start 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.779+0000 7fcab7c8d700 1 -- start start 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.779+0000 7fcab7c8d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab0068580 0x7fcab019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.779+0000 7fcab7c8d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 0x7fcab019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.779+0000 7fcab7c8d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab019d230 con 0x7fcab0106cd0 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.779+0000 7fcab7c8d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab01966e0 con 0x7fcab0068580 2024-04-02T04:51:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab5228700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 0x7fcab019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab5228700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 0x7fcab019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60040/0 (socket says 172.21.15.70:60040) 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab5228700 1 -- 172.21.15.70:0/469687189 learned_addr learned my addr 172.21.15.70:0/469687189 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab5a29700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab0068580 0x7fcab019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab5228700 1 -- 172.21.15.70:0/469687189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab0068580 msgr2=0x7fcab019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab5228700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab0068580 0x7fcab019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab5228700 1 -- 172.21.15.70:0/469687189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac009d00 con 0x7fcab0106cd0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab5228700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 0x7fcab019cba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000c8a0 tx=0x7fcaa000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcaa6ffd700 1 -- 172.21.15.70:0/469687189 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaa00078e0 con 0x7fcab0106cd0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcaa6ffd700 1 -- 172.21.15.70:0/469687189 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcaa000f450 con 0x7fcab0106cd0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcaa6ffd700 1 -- 172.21.15.70:0/469687189 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaa000e5e0 con 0x7fcab0106cd0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab01969c0 con 0x7fcab0106cd0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.780+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab0196f10 con 0x7fcab0106cd0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.781+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab01091d0 con 0x7fcab0106cd0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.782+0000 7fcaa6ffd700 1 -- 172.21.15.70:0/469687189 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7fcaa000f5c0 con 0x7fcab0106cd0 2024-04-02T04:51:44.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.782+0000 7fcaa6ffd700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fca9c06d9e0 0x7fca9c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:44.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.782+0000 7fcaa6ffd700 1 -- 172.21.15.70:0/469687189 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fcaa008d460 con 0x7fcab0106cd0 2024-04-02T04:51:44.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.782+0000 7fcab5a29700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fca9c06d9e0 0x7fca9c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:44.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.783+0000 7fcab5a29700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fca9c06d9e0 0x7fca9c06fea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcaac000f80 tx=0x7fcaac0060c0 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:44.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.787+0000 7fcaa6ffd700 1 -- 172.21.15.70:0/469687189 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fcaa005aad0 con 0x7fcab0106cd0 2024-04-02T04:51:44.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.980+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fcab004f2e0 con 0x7fcab0106cd0 2024-04-02T04:51:44.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.980+0000 7fcaa6ffd700 1 -- 172.21.15.70:0/469687189 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7fcaa005a660 con 0x7fcab0106cd0 2024-04-02T04:51:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fca9c06d9e0 msgr2=0x7fca9c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fca9c06d9e0 0x7fca9c06fea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcaac000f80 tx=0x7fcaac0060c0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 msgr2=0x7fcab019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 0x7fcab019cba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000c8a0 tx=0x7fcaa000cbb0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 shutdown_connections 2024-04-02T04:51:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fca9c06d9e0 0x7fca9c06fea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab0068580 0x7fcab019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 --2- 172.21.15.70:0/469687189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab0106cd0 0x7fcab019cba0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 >> 172.21.15.70:0/469687189 conn(0x7fcab0075670 msgr2=0x7fcab00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 shutdown_connections 2024-04-02T04:51:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:44.982+0000 7fcab7c8d700 1 -- 172.21.15.70:0/469687189 wait complete. 2024-04-02T04:51:45.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:45 smithi070 ceph-mon[29212]: pgmap v94: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:45.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:45 smithi179 ceph-mon[36603]: pgmap v94: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:45.671 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-02T04:51:45.880 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:47.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:47 smithi070 ceph-mon[29212]: pgmap v95: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:47.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.524+0000 7fdc0dab0700 1 -- 172.21.15.70:0/1071173774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 msgr2=0x7fdc08100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.524+0000 7fdc0dab0700 1 --2- 172.21.15.70:0/1071173774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 0x7fdc08100180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009910 tx=0x7fdbf8009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.524+0000 7fdc0dab0700 1 -- 172.21.15.70:0/1071173774 shutdown_connections 2024-04-02T04:51:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.524+0000 7fdc0dab0700 1 --2- 172.21.15.70:0/1071173774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08100750 0x7fdc081047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.524+0000 7fdc0dab0700 1 --2- 172.21.15.70:0/1071173774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 0x7fdc08100180 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.524+0000 7fdc0dab0700 1 -- 172.21.15.70:0/1071173774 >> 172.21.15.70:0/1071173774 conn(0x7fdc080fb650 msgr2=0x7fdc080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.524+0000 7fdc0dab0700 1 -- 172.21.15.70:0/1071173774 shutdown_connections 2024-04-02T04:51:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.525+0000 7fdc0dab0700 1 -- 172.21.15.70:0/1071173774 wait complete. 2024-04-02T04:51:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.525+0000 7fdc0dab0700 1 Processor -- start 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.525+0000 7fdc0dab0700 1 -- start start 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.525+0000 7fdc0dab0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 0x7fdc08197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.525+0000 7fdc0dab0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08100750 0x7fdc08197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.525+0000 7fdc0dab0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc08198360 con 0x7fdc080ffda0 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.525+0000 7fdc0dab0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc081984d0 con 0x7fdc08100750 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc077fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 0x7fdc08197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc077fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 0x7fdc08197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42452/0 (socket says 172.21.15.70:42452) 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc077fe700 1 -- 172.21.15.70:0/732082449 learned_addr learned my addr 172.21.15.70:0/732082449 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc06ffd700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08100750 0x7fdc08197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc077fe700 1 -- 172.21.15.70:0/732082449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08100750 msgr2=0x7fdc08197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc077fe700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08100750 0x7fdc08197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:47.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc077fe700 1 -- 172.21.15.70:0/732082449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf80095f0 con 0x7fdc080ffda0 2024-04-02T04:51:47.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc06ffd700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08100750 0x7fdc08197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:51:47.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc077fe700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 0x7fdc08197800 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006010 tx=0x7fdbf800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:47.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc04ff9700 1 -- 172.21.15.70:0/732082449 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf801d070 con 0x7fdc080ffda0 2024-04-02T04:51:47.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc081a0870 con 0x7fdc080ffda0 2024-04-02T04:51:47.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc04ff9700 1 -- 172.21.15.70:0/732082449 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbf8004fb0 con 0x7fdc080ffda0 2024-04-02T04:51:47.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc04ff9700 1 -- 172.21.15.70:0/732082449 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf8022ea0 con 0x7fdc080ffda0 2024-04-02T04:51:47.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.526+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc081a0ce0 con 0x7fdc080ffda0 2024-04-02T04:51:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.527+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc08108140 con 0x7fdc080ffda0 2024-04-02T04:51:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.528+0000 7fdc04ff9700 1 -- 172.21.15.70:0/732082449 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7fdbf8022470 con 0x7fdc080ffda0 2024-04-02T04:51:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.528+0000 7fdc04ff9700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fdbf006d9e0 0x7fdbf006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.528+0000 7fdc04ff9700 1 -- 172.21.15.70:0/732082449 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fdbf808eaf0 con 0x7fdc080ffda0 2024-04-02T04:51:47.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.528+0000 7fdc06ffd700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fdbf006d9e0 0x7fdbf006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:47.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.529+0000 7fdc06ffd700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fdbf006d9e0 0x7fdbf006fea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc080696e0 tx=0x7fdbfc008040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.533+0000 7fdc04ff9700 1 -- 172.21.15.70:0/732082449 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fdbf8058780 con 0x7fdc080ffda0 2024-04-02T04:51:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:47 smithi179 ceph-mon[36603]: pgmap v95: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:47.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.725+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fdc0802cc70 con 0x7fdc080ffda0 2024-04-02T04:51:47.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.726+0000 7fdc04ff9700 1 -- 172.21.15.70:0/732082449 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7fdbf8027020 con 0x7fdc080ffda0 2024-04-02T04:51:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fdbf006d9e0 msgr2=0x7fdbf006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fdbf006d9e0 0x7fdbf006fea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc080696e0 tx=0x7fdbfc008040 comp rx=0 tx=0).stop 2024-04-02T04:51:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 msgr2=0x7fdc08197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 0x7fdc08197800 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006010 tx=0x7fdbf800be20 comp rx=0 tx=0).stop 2024-04-02T04:51:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 shutdown_connections 2024-04-02T04:51:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fdbf006d9e0 0x7fdbf006fea0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc080ffda0 0x7fdc08197800 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 --2- 172.21.15.70:0/732082449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08100750 0x7fdc08197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 >> 172.21.15.70:0/732082449 conn(0x7fdc080fb650 msgr2=0x7fdc080fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 shutdown_connections 2024-04-02T04:51:47.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:47.728+0000 7fdc0dab0700 1 -- 172.21.15.70:0/732082449 wait complete. 2024-04-02T04:51:48.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-02T04:51:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:49.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:49 smithi070 ceph-mon[29212]: pgmap v96: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:49.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:49 smithi179 ceph-mon[36603]: pgmap v96: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.218+0000 7f927ea6a700 1 -- 172.21.15.70:0/546386621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92780ffa10 msgr2=0x7f92780ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.218+0000 7f927ea6a700 1 --2- 172.21.15.70:0/546386621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92780ffa10 0x7f92780ffdf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f926c009910 tx=0x7f926c009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.218+0000 7f927ea6a700 1 -- 172.21.15.70:0/546386621 shutdown_connections 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.218+0000 7f927ea6a700 1 --2- 172.21.15.70:0/546386621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9278100330 0x7f927810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.218+0000 7f927ea6a700 1 --2- 172.21.15.70:0/546386621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92780ffa10 0x7f92780ffdf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.218+0000 7f927ea6a700 1 -- 172.21.15.70:0/546386621 >> 172.21.15.70:0/546386621 conn(0x7f92780fb650 msgr2=0x7f92780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.218+0000 7f927ea6a700 1 -- 172.21.15.70:0/546386621 shutdown_connections 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.218+0000 7f927ea6a700 1 -- 172.21.15.70:0/546386621 wait complete. 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f927ea6a700 1 Processor -- start 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f927ea6a700 1 -- start start 2024-04-02T04:51:50.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f927ea6a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92780ffa10 0x7f927819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f927ea6a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9278100330 0x7f927819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f927ea6a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927819f440 con 0x7f9278100330 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f927ea6a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92781988f0 con 0x7f92780ffa10 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f9277fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9278100330 0x7f927819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f9277fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9278100330 0x7f927819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42474/0 (socket says 172.21.15.70:42474) 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.219+0000 7f9277fff700 1 -- 172.21.15.70:0/1104930276 learned_addr learned my addr 172.21.15.70:0/1104930276 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f927c806700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92780ffa10 0x7f927819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f9277fff700 1 -- 172.21.15.70:0/1104930276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92780ffa10 msgr2=0x7f927819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f9277fff700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92780ffa10 0x7f927819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f9277fff700 1 -- 172.21.15.70:0/1104930276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0095f0 con 0x7f9278100330 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f9277fff700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9278100330 0x7f927819edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f926400d8d0 tx=0x7f926400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f9275ffb700 1 -- 172.21.15.70:0/1104930276 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92640098a0 con 0x7f9278100330 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f9275ffb700 1 -- 172.21.15.70:0/1104930276 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9264010460 con 0x7f9278100330 2024-04-02T04:51:50.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9278198bd0 con 0x7f9278100330 2024-04-02T04:51:50.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f9275ffb700 1 -- 172.21.15.70:0/1104930276 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f926400f5f0 con 0x7f9278100330 2024-04-02T04:51:50.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.220+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9278199120 con 0x7f9278100330 2024-04-02T04:51:50.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.221+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927810a7f0 con 0x7f9278100330 2024-04-02T04:51:50.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.222+0000 7f9275ffb700 1 -- 172.21.15.70:0/1104930276 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f9264009a00 con 0x7f9278100330 2024-04-02T04:51:50.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.222+0000 7f9275ffb700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f926806d9e0 0x7f926806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:50.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.222+0000 7f9275ffb700 1 -- 172.21.15.70:0/1104930276 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f926408d410 con 0x7f9278100330 2024-04-02T04:51:50.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.222+0000 7f927c806700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f926806d9e0 0x7f926806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:50.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.223+0000 7f927c806700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f926806d9e0 0x7f926806fea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f926c006010 tx=0x7f926c005a90 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:50.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.227+0000 7f9275ffb700 1 -- 172.21.15.70:0/1104930276 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f926405aa00 con 0x7f9278100330 2024-04-02T04:51:50.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.419+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f9278199340 con 0x7f9278100330 2024-04-02T04:51:50.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.419+0000 7f9275ffb700 1 -- 172.21.15.70:0/1104930276 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7f926405a590 con 0x7f9278100330 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.421+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f926806d9e0 msgr2=0x7f926806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.421+0000 7f927ea6a700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f926806d9e0 0x7f926806fea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f926c006010 tx=0x7f926c005a90 comp rx=0 tx=0).stop 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.421+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9278100330 msgr2=0x7f927819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.421+0000 7f927ea6a700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9278100330 0x7f927819edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f926400d8d0 tx=0x7f926400dbe0 comp rx=0 tx=0).stop 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.422+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 shutdown_connections 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.422+0000 7f927ea6a700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f926806d9e0 0x7f926806fea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.422+0000 7f927ea6a700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92780ffa10 0x7f927819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.422+0000 7f927ea6a700 1 --2- 172.21.15.70:0/1104930276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9278100330 0x7f927819edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:50.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.422+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 >> 172.21.15.70:0/1104930276 conn(0x7f92780fb650 msgr2=0x7f92780fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:50.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.422+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 shutdown_connections 2024-04-02T04:51:50.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:50.422+0000 7f927ea6a700 1 -- 172.21.15.70:0/1104930276 wait complete. 2024-04-02T04:51:51.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-02T04:51:51.272 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:51.386 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:51 smithi070 ceph-mon[29212]: pgmap v97: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:51 smithi179 ceph-mon[36603]: pgmap v97: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 -- 172.21.15.70:0/38588766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8100ea0 msgr2=0x7f09b8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 --2- 172.21.15.70:0/38588766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8100ea0 0x7f09b8101280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009910 tx=0x7f09ac009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 -- 172.21.15.70:0/38588766 shutdown_connections 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 --2- 172.21.15.70:0/38588766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b8101850 0x7f09b81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 --2- 172.21.15.70:0/38588766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8100ea0 0x7f09b8101280 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 -- 172.21.15.70:0/38588766 >> 172.21.15.70:0/38588766 conn(0x7f09b80fc750 msgr2=0x7f09b80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 -- 172.21.15.70:0/38588766 shutdown_connections 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 -- 172.21.15.70:0/38588766 wait complete. 2024-04-02T04:51:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 Processor -- start 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.881+0000 7f09be9fe700 1 -- start start 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09be9fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b8100ea0 0x7f09b819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09be9fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8101850 0x7f09b819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09be9fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b819d2c0 con 0x7f09b8101850 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09be9fe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8196770 con 0x7f09b8100ea0 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09b77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8101850 0x7f09b819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09b77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8101850 0x7f09b819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42492/0 (socket says 172.21.15.70:42492) 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09b77fe700 1 -- 172.21.15.70:0/881510955 learned_addr learned my addr 172.21.15.70:0/881510955 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09b7fff700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b8100ea0 0x7f09b819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09b77fe700 1 -- 172.21.15.70:0/881510955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b8100ea0 msgr2=0x7f09b819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09b77fe700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b8100ea0 0x7f09b819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09b77fe700 1 -- 172.21.15.70:0/881510955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0095f0 con 0x7f09b8101850 2024-04-02T04:51:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.882+0000 7f09b77fe700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8101850 0x7f09b819cc30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c8f0 tx=0x7f09a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.883+0000 7f09b57fa700 1 -- 172.21.15.70:0/881510955 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a8007a50 con 0x7f09b8101850 2024-04-02T04:51:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.883+0000 7f09b57fa700 1 -- 172.21.15.70:0/881510955 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09a8007bb0 con 0x7f09b8101850 2024-04-02T04:51:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.883+0000 7f09b57fa700 1 -- 172.21.15.70:0/881510955 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a8018630 con 0x7f09b8101850 2024-04-02T04:51:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.883+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b8196a50 con 0x7f09b8101850 2024-04-02T04:51:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.883+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b8196f50 con 0x7f09b8101850 2024-04-02T04:51:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.884+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b8109240 con 0x7f09b8101850 2024-04-02T04:51:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.884+0000 7f09b57fa700 1 -- 172.21.15.70:0/881510955 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f09a800f450 con 0x7f09b8101850 2024-04-02T04:51:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.885+0000 7f09b57fa700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f09a006d9e0 0x7f09a006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.885+0000 7f09b57fa700 1 -- 172.21.15.70:0/881510955 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f09a808d560 con 0x7f09b8101850 2024-04-02T04:51:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.885+0000 7f09b7fff700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f09a006d9e0 0x7f09a006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.885+0000 7f09b7fff700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f09a006d9e0 0x7f09a006fea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00b5c0 tx=0x7f09ac01a040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:52.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:52.890+0000 7f09b57fa700 1 -- 172.21.15.70:0/881510955 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f09a805ab50 con 0x7f09b8101850 2024-04-02T04:51:53.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.117+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}) v1 -- 0x7f09b8197d90 con 0x7f09a006d9e0 2024-04-02T04:51:53.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.124+0000 7f09b57fa700 1 -- 172.21.15.70:0/881510955 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f09b8197d90 con 0x7f09a006d9e0 2024-04-02T04:51:53.128 INFO:teuthology.orchestra.run.smithi070.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:51:53.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f09a006d9e0 msgr2=0x7f09a006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:53.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f09a006d9e0 0x7f09a006fea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00b5c0 tx=0x7f09ac01a040 comp rx=0 tx=0).stop 2024-04-02T04:51:53.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8101850 msgr2=0x7f09b819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:53.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8101850 0x7f09b819cc30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c8f0 tx=0x7f09a800ccb0 comp rx=0 tx=0).stop 2024-04-02T04:51:53.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 shutdown_connections 2024-04-02T04:51:53.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f09a006d9e0 0x7f09a006fea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:53.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b8100ea0 0x7f09b819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:53.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 --2- 172.21.15.70:0/881510955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09b8101850 0x7f09b819cc30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:53.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 >> 172.21.15.70:0/881510955 conn(0x7f09b80fc750 msgr2=0x7f09b80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:53.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 shutdown_connections 2024-04-02T04:51:53.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:53.126+0000 7f09be9fe700 1 -- 172.21.15.70:0/881510955 wait complete. 2024-04-02T04:51:53.492 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:53 smithi070 ceph-mon[29212]: pgmap v98: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:53.492 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:53 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:53.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:53 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:53.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:53 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:53.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:53 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:53.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:53 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:53 smithi179 ceph-mon[36603]: pgmap v98: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:53 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:53 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:53 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:53 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:53 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:53.831 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-02T04:51:53.832 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T04:51:53.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-02T04:51:54.051 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:51:54.448 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:54 smithi070 ceph-mon[29212]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:54.448 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:54 smithi070 ceph-mon[29212]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:51:54.449 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:54 smithi070 ceph-mon[29212]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:51:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:54 smithi179 ceph-mon[36603]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:54 smithi179 ceph-mon[36603]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:51:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:54 smithi179 ceph-mon[36603]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:51:55.477 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:55 smithi070 ceph-mon[29212]: pgmap v99: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:55.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:55 smithi179 ceph-mon[36603]: pgmap v99: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 -- 172.21.15.70:0/1284262732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 msgr2=0x7f99b41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 --2- 172.21.15.70:0/1284262732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 0x7f99b41047a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009910 tx=0x7f99a8009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 -- 172.21.15.70:0/1284262732 shutdown_connections 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 --2- 172.21.15.70:0/1284262732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 0x7f99b41047a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 --2- 172.21.15.70:0/1284262732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99b40ffda0 0x7f99b4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 -- 172.21.15.70:0/1284262732 >> 172.21.15.70:0/1284262732 conn(0x7f99b40fb650 msgr2=0x7f99b40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 -- 172.21.15.70:0/1284262732 shutdown_connections 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 -- 172.21.15.70:0/1284262732 wait complete. 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 Processor -- start 2024-04-02T04:51:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.673+0000 7f99ba530700 1 -- start start 2024-04-02T04:51:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99ba530700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99b40ffda0 0x7f99b419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99ba530700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 0x7f99b419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99ba530700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b419c1f0 con 0x7f99b4100750 2024-04-02T04:51:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99ba530700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b41956a0 con 0x7f99b40ffda0 2024-04-02T04:51:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99b37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 0x7f99b419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99b37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 0x7f99b419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48160/0 (socket says 172.21.15.70:48160) 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99b37fe700 1 -- 172.21.15.70:0/2376612329 learned_addr learned my addr 172.21.15.70:0/2376612329 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99b3fff700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99b40ffda0 0x7f99b419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99b37fe700 1 -- 172.21.15.70:0/2376612329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99b40ffda0 msgr2=0x7f99b419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99b37fe700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99b40ffda0 0x7f99b419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99b37fe700 1 -- 172.21.15.70:0/2376612329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a80095f0 con 0x7f99b4100750 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.674+0000 7f99b3fff700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99b40ffda0 0x7f99b419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.675+0000 7f99b37fe700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 0x7f99b419bb60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f99a80092e0 tx=0x7f99a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.675+0000 7f99b17fa700 1 -- 172.21.15.70:0/2376612329 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99a801d070 con 0x7f99b4100750 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.675+0000 7f99b17fa700 1 -- 172.21.15.70:0/2376612329 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99a8005dc0 con 0x7f99b4100750 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.675+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b4195920 con 0x7f99b4100750 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.675+0000 7f99b17fa700 1 -- 172.21.15.70:0/2376612329 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99a8017400 con 0x7f99b4100750 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.675+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b4195e10 con 0x7f99b4100750 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.676+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b404f2e0 con 0x7f99b4100750 2024-04-02T04:51:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.676+0000 7f99b17fa700 1 -- 172.21.15.70:0/2376612329 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f99a800f460 con 0x7f99b4100750 2024-04-02T04:51:56.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.677+0000 7f99b17fa700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f999c06d9e0 0x7f999c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.677+0000 7f99b17fa700 1 -- 172.21.15.70:0/2376612329 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f99a808ea80 con 0x7f99b4100750 2024-04-02T04:51:56.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.677+0000 7f99b3fff700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f999c06d9e0 0x7f999c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.677+0000 7f99b3fff700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f999c06d9e0 0x7f999c06fea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99a4005950 tx=0x7f99a40058e0 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.681+0000 7f99b17fa700 1 -- 172.21.15.70:0/2376612329 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f99a805c070 con 0x7f99b4100750 2024-04-02T04:51:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.903+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99b41960a0 con 0x7f999c06d9e0 2024-04-02T04:51:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.904+0000 7f99b17fa700 1 -- 172.21.15.70:0/2376612329 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f99b41960a0 con 0x7f999c06d9e0 2024-04-02T04:51:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f999c06d9e0 msgr2=0x7f999c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f999c06d9e0 0x7f999c06fea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99a4005950 tx=0x7f99a40058e0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 msgr2=0x7f99b419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 0x7f99b419bb60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f99a80092e0 tx=0x7f99a800be20 comp rx=0 tx=0).stop 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 shutdown_connections 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f999c06d9e0 0x7f999c06fea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99b40ffda0 0x7f99b419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 --2- 172.21.15.70:0/2376612329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b4100750 0x7f99b419bb60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 >> 172.21.15.70:0/2376612329 conn(0x7f99b40fb650 msgr2=0x7f99b40fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 shutdown_connections 2024-04-02T04:51:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:56.906+0000 7f99ba530700 1 -- 172.21.15.70:0/2376612329 wait complete. 2024-04-02T04:51:56.921 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:51:57.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.012+0000 7f565fe92700 1 -- 172.21.15.70:0/3266534157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580ff5a0 msgr2=0x7f56580ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.012+0000 7f565fe92700 1 --2- 172.21.15.70:0/3266534157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580ff5a0 0x7f56580ff980 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5654009910 tx=0x7f5654009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:57.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.012+0000 7f565fe92700 1 -- 172.21.15.70:0/3266534157 shutdown_connections 2024-04-02T04:51:57.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.012+0000 7f565fe92700 1 --2- 172.21.15.70:0/3266534157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56580fff50 0x7f5658105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.012+0000 7f565fe92700 1 --2- 172.21.15.70:0/3266534157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580ff5a0 0x7f56580ff980 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.012+0000 7f565fe92700 1 -- 172.21.15.70:0/3266534157 >> 172.21.15.70:0/3266534157 conn(0x7f5658074e30 msgr2=0x7f5658075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.012+0000 7f565fe92700 1 -- 172.21.15.70:0/3266534157 shutdown_connections 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.012+0000 7f565fe92700 1 -- 172.21.15.70:0/3266534157 wait complete. 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565fe92700 1 Processor -- start 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565fe92700 1 -- start start 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565fe92700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56580ff5a0 0x7f5658100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565fe92700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580fff50 0x7f5658100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565fe92700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5658100fa0 con 0x7f56580fff50 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565fe92700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581010e0 con 0x7f56580ff5a0 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565d42d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580fff50 0x7f5658100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565d42d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580fff50 0x7f5658100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48174/0 (socket says 172.21.15.70:48174) 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565d42d700 1 -- 172.21.15.70:0/4067871518 learned_addr learned my addr 172.21.15.70:0/4067871518 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565dc2e700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56580ff5a0 0x7f5658100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565d42d700 1 -- 172.21.15.70:0/4067871518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56580ff5a0 msgr2=0x7f5658100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565d42d700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56580ff5a0 0x7f5658100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565d42d700 1 -- 172.21.15.70:0/4067871518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56540095f0 con 0x7f56580fff50 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.013+0000 7f565d42d700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580fff50 0x7f5658100a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f564800c960 tx=0x7f564800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.014+0000 7f564effd700 1 -- 172.21.15.70:0/4067871518 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5648007a70 con 0x7f56580fff50 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.014+0000 7f564effd700 1 -- 172.21.15.70:0/4067871518 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5648007bd0 con 0x7f56580fff50 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.014+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56581a2b00 con 0x7f56580fff50 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.014+0000 7f564effd700 1 -- 172.21.15.70:0/4067871518 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5648018630 con 0x7f56580fff50 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.014+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56581a2e60 con 0x7f56580fff50 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.015+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5658109270 con 0x7f56580fff50 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.015+0000 7f564effd700 1 -- 172.21.15.70:0/4067871518 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f564800f450 con 0x7f56580fff50 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.016+0000 7f564effd700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f564406d9e0 0x7f564406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.016+0000 7f564effd700 1 -- 172.21.15.70:0/4067871518 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f564808d400 con 0x7f56580fff50 2024-04-02T04:51:57.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.016+0000 7f565dc2e700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f564406d9e0 0x7f564406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.016+0000 7f565dc2e700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f564406d9e0 0x7f564406fea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5654006010 tx=0x7f565401a040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.021+0000 7f564effd700 1 -- 172.21.15.70:0/4067871518 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f5648056ff0 con 0x7f56580fff50 2024-04-02T04:51:57.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.244+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56580611d0 con 0x7f564406d9e0 2024-04-02T04:51:57.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.246+0000 7f564effd700 1 -- 172.21.15.70:0/4067871518 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f56580611d0 con 0x7f564406d9e0 2024-04-02T04:51:57.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f564406d9e0 msgr2=0x7f564406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f564406d9e0 0x7f564406fea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5654006010 tx=0x7f565401a040 comp rx=0 tx=0).stop 2024-04-02T04:51:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580fff50 msgr2=0x7f5658100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580fff50 0x7f5658100a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f564800c960 tx=0x7f564800cc70 comp rx=0 tx=0).stop 2024-04-02T04:51:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 shutdown_connections 2024-04-02T04:51:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f564406d9e0 0x7f564406fea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56580ff5a0 0x7f5658100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 --2- 172.21.15.70:0/4067871518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56580fff50 0x7f5658100a60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 >> 172.21.15.70:0/4067871518 conn(0x7f5658074e30 msgr2=0x7f56580fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 shutdown_connections 2024-04-02T04:51:57.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.248+0000 7f565fe92700 1 -- 172.21.15.70:0/4067871518 wait complete. 2024-04-02T04:51:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:57 smithi070 ceph-mon[29212]: pgmap v100: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:57.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:57.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:57 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-02T04:51:57.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 -- 172.21.15.70:0/1275268805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1990105bd0 msgr2=0x7f1990105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 --2- 172.21.15.70:0/1275268805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1990105bd0 0x7f1990105fb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f198c009910 tx=0x7f198c009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:57.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 -- 172.21.15.70:0/1275268805 shutdown_connections 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 --2- 172.21.15.70:0/1275268805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19900ffc20 0x7f1990100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 --2- 172.21.15.70:0/1275268805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1990105bd0 0x7f1990105fb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 -- 172.21.15.70:0/1275268805 >> 172.21.15.70:0/1275268805 conn(0x7f1990075670 msgr2=0x7f1990075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 -- 172.21.15.70:0/1275268805 shutdown_connections 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 -- 172.21.15.70:0/1275268805 wait complete. 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 Processor -- start 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 -- start start 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.356+0000 7f1998784700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19900ffc20 0x7f199019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1998784700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1990105bd0 0x7f199019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1998784700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199019d240 con 0x7f19900ffc20 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1998784700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19901966f0 con 0x7f1990105bd0 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1996520700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19900ffc20 0x7f199019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1996520700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19900ffc20 0x7f199019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48182/0 (socket says 172.21.15.70:48182) 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1996520700 1 -- 172.21.15.70:0/3144841229 learned_addr learned my addr 172.21.15.70:0/3144841229 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1995d1f700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1990105bd0 0x7f199019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1996520700 1 -- 172.21.15.70:0/3144841229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1990105bd0 msgr2=0x7f199019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1996520700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1990105bd0 0x7f199019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1996520700 1 -- 172.21.15.70:0/3144841229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c0095f0 con 0x7f19900ffc20 2024-04-02T04:51:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1996520700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19900ffc20 0x7f199019c670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f198c006010 tx=0x7f198c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f19877fe700 1 -- 172.21.15.70:0/3144841229 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f198c01d070 con 0x7f19900ffc20 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1990196970 con 0x7f19900ffc20 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f19877fe700 1 -- 172.21.15.70:0/3144841229 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f198c00f460 con 0x7f19900ffc20 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f19877fe700 1 -- 172.21.15.70:0/3144841229 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f198c017630 con 0x7f19900ffc20 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.357+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1990196e60 con 0x7f19900ffc20 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.358+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199004f2e0 con 0x7f19900ffc20 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.359+0000 7f19877fe700 1 -- 172.21.15.70:0/3144841229 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f198c00f5d0 con 0x7f19900ffc20 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.359+0000 7f19877fe700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f197c06d9e0 0x7f197c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.359+0000 7f19877fe700 1 -- 172.21.15.70:0/3144841229 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f198c08ea10 con 0x7f19900ffc20 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.359+0000 7f1995d1f700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f197c06d9e0 0x7f197c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.360+0000 7f1995d1f700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f197c06d9e0 0x7f197c06fea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1980005950 tx=0x7f19800058e0 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.364+0000 7f19877fe700 1 -- 172.21.15.70:0/3144841229 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f198c05c000 con 0x7f19900ffc20 2024-04-02T04:51:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:57 smithi179 ceph-mon[36603]: pgmap v100: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:57 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-02T04:51:57.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.575+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1990197140 con 0x7f197c06d9e0 2024-04-02T04:51:57.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.596+0000 7f19877fe700 1 -- 172.21.15.70:0/3144841229 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1990197140 con 0x7f197c06d9e0 2024-04-02T04:51:57.600 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:51:57.600 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (93s) 62s ago 12m 16.9M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (12m) 62s ago 12m 7960k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (11m) 2m ago 11m 7545k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (12m) 62s ago 12m 6669k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (11m) 2m ago 11m 6665k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (68s) 62s ago 12m 52.3M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (7m) 62s ago 7m 154M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (6m) 62s ago 6m 17.8M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (6m) 2m ago 6m 15.9M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (6m) 2m ago 6m 17.5M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (3m) 62s ago 13m 507M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (2m) 2m ago 11m 448M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (14m) 62s ago 14m 56.0M 2048M 18.2.1 5be31c24972a 6fceaaf3da10 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (11m) 2m ago 11m 48.4M 2048M 18.2.1 5be31c24972a eeb4d97171fb 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (2m) 62s ago 12m 13.4M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:51:57.601 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (2m) 2m ago 11m 3439k - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:51:57.602 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (10m) 62s ago 10m 228M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:51:57.602 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (10m) 62s ago 10m 263M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:51:57.602 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (9m) 62s ago 9m 211M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:51:57.602 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (9m) 2m ago 9m 296M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:51:57.602 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (8m) 2m ago 8m 272M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:51:57.602 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (8m) 2m ago 8m 226M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:51:57.602 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (111s) 62s ago 12m 49.3M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f197c06d9e0 msgr2=0x7f197c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f197c06d9e0 0x7f197c06fea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1980005950 tx=0x7f19800058e0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19900ffc20 msgr2=0x7f199019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19900ffc20 0x7f199019c670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f198c006010 tx=0x7f198c00be20 comp rx=0 tx=0).stop 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 shutdown_connections 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f197c06d9e0 0x7f197c06fea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19900ffc20 0x7f199019c670 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 --2- 172.21.15.70:0/3144841229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1990105bd0 0x7f199019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 >> 172.21.15.70:0/3144841229 conn(0x7f1990075670 msgr2=0x7f1990068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 shutdown_connections 2024-04-02T04:51:57.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.598+0000 7f1998784700 1 -- 172.21.15.70:0/3144841229 wait complete. 2024-04-02T04:51:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.711+0000 7f589ff89700 1 -- 172.21.15.70:0/3260770616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 msgr2=0x7f5898101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.711+0000 7f589ff89700 1 --2- 172.21.15.70:0/3260770616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 0x7f5898101280 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5890009910 tx=0x7f5890009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.711+0000 7f589ff89700 1 -- 172.21.15.70:0/3260770616 shutdown_connections 2024-04-02T04:51:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.711+0000 7f589ff89700 1 --2- 172.21.15.70:0/3260770616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5898101850 0x7f58981058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.711+0000 7f589ff89700 1 --2- 172.21.15.70:0/3260770616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 0x7f5898101280 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.711+0000 7f589ff89700 1 -- 172.21.15.70:0/3260770616 >> 172.21.15.70:0/3260770616 conn(0x7f58980fc750 msgr2=0x7f58980feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.711+0000 7f589ff89700 1 -- 172.21.15.70:0/3260770616 shutdown_connections 2024-04-02T04:51:57.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.711+0000 7f589ff89700 1 -- 172.21.15.70:0/3260770616 wait complete. 2024-04-02T04:51:57.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589ff89700 1 Processor -- start 2024-04-02T04:51:57.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589ff89700 1 -- start start 2024-04-02T04:51:57.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589ff89700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 0x7f589819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589ff89700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5898101850 0x7f589819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589ff89700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589819d350 con 0x7f5898100ea0 2024-04-02T04:51:57.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589ff89700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5898196800 con 0x7f5898101850 2024-04-02T04:51:57.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589dd25700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 0x7f589819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589dd25700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 0x7f589819c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48208/0 (socket says 172.21.15.70:48208) 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589dd25700 1 -- 172.21.15.70:0/1229857736 learned_addr learned my addr 172.21.15.70:0/1229857736 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.712+0000 7f589d524700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5898101850 0x7f589819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f589dd25700 1 -- 172.21.15.70:0/1229857736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5898101850 msgr2=0x7f589819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f589dd25700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5898101850 0x7f589819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f589dd25700 1 -- 172.21.15.70:0/1229857736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900095f0 con 0x7f5898100ea0 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f589dd25700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 0x7f589819c780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5890006010 tx=0x7f589000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f588effd700 1 -- 172.21.15.70:0/1229857736 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f589001d070 con 0x7f5898100ea0 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f588effd700 1 -- 172.21.15.70:0/1229857736 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5890004fb0 con 0x7f5898100ea0 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5898196a80 con 0x7f5898100ea0 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f588effd700 1 -- 172.21.15.70:0/1229857736 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5890022ea0 con 0x7f5898100ea0 2024-04-02T04:51:57.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.713+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5898196f70 con 0x7f5898100ea0 2024-04-02T04:51:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.714+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898109240 con 0x7f5898100ea0 2024-04-02T04:51:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.714+0000 7f588effd700 1 -- 172.21.15.70:0/1229857736 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f5890022470 con 0x7f5898100ea0 2024-04-02T04:51:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.715+0000 7f588effd700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f588406d9e0 0x7f588406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.715+0000 7f589d524700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f588406d9e0 0x7f588406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.715+0000 7f588effd700 1 -- 172.21.15.70:0/1229857736 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f589008ec80 con 0x7f5898100ea0 2024-04-02T04:51:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.716+0000 7f589d524700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f588406d9e0 0x7f588406fea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5888006fd0 tx=0x7f5888008040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:57.720+0000 7f588effd700 1 -- 172.21.15.70:0/1229857736 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f589005c4c0 con 0x7f5898100ea0 2024-04-02T04:51:58.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.009+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f589804f2e0 con 0x7f5898100ea0 2024-04-02T04:51:58.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.010+0000 7f588effd700 1 -- 172.21.15.70:0/1229857736 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f58900272c0 con 0x7f5898100ea0 2024-04-02T04:51:58.014 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:51:58.014 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:51:58.014 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-02T04:51:58.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:51:58.015 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f588406d9e0 msgr2=0x7f588406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f588406d9e0 0x7f588406fea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5888006fd0 tx=0x7f5888008040 comp rx=0 tx=0).stop 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 msgr2=0x7f589819c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 0x7f589819c780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5890006010 tx=0x7f589000be20 comp rx=0 tx=0).stop 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 shutdown_connections 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f588406d9e0 0x7f588406fea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5898100ea0 0x7f589819c780 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 --2- 172.21.15.70:0/1229857736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5898101850 0x7f589819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 >> 172.21.15.70:0/1229857736 conn(0x7f58980fc750 msgr2=0x7f58980fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 shutdown_connections 2024-04-02T04:51:58.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.012+0000 7f589ff89700 1 -- 172.21.15.70:0/1229857736 wait complete. 2024-04-02T04:51:58.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.118+0000 7f7212cdb700 1 -- 172.21.15.70:0/1565847833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 msgr2=0x7f720c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.118+0000 7f7212cdb700 1 --2- 172.21.15.70:0/1565847833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 0x7f720c105fd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7200009910 tx=0x7f7200009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:58.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.118+0000 7f7212cdb700 1 -- 172.21.15.70:0/1565847833 shutdown_connections 2024-04-02T04:51:58.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.118+0000 7f7212cdb700 1 --2- 172.21.15.70:0/1565847833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f720c0ffb40 0x7f720c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.118+0000 7f7212cdb700 1 --2- 172.21.15.70:0/1565847833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 0x7f720c105fd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.118+0000 7f7212cdb700 1 -- 172.21.15.70:0/1565847833 >> 172.21.15.70:0/1565847833 conn(0x7f720c0fb610 msgr2=0x7f720c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:58.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.118+0000 7f7212cdb700 1 -- 172.21.15.70:0/1565847833 shutdown_connections 2024-04-02T04:51:58.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.118+0000 7f7212cdb700 1 -- 172.21.15.70:0/1565847833 wait complete. 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f7212cdb700 1 Processor -- start 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f7212cdb700 1 -- start start 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f7212cdb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f720c0ffb40 0x7f720c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f7212cdb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 0x7f720c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f7212cdb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c19d220 con 0x7f720c105bf0 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f7212cdb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c196740 con 0x7f720c0ffb40 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f720bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 0x7f720c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f720bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 0x7f720c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48232/0 (socket says 172.21.15.70:48232) 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f720bfff700 1 -- 172.21.15.70:0/1141184667 learned_addr learned my addr 172.21.15.70:0/1141184667 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f7210a77700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f720c0ffb40 0x7f720c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f720bfff700 1 -- 172.21.15.70:0/1141184667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f720c0ffb40 msgr2=0x7f720c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f720bfff700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f720c0ffb40 0x7f720c19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f720bfff700 1 -- 172.21.15.70:0/1141184667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000095f0 con 0x7f720c105bf0 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.119+0000 7f7210a77700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f720c0ffb40 0x7f720c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.120+0000 7f720bfff700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 0x7f720c19cb90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f71f800b4b0 tx=0x7f71f800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.120+0000 7f7209ffb700 1 -- 172.21.15.70:0/1141184667 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71f8011800 con 0x7f720c105bf0 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.120+0000 7f7209ffb700 1 -- 172.21.15.70:0/1141184667 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71f8011e20 con 0x7f720c105bf0 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.120+0000 7f7209ffb700 1 -- 172.21.15.70:0/1141184667 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71f800f2e0 con 0x7f720c105bf0 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.120+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c196a20 con 0x7f720c105bf0 2024-04-02T04:51:58.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.120+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c196f20 con 0x7f720c105bf0 2024-04-02T04:51:58.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.121+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c0679c0 con 0x7f720c105bf0 2024-04-02T04:51:58.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.122+0000 7f7209ffb700 1 -- 172.21.15.70:0/1141184667 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f71f8010380 con 0x7f720c105bf0 2024-04-02T04:51:58.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.122+0000 7f7209ffb700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f71fc06d9e0 0x7f71fc06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.122+0000 7f7209ffb700 1 -- 172.21.15.70:0/1141184667 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f71f808d160 con 0x7f720c105bf0 2024-04-02T04:51:58.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.122+0000 7f7210a77700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f71fc06d9e0 0x7f71fc06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.123+0000 7f7210a77700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f71fc06d9e0 0x7f71fc06fea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f720000b5c0 tx=0x7f7200005480 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:58.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.127+0000 7f7209ffb700 1 -- 172.21.15.70:0/1141184667 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f71f805a750 con 0x7f720c105bf0 2024-04-02T04:51:58.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:58 smithi070 ceph-mon[29212]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:58.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:58 smithi070 ceph-mon[29212]: Upgrade: Target is version 19.0.0-1979-g6cec793d (squid) 2024-04-02T04:51:58.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:58 smithi070 ceph-mon[29212]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae'] 2024-04-02T04:51:58.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:58 smithi070 ceph-mon[29212]: Upgrade: Setting container_image for all mgr 2024-04-02T04:51:58.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:58 smithi070 ceph-mon[29212]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:58.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:58 smithi070 ceph-mon[29212]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:58.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:58 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1229857736' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:58.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.370+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f720c04f2e0 con 0x7f720c105bf0 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.370+0000 7f7209ffb700 1 -- 172.21.15.70:0/1141184667 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1826 (secure 0 0 0) 0x7f71f805a2e0 con 0x7f720c105bf0 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:51:58.376 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:51:58.377 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:51:58.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:51:58.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:51:58.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f71fc06d9e0 msgr2=0x7f71fc06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f71fc06d9e0 0x7f71fc06fea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f720000b5c0 tx=0x7f7200005480 comp rx=0 tx=0).stop 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 msgr2=0x7f720c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 0x7f720c19cb90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f71f800b4b0 tx=0x7f71f800b870 comp rx=0 tx=0).stop 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 shutdown_connections 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f71fc06d9e0 0x7f71fc06fea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f720c0ffb40 0x7f720c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 --2- 172.21.15.70:0/1141184667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f720c105bf0 0x7f720c19cb90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.372+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 >> 172.21.15.70:0/1141184667 conn(0x7f720c0fb610 msgr2=0x7f720c068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:58.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.373+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 shutdown_connections 2024-04-02T04:51:58.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.373+0000 7f7212cdb700 1 -- 172.21.15.70:0/1141184667 wait complete. 2024-04-02T04:51:58.380 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:51:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 -- 172.21.15.70:0/2569901759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c0ffc20 msgr2=0x7f2e4c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 --2- 172.21.15.70:0/2569901759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c0ffc20 0x7f2e4c100080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009910 tx=0x7f2e48009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 -- 172.21.15.70:0/2569901759 shutdown_connections 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 --2- 172.21.15.70:0/2569901759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c0ffc20 0x7f2e4c100080 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 --2- 172.21.15.70:0/2569901759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e4c105bd0 0x7f2e4c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 -- 172.21.15.70:0/2569901759 >> 172.21.15.70:0/2569901759 conn(0x7f2e4c075670 msgr2=0x7f2e4c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 -- 172.21.15.70:0/2569901759 shutdown_connections 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 -- 172.21.15.70:0/2569901759 wait complete. 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.478+0000 7f2e53514700 1 Processor -- start 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e53514700 1 -- start start 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e53514700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e4c0ffc20 0x7f2e4c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e53514700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c105bd0 0x7f2e4c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e53514700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c198380 con 0x7f2e4c105bd0 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e53514700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c1984f0 con 0x7f2e4c0ffc20 2024-04-02T04:51:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e50aaf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c105bd0 0x7f2e4c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e50aaf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c105bd0 0x7f2e4c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48258/0 (socket says 172.21.15.70:48258) 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e50aaf700 1 -- 172.21.15.70:0/2903315581 learned_addr learned my addr 172.21.15.70:0/2903315581 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e512b0700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e4c0ffc20 0x7f2e4c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e50aaf700 1 -- 172.21.15.70:0/2903315581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e4c0ffc20 msgr2=0x7f2e4c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e50aaf700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e4c0ffc20 0x7f2e4c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e50aaf700 1 -- 172.21.15.70:0/2903315581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e480095f0 con 0x7f2e4c105bd0 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.479+0000 7f2e50aaf700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c105bd0 0x7f2e4c197ca0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2e48006010 tx=0x7f2e4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.480+0000 7f2e427fc700 1 -- 172.21.15.70:0/2903315581 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e4801d070 con 0x7f2e4c105bd0 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.480+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c1a07d0 con 0x7f2e4c105bd0 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.480+0000 7f2e427fc700 1 -- 172.21.15.70:0/2903315581 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e48005dc0 con 0x7f2e4c105bd0 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.480+0000 7f2e427fc700 1 -- 172.21.15.70:0/2903315581 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e48017400 con 0x7f2e4c105bd0 2024-04-02T04:51:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.480+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c1a0c40 con 0x7f2e4c105bd0 2024-04-02T04:51:58.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.481+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c1080d0 con 0x7f2e4c105bd0 2024-04-02T04:51:58.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.481+0000 7f2e427fc700 1 -- 172.21.15.70:0/2903315581 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7f2e48017560 con 0x7f2e4c105bd0 2024-04-02T04:51:58.488 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.481+0000 7f2e427fc700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f2e3806d9e0 0x7f2e3806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.488 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.481+0000 7f2e427fc700 1 -- 172.21.15.70:0/2903315581 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f2e4808ea20 con 0x7f2e4c105bd0 2024-04-02T04:51:58.488 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.481+0000 7f2e512b0700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f2e3806d9e0 0x7f2e3806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.488 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.482+0000 7f2e512b0700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f2e3806d9e0 0x7f2e3806fea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005950 tx=0x7f2e3c00b410 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:58.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.486+0000 7f2e427fc700 1 -- 172.21.15.70:0/2903315581 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f2e48058550 con 0x7f2e4c105bd0 2024-04-02T04:51:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:58 smithi179 ceph-mon[36603]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:58 smithi179 ceph-mon[36603]: Upgrade: Target is version 19.0.0-1979-g6cec793d (squid) 2024-04-02T04:51:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:58 smithi179 ceph-mon[36603]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae'] 2024-04-02T04:51:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:58 smithi179 ceph-mon[36603]: Upgrade: Setting container_image for all mgr 2024-04-02T04:51:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:58 smithi179 ceph-mon[36603]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:58 smithi179 ceph-mon[36603]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:58 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1229857736' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:58.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.706+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 --> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e4c0611d0 con 0x7f2e3806d9e0 2024-04-02T04:51:58.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.707+0000 7f2e427fc700 1 -- 172.21.15.70:0/2903315581 <== mgr.24423 v2:172.21.15.70:6800/146104126 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f2e4c0611d0 con 0x7f2e3806d9e0 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/23 daemons upgraded", 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mon daemons", 2024-04-02T04:51:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:51:58.713 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.709+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f2e3806d9e0 msgr2=0x7f2e3806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.709+0000 7f2e53514700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f2e3806d9e0 0x7f2e3806fea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005950 tx=0x7f2e3c00b410 comp rx=0 tx=0).stop 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.709+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c105bd0 msgr2=0x7f2e4c197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.709+0000 7f2e53514700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c105bd0 0x7f2e4c197ca0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2e48006010 tx=0x7f2e4800bd70 comp rx=0 tx=0).stop 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.709+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 shutdown_connections 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.710+0000 7f2e53514700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7f2e3806d9e0 0x7f2e3806fea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.710+0000 7f2e53514700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e4c0ffc20 0x7f2e4c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.710+0000 7f2e53514700 1 --2- 172.21.15.70:0/2903315581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e4c105bd0 0x7f2e4c197ca0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.710+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 >> 172.21.15.70:0/2903315581 conn(0x7f2e4c075670 msgr2=0x7f2e4c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.710+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 shutdown_connections 2024-04-02T04:51:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.710+0000 7f2e53514700 1 -- 172.21.15.70:0/2903315581 wait complete. 2024-04-02T04:51:58.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.813+0000 7fd407d8a700 1 -- 172.21.15.70:0/3419756198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4000ffc20 msgr2=0x7fd400100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.813+0000 7fd407d8a700 1 --2- 172.21.15.70:0/3419756198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4000ffc20 0x7fd400100080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009910 tx=0x7fd3f4009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:58.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.813+0000 7fd407d8a700 1 -- 172.21.15.70:0/3419756198 shutdown_connections 2024-04-02T04:51:58.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.813+0000 7fd407d8a700 1 --2- 172.21.15.70:0/3419756198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4000ffc20 0x7fd400100080 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.813+0000 7fd407d8a700 1 --2- 172.21.15.70:0/3419756198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd400105bd0 0x7fd400105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.813+0000 7fd407d8a700 1 -- 172.21.15.70:0/3419756198 >> 172.21.15.70:0/3419756198 conn(0x7fd400075670 msgr2=0x7fd400075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:58.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.813+0000 7fd407d8a700 1 -- 172.21.15.70:0/3419756198 shutdown_connections 2024-04-02T04:51:58.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.813+0000 7fd407d8a700 1 -- 172.21.15.70:0/3419756198 wait complete. 2024-04-02T04:51:58.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd407d8a700 1 Processor -- start 2024-04-02T04:51:58.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd407d8a700 1 -- start start 2024-04-02T04:51:58.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd407d8a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4000ffc20 0x7fd4001977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd407d8a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd400105bd0 0x7fd400197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd407d8a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd400198350 con 0x7fd400105bd0 2024-04-02T04:51:58.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd407d8a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd400198490 con 0x7fd4000ffc20 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd405325700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd400105bd0 0x7fd400197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd405325700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd400105bd0 0x7fd400197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48286/0 (socket says 172.21.15.70:48286) 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd405325700 1 -- 172.21.15.70:0/563067253 learned_addr learned my addr 172.21.15.70:0/563067253 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd405b26700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4000ffc20 0x7fd4001977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd405325700 1 -- 172.21.15.70:0/563067253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4000ffc20 msgr2=0x7fd4001977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd405325700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4000ffc20 0x7fd4001977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd405325700 1 -- 172.21.15.70:0/563067253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f40095f0 con 0x7fd400105bd0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.814+0000 7fd405b26700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4000ffc20 0x7fd4001977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.815+0000 7fd405325700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd400105bd0 0x7fd400197d30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4006010 tx=0x7fd3f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.815+0000 7fd3feffd700 1 -- 172.21.15.70:0/563067253 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f401d070 con 0x7fd400105bd0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.815+0000 7fd3feffd700 1 -- 172.21.15.70:0/563067253 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3f4004fb0 con 0x7fd400105bd0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.815+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4001a0860 con 0x7fd400105bd0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.815+0000 7fd3feffd700 1 -- 172.21.15.70:0/563067253 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f4022ea0 con 0x7fd400105bd0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.815+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4001a0c70 con 0x7fd400105bd0 2024-04-02T04:51:58.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.816+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4001080f0 con 0x7fd400105bd0 2024-04-02T04:51:58.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.816+0000 7fd3feffd700 1 -- 172.21.15.70:0/563067253 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 91714+0+0 (secure 0 0 0) 0x7fd3f4022470 con 0x7fd400105bd0 2024-04-02T04:51:58.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.817+0000 7fd3feffd700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fd3e806d9e0 0x7fd3e806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:58.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.817+0000 7fd3feffd700 1 -- 172.21.15.70:0/563067253 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fd3f408ec60 con 0x7fd400105bd0 2024-04-02T04:51:58.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.817+0000 7fd405b26700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fd3e806d9e0 0x7fd3e806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:58.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.817+0000 7fd405b26700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fd3e806d9e0 0x7fd3e806fea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8007900 tx=0x7fd3f8008040 comp rx=0 tx=0).ready entity=mgr.24423 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:58.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:58.821+0000 7fd3feffd700 1 -- 172.21.15.70:0/563067253 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fd3f4058970 con 0x7fd400105bd0 2024-04-02T04:51:59.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.120+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd40004f2e0 con 0x7fd400105bd0 2024-04-02T04:51:59.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.121+0000 7fd3feffd700 1 -- 172.21.15.70:0/563067253 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd3f4027090 con 0x7fd400105bd0 2024-04-02T04:51:59.126 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fd3e806d9e0 msgr2=0x7fd3e806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fd3e806d9e0 0x7fd3e806fea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8007900 tx=0x7fd3f8008040 comp rx=0 tx=0).stop 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd400105bd0 msgr2=0x7fd400197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd400105bd0 0x7fd400197d30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4006010 tx=0x7fd3f400be20 comp rx=0 tx=0).stop 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 shutdown_connections 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:6800/146104126,v1:172.21.15.70:6801/146104126] conn(0x7fd3e806d9e0 0x7fd3e806fea0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4000ffc20 0x7fd4001977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 --2- 172.21.15.70:0/563067253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd400105bd0 0x7fd400197d30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 >> 172.21.15.70:0/563067253 conn(0x7fd400075670 msgr2=0x7fd4000fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 shutdown_connections 2024-04-02T04:51:59.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:51:59.123+0000 7fd407d8a700 1 -- 172.21.15.70:0/563067253 wait complete. 2024-04-02T04:51:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:59 smithi070 ceph-mon[29212]: pgmap v101: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:59.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:59 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/1141184667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:51:59.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:59.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:51:59.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:51:59.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:59 smithi070 ceph-mon[29212]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:59.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:51:59 smithi070 ceph-mon[29212]: from='client.? 172.21.15.70:0/563067253' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:51:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:59 smithi179 ceph-mon[36603]: pgmap v101: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:59 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/1141184667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:51:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' 2024-04-02T04:51:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:51:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:51:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:59 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:51:59 smithi179 ceph-mon[36603]: from='client.? 172.21.15.70:0/563067253' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:52:00.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:00 smithi070 ceph-mon[29212]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:00 smithi070 ceph-mon[29212]: Upgrade: Updating mon.smithi070 2024-04-02T04:52:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:00 smithi070 ceph-mon[29212]: Deploying daemon mon.smithi070 on smithi070 2024-04-02T04:52:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:00 smithi179 ceph-mon[36603]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:00 smithi179 ceph-mon[36603]: Upgrade: Updating mon.smithi070 2024-04-02T04:52:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:00 smithi179 ceph-mon[36603]: Deploying daemon mon.smithi070 on smithi070 2024-04-02T04:52:01.141 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:00 smithi070 systemd[1]: Stopping Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:52:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:01 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[29208]: 2024-04-02T04:52:01.136+0000 7efefef59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi070 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:52:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:01 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[29208]: 2024-04-02T04:52:01.136+0000 7efefef59700 -1 mon.smithi070@0(leader) e2 *** Got Signal Terminated *** 2024-04-02T04:52:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:01 smithi070 podman[113460]: 2024-04-02 04:52:01.198333856 +0000 UTC m=+0.187554713 container died 6fceaaf3da10741c2e3cc8c86ed5c9b2b69b77d6fa89d6c863411f0c3608a0ed (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, org.label-schema.vendor=CentOS, RELEASE=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.29.1) 2024-04-02T04:52:01.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:01 smithi070 podman[113460]: 2024-04-02 04:52:01.620467526 +0000 UTC m=+0.609688370 container cleanup 6fceaaf3da10741c2e3cc8c86ed5c9b2b69b77d6fa89d6c863411f0c3608a0ed (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-02T04:52:01.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:01 smithi070 bash[113460]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070 2024-04-02T04:52:02.101 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:01 smithi070 podman[113485]: 2024-04-02 04:52:01.838170712 +0000 UTC m=+0.639996205 container remove 6fceaaf3da10741c2e3cc8c86ed5c9b2b69b77d6fa89d6c863411f0c3608a0ed (image=quay.io/ceph/ceph:v18.2.1, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.29.1, RELEASE=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-02T04:52:02.101 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070.service: Deactivated successfully. 2024-04-02T04:52:02.101 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 systemd[1]: Stopped Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:52:02.101 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070.service: Consumed 13.977s CPU time. 2024-04-02T04:52:02.503 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 systemd[1]: Starting Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:52:02.773 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 podman[113616]: 2024-04-02 04:52:02.402087928 +0000 UTC m=+0.017809219 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:52:02.773 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 podman[113616]: 2024-04-02 04:52:02.531283429 +0000 UTC m=+0.147004709 container create e6a7191c4a7f10e452b2625b6d3ecc0e4bb9d72152f0878614ff608d02c1fd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T04:52:02.773 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 podman[113616]: 2024-04-02 04:52:02.743451265 +0000 UTC m=+0.359172561 container init e6a7191c4a7f10e452b2625b6d3ecc0e4bb9d72152f0878614ff608d02c1fd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:52:02.773 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 podman[113616]: 2024-04-02 04:52:02.746821128 +0000 UTC m=+0.362542419 container start e6a7191c4a7f10e452b2625b6d3ecc0e4bb9d72152f0878614ff608d02c1fd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:52:03.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T04:52:03.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev), process ceph-mon, pid 2 2024-04-02T04:52:03.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: pidfile_write: ignore empty --pid-file 2024-04-02T04:52:03.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: load: jerasure load: lrc 2024-04-02T04:52:03.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: RocksDB version: 7.9.2 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Git sha 0 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Compile date 2024-04-01 15:23:26 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: DB SUMMARY 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: DB Session ID: 3HHXVAAIGDVO2U5VVDR3 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: CURRENT file: CURRENT 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi070/store.db dir, Total Num: 1, files: 000032.sst 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi070/store.db: 000030.log size: 3749068 ; 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.error_if_exists: 0 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.create_if_missing: 0 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.paranoid_checks: 1 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T04:52:03.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.env: 0x55da201eac60 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.info_log: 0x55da215ed9a0 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.statistics: (nil) 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.use_fsync: 0 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_log_file_size: 0 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.allow_fallocate: 1 2024-04-02T04:52:03.032 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T04:52:03.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T04:52:03.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.use_direct_reads: 0 2024-04-02T04:52:03.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T04:52:03.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T04:52:03.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.db_log_dir: 2024-04-02T04:52:03.033 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.wal_dir: 2024-04-02T04:52:03.034 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.write_buffer_manager: 0x55da215c01e0 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T04:52:03.035 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.unordered_write: 0 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.row_cache: None 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.wal_filter: None 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.two_write_queues: 0 2024-04-02T04:52:03.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.wal_compression: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.atomic_flush: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.log_readahead_size: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T04:52:03.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_background_jobs: 2 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_background_compactions: -1 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_subcompactions: 1 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_open_files: -1 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T04:52:03.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_background_flushes: -1 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Compression algorithms supported: 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: kZSTD supported: 0 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: kXpressCompression supported: 0 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: kZlibCompression supported: 1 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: kSnappyCompression supported: 1 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: kLZ4Compression supported: 1 2024-04-02T04:52:03.039 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: kBZip2Compression supported: 0 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000015 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.merge_operator: 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_filter: None 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55da215edaa0) 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-04-02T04:52:03.040 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_type: 0 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_index_type: 0 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_shortening: 1 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: checksum: 4 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: no_block_cache: 0 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache: 0x55da215f1090 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_options: 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: capacity : 536870912 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_shard_bits : 4 2024-04-02T04:52:03.041 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: strict_capacity_limit : 0 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_compressed: (nil) 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: persistent_cache: (nil) 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size: 4096 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size_deviation: 10 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_restart_interval: 16 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_block_restart_interval: 1 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_block_size: 4096 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: partition_filters: 0 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: use_delta_encoding: 1 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: filter_policy: bloomfilter 2024-04-02T04:52:03.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: whole_key_filtering: 1 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: verify_compression: 0 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: format_version: 5 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_index_compression: 1 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_align: 0 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_auto_readahead_size: 262144 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: prepopulate_block_cache: 0 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: initial_auto_readahead_size: 8192 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression: NoCompression 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T04:52:03.043 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.num_levels: 7 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:52:03.044 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T04:52:03.045 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T04:52:03.046 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T04:52:03.047 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.table_properties_collectors: 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.inplace_update_support: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.bloom_locality: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.max_successive_merges: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.ttl: 2592000 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T04:52:03.048 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.enable_blob_files: false 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.min_blob_size: 0 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12067, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-04-02T04:52:03.049 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0c88179a-63b2-4520-a499-6be02674e2c2 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033522775634, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033522787180, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 3251656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12068, "largest_seqno": 15289, "table_properties": {"data_size": 3239428, "index_size": 7623, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3589, "raw_key_size": 31198, "raw_average_key_size": 21, "raw_value_size": 3211569, "raw_average_value_size": 2253, "num_data_blocks": 347, "num_entries": 1425, "num_filter_entries": 1425, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712033522, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0c88179a-63b2-4520-a499-6be02674e2c2", "db_session_id": "3HHXVAAIGDVO2U5VVDR3", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033522787305, "job": 1, "event": "recovery_finished"} 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi070/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55da216e2000 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: DB pointer 0x55da216cc000 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** DB Stats ** 2024-04-02T04:52:03.050 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L0 1/0 3.10 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 599.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L6 1/0 5.66 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Sum 2/0 8.76 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 599.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-02T04:52:03.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 599.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 599.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T04:52:03.052 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative compaction: 0.00 GB write, 222.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval compaction: 0.00 GB write, 222.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache BinnedLRUCache@0x55da215f1090#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: starting mon.smithi070 rank 0 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi070 fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: mon.smithi070@-1(???) e2 preinit fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: mon.smithi070@-1(???).mds e14 new map 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: mon.smithi070@-1(???).mds e14 print_map 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: e14 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:52:03.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: legacy client fscid: 1 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Filesystem 'cephfs' (1) 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: fs_name cephfs 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: epoch 13 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: tableserver 0 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: root 0 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: session_timeout 60 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: session_autoclose 300 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_file_size 1099511627776 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_xattr_size 65536 2024-04-02T04:52:03.054 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: required_client_features {} 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: last_failure 0 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: last_failure_osd_epoch 0 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_mds 1 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: in 0 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: up {0=14468} 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: failed 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: damaged 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: stopped 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_pools [3] 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_pool 2 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: inline_data disabled 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: balancer 2024-04-02T04:52:03.055 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: bal_rank_mask -1 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: standby_count_wanted 1 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: qdb_cluster leader: 0 members: 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Standby daemons: 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: mon.smithi070@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: mon.smithi070@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: mon.smithi070@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: mon.smithi070@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:52:03.056 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 ceph-mon[113635]: mon.smithi070@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-04-02T04:52:03.057 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 bash[113616]: e6a7191c4a7f10e452b2625b6d3ecc0e4bb9d72152f0878614ff608d02c1fd14 2024-04-02T04:52:03.057 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:02 smithi070 systemd[1]: Started Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: pgmap v103: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: mon.smithi070 calling monitor election 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: mon.smithi070 is new leader, mons smithi070,smithi179 in quorum (ranks 0,1) 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: monmap epoch 2 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: last_changed 2024-04-02T04:40:26.229830+0000 2024-04-02T04:52:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: created 2024-04-02T04:37:39.761198+0000 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: min_mon_release 18 (reef) 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: election_strategy: 1 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.smithi179 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: osdmap e48: 6 total, 6 up, 6 in 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: mgrmap e31: smithi070.igidvr(active, since 3m), standbys: smithi179.labmvs 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: overall HEALTH_OK 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: from='mgr.24423 ' entity='' 2024-04-02T04:52:04.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:04 smithi070 ceph-mon[113635]: mgrmap e32: smithi070.igidvr(active, since 3m), standbys: smithi179.labmvs 2024-04-02T04:52:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:52:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: from='mgr.24423 172.21.15.70:0/2106770499' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:52:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: pgmap v103: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: mon.smithi070 calling monitor election 2024-04-02T04:52:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: mon.smithi070 is new leader, mons smithi070,smithi179 in quorum (ranks 0,1) 2024-04-02T04:52:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: monmap epoch 2 2024-04-02T04:52:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:52:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: last_changed 2024-04-02T04:40:26.229830+0000 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: created 2024-04-02T04:37:39.761198+0000 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: min_mon_release 18 (reef) 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: election_strategy: 1 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.smithi179 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: osdmap e48: 6 total, 6 up, 6 in 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: mgrmap e31: smithi070.igidvr(active, since 3m), standbys: smithi179.labmvs 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: overall HEALTH_OK 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: from='mgr.24423 ' entity='' 2024-04-02T04:52:04.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:04 smithi179 ceph-mon[36603]: mgrmap e32: smithi070.igidvr(active, since 3m), standbys: smithi179.labmvs 2024-04-02T04:52:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: Active manager daemon smithi070.igidvr restarted 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: Activating manager daemon smithi070.igidvr 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: mgrmap e33: smithi070.igidvr(active, starting, since 0.00574544s), standbys: smithi179.labmvs 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi179.labmvs", "id": "smithi179.labmvs"}]: dispatch 2024-04-02T04:52:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: Standby manager daemon smithi179.labmvs restarted 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: Manager daemon smithi070.igidvr is now available 2024-04-02T04:52:11.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.? 172.21.15.179:0/3402672908' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:52:11.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.? 172.21.15.179:0/3402672908' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:52:11.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.? 172.21.15.179:0/3402672908' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:52:11.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.? 172.21.15.179:0/3402672908' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:52:11.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: Active manager daemon smithi070.igidvr restarted 2024-04-02T04:52:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: Activating manager daemon smithi070.igidvr 2024-04-02T04:52:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: mgrmap e33: smithi070.igidvr(active, starting, since 0.00574544s), standbys: smithi179.labmvs 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi070.igidvr", "id": "smithi070.igidvr"}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr metadata", "who": "smithi179.labmvs", "id": "smithi179.labmvs"}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:52:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: Standby manager daemon smithi179.labmvs restarted 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: Standby manager daemon smithi179.labmvs started 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: Manager daemon smithi070.igidvr is now available 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/3402672908' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/crt"}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/3402672908' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/3402672908' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi179.labmvs/key"}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.? 172.21.15.179:0/3402672908' entity='mgr.smithi179.labmvs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:52:11.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:11 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:52:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/trash_purge_schedule"}]: dispatch 2024-04-02T04:52:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:12 smithi070 ceph-mon[113635]: mgrmap e34: smithi070.igidvr(active, since 1.01005s), standbys: smithi179.labmvs 2024-04-02T04:52:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:12 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:52:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:12 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:12 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.igidvr/trash_purge_schedule"}]: dispatch 2024-04-02T04:52:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:12 smithi179 ceph-mon[36603]: mgrmap e34: smithi070.igidvr(active, since 1.01005s), standbys: smithi179.labmvs 2024-04-02T04:52:13.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:13 smithi070 ceph-mon[113635]: [02/Apr/2024:04:52:12] ENGINE Bus STARTING 2024-04-02T04:52:13.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:13 smithi070 ceph-mon[113635]: [02/Apr/2024:04:52:12] ENGINE Serving on http://172.21.15.70:8765 2024-04-02T04:52:13.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:13 smithi179 ceph-mon[36603]: [02/Apr/2024:04:52:12] ENGINE Bus STARTING 2024-04-02T04:52:13.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:13 smithi179 ceph-mon[36603]: [02/Apr/2024:04:52:12] ENGINE Serving on http://172.21.15.70:8765 2024-04-02T04:52:14.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:14 smithi070 ceph-mon[113635]: [02/Apr/2024:04:52:12] ENGINE Serving on https://172.21.15.70:7150 2024-04-02T04:52:14.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:14 smithi070 ceph-mon[113635]: [02/Apr/2024:04:52:12] ENGINE Bus STARTED 2024-04-02T04:52:14.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:14 smithi070 ceph-mon[113635]: pgmap v4: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:14.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:14 smithi070 ceph-mon[113635]: mgrmap e35: smithi070.igidvr(active, since 2s), standbys: smithi179.labmvs 2024-04-02T04:52:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:14 smithi179 ceph-mon[36603]: [02/Apr/2024:04:52:12] ENGINE Serving on https://172.21.15.70:7150 2024-04-02T04:52:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:14 smithi179 ceph-mon[36603]: [02/Apr/2024:04:52:12] ENGINE Bus STARTED 2024-04-02T04:52:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:14 smithi179 ceph-mon[36603]: pgmap v4: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:14 smithi179 ceph-mon[36603]: mgrmap e35: smithi070.igidvr(active, since 2s), standbys: smithi179.labmvs 2024-04-02T04:52:15.379 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:15 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:15.379 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:15 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:15.379 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:15 smithi070 ceph-mon[113635]: pgmap v5: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:15.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:15 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:15.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:15 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:15.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:15 smithi179 ceph-mon[36603]: pgmap v5: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:16.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:16 smithi070 ceph-mon[113635]: mgrmap e36: smithi070.igidvr(active, since 4s), standbys: smithi179.labmvs 2024-04-02T04:52:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:16 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:16 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:16.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:16 smithi179 ceph-mon[36603]: mgrmap e36: smithi070.igidvr(active, since 4s), standbys: smithi179.labmvs 2024-04-02T04:52:16.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:16 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:16.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:16 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:18.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:18.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:17 smithi070 ceph-mon[113635]: pgmap v6: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:17 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:17 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:17 smithi179 ceph-mon[36603]: pgmap v6: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:19.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:18 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:19.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:18 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:19.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:18 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:19.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:18 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:19.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:19 smithi070 ceph-mon[113635]: pgmap v7: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-02T04:52:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:19 smithi179 ceph-mon[36603]: pgmap v7: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-02T04:52:22.303 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: pgmap v8: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: Detected new or changed devices on smithi070 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: pgmap v8: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: Detected new or changed devices on smithi070 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: Updating smithi070:/etc/ceph/ceph.conf 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-02T04:52:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:22 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:23 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: Updating smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: Updating smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:23 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-02T04:52:24.504 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:24 smithi179 ceph-mon[36603]: pgmap v9: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T04:52:24.504 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:24 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:24.504 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:24 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:52:24.504 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:24 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:52:24.504 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:24 smithi179 ceph-mon[36603]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:24 smithi070 ceph-mon[113635]: pgmap v9: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T04:52:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:52:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:52:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:25.526 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:25 smithi179 ceph-mon[36603]: Upgrade: Updating mon.smithi179 2024-04-02T04:52:25.527 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:25 smithi179 ceph-mon[36603]: Deploying daemon mon.smithi179 on smithi179 2024-04-02T04:52:25.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:25 smithi070 ceph-mon[113635]: Upgrade: Updating mon.smithi179 2024-04-02T04:52:25.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:25 smithi070 ceph-mon[113635]: Deploying daemon mon.smithi179 on smithi179 2024-04-02T04:52:26.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:25 smithi179 systemd[1]: Stopping Ceph mon.smithi179 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:52:26.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179[36585]: 2024-04-02T04:52:26.051+0000 7f9f75356700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi179 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:52:26.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179[36585]: 2024-04-02T04:52:26.051+0000 7f9f75356700 -1 mon.smithi179@1(peon) e2 *** Got Signal Terminated *** 2024-04-02T04:52:26.498 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 podman[84123]: 2024-04-02 04:52:26.226828174 +0000 UTC m=+0.277193298 container died eeb4d97171fb2c62fdae72247f6dd1fcad069290409946e2d8fbc42183b5846b (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179, RELEASE=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-02T04:52:26.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 podman[84123]: 2024-04-02 04:52:26.594244964 +0000 UTC m=+0.644610123 container cleanup eeb4d97171fb2c62fdae72247f6dd1fcad069290409946e2d8fbc42183b5846b (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179, ceph=True, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-02T04:52:26.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 bash[84123]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179 2024-04-02T04:52:27.170 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 podman[84136]: 2024-04-02 04:52:26.845218204 +0000 UTC m=+0.614323257 container remove eeb4d97171fb2c62fdae72247f6dd1fcad069290409946e2d8fbc42183b5846b (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-04-02T04:52:27.170 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi179.service: Deactivated successfully. 2024-04-02T04:52:27.171 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 systemd[1]: Stopped Ceph mon.smithi179 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:52:27.171 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:26 smithi179 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi179.service: Consumed 7.811s CPU time. 2024-04-02T04:52:27.491 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 systemd[1]: Starting Ceph mon.smithi179 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:52:27.491 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 podman[84268]: 2024-04-02 04:52:27.371168952 +0000 UTC m=+0.105833985 container create 1cbe8f43e3c6580d1fcb86827be37da2443e614db854edce97e8bc203368bcc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:52:27.491 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 podman[84268]: 2024-04-02 04:52:27.284260275 +0000 UTC m=+0.018925333 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:52:27.651 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 podman[84268]: 2024-04-02 04:52:27.615539632 +0000 UTC m=+0.350204683 container init 1cbe8f43e3c6580d1fcb86827be37da2443e614db854edce97e8bc203368bcc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux ) 2024-04-02T04:52:27.651 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 podman[84268]: 2024-04-02 04:52:27.618456097 +0000 UTC m=+0.353121132 container start 1cbe8f43e3c6580d1fcb86827be37da2443e614db854edce97e8bc203368bcc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-02T04:52:27.651 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev), process ceph-mon, pid 2 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: pidfile_write: ignore empty --pid-file 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: load: jerasure load: lrc 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: RocksDB version: 7.9.2 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Git sha 0 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Compile date 2024-04-01 15:23:26 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: DB SUMMARY 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: DB Session ID: 61NUTSRHNCO0ZD1JDSTC 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: CURRENT file: CURRENT 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi179/store.db dir, Total Num: 1, files: 000027.sst 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi179/store.db: 000025.log size: 8239827 ; 2024-04-02T04:52:27.652 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.error_if_exists: 0 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.create_if_missing: 0 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.paranoid_checks: 1 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.env: 0x5625269d1c60 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.info_log: 0x562527c3f9a0 2024-04-02T04:52:27.653 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.statistics: (nil) 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.use_fsync: 0 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_log_file_size: 0 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.allow_fallocate: 1 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.use_direct_reads: 0 2024-04-02T04:52:27.654 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.db_log_dir: 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.wal_dir: 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.write_buffer_manager: 0x562527c121e0 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T04:52:27.655 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.unordered_write: 0 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.row_cache: None 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.wal_filter: None 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.two_write_queues: 0 2024-04-02T04:52:27.656 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T04:52:27.657 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.wal_compression: 0 2024-04-02T04:52:27.657 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.atomic_flush: 0 2024-04-02T04:52:27.657 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T04:52:27.657 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T04:52:27.657 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T04:52:27.746 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.log_readahead_size: 0 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_background_jobs: 2 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_background_compactions: -1 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_subcompactions: 1 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T04:52:27.747 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_open_files: -1 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_background_flushes: -1 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Compression algorithms supported: 2024-04-02T04:52:27.748 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: kZSTD supported: 0 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: kXpressCompression supported: 0 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: kZlibCompression supported: 1 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: kSnappyCompression supported: 1 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: kLZ4Compression supported: 1 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: kBZip2Compression supported: 0 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi179/store.db/MANIFEST-000010 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T04:52:27.749 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.merge_operator: 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_filter: None 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562527c3faa0) 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: cache_index_and_filter_blocks: 1 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: pin_top_level_index_and_filter: 1 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: index_type: 0 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: data_block_index_type: 0 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: index_shortening: 1 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T04:52:27.750 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: checksum: 4 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: no_block_cache: 0 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: block_cache: 0x562527c43090 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: block_cache_name: BinnedLRUCache 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: block_cache_options: 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: capacity : 536870912 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: num_shard_bits : 4 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: strict_capacity_limit : 0 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: high_pri_pool_ratio: 0.000 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: block_cache_compressed: (nil) 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: persistent_cache: (nil) 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: block_size: 4096 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: block_size_deviation: 10 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: block_restart_interval: 16 2024-04-02T04:52:27.751 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: index_block_restart_interval: 1 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: metadata_block_size: 4096 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: partition_filters: 0 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: use_delta_encoding: 1 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: filter_policy: bloomfilter 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: whole_key_filtering: 1 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: verify_compression: 0 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: read_amp_bytes_per_bit: 0 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: format_version: 5 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: enable_index_compression: 1 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: block_align: 0 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: max_auto_readahead_size: 262144 2024-04-02T04:52:27.752 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: prepopulate_block_cache: 0 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: initial_auto_readahead_size: 8192 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression: NoCompression 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.num_levels: 7 2024-04-02T04:52:27.753 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T04:52:27.754 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T04:52:27.754 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T04:52:27.754 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T04:52:27.754 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:52:27.755 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T04:52:27.756 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T04:52:27.757 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.table_properties_collectors: 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.inplace_update_support: 0 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.bloom_locality: 0 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.max_successive_merges: 0 2024-04-02T04:52:27.758 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.ttl: 2592000 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.enable_blob_files: false 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.min_blob_size: 0 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T04:52:27.759 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi179/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12129, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e8326977-cccb-4df3-aba1-6894e1308134 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033547647831, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033547669028, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4885224, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12134, "largest_seqno": 16185, "table_properties": {"data_size": 4871033, "index_size": 9074, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4101, "raw_key_size": 35964, "raw_average_key_size": 22, "raw_value_size": 4838801, "raw_average_value_size": 2970, "num_data_blocks": 411, "num_entries": 1629, "num_filter_entries": 1629, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712033547, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e8326977-cccb-4df3-aba1-6894e1308134", "db_session_id": "61NUTSRHNCO0ZD1JDSTC", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033547669142, "job": 1, "event": "recovery_finished"} 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi179/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562527d34000 2024-04-02T04:52:27.760 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: DB pointer 0x562527d1e000 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: ** DB Stats ** 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: ** Compaction Stats [default] ** 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T04:52:27.761 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: L0 1/0 4.66 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 529.3 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: L6 1/0 5.66 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Sum 2/0 10.32 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 529.3 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 529.3 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: ** Compaction Stats [default] ** 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 529.3 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T04:52:27.762 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Cumulative compaction: 0.00 GB write, 195.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Interval compaction: 0.00 GB write, 195.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Block cache BinnedLRUCache@0x562527c43090#2 capacity: 512.00 MB usage: 13.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: starting mon.smithi179 rank 1 at public addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] at bind addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi179 fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: mon.smithi179@-1(???) e2 preinit fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: mon.smithi179@-1(???).mds e14 new map 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: mon.smithi179@-1(???).mds e14 print_map 2024-04-02T04:52:27.763 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: e14 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: legacy client fscid: 1 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Filesystem 'cephfs' (1) 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: fs_name cephfs 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: epoch 13 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: tableserver 0 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: root 0 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: session_timeout 60 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: session_autoclose 300 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: max_file_size 1099511627776 2024-04-02T04:52:27.764 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: max_xattr_size 65536 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: required_client_features {} 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: last_failure 0 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: last_failure_osd_epoch 0 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: max_mds 1 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: in 0 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: up {0=14468} 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: failed 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: damaged 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: stopped 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: data_pools [3] 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: metadata_pool 2 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: inline_data disabled 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: balancer 2024-04-02T04:52:27.765 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: bal_rank_mask -1 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: standby_count_wanted 1 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: qdb_cluster leader: 0 members: 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: [mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: Standby daemons: 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: [mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: [mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout: [mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: mon.smithi179@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: mon.smithi179@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: mon.smithi179@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:52:27.766 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: mon.smithi179@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:52:27.767 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 ceph-mon[84288]: mon.smithi179@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-04-02T04:52:27.767 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 bash[84268]: 1cbe8f43e3c6580d1fcb86827be37da2443e614db854edce97e8bc203368bcc7 2024-04-02T04:52:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:27 smithi179 systemd[1]: Started Ceph mon.smithi179 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.247+0000 7fdf77192700 1 -- 172.21.15.70:0/883892423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf70101ea0 msgr2=0x7fdf70102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.247+0000 7fdf77192700 1 --2- 172.21.15.70:0/883892423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf70101ea0 0x7fdf70102280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009910 tx=0x7fdf64009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.247+0000 7fdf77192700 1 -- 172.21.15.70:0/883892423 shutdown_connections 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.247+0000 7fdf77192700 1 --2- 172.21.15.70:0/883892423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf701027c0 0x7fdf7010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.247+0000 7fdf77192700 1 --2- 172.21.15.70:0/883892423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf70101ea0 0x7fdf70102280 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.247+0000 7fdf77192700 1 -- 172.21.15.70:0/883892423 >> 172.21.15.70:0/883892423 conn(0x7fdf700fb610 msgr2=0x7fdf700fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.247+0000 7fdf77192700 1 -- 172.21.15.70:0/883892423 shutdown_connections 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.247+0000 7fdf77192700 1 -- 172.21.15.70:0/883892423 wait complete. 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf77192700 1 Processor -- start 2024-04-02T04:52:29.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf77192700 1 -- start start 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf77192700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 0x7fdf7019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf77192700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701027c0 0x7fdf7019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf77192700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7019f4c0 con 0x7fdf701027c0 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf77192700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70198970 con 0x7fdf70101ea0 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf6ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701027c0 0x7fdf7019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf6ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701027c0 0x7fdf7019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36154/0 (socket says 172.21.15.70:36154) 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf6ffff700 1 -- 172.21.15.70:0/3768844660 learned_addr learned my addr 172.21.15.70:0/3768844660 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf74f2e700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 0x7fdf7019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf74f2e700 1 -- 172.21.15.70:0/3768844660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 msgr2=0x7fdf7019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf74f2e700 1 -- 172.21.15.70:0/3768844660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 msgr2=0x7fdf7019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf74f2e700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 0x7fdf7019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf74f2e700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 0x7fdf7019e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf6ffff700 1 -- 172.21.15.70:0/3768844660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 msgr2=0x7fdf7019e8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf6ffff700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 0x7fdf7019e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.248+0000 7fdf6ffff700 1 -- 172.21.15.70:0/3768844660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640095f0 con 0x7fdf701027c0 2024-04-02T04:52:29.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.249+0000 7fdf6ffff700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701027c0 0x7fdf7019ee30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c8f0 tx=0x7fdf5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:29.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.250+0000 7fdf6dffb700 1 -- 172.21.15.70:0/3768844660 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf5c007a50 con 0x7fdf701027c0 2024-04-02T04:52:29.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.250+0000 7fdf6dffb700 1 -- 172.21.15.70:0/3768844660 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf5c007bb0 con 0x7fdf701027c0 2024-04-02T04:52:29.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.250+0000 7fdf6dffb700 1 -- 172.21.15.70:0/3768844660 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf5c018630 con 0x7fdf701027c0 2024-04-02T04:52:29.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.250+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf70198c50 con 0x7fdf701027c0 2024-04-02T04:52:29.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.250+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf701991a0 con 0x7fdf701027c0 2024-04-02T04:52:29.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.251+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7010a810 con 0x7fdf701027c0 2024-04-02T04:52:29.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.251+0000 7fdf6dffb700 1 -- 172.21.15.70:0/3768844660 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fdf5c007d20 con 0x7fdf701027c0 2024-04-02T04:52:29.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.252+0000 7fdf6dffb700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006de90 0x7fdf60070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.252+0000 7fdf6dffb700 1 -- 172.21.15.70:0/3768844660 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdf5c08dc00 con 0x7fdf701027c0 2024-04-02T04:52:29.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.252+0000 7fdf74f2e700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006de90 0x7fdf60070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.252+0000 7fdf74f2e700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006de90 0x7fdf60070350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf6401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:29.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.257+0000 7fdf6dffb700 1 -- 172.21.15.70:0/3768844660 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdf5c093020 con 0x7fdf701027c0 2024-04-02T04:52:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.470+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf70199f20 con 0x7fdf6006de90 2024-04-02T04:52:29.475 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:29 smithi070 ceph-mon[113635]: pgmap v10: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-02T04:52:29.475 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:29 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.472+0000 7fdf6dffb700 1 -- 172.21.15.70:0/3768844660 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fdf70199f20 con 0x7fdf6006de90 2024-04-02T04:52:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006de90 msgr2=0x7fdf60070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006de90 0x7fdf60070350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf6401a040 comp rx=0 tx=0).stop 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701027c0 msgr2=0x7fdf7019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701027c0 0x7fdf7019ee30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c8f0 tx=0x7fdf5c00ccb0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 shutdown_connections 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006de90 0x7fdf60070350 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf70101ea0 0x7fdf7019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 --2- 172.21.15.70:0/3768844660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf701027c0 0x7fdf7019ee30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 >> 172.21.15.70:0/3768844660 conn(0x7fdf700fb610 msgr2=0x7fdf701002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 shutdown_connections 2024-04-02T04:52:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.474+0000 7fdf77192700 1 -- 172.21.15.70:0/3768844660 wait complete. 2024-04-02T04:52:29.488 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:52:29.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:29 smithi179 ceph-mon[84288]: pgmap v10: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-02T04:52:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:29 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:29.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.577+0000 7f1dbf48b700 1 -- 172.21.15.70:0/1384510801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db80ffda0 msgr2=0x7f1db8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.577+0000 7f1dbf48b700 1 --2- 172.21.15.70:0/1384510801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db80ffda0 0x7f1db8100180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009910 tx=0x7f1da8009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.577+0000 7f1dbf48b700 1 -- 172.21.15.70:0/1384510801 shutdown_connections 2024-04-02T04:52:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.577+0000 7f1dbf48b700 1 --2- 172.21.15.70:0/1384510801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db8100750 0x7f1db81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.577+0000 7f1dbf48b700 1 --2- 172.21.15.70:0/1384510801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db80ffda0 0x7f1db8100180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.577+0000 7f1dbf48b700 1 -- 172.21.15.70:0/1384510801 >> 172.21.15.70:0/1384510801 conn(0x7f1db80fb650 msgr2=0x7f1db80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.577+0000 7f1dbf48b700 1 -- 172.21.15.70:0/1384510801 shutdown_connections 2024-04-02T04:52:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.577+0000 7f1dbf48b700 1 -- 172.21.15.70:0/1384510801 wait complete. 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbf48b700 1 Processor -- start 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbf48b700 1 -- start start 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbf48b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 0x7f1db8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbf48b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db8100750 0x7f1db8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbf48b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db81022c0 con 0x7f1db8100750 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbf48b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db8102400 con 0x7f1db80ffda0 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbca26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db8100750 0x7f1db8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbca26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db8100750 0x7f1db8101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36174/0 (socket says 172.21.15.70:36174) 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbca26700 1 -- 172.21.15.70:0/775022325 learned_addr learned my addr 172.21.15.70:0/775022325 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.578+0000 7f1dbd227700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 0x7f1db8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbd227700 1 -- 172.21.15.70:0/775022325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 msgr2=0x7f1db8101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbd227700 1 -- 172.21.15.70:0/775022325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 msgr2=0x7f1db8101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T04:52:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbd227700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 0x7f1db8101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbd227700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 0x7f1db8101840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbca26700 1 -- 172.21.15.70:0/775022325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 msgr2=0x7f1db8101840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbca26700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 0x7f1db8101840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbca26700 1 -- 172.21.15.70:0/775022325 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da80095f0 con 0x7f1db8100750 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbca26700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db8100750 0x7f1db8101d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1db400d900 tx=0x7f1db400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dae7fc700 1 -- 172.21.15.70:0/775022325 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1db4004a00 con 0x7f1db8100750 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db81a2af0 con 0x7f1db8100750 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dae7fc700 1 -- 172.21.15.70:0/775022325 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1db40054e0 con 0x7f1db8100750 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dae7fc700 1 -- 172.21.15.70:0/775022325 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1db4009d00 con 0x7f1db8100750 2024-04-02T04:52:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.579+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db81a2f00 con 0x7f1db8100750 2024-04-02T04:52:29.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.580+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db804f2e0 con 0x7f1db8100750 2024-04-02T04:52:29.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.580+0000 7f1dae7fc700 1 -- 172.21.15.70:0/775022325 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f1db4005020 con 0x7f1db8100750 2024-04-02T04:52:29.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.581+0000 7f1dae7fc700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1da406de90 0x7f1da4070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.581+0000 7f1dbd227700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1da406de90 0x7f1da4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.581+0000 7f1dae7fc700 1 -- 172.21.15.70:0/775022325 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1db408df60 con 0x7f1db8100750 2024-04-02T04:52:29.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.581+0000 7f1dbd227700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1da406de90 0x7f1da4070350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1da8006010 tx=0x7f1da801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:29.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.586+0000 7f1dae7fc700 1 -- 172.21.15.70:0/775022325 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1db40c2280 con 0x7f1db8100750 2024-04-02T04:52:29.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.800+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1db80fc460 con 0x7f1da406de90 2024-04-02T04:52:29.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.802+0000 7f1dae7fc700 1 -- 172.21.15.70:0/775022325 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f1db80fc460 con 0x7f1da406de90 2024-04-02T04:52:29.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1da406de90 msgr2=0x7f1da4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1da406de90 0x7f1da4070350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1da8006010 tx=0x7f1da801a040 comp rx=0 tx=0).stop 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db8100750 msgr2=0x7f1db8101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db8100750 0x7f1db8101d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1db400d900 tx=0x7f1db400dcc0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 shutdown_connections 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1da406de90 0x7f1da4070350 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1db80ffda0 0x7f1db8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 --2- 172.21.15.70:0/775022325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1db8100750 0x7f1db8101d80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 >> 172.21.15.70:0/775022325 conn(0x7f1db80fb650 msgr2=0x7f1db8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 shutdown_connections 2024-04-02T04:52:29.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.804+0000 7f1dbf48b700 1 -- 172.21.15.70:0/775022325 wait complete. 2024-04-02T04:52:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 -- 172.21.15.70:0/3718498467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 msgr2=0x7f3be0105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 --2- 172.21.15.70:0/3718498467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 0x7f3be0105730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009910 tx=0x7f3bd0009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 -- 172.21.15.70:0/3718498467 shutdown_connections 2024-04-02T04:52:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 --2- 172.21.15.70:0/3718498467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 0x7f3be00ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 --2- 172.21.15.70:0/3718498467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 0x7f3be0105730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 -- 172.21.15.70:0/3718498467 >> 172.21.15.70:0/3718498467 conn(0x7f3be0074e30 msgr2=0x7f3be0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 -- 172.21.15.70:0/3718498467 shutdown_connections 2024-04-02T04:52:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 -- 172.21.15.70:0/3718498467 wait complete. 2024-04-02T04:52:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 Processor -- start 2024-04-02T04:52:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.909+0000 7f3be6642700 1 -- start start 2024-04-02T04:52:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3be6642700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 0x7f3be019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3be6642700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 0x7f3be019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3be6642700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019f4b0 con 0x7f3be0105350 2024-04-02T04:52:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3be6642700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be0198960 con 0x7f3be00ff340 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 0x7f3be019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 0x7f3be019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36194/0 (socket says 172.21.15.70:36194) 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdf7fe700 1 -- 172.21.15.70:0/4091369319 learned_addr learned my addr 172.21.15.70:0/4091369319 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdffff700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 0x7f3be019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdffff700 1 -- 172.21.15.70:0/4091369319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 msgr2=0x7f3be019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdffff700 1 -- 172.21.15.70:0/4091369319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 msgr2=0x7f3be019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdffff700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 0x7f3be019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdffff700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 0x7f3be019e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdf7fe700 1 -- 172.21.15.70:0/4091369319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 msgr2=0x7f3be019e8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdf7fe700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 0x7f3be019e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdf7fe700 1 -- 172.21.15.70:0/4091369319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00095f0 con 0x7f3be0105350 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdf7fe700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 0x7f3be019ee20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400d8d0 tx=0x7f3bd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.910+0000 7f3bdd7fa700 1 -- 172.21.15.70:0/4091369319 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bd4009940 con 0x7f3be0105350 2024-04-02T04:52:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.911+0000 7f3bdd7fa700 1 -- 172.21.15.70:0/4091369319 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bd4010460 con 0x7f3be0105350 2024-04-02T04:52:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.911+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be0198c40 con 0x7f3be0105350 2024-04-02T04:52:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.911+0000 7f3bdd7fa700 1 -- 172.21.15.70:0/4091369319 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bd400f5f0 con 0x7f3be0105350 2024-04-02T04:52:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.911+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be0199190 con 0x7f3be0105350 2024-04-02T04:52:29.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.911+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be004f2e0 con 0x7f3be0105350 2024-04-02T04:52:29.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.912+0000 7f3bdd7fa700 1 -- 172.21.15.70:0/4091369319 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f3bd40105d0 con 0x7f3be0105350 2024-04-02T04:52:29.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.912+0000 7f3bdd7fa700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3bc806de90 0x7f3bc8070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:29.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.912+0000 7f3bdd7fa700 1 -- 172.21.15.70:0/4091369319 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3bd408dd20 con 0x7f3be0105350 2024-04-02T04:52:29.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.912+0000 7f3bdffff700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3bc806de90 0x7f3bc8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:29.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.913+0000 7f3bdffff700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3bc806de90 0x7f3bc8070350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0006010 tx=0x7f3bd001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:29.917+0000 7f3bdd7fa700 1 -- 172.21.15.70:0/4091369319 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3bd405ac70 con 0x7f3be0105350 2024-04-02T04:52:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.119+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3be01993b0 con 0x7f3bc806de90 2024-04-02T04:52:30.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.130+0000 7f3bdd7fa700 1 -- 172.21.15.70:0/4091369319 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f3be01993b0 con 0x7f3bc806de90 2024-04-02T04:52:30.134 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:52:30.134 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (2m) 13s ago 13m 17.8M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (13m) 13s ago 13m 8070k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (12m) 15s ago 12m 7734k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (13m) 13s ago 13m 6669k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (12m) 15s ago 12m 6669k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (101s) 13s ago 12m 75.9M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (7m) 13s ago 7m 153M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (7m) 13s ago 7m 18.9M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (7m) 15s ago 7m 16.9M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (7m) 15s ago 7m 18.5M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (4m) 13s ago 14m 471M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (3m) 15s ago 12m 447M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (27s) 13s ago 14m 29.1M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:52:30.135 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 starting - - - 2048M 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (2m) 13s ago 13m 15.7M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (2m) 15s ago 12m 16.9M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (11m) 13s ago 11m 229M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (10m) 13s ago 10m 264M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (10m) 13s ago 10m 211M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (9m) 15s ago 9m 298M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (9m) 15s ago 9m 274M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (9m) 15s ago 9m 229M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:52:30.136 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (2m) 13s ago 12m 60.7M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:52:30.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3bc806de90 msgr2=0x7f3bc8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3bc806de90 0x7f3bc8070350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0006010 tx=0x7f3bd001a040 comp rx=0 tx=0).stop 2024-04-02T04:52:30.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 msgr2=0x7f3be019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 0x7f3be019ee20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400d8d0 tx=0x7f3bd400dc90 comp rx=0 tx=0).stop 2024-04-02T04:52:30.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 shutdown_connections 2024-04-02T04:52:30.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3bc806de90 0x7f3bc8070350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3be00ff340 0x7f3be019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 --2- 172.21.15.70:0/4091369319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3be0105350 0x7f3be019ee20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 >> 172.21.15.70:0/4091369319 conn(0x7f3be0074e30 msgr2=0x7f3be00fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:30.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 shutdown_connections 2024-04-02T04:52:30.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.132+0000 7f3be6642700 1 -- 172.21.15.70:0/4091369319 wait complete. 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 -- 172.21.15.70:0/3472508716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098106cd0 msgr2=0x7f20981070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 --2- 172.21.15.70:0/3472508716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098106cd0 0x7f20981070b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f209000b0a0 tx=0x7f209000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 -- 172.21.15.70:0/3472508716 shutdown_connections 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 --2- 172.21.15.70:0/3472508716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098068580 0x7f20980689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 --2- 172.21.15.70:0/3472508716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098106cd0 0x7f20981070b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 -- 172.21.15.70:0/3472508716 >> 172.21.15.70:0/3472508716 conn(0x7f2098075670 msgr2=0x7f2098075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 -- 172.21.15.70:0/3472508716 shutdown_connections 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 -- 172.21.15.70:0/3472508716 wait complete. 2024-04-02T04:52:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.236+0000 7f209d93f700 1 Processor -- start 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f209d93f700 1 -- start start 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f209d93f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098068580 0x7f209819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f209d93f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 0x7f209819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f209d93f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209819d250 con 0x7f2098068580 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f209d93f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098196700 con 0x7f2098106cd0 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f2096ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098068580 0x7f209819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f2096ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098068580 0x7f209819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36216/0 (socket says 172.21.15.70:36216) 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f2096ffd700 1 -- 172.21.15.70:0/1909121299 learned_addr learned my addr 172.21.15.70:0/1909121299 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f20967fc700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 0x7f209819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f20967fc700 1 -- 172.21.15.70:0/1909121299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 msgr2=0x7f209819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f20967fc700 1 -- 172.21.15.70:0/1909121299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 msgr2=0x7f209819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f20967fc700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 0x7f209819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T04:52:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f20967fc700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 0x7f209819cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f2096ffd700 1 -- 172.21.15.70:0/1909121299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 msgr2=0x7f209819cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f2096ffd700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 0x7f209819cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f2096ffd700 1 -- 172.21.15.70:0/1909121299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090009d00 con 0x7f2098068580 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f2096ffd700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098068580 0x7f209819c680 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f209000b070 tx=0x7f209000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.237+0000 7f209c93d700 1 -- 172.21.15.70:0/1909121299 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209000e040 con 0x7f2098068580 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.238+0000 7f209c93d700 1 -- 172.21.15.70:0/1909121299 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2090003de0 con 0x7f2098068580 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.238+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2098196980 con 0x7f2098068580 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.238+0000 7f209c93d700 1 -- 172.21.15.70:0/1909121299 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209001d8e0 con 0x7f2098068580 2024-04-02T04:52:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.238+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2098196e70 con 0x7f2098068580 2024-04-02T04:52:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.238+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20981091f0 con 0x7f2098068580 2024-04-02T04:52:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.239+0000 7f209c93d700 1 -- 172.21.15.70:0/1909121299 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f2090004040 con 0x7f2098068580 2024-04-02T04:52:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.239+0000 7f209c93d700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f208406de90 0x7f2084070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.239+0000 7f209c93d700 1 -- 172.21.15.70:0/1909121299 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f209008f450 con 0x7f2098068580 2024-04-02T04:52:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.239+0000 7f20967fc700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f208406de90 0x7f2084070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.240+0000 7f20967fc700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f208406de90 0x7f2084070350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2080005950 tx=0x7f20800058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:30.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.244+0000 7f209c93d700 1 -- 172.21.15.70:0/1909121299 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f209005c420 con 0x7f2098068580 2024-04-02T04:52:30.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.520+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f20981971a0 con 0x7f2098068580 2024-04-02T04:52:30.524 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: pgmap v12: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-02T04:52:30.524 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:52:30.524 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: mon.smithi070 calling monitor election 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: mon.smithi179 calling monitor election 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: mon.smithi070 is new leader, mons smithi070,smithi179 in quorum (ranks 0,1) 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: monmap epoch 3 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: last_changed 2024-04-02T04:52:29.222333+0000 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: created 2024-04-02T04:37:39.761198+0000 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: min_mon_release 19 (squid) 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: election_strategy: 1 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.smithi179 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:52:30.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:52:30.526 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: mgrmap e36: smithi070.igidvr(active, since 18s), standbys: smithi179.labmvs 2024-04-02T04:52:30.526 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: overall HEALTH_OK 2024-04-02T04:52:30.526 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:30.526 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:30.526 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:30 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.520+0000 7f209c93d700 1 -- 172.21.15.70:0/1909121299 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f209005bfb0 con 0x7f2098068580 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:52:30.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:52:30.528 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:52:30.528 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:52:30.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:52:30.528 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:52:30.528 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-04-02T04:52:30.528 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:52:30.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:52:30.528 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:52:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f208406de90 msgr2=0x7f2084070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f208406de90 0x7f2084070350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2080005950 tx=0x7f20800058e0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098068580 msgr2=0x7f209819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098068580 0x7f209819c680 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f209000b070 tx=0x7f209000cdd0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 shutdown_connections 2024-04-02T04:52:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f208406de90 0x7f2084070350 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2098068580 0x7f209819c680 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 --2- 172.21.15.70:0/1909121299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098106cd0 0x7f209819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 >> 172.21.15.70:0/1909121299 conn(0x7f2098075670 msgr2=0x7f20980fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 shutdown_connections 2024-04-02T04:52:30.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.523+0000 7f209d93f700 1 -- 172.21.15.70:0/1909121299 wait complete. 2024-04-02T04:52:30.552 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: pgmap v12: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mon metadata", "id": "smithi179"}]: dispatch 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: mon.smithi070 calling monitor election 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: mon.smithi179 calling monitor election 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: mon.smithi070 is new leader, mons smithi070,smithi179 in quorum (ranks 0,1) 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: monmap epoch 3 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: last_changed 2024-04-02T04:52:29.222333+0000 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: created 2024-04-02T04:37:39.761198+0000 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: min_mon_release 19 (squid) 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: election_strategy: 1 2024-04-02T04:52:30.553 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-04-02T04:52:30.554 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.smithi179 2024-04-02T04:52:30.554 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:52:30.554 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:52:30.554 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: mgrmap e36: smithi070.igidvr(active, since 18s), standbys: smithi179.labmvs 2024-04-02T04:52:30.554 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: overall HEALTH_OK 2024-04-02T04:52:30.554 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:30.554 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:30.554 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:30 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:30.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.626+0000 7faf630de700 1 -- 172.21.15.70:0/819163880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 msgr2=0x7faf5c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.626+0000 7faf630de700 1 --2- 172.21.15.70:0/819163880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 0x7faf5c105fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf50009910 tx=0x7faf50009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:30.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.626+0000 7faf630de700 1 -- 172.21.15.70:0/819163880 shutdown_connections 2024-04-02T04:52:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.626+0000 7faf630de700 1 --2- 172.21.15.70:0/819163880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 0x7faf5c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.626+0000 7faf630de700 1 --2- 172.21.15.70:0/819163880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 0x7faf5c105fb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.626+0000 7faf630de700 1 -- 172.21.15.70:0/819163880 >> 172.21.15.70:0/819163880 conn(0x7faf5c075670 msgr2=0x7faf5c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.626+0000 7faf630de700 1 -- 172.21.15.70:0/819163880 shutdown_connections 2024-04-02T04:52:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.626+0000 7faf630de700 1 -- 172.21.15.70:0/819163880 wait complete. 2024-04-02T04:52:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf630de700 1 Processor -- start 2024-04-02T04:52:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf630de700 1 -- start start 2024-04-02T04:52:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf630de700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 0x7faf5c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf630de700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 0x7faf5c197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf630de700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1982f0 con 0x7faf5c105bd0 2024-04-02T04:52:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf630de700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c198460 con 0x7faf5c0ffc20 2024-04-02T04:52:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf5bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 0x7faf5c197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf5bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 0x7faf5c197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36228/0 (socket says 172.21.15.70:36228) 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf5bfff700 1 -- 172.21.15.70:0/2552705829 learned_addr learned my addr 172.21.15.70:0/2552705829 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf60e7a700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 0x7faf5c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf60e7a700 1 -- 172.21.15.70:0/2552705829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 msgr2=0x7faf5c1976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf60e7a700 1 -- 172.21.15.70:0/2552705829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 msgr2=0x7faf5c1976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf60e7a700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 0x7faf5c1976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf60e7a700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 0x7faf5c1976d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf5bfff700 1 -- 172.21.15.70:0/2552705829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 msgr2=0x7faf5c1976d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf5bfff700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 0x7faf5c1976d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf5bfff700 1 -- 172.21.15.70:0/2552705829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500095f0 con 0x7faf5c105bd0 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf5bfff700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 0x7faf5c197c10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf4800d8d0 tx=0x7faf4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.627+0000 7faf59ffb700 1 -- 172.21.15.70:0/2552705829 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf48009940 con 0x7faf5c105bd0 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.628+0000 7faf59ffb700 1 -- 172.21.15.70:0/2552705829 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf48010460 con 0x7faf5c105bd0 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.628+0000 7faf59ffb700 1 -- 172.21.15.70:0/2552705829 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf4800f5f0 con 0x7faf5c105bd0 2024-04-02T04:52:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.628+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c1a0740 con 0x7faf5c105bd0 2024-04-02T04:52:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.628+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c1a0c90 con 0x7faf5c105bd0 2024-04-02T04:52:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.629+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c1080d0 con 0x7faf5c105bd0 2024-04-02T04:52:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.629+0000 7faf59ffb700 1 -- 172.21.15.70:0/2552705829 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7faf480105d0 con 0x7faf5c105bd0 2024-04-02T04:52:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.629+0000 7faf59ffb700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf4c06de90 0x7faf4c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.630+0000 7faf59ffb700 1 -- 172.21.15.70:0/2552705829 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faf4808dd30 con 0x7faf5c105bd0 2024-04-02T04:52:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.630+0000 7faf60e7a700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf4c06de90 0x7faf4c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.630+0000 7faf60e7a700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf4c06de90 0x7faf4c070350 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf50006010 tx=0x7faf5001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.634+0000 7faf59ffb700 1 -- 172.21.15.70:0/2552705829 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faf48057150 con 0x7faf5c105bd0 2024-04-02T04:52:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.876+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf5c04f2e0 con 0x7faf5c105bd0 2024-04-02T04:52:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.877+0000 7faf59ffb700 1 -- 172.21.15.70:0/2552705829 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7faf48016770 con 0x7faf5c105bd0 2024-04-02T04:52:30.881 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:52:30.881 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:52:30.881 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:52:30.882 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:52:30.883 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:30.884 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:52:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf4c06de90 msgr2=0x7faf4c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf4c06de90 0x7faf4c070350 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf50006010 tx=0x7faf5001a040 comp rx=0 tx=0).stop 2024-04-02T04:52:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 msgr2=0x7faf5c197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 0x7faf5c197c10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf4800d8d0 tx=0x7faf4800dc90 comp rx=0 tx=0).stop 2024-04-02T04:52:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 shutdown_connections 2024-04-02T04:52:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf4c06de90 0x7faf4c070350 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf5c0ffc20 0x7faf5c1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 --2- 172.21.15.70:0/2552705829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf5c105bd0 0x7faf5c197c10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 >> 172.21.15.70:0/2552705829 conn(0x7faf5c075670 msgr2=0x7faf5c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 shutdown_connections 2024-04-02T04:52:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.880+0000 7faf630de700 1 -- 172.21.15.70:0/2552705829 wait complete. 2024-04-02T04:52:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:52:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 -- 172.21.15.70:0/1647296233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8101840 msgr2=0x7f92c8105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 --2- 172.21.15.70:0/1647296233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8101840 0x7f92c8105890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009910 tx=0x7f92b8009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 -- 172.21.15.70:0/1647296233 shutdown_connections 2024-04-02T04:52:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 --2- 172.21.15.70:0/1647296233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8101840 0x7f92c8105890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 --2- 172.21.15.70:0/1647296233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8100e90 0x7f92c8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 -- 172.21.15.70:0/1647296233 >> 172.21.15.70:0/1647296233 conn(0x7f92c80fc740 msgr2=0x7f92c80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:30.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 -- 172.21.15.70:0/1647296233 shutdown_connections 2024-04-02T04:52:30.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 -- 172.21.15.70:0/1647296233 wait complete. 2024-04-02T04:52:30.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 Processor -- start 2024-04-02T04:52:30.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.984+0000 7f92cf129700 1 -- start start 2024-04-02T04:52:30.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92cf129700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8100e90 0x7f92c819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92cf129700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 0x7f92c819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92cf129700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c819d240 con 0x7f92c8100e90 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92cf129700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c81966f0 con 0x7f92c8101840 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92ccec5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8100e90 0x7f92c819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92ccec5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8100e90 0x7f92c819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36244/0 (socket says 172.21.15.70:36244) 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92ccec5700 1 -- 172.21.15.70:0/1709701300 learned_addr learned my addr 172.21.15.70:0/1709701300 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92bffff700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 0x7f92c819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92bffff700 1 -- 172.21.15.70:0/1709701300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 msgr2=0x7f92c819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92bffff700 1 -- 172.21.15.70:0/1709701300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 msgr2=0x7f92c819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92bffff700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 0x7f92c819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92bffff700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 0x7f92c819cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92ccec5700 1 -- 172.21.15.70:0/1709701300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 msgr2=0x7f92c819cbb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:52:30.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92ccec5700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 0x7f92c819cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92ccec5700 1 -- 172.21.15.70:0/1709701300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b80095f0 con 0x7f92c8100e90 2024-04-02T04:52:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92ccec5700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8100e90 0x7f92c819c670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92c400d8d0 tx=0x7f92c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92bdffb700 1 -- 172.21.15.70:0/1709701300 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c4009940 con 0x7f92c8100e90 2024-04-02T04:52:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c81969d0 con 0x7f92c8100e90 2024-04-02T04:52:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92bdffb700 1 -- 172.21.15.70:0/1709701300 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92c4010460 con 0x7f92c8100e90 2024-04-02T04:52:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.985+0000 7f92bdffb700 1 -- 172.21.15.70:0/1709701300 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c400f5f0 con 0x7f92c8100e90 2024-04-02T04:52:30.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.986+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c8196f20 con 0x7f92c8100e90 2024-04-02T04:52:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.987+0000 7f92bdffb700 1 -- 172.21.15.70:0/1709701300 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f92c40105d0 con 0x7f92c8100e90 2024-04-02T04:52:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.987+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c8109290 con 0x7f92c8100e90 2024-04-02T04:52:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.987+0000 7f92bdffb700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f92b006dea0 0x7f92b0070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.987+0000 7f92bdffb700 1 -- 172.21.15.70:0/1709701300 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f92c408d780 con 0x7f92c8100e90 2024-04-02T04:52:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.987+0000 7f92bffff700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f92b006dea0 0x7f92b0070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.988+0000 7f92bffff700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f92b006dea0 0x7f92b0070360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f92c8197b60 tx=0x7f92b801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:30.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:30.992+0000 7f92bdffb700 1 -- 172.21.15.70:0/1709701300 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f92c405a6d0 con 0x7f92c8100e90 2024-04-02T04:52:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.203+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92c8197110 con 0x7f92b006dea0 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.205+0000 7f92bdffb700 1 -- 172.21.15.70:0/1709701300 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f92c8197110 con 0x7f92b006dea0 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "3/23 daemons upgraded", 2024-04-02T04:52:31.209 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mon daemons", 2024-04-02T04:52:31.210 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:52:31.210 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:52:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f92b006dea0 msgr2=0x7f92b0070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f92b006dea0 0x7f92b0070360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f92c8197b60 tx=0x7f92b801a040 comp rx=0 tx=0).stop 2024-04-02T04:52:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8100e90 msgr2=0x7f92c819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8100e90 0x7f92c819c670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92c400d8d0 tx=0x7f92c400dc90 comp rx=0 tx=0).stop 2024-04-02T04:52:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 shutdown_connections 2024-04-02T04:52:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f92b006dea0 0x7f92b0070360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c8100e90 0x7f92c819c670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 --2- 172.21.15.70:0/1709701300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92c8101840 0x7f92c819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 >> 172.21.15.70:0/1709701300 conn(0x7f92c80fc740 msgr2=0x7f92c80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 shutdown_connections 2024-04-02T04:52:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.207+0000 7f92cf129700 1 -- 172.21.15.70:0/1709701300 wait complete. 2024-04-02T04:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.311+0000 7f4a4b64a700 1 -- 172.21.15.70:0/2023748654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440ff5a0 msgr2=0x7f4a440ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.311+0000 7f4a4b64a700 1 --2- 172.21.15.70:0/2023748654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440ff5a0 0x7f4a440ff980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009910 tx=0x7f4a34009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.311+0000 7f4a4b64a700 1 -- 172.21.15.70:0/2023748654 shutdown_connections 2024-04-02T04:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.311+0000 7f4a4b64a700 1 --2- 172.21.15.70:0/2023748654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440fff50 0x7f4a44105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.311+0000 7f4a4b64a700 1 --2- 172.21.15.70:0/2023748654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440ff5a0 0x7f4a440ff980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.311+0000 7f4a4b64a700 1 -- 172.21.15.70:0/2023748654 >> 172.21.15.70:0/2023748654 conn(0x7f4a44074e30 msgr2=0x7f4a44075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.312+0000 7f4a4b64a700 1 -- 172.21.15.70:0/2023748654 shutdown_connections 2024-04-02T04:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.312+0000 7f4a4b64a700 1 -- 172.21.15.70:0/2023748654 wait complete. 2024-04-02T04:52:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.312+0000 7f4a4b64a700 1 Processor -- start 2024-04-02T04:52:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.312+0000 7f4a4b64a700 1 -- start start 2024-04-02T04:52:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.312+0000 7f4a4b64a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 0x7f4a44100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.312+0000 7f4a4b64a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440fff50 0x7f4a44100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.312+0000 7f4a4b64a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a44100fa0 con 0x7f4a440fff50 2024-04-02T04:52:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.312+0000 7f4a4b64a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a441010e0 con 0x7f4a440ff5a0 2024-04-02T04:52:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a48be5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440fff50 0x7f4a44100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a48be5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440fff50 0x7f4a44100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36266/0 (socket says 172.21.15.70:36266) 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a48be5700 1 -- 172.21.15.70:0/21204677 learned_addr learned my addr 172.21.15.70:0/21204677 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a493e6700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 0x7f4a44100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a493e6700 1 -- 172.21.15.70:0/21204677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 msgr2=0x7f4a44100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a493e6700 1 -- 172.21.15.70:0/21204677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 msgr2=0x7f4a44100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a493e6700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 0x7f4a44100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a493e6700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 0x7f4a44100520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a48be5700 1 -- 172.21.15.70:0/21204677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 msgr2=0x7f4a44100520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a48be5700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 0x7f4a44100520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a48be5700 1 -- 172.21.15.70:0/21204677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a340095f0 con 0x7f4a440fff50 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a48be5700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440fff50 0x7f4a44100a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4a38007e90 tx=0x7f4a3800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a427fc700 1 -- 172.21.15.70:0/21204677 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a3800dda0 con 0x7f4a440fff50 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a427fc700 1 -- 172.21.15.70:0/21204677 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a3800f040 con 0x7f4a440fff50 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a441a2a80 con 0x7f4a440fff50 2024-04-02T04:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a427fc700 1 -- 172.21.15.70:0/21204677 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a380125f0 con 0x7f4a440fff50 2024-04-02T04:52:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.313+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a441a2e90 con 0x7f4a440fff50 2024-04-02T04:52:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.314+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a44109270 con 0x7f4a440fff50 2024-04-02T04:52:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.315+0000 7f4a427fc700 1 -- 172.21.15.70:0/21204677 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f4a380128b0 con 0x7f4a440fff50 2024-04-02T04:52:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.315+0000 7f4a427fc700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4a2c06dea0 0x7f4a2c070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:31.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.315+0000 7f4a427fc700 1 -- 172.21.15.70:0/21204677 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4a3808d660 con 0x7f4a440fff50 2024-04-02T04:52:31.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.315+0000 7f4a493e6700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4a2c06dea0 0x7f4a2c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:31.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.316+0000 7f4a493e6700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4a2c06dea0 0x7f4a2c070360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400b5c0 tx=0x7f4a3401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:31.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.320+0000 7f4a427fc700 1 -- 172.21.15.70:0/21204677 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4a38056a80 con 0x7f4a440fff50 2024-04-02T04:52:31.486 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:31 smithi179 ceph-mon[84288]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:31.487 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:31 smithi179 ceph-mon[84288]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:31.487 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:31 smithi179 ceph-mon[84288]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:31.487 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:31 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1909121299' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:31.487 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:31 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2552705829' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:52:31.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:31 smithi070 ceph-mon[113635]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:31.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:31 smithi070 ceph-mon[113635]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:31.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:31 smithi070 ceph-mon[113635]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:31.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:31 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1909121299' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:31.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:31 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2552705829' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:52:31.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.592+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a4404f2e0 con 0x7f4a440fff50 2024-04-02T04:52:31.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.593+0000 7f4a427fc700 1 -- 172.21.15.70:0/21204677 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4a3805a140 con 0x7f4a440fff50 2024-04-02T04:52:31.597 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:52:31.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4a2c06dea0 msgr2=0x7f4a2c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:31.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4a2c06dea0 0x7f4a2c070360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400b5c0 tx=0x7f4a3401a040 comp rx=0 tx=0).stop 2024-04-02T04:52:31.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440fff50 msgr2=0x7f4a44100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:31.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440fff50 0x7f4a44100a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4a38007e90 tx=0x7f4a3800d3b0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 shutdown_connections 2024-04-02T04:52:31.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4a2c06dea0 0x7f4a2c070360 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a440ff5a0 0x7f4a44100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 --2- 172.21.15.70:0/21204677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a440fff50 0x7f4a44100a60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 >> 172.21.15.70:0/21204677 conn(0x7f4a44074e30 msgr2=0x7f4a440fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:31.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 shutdown_connections 2024-04-02T04:52:31.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:52:31.595+0000 7f4a4b64a700 1 -- 172.21.15.70:0/21204677 wait complete. 2024-04-02T04:52:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:32 smithi070 ceph-mon[113635]: pgmap v13: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:32 smithi070 ceph-mon[113635]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:32 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/21204677' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:52:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:32 smithi179 ceph-mon[84288]: pgmap v13: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:32 smithi179 ceph-mon[84288]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:32 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/21204677' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:52:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:34.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:34 smithi070 ceph-mon[113635]: pgmap v14: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:34.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:34 smithi179 ceph-mon[84288]: pgmap v14: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:36.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:36 smithi070 ceph-mon[113635]: pgmap v15: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:36 smithi179 ceph-mon[84288]: pgmap v15: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:38.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: Detected new or changed devices on smithi179 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: Reconfiguring mon.smithi070 (monmap changed)... 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: Reconfiguring daemon mon.smithi070 on smithi070 2024-04-02T04:52:38.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:37 smithi070 ceph-mon[113635]: pgmap v16: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: Detected new or changed devices on smithi179 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: Reconfiguring mon.smithi070 (monmap changed)... 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:52:38.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:52:38.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:38.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: Reconfiguring daemon mon.smithi070 on smithi070 2024-04-02T04:52:38.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:37 smithi179 ceph-mon[84288]: pgmap v16: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:39 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:39 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:39 smithi070 ceph-mon[113635]: Reconfiguring mgr.smithi070.igidvr (monmap changed)... 2024-04-02T04:52:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:39 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.igidvr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:52:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:39 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:52:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:39 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:39 smithi070 ceph-mon[113635]: Reconfiguring daemon mgr.smithi070.igidvr on smithi070 2024-04-02T04:52:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:39 smithi070 ceph-mon[113635]: pgmap v17: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:39.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:39 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:39.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:39 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:39.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:39 smithi179 ceph-mon[84288]: Reconfiguring mgr.smithi070.igidvr (monmap changed)... 2024-04-02T04:52:39.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:39 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.igidvr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:52:39.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:39 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:52:39.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:39 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:39.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:39 smithi179 ceph-mon[84288]: Reconfiguring daemon mgr.smithi070.igidvr on smithi070 2024-04-02T04:52:39.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:39 smithi179 ceph-mon[84288]: pgmap v17: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: Reconfiguring ceph-exporter.smithi070 (monmap changed)... 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: Unable to update caps for client.ceph-exporter.smithi070 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi070"}]: dispatch 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:41 smithi070 ceph-mon[113635]: Reconfiguring daemon ceph-exporter.smithi070 on smithi070 2024-04-02T04:52:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: Reconfiguring ceph-exporter.smithi070 (monmap changed)... 2024-04-02T04:52:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:52:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:52:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:52:41.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: Unable to update caps for client.ceph-exporter.smithi070 2024-04-02T04:52:41.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi070"}]: dispatch 2024-04-02T04:52:41.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:41.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:41 smithi179 ceph-mon[84288]: Reconfiguring daemon ceph-exporter.smithi070 on smithi070 2024-04-02T04:52:42.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:42 smithi179 ceph-mon[84288]: pgmap v18: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:42.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:42.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:42.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:42.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:52:42.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:42.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:42 smithi070 ceph-mon[113635]: pgmap v18: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:42.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:42.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:42.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:42.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:52:42.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:43.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:43 smithi179 ceph-mon[84288]: Reconfiguring crash.smithi070 (monmap changed)... 2024-04-02T04:52:43.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:43 smithi179 ceph-mon[84288]: Reconfiguring daemon crash.smithi070 on smithi070 2024-04-02T04:52:43.400 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:43 smithi070 ceph-mon[113635]: Reconfiguring crash.smithi070 (monmap changed)... 2024-04-02T04:52:43.401 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:43 smithi070 ceph-mon[113635]: Reconfiguring daemon crash.smithi070 on smithi070 2024-04-02T04:52:44.198 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:44 smithi179 ceph-mon[84288]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:44.198 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:44.198 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:44.198 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:52:44.198 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:44.312 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:44 smithi070 ceph-mon[113635]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:44.312 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:44.312 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:44.312 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:52:44.312 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:45.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:45 smithi179 ceph-mon[84288]: Reconfiguring osd.0 (monmap changed)... 2024-04-02T04:52:45.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:45 smithi179 ceph-mon[84288]: Reconfiguring daemon osd.0 on smithi070 2024-04-02T04:52:45.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:45 smithi070 ceph-mon[113635]: Reconfiguring osd.0 (monmap changed)... 2024-04-02T04:52:45.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:45 smithi070 ceph-mon[113635]: Reconfiguring daemon osd.0 on smithi070 2024-04-02T04:52:46.447 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:46 smithi070 ceph-mon[113635]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:46.448 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:46.448 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:46.448 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:46 smithi070 ceph-mon[113635]: Reconfiguring osd.1 (monmap changed)... 2024-04-02T04:52:46.448 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:52:46.448 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:46.448 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:46 smithi070 ceph-mon[113635]: Reconfiguring daemon osd.1 on smithi070 2024-04-02T04:52:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:46 smithi179 ceph-mon[84288]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:46 smithi179 ceph-mon[84288]: Reconfiguring osd.1 (monmap changed)... 2024-04-02T04:52:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:52:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:46 smithi179 ceph-mon[84288]: Reconfiguring daemon osd.1 on smithi070 2024-04-02T04:52:48.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:48.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:48.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:47 smithi070 ceph-mon[113635]: Reconfiguring osd.2 (monmap changed)... 2024-04-02T04:52:48.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:52:48.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:48.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:47 smithi070 ceph-mon[113635]: Reconfiguring daemon osd.2 on smithi070 2024-04-02T04:52:48.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:47 smithi070 ceph-mon[113635]: pgmap v21: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:48.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:48.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:48.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:47 smithi179 ceph-mon[84288]: Reconfiguring osd.2 (monmap changed)... 2024-04-02T04:52:48.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:52:48.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:48.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:47 smithi179 ceph-mon[84288]: Reconfiguring daemon osd.2 on smithi070 2024-04-02T04:52:48.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:47 smithi179 ceph-mon[84288]: pgmap v21: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:50.019 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:49 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:50.020 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:49 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:50.020 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:49 smithi070 ceph-mon[113635]: Reconfiguring mds.cephfs.smithi070.mnzrwt (monmap changed)... 2024-04-02T04:52:50.020 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:49 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.mnzrwt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:52:50.020 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:49 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:50.020 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:49 smithi070 ceph-mon[113635]: Reconfiguring daemon mds.cephfs.smithi070.mnzrwt on smithi070 2024-04-02T04:52:50.020 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:49 smithi070 ceph-mon[113635]: pgmap v22: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:50.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:49 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:50.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:49 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:50.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:49 smithi179 ceph-mon[84288]: Reconfiguring mds.cephfs.smithi070.mnzrwt (monmap changed)... 2024-04-02T04:52:50.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:49 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.mnzrwt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:52:50.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:49 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:50.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:49 smithi179 ceph-mon[84288]: Reconfiguring daemon mds.cephfs.smithi070.mnzrwt on smithi070 2024-04-02T04:52:50.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:49 smithi179 ceph-mon[84288]: pgmap v22: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:51 smithi070 ceph-mon[113635]: Reconfiguring mds.cephfs.smithi070.zvraae (monmap changed)... 2024-04-02T04:52:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.zvraae", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:52:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:51 smithi070 ceph-mon[113635]: Reconfiguring daemon mds.cephfs.smithi070.zvraae on smithi070 2024-04-02T04:52:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:51 smithi070 ceph-mon[113635]: pgmap v23: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:51.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:51.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:51.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:51 smithi179 ceph-mon[84288]: Reconfiguring mds.cephfs.smithi070.zvraae (monmap changed)... 2024-04-02T04:52:51.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.zvraae", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:52:51.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:51.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:51 smithi179 ceph-mon[84288]: Reconfiguring daemon mds.cephfs.smithi070.zvraae on smithi070 2024-04-02T04:52:51.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:51 smithi179 ceph-mon[84288]: pgmap v23: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: Reconfiguring ceph-exporter.smithi179 (monmap changed)... 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: Unable to update caps for client.ceph-exporter.smithi179 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi179"}]: dispatch 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:53.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:53 smithi070 ceph-mon[113635]: Reconfiguring daemon ceph-exporter.smithi179 on smithi179 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: Reconfiguring ceph-exporter.smithi179 (monmap changed)... 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: Unable to update caps for client.ceph-exporter.smithi179 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi179"}]: dispatch 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:53.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:53 smithi179 ceph-mon[84288]: Reconfiguring daemon ceph-exporter.smithi179 on smithi179 2024-04-02T04:52:54.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:54 smithi070 ceph-mon[113635]: pgmap v24: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:54.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:54 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:54.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:54 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:54.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:54 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi179", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:52:54.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:54 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:54.304 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:54 smithi179 ceph-mon[84288]: pgmap v24: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:54.304 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:54 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:54.305 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:54 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:54.305 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:54 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi179", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:52:54.305 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:54 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:55.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:55 smithi070 ceph-mon[113635]: Reconfiguring crash.smithi179 (monmap changed)... 2024-04-02T04:52:55.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:55 smithi070 ceph-mon[113635]: Reconfiguring daemon crash.smithi179 on smithi179 2024-04-02T04:52:55.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:55 smithi179 ceph-mon[84288]: Reconfiguring crash.smithi179 (monmap changed)... 2024-04-02T04:52:55.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:55 smithi179 ceph-mon[84288]: Reconfiguring daemon crash.smithi179 on smithi179 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: pgmap v25: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: Reconfiguring mgr.smithi179.labmvs (monmap changed)... 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: Reconfiguring daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:52:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: pgmap v25: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: Reconfiguring mgr.smithi179.labmvs (monmap changed)... 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi179.labmvs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: Reconfiguring daemon mgr.smithi179.labmvs on smithi179 2024-04-02T04:52:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:57.996 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:57.996 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:57.996 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:57 smithi179 ceph-mon[84288]: Reconfiguring mon.smithi179 (monmap changed)... 2024-04-02T04:52:57.997 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:52:57.997 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:52:57.997 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:57.997 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:57 smithi179 ceph-mon[84288]: Reconfiguring daemon mon.smithi179 on smithi179 2024-04-02T04:52:57.997 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:57 smithi179 ceph-mon[84288]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:58.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:58.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:58.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:57 smithi070 ceph-mon[113635]: Reconfiguring mon.smithi179 (monmap changed)... 2024-04-02T04:52:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:52:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:52:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:57 smithi070 ceph-mon[113635]: Reconfiguring daemon mon.smithi179 on smithi179 2024-04-02T04:52:58.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:57 smithi070 ceph-mon[113635]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:59 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:59 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:59 smithi179 ceph-mon[84288]: Reconfiguring osd.3 (monmap changed)... 2024-04-02T04:52:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:59 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:52:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:59 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:59.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:59 smithi179 ceph-mon[84288]: Reconfiguring daemon osd.3 on smithi179 2024-04-02T04:52:59.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:52:59 smithi179 ceph-mon[84288]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:59.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:59 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:59.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:59 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:52:59.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:59 smithi070 ceph-mon[113635]: Reconfiguring osd.3 (monmap changed)... 2024-04-02T04:52:59.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:59 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:52:59.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:59 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:59.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:59 smithi070 ceph-mon[113635]: Reconfiguring daemon osd.3 on smithi179 2024-04-02T04:52:59.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:52:59 smithi070 ceph-mon[113635]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:01.193 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:00 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:01.194 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:00 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:01.194 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:00 smithi179 ceph-mon[84288]: Reconfiguring osd.4 (monmap changed)... 2024-04-02T04:53:01.194 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:00 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:53:01.194 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:00 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:01.194 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:00 smithi179 ceph-mon[84288]: Reconfiguring daemon osd.4 on smithi179 2024-04-02T04:53:01.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:00 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:01.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:00 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:01.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:00 smithi070 ceph-mon[113635]: Reconfiguring osd.4 (monmap changed)... 2024-04-02T04:53:01.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:00 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:53:01.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:00 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:01.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:00 smithi070 ceph-mon[113635]: Reconfiguring daemon osd.4 on smithi179 2024-04-02T04:53:01.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 -- 172.21.15.70:0/3774352130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8068580 msgr2=0x7f67b80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:01.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 --2- 172.21.15.70:0/3774352130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8068580 0x7f67b80689e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009910 tx=0x7f67b4009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 -- 172.21.15.70:0/3774352130 shutdown_connections 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 --2- 172.21.15.70:0/3774352130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8068580 0x7f67b80689e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 --2- 172.21.15.70:0/3774352130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b8106cd0 0x7f67b81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 -- 172.21.15.70:0/3774352130 >> 172.21.15.70:0/3774352130 conn(0x7f67b8075670 msgr2=0x7f67b8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 -- 172.21.15.70:0/3774352130 shutdown_connections 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 -- 172.21.15.70:0/3774352130 wait complete. 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 Processor -- start 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 -- start start 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.713+0000 7f67bf972700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b8068580 0x7f67b819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bf972700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8106cd0 0x7f67b819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:01.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bf972700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b819d120 con 0x7f67b8106cd0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bf972700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b81966b0 con 0x7f67b8068580 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bcf0d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8106cd0 0x7f67b819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bcf0d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8106cd0 0x7f67b819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38170/0 (socket says 172.21.15.70:38170) 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bcf0d700 1 -- 172.21.15.70:0/4149110435 learned_addr learned my addr 172.21.15.70:0/4149110435 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bd70e700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b8068580 0x7f67b819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bcf0d700 1 -- 172.21.15.70:0/4149110435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b8068580 msgr2=0x7f67b819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bcf0d700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b8068580 0x7f67b819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bcf0d700 1 -- 172.21.15.70:0/4149110435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40095f0 con 0x7f67b8106cd0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bcf0d700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8106cd0 0x7f67b819cb00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006010 tx=0x7f67b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67ae7fc700 1 -- 172.21.15.70:0/4149110435 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b401d070 con 0x7f67b8106cd0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b8196930 con 0x7f67b8106cd0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67ae7fc700 1 -- 172.21.15.70:0/4149110435 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67b4005dc0 con 0x7f67b8106cd0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67ae7fc700 1 -- 172.21.15.70:0/4149110435 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b4017400 con 0x7f67b8106cd0 2024-04-02T04:53:01.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.714+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b8196e20 con 0x7f67b8106cd0 2024-04-02T04:53:01.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.716+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b81091d0 con 0x7f67b8106cd0 2024-04-02T04:53:01.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.716+0000 7f67ae7fc700 1 -- 172.21.15.70:0/4149110435 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f67b4003a60 con 0x7f67b8106cd0 2024-04-02T04:53:01.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.716+0000 7f67ae7fc700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f67a406dea0 0x7f67a4070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:01.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.717+0000 7f67ae7fc700 1 -- 172.21.15.70:0/4149110435 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f67b408f150 con 0x7f67b8106cd0 2024-04-02T04:53:01.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.717+0000 7f67bd70e700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f67a406dea0 0x7f67a4070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:01.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.717+0000 7f67bd70e700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f67a406dea0 0x7f67a4070360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005950 tx=0x7f67a800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:01.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.721+0000 7f67ae7fc700 1 -- 172.21.15.70:0/4149110435 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f67b405c0a0 con 0x7f67b8106cd0 2024-04-02T04:53:01.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.938+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67b8197040 con 0x7f67a406dea0 2024-04-02T04:53:01.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.939+0000 7f67ae7fc700 1 -- 172.21.15.70:0/4149110435 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f67b8197040 con 0x7f67a406dea0 2024-04-02T04:53:01.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f67a406dea0 msgr2=0x7f67a4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:01.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f67a406dea0 0x7f67a4070360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005950 tx=0x7f67a800b410 comp rx=0 tx=0).stop 2024-04-02T04:53:01.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8106cd0 msgr2=0x7f67b819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:01.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8106cd0 0x7f67b819cb00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006010 tx=0x7f67b400bd70 comp rx=0 tx=0).stop 2024-04-02T04:53:01.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 shutdown_connections 2024-04-02T04:53:01.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f67a406dea0 0x7f67a4070360 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:01.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b8068580 0x7f67b819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:01.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 --2- 172.21.15.70:0/4149110435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b8106cd0 0x7f67b819cb00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:01.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 >> 172.21.15.70:0/4149110435 conn(0x7f67b8075670 msgr2=0x7f67b80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:01.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 shutdown_connections 2024-04-02T04:53:01.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:01.942+0000 7f67bf972700 1 -- 172.21.15.70:0/4149110435 wait complete. 2024-04-02T04:53:01.955 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:53:02.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:01 smithi070 ceph-mon[113635]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:02.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:02.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:02.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:53:02.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:02.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.047+0000 7fe241919700 1 -- 172.21.15.70:0/3863974904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 msgr2=0x7fe23c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.047+0000 7fe241919700 1 --2- 172.21.15.70:0/3863974904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 0x7fe23c1058a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009910 tx=0x7fe22c009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:02.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.047+0000 7fe241919700 1 -- 172.21.15.70:0/3863974904 shutdown_connections 2024-04-02T04:53:02.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.047+0000 7fe241919700 1 --2- 172.21.15.70:0/3863974904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 0x7fe23c1058a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.047+0000 7fe241919700 1 --2- 172.21.15.70:0/3863974904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe23c100ea0 0x7fe23c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.047+0000 7fe241919700 1 -- 172.21.15.70:0/3863974904 >> 172.21.15.70:0/3863974904 conn(0x7fe23c0fc750 msgr2=0x7fe23c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:02.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.047+0000 7fe241919700 1 -- 172.21.15.70:0/3863974904 shutdown_connections 2024-04-02T04:53:02.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.047+0000 7fe241919700 1 -- 172.21.15.70:0/3863974904 wait complete. 2024-04-02T04:53:02.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe241919700 1 Processor -- start 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe241919700 1 -- start start 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe241919700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe23c100ea0 0x7fe23c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe241919700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 0x7fe23c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe241919700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c19d2d0 con 0x7fe23c101850 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe241919700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c196780 con 0x7fe23c100ea0 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe23a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 0x7fe23c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe23a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 0x7fe23c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38188/0 (socket says 172.21.15.70:38188) 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe23a7fc700 1 -- 172.21.15.70:0/3945847334 learned_addr learned my addr 172.21.15.70:0/3945847334 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe23affd700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe23c100ea0 0x7fe23c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe23a7fc700 1 -- 172.21.15.70:0/3945847334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe23c100ea0 msgr2=0x7fe23c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe23a7fc700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe23c100ea0 0x7fe23c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.048+0000 7fe23a7fc700 1 -- 172.21.15.70:0/3945847334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0095f0 con 0x7fe23c101850 2024-04-02T04:53:02.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.049+0000 7fe23a7fc700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 0x7fe23c19cc40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0092e0 tx=0x7fe22c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:02.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.049+0000 7fe240917700 1 -- 172.21.15.70:0/3945847334 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe22c01d070 con 0x7fe23c101850 2024-04-02T04:53:02.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.049+0000 7fe240917700 1 -- 172.21.15.70:0/3945847334 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe22c005dc0 con 0x7fe23c101850 2024-04-02T04:53:02.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.049+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c196a00 con 0x7fe23c101850 2024-04-02T04:53:02.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.049+0000 7fe240917700 1 -- 172.21.15.70:0/3945847334 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe22c017400 con 0x7fe23c101850 2024-04-02T04:53:02.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.049+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c196ef0 con 0x7fe23c101850 2024-04-02T04:53:02.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.050+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c1092a0 con 0x7fe23c101850 2024-04-02T04:53:02.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.050+0000 7fe240917700 1 -- 172.21.15.70:0/3945847334 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fe22c00f6f0 con 0x7fe23c101850 2024-04-02T04:53:02.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.051+0000 7fe240917700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe22806dea0 0x7fe228070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.051+0000 7fe240917700 1 -- 172.21.15.70:0/3945847334 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe22c08f0c0 con 0x7fe23c101850 2024-04-02T04:53:02.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.051+0000 7fe23affd700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe22806dea0 0x7fe228070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.051+0000 7fe23affd700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe22806dea0 0x7fe228070360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe224007be0 tx=0x7fe2240073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:02.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.055+0000 7fe240917700 1 -- 172.21.15.70:0/3945847334 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe22c05c010 con 0x7fe23c101850 2024-04-02T04:53:02.268 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:01 smithi179 ceph-mon[84288]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:02.268 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:02.269 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:02.269 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:53:02.269 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:02.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.271+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe23c197110 con 0x7fe22806dea0 2024-04-02T04:53:02.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.273+0000 7fe240917700 1 -- 172.21.15.70:0/3945847334 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fe23c197110 con 0x7fe22806dea0 2024-04-02T04:53:02.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe22806dea0 msgr2=0x7fe228070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe22806dea0 0x7fe228070360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe224007be0 tx=0x7fe2240073d0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 msgr2=0x7fe23c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 0x7fe23c19cc40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0092e0 tx=0x7fe22c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 shutdown_connections 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe22806dea0 0x7fe228070360 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe23c100ea0 0x7fe23c19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 --2- 172.21.15.70:0/3945847334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe23c101850 0x7fe23c19cc40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 >> 172.21.15.70:0/3945847334 conn(0x7fe23c0fc750 msgr2=0x7fe23c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 shutdown_connections 2024-04-02T04:53:02.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.275+0000 7fe241919700 1 -- 172.21.15.70:0/3945847334 wait complete. 2024-04-02T04:53:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.379+0000 7f0aabfe3700 1 -- 172.21.15.70:0/2222154766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4100750 msgr2=0x7f0aa41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.379+0000 7f0aabfe3700 1 --2- 172.21.15.70:0/2222154766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4100750 0x7f0aa41047a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009910 tx=0x7f0aa0009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.379+0000 7f0aabfe3700 1 -- 172.21.15.70:0/2222154766 shutdown_connections 2024-04-02T04:53:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.379+0000 7f0aabfe3700 1 --2- 172.21.15.70:0/2222154766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa4100750 0x7f0aa41047a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.379+0000 7f0aabfe3700 1 --2- 172.21.15.70:0/2222154766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0aa40ffda0 0x7f0aa4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.379+0000 7f0aabfe3700 1 -- 172.21.15.70:0/2222154766 >> 172.21.15.70:0/2222154766 conn(0x7f0aa40fb650 msgr2=0x7f0aa40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.379+0000 7f0aabfe3700 1 -- 172.21.15.70:0/2222154766 shutdown_connections 2024-04-02T04:53:02.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.379+0000 7f0aabfe3700 1 -- 172.21.15.70:0/2222154766 wait complete. 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aabfe3700 1 Processor -- start 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aabfe3700 1 -- start start 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aabfe3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa40ffda0 0x7f0aa41977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aabfe3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0aa4100750 0x7f0aa4197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aabfe3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4198320 con 0x7f0aa40ffda0 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aabfe3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4198460 con 0x7f0aa4100750 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aa9d7f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa40ffda0 0x7f0aa41977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aa9d7f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa40ffda0 0x7f0aa41977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38212/0 (socket says 172.21.15.70:38212) 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aa9d7f700 1 -- 172.21.15.70:0/3000092068 learned_addr learned my addr 172.21.15.70:0/3000092068 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:02.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.380+0000 7f0aa957e700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0aa4100750 0x7f0aa4197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0aa9d7f700 1 -- 172.21.15.70:0/3000092068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0aa4100750 msgr2=0x7f0aa4197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0aa9d7f700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0aa4100750 0x7f0aa4197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0aa9d7f700 1 -- 172.21.15.70:0/3000092068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa00095f0 con 0x7f0aa40ffda0 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0aa9d7f700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa40ffda0 0x7f0aa41977c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d8d0 tx=0x7f0a9400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0a9affd700 1 -- 172.21.15.70:0/3000092068 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a940098a0 con 0x7f0aa40ffda0 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0a9affd700 1 -- 172.21.15.70:0/3000092068 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a94010460 con 0x7f0aa40ffda0 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa41a0830 con 0x7f0aa40ffda0 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0a9affd700 1 -- 172.21.15.70:0/3000092068 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a9400f5f0 con 0x7f0aa40ffda0 2024-04-02T04:53:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.381+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa41a0cf0 con 0x7f0aa40ffda0 2024-04-02T04:53:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.382+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa41081a0 con 0x7f0aa40ffda0 2024-04-02T04:53:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.383+0000 7f0a9affd700 1 -- 172.21.15.70:0/3000092068 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f0a94009a00 con 0x7f0aa40ffda0 2024-04-02T04:53:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.383+0000 7f0a9affd700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0a9006dea0 0x7f0a90070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.383+0000 7f0a9affd700 1 -- 172.21.15.70:0/3000092068 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0a9408daf0 con 0x7f0aa40ffda0 2024-04-02T04:53:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.383+0000 7f0aa957e700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0a9006dea0 0x7f0a90070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.383+0000 7f0aa957e700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0a9006dea0 0x7f0a90070360 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0006010 tx=0x7f0aa001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.388+0000 7f0a9affd700 1 -- 172.21.15.70:0/3000092068 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0a94056f90 con 0x7f0aa40ffda0 2024-04-02T04:53:02.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.592+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0aa40611d0 con 0x7f0a9006dea0 2024-04-02T04:53:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.602+0000 7f0a9affd700 1 -- 172.21.15.70:0/3000092068 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0aa40611d0 con 0x7f0a9006dea0 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (2m) 45s ago 13m 17.8M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (13m) 45s ago 13m 8070k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (12m) 30s ago 12m 7759k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (13m) 45s ago 13m 6669k - 18.2.1 5be31c24972a c803edb54bf7 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (12m) 30s ago 12m 6669k - 18.2.1 5be31c24972a 4cfff30719ef 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (2m) 45s ago 13m 75.9M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (8m) 45s ago 8m 153M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (8m) 45s ago 7m 18.9M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (7m) 30s ago 7m 17.4M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (8m) 30s ago 8m 18.7M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:53:02.607 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (4m) 45s ago 15m 471M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (3m) 30s ago 12m 447M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (60s) 45s ago 15m 29.1M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (35s) 30s ago 12m 28.0M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (3m) 45s ago 13m 15.7M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (3m) 30s ago 12m 16.8M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (11m) 45s ago 11m 229M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (11m) 45s ago 11m 264M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (10m) 45s ago 10m 211M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (10m) 30s ago 10m 298M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (10m) 30s ago 10m 274M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (9m) 30s ago 9m 228M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:53:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (2m) 45s ago 13m 60.7M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:53:02.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0a9006dea0 msgr2=0x7f0a90070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0a9006dea0 0x7f0a90070360 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0006010 tx=0x7f0aa001a040 comp rx=0 tx=0).stop 2024-04-02T04:53:02.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa40ffda0 msgr2=0x7f0aa41977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa40ffda0 0x7f0aa41977c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d8d0 tx=0x7f0a9400dbe0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 shutdown_connections 2024-04-02T04:53:02.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0a9006dea0 0x7f0a90070360 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aa40ffda0 0x7f0aa41977c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 --2- 172.21.15.70:0/3000092068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0aa4100750 0x7f0aa4197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.604+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 >> 172.21.15.70:0/3000092068 conn(0x7f0aa40fb650 msgr2=0x7f0aa40fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:02.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.605+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 shutdown_connections 2024-04-02T04:53:02.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.605+0000 7f0aabfe3700 1 -- 172.21.15.70:0/3000092068 wait complete. 2024-04-02T04:53:02.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1277178920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278102780 msgr2=0x7fb27810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 --2- 172.21.15.70:0/1277178920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278102780 0x7fb27810ac70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009910 tx=0x7fb26c009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:02.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1277178920 shutdown_connections 2024-04-02T04:53:02.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 --2- 172.21.15.70:0/1277178920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278102780 0x7fb27810ac70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 --2- 172.21.15.70:0/1277178920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb278101e60 0x7fb278102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1277178920 >> 172.21.15.70:0/1277178920 conn(0x7fb2780fb650 msgr2=0x7fb2780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:02.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1277178920 shutdown_connections 2024-04-02T04:53:02.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1277178920 wait complete. 2024-04-02T04:53:02.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 Processor -- start 2024-04-02T04:53:02.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.707+0000 7fb27e4d8700 1 -- start start 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb27e4d8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278101e60 0x7fb2780ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb27e4d8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb278102780 0x7fb278100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb27e4d8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278100540 con 0x7fb278101e60 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb27e4d8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278100680 con 0x7fb278102780 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb277fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278101e60 0x7fb2780ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb277fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278101e60 0x7fb2780ffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38234/0 (socket says 172.21.15.70:38234) 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb277fff700 1 -- 172.21.15.70:0/1699199324 learned_addr learned my addr 172.21.15.70:0/1699199324 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb2777fe700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb278102780 0x7fb278100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb277fff700 1 -- 172.21.15.70:0/1699199324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb278102780 msgr2=0x7fb278100000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb277fff700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb278102780 0x7fb278100000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb277fff700 1 -- 172.21.15.70:0/1699199324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c0095f0 con 0x7fb278101e60 2024-04-02T04:53:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb277fff700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278101e60 0x7fb2780ffac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d900 tx=0x7fb26800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.708+0000 7fb2757fa700 1 -- 172.21.15.70:0/1699199324 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb268004a00 con 0x7fb278101e60 2024-04-02T04:53:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.709+0000 7fb2757fa700 1 -- 172.21.15.70:0/1699199324 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2680054e0 con 0x7fb278101e60 2024-04-02T04:53:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.709+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2781a2aa0 con 0x7fb278101e60 2024-04-02T04:53:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.709+0000 7fb2757fa700 1 -- 172.21.15.70:0/1699199324 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb268009d00 con 0x7fb278101e60 2024-04-02T04:53:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.709+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2781a2e00 con 0x7fb278101e60 2024-04-02T04:53:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.710+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb278108300 con 0x7fb278101e60 2024-04-02T04:53:02.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.710+0000 7fb2757fa700 1 -- 172.21.15.70:0/1699199324 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fb268005020 con 0x7fb278101e60 2024-04-02T04:53:02.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.710+0000 7fb2757fa700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb26006dea0 0x7fb260070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:02.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.711+0000 7fb2757fa700 1 -- 172.21.15.70:0/1699199324 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb26808dcb0 con 0x7fb278101e60 2024-04-02T04:53:02.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.711+0000 7fb2777fe700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb26006dea0 0x7fb260070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:02.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.711+0000 7fb2777fe700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb26006dea0 0x7fb260070360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb278101240 tx=0x7fb26c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:02.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.715+0000 7fb2757fa700 1 -- 172.21.15.70:0/1699199324 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb2680570d0 con 0x7fb278101e60 2024-04-02T04:53:02.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.995+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb278107ce0 con 0x7fb278101e60 2024-04-02T04:53:02.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:02 smithi070 ceph-mon[113635]: Reconfiguring osd.5 (monmap changed)... 2024-04-02T04:53:03.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:02 smithi070 ceph-mon[113635]: Reconfiguring daemon osd.5 on smithi179 2024-04-02T04:53:03.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:02 smithi070 ceph-mon[113635]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:03.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:02 smithi070 ceph-mon[113635]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:03.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.995+0000 7fb2757fa700 1 -- 172.21.15.70:0/1699199324 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fb2680172c0 con 0x7fb278101e60 2024-04-02T04:53:03.000 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:53:03.000 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:53:03.000 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:53:03.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:53:03.001 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:53:03.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb26006dea0 msgr2=0x7fb260070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb26006dea0 0x7fb260070360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb278101240 tx=0x7fb26c01a040 comp rx=0 tx=0).stop 2024-04-02T04:53:03.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278101e60 msgr2=0x7fb2780ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278101e60 0x7fb2780ffac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d900 tx=0x7fb26800dc10 comp rx=0 tx=0).stop 2024-04-02T04:53:03.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 shutdown_connections 2024-04-02T04:53:03.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb26006dea0 0x7fb260070360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb278101e60 0x7fb2780ffac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 --2- 172.21.15.70:0/1699199324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb278102780 0x7fb278100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 >> 172.21.15.70:0/1699199324 conn(0x7fb2780fb650 msgr2=0x7fb278105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:03.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 shutdown_connections 2024-04-02T04:53:03.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:02.998+0000 7fb27e4d8700 1 -- 172.21.15.70:0/1699199324 wait complete. 2024-04-02T04:53:03.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.102+0000 7f5d5209f700 1 -- 172.21.15.70:0/955703021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ffe70 msgr2=0x7f5d4c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.102+0000 7f5d5209f700 1 --2- 172.21.15.70:0/955703021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ffe70 0x7f5d4c1057a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009910 tx=0x7f5d34009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.102+0000 7f5d5209f700 1 -- 172.21.15.70:0/955703021 shutdown_connections 2024-04-02T04:53:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.102+0000 7f5d5209f700 1 --2- 172.21.15.70:0/955703021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ffe70 0x7f5d4c1057a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.102+0000 7f5d5209f700 1 --2- 172.21.15.70:0/955703021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d4c0ff4c0 0x7f5d4c0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.102+0000 7f5d5209f700 1 -- 172.21.15.70:0/955703021 >> 172.21.15.70:0/955703021 conn(0x7f5d4c074cb0 msgr2=0x7f5d4c0750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.102+0000 7f5d5209f700 1 -- 172.21.15.70:0/955703021 shutdown_connections 2024-04-02T04:53:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.102+0000 7f5d5209f700 1 -- 172.21.15.70:0/955703021 wait complete. 2024-04-02T04:53:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d5209f700 1 Processor -- start 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d5209f700 1 -- start start 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d5209f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ff4c0 0x7f5d4c100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d5209f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d4c0ffe70 0x7f5d4c100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d5209f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c100ec0 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d5209f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c101000 con 0x7f5d4c0ffe70 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d4b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ff4c0 0x7f5d4c100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d4b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ff4c0 0x7f5d4c100440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38248/0 (socket says 172.21.15.70:38248) 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d4b7fe700 1 -- 172.21.15.70:0/3327350653 learned_addr learned my addr 172.21.15.70:0/3327350653 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d4affd700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d4c0ffe70 0x7f5d4c100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d4b7fe700 1 -- 172.21.15.70:0/3327350653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d4c0ffe70 msgr2=0x7f5d4c100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d4b7fe700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d4c0ffe70 0x7f5d4c100980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.103+0000 7f5d4b7fe700 1 -- 172.21.15.70:0/3327350653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d340095f0 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.104+0000 7f5d4b7fe700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ff4c0 0x7f5d4c100440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400cfd0 tx=0x7f5d4400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.104+0000 7f5d48ff9700 1 -- 172.21.15.70:0/3327350653 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d4400ad70 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.104+0000 7f5d48ff9700 1 -- 172.21.15.70:0/3327350653 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d44007e70 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.104+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c1a28f0 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.104+0000 7f5d48ff9700 1 -- 172.21.15.70:0/3327350653 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d44011810 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.104+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c1a2c50 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.105+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c1090a0 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.105+0000 7f5d48ff9700 1 -- 172.21.15.70:0/3327350653 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f5d440119b0 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.106+0000 7f5d48ff9700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5d3806de90 0x7f5d38070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.106+0000 7f5d4affd700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5d3806de90 0x7f5d38070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.106+0000 7f5d48ff9700 1 -- 172.21.15.70:0/3327350653 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5d4408dc60 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.106+0000 7f5d4affd700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5d3806de90 0x7f5d38070350 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c101c30 tx=0x7f5d3401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:03.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.110+0000 7f5d48ff9700 1 -- 172.21.15.70:0/3327350653 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5d44057190 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:02 smithi179 ceph-mon[84288]: Reconfiguring osd.5 (monmap changed)... 2024-04-02T04:53:03.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:02 smithi179 ceph-mon[84288]: Reconfiguring daemon osd.5 on smithi179 2024-04-02T04:53:03.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:02 smithi179 ceph-mon[84288]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:03.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:02 smithi179 ceph-mon[84288]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:03.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.355+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5d4c02cd60 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.355+0000 7f5d48ff9700 1 -- 172.21.15.70:0/3327350653 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f5d44016770 con 0x7f5d4c0ff4c0 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:53:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:53:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:53:03.362 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:53:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5d3806de90 msgr2=0x7f5d38070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5d3806de90 0x7f5d38070350 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c101c30 tx=0x7f5d3401a040 comp rx=0 tx=0).stop 2024-04-02T04:53:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ff4c0 msgr2=0x7f5d4c100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ff4c0 0x7f5d4c100440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400cfd0 tx=0x7f5d4400a340 comp rx=0 tx=0).stop 2024-04-02T04:53:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 shutdown_connections 2024-04-02T04:53:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5d3806de90 0x7f5d38070350 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d4c0ff4c0 0x7f5d4c100440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 --2- 172.21.15.70:0/3327350653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d4c0ffe70 0x7f5d4c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 >> 172.21.15.70:0/3327350653 conn(0x7f5d4c074cb0 msgr2=0x7f5d4c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 shutdown_connections 2024-04-02T04:53:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.357+0000 7f5d5209f700 1 -- 172.21.15.70:0/3327350653 wait complete. 2024-04-02T04:53:03.364 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:53:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.462+0000 7fbe6a26e700 1 -- 172.21.15.70:0/3831418737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 msgr2=0x7fbe64105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.462+0000 7fbe6a26e700 1 --2- 172.21.15.70:0/3831418737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 0x7fbe64105fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009910 tx=0x7fbe58009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.462+0000 7fbe6a26e700 1 -- 172.21.15.70:0/3831418737 shutdown_connections 2024-04-02T04:53:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.462+0000 7fbe6a26e700 1 --2- 172.21.15.70:0/3831418737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640ffc20 0x7fbe64100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.462+0000 7fbe6a26e700 1 --2- 172.21.15.70:0/3831418737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 0x7fbe64105fb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.462+0000 7fbe6a26e700 1 -- 172.21.15.70:0/3831418737 >> 172.21.15.70:0/3831418737 conn(0x7fbe64075670 msgr2=0x7fbe64075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.462+0000 7fbe6a26e700 1 -- 172.21.15.70:0/3831418737 shutdown_connections 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.462+0000 7fbe6a26e700 1 -- 172.21.15.70:0/3831418737 wait complete. 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe6a26e700 1 Processor -- start 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe6a26e700 1 -- start start 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe6a26e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640ffc20 0x7fbe6419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe6a26e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 0x7fbe6419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe6a26e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6419d220 con 0x7fbe64105bd0 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe6a26e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe641966a0 con 0x7fbe640ffc20 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe637fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 0x7fbe6419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe637fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 0x7fbe6419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38258/0 (socket says 172.21.15.70:38258) 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe637fe700 1 -- 172.21.15.70:0/4082135442 learned_addr learned my addr 172.21.15.70:0/4082135442 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe63fff700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640ffc20 0x7fbe6419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe637fe700 1 -- 172.21.15.70:0/4082135442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640ffc20 msgr2=0x7fbe6419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe637fe700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640ffc20 0x7fbe6419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.463+0000 7fbe637fe700 1 -- 172.21.15.70:0/4082135442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580095f0 con 0x7fbe64105bd0 2024-04-02T04:53:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.464+0000 7fbe637fe700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 0x7fbe6419cb40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400d8d0 tx=0x7fbe5400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.464+0000 7fbe617fa700 1 -- 172.21.15.70:0/4082135442 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe540098a0 con 0x7fbe64105bd0 2024-04-02T04:53:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.464+0000 7fbe617fa700 1 -- 172.21.15.70:0/4082135442 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe54010460 con 0x7fbe64105bd0 2024-04-02T04:53:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.464+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe64196980 con 0x7fbe64105bd0 2024-04-02T04:53:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.464+0000 7fbe617fa700 1 -- 172.21.15.70:0/4082135442 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5400f5f0 con 0x7fbe64105bd0 2024-04-02T04:53:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.464+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe64196ed0 con 0x7fbe64105bd0 2024-04-02T04:53:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.465+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6404f2e0 con 0x7fbe64105bd0 2024-04-02T04:53:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.465+0000 7fbe617fa700 1 -- 172.21.15.70:0/4082135442 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fbe54009a00 con 0x7fbe64105bd0 2024-04-02T04:53:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.466+0000 7fbe617fa700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06de90 0x7fbe4c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.466+0000 7fbe63fff700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06de90 0x7fbe4c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.466+0000 7fbe617fa700 1 -- 172.21.15.70:0/4082135442 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbe5408dba0 con 0x7fbe64105bd0 2024-04-02T04:53:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.466+0000 7fbe63fff700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06de90 0x7fbe4c070350 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800b5c0 tx=0x7fbe5801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.470+0000 7fbe617fa700 1 -- 172.21.15.70:0/4082135442 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe5405aaf0 con 0x7fbe64105bd0 2024-04-02T04:53:03.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.682+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe64197200 con 0x7fbe4c06de90 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.683+0000 7fbe617fa700 1 -- 172.21.15.70:0/4082135442 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fbe64197200 con 0x7fbe4c06de90 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "mon" 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "4/23 daemons upgraded", 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mon daemons", 2024-04-02T04:53:03.688 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:53:03.689 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:53:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06de90 msgr2=0x7fbe4c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06de90 0x7fbe4c070350 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800b5c0 tx=0x7fbe5801a040 comp rx=0 tx=0).stop 2024-04-02T04:53:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 msgr2=0x7fbe6419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 0x7fbe6419cb40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400d8d0 tx=0x7fbe5400dbe0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 shutdown_connections 2024-04-02T04:53:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06de90 0x7fbe4c070350 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640ffc20 0x7fbe6419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 --2- 172.21.15.70:0/4082135442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64105bd0 0x7fbe6419cb40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 >> 172.21.15.70:0/4082135442 conn(0x7fbe64075670 msgr2=0x7fbe64103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 shutdown_connections 2024-04-02T04:53:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.686+0000 7fbe6a26e700 1 -- 172.21.15.70:0/4082135442 wait complete. 2024-04-02T04:53:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.807+0000 7fc4eb82e700 1 -- 172.21.15.70:0/3966672118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4101850 msgr2=0x7fc4e41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.807+0000 7fc4eb82e700 1 --2- 172.21.15.70:0/3966672118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4101850 0x7fc4e41058a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009910 tx=0x7fc4e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.807+0000 7fc4eb82e700 1 -- 172.21.15.70:0/3966672118 shutdown_connections 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.807+0000 7fc4eb82e700 1 --2- 172.21.15.70:0/3966672118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4101850 0x7fc4e41058a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.807+0000 7fc4eb82e700 1 --2- 172.21.15.70:0/3966672118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4e4100ea0 0x7fc4e4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.807+0000 7fc4eb82e700 1 -- 172.21.15.70:0/3966672118 >> 172.21.15.70:0/3966672118 conn(0x7fc4e40fc750 msgr2=0x7fc4e40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.807+0000 7fc4eb82e700 1 -- 172.21.15.70:0/3966672118 shutdown_connections 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.807+0000 7fc4eb82e700 1 -- 172.21.15.70:0/3966672118 wait complete. 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4eb82e700 1 Processor -- start 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4eb82e700 1 -- start start 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4eb82e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4100ea0 0x7fc4e419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4eb82e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4e4101850 0x7fc4e419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4eb82e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e419d2d0 con 0x7fc4e4100ea0 2024-04-02T04:53:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4eb82e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e4196780 con 0x7fc4e4101850 2024-04-02T04:53:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4e95ca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4100ea0 0x7fc4e419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4e95ca700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4100ea0 0x7fc4e419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38274/0 (socket says 172.21.15.70:38274) 2024-04-02T04:53:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4e95ca700 1 -- 172.21.15.70:0/1018353091 learned_addr learned my addr 172.21.15.70:0/1018353091 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.808+0000 7fc4e8dc9700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4e4101850 0x7fc4e419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4e95ca700 1 -- 172.21.15.70:0/1018353091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4e4101850 msgr2=0x7fc4e419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4e95ca700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4e4101850 0x7fc4e419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4e95ca700 1 -- 172.21.15.70:0/1018353091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00095f0 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4e95ca700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4100ea0 0x7fc4e419c700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400d8d0 tx=0x7fc4d400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4da7fc700 1 -- 172.21.15.70:0/1018353091 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4d40098a0 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4da7fc700 1 -- 172.21.15.70:0/1018353091 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4d4010460 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e4196a60 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4da7fc700 1 -- 172.21.15.70:0/1018353091 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4d400f5f0 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.809+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e4196fb0 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.810+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e41092a0 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.811+0000 7fc4da7fc700 1 -- 172.21.15.70:0/1018353091 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fc4d4009a00 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.811+0000 7fc4da7fc700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc4d006dea0 0x7fc4d0070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.811+0000 7fc4da7fc700 1 -- 172.21.15.70:0/1018353091 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc4d408dba0 con 0x7fc4e4100ea0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.811+0000 7fc4e8dc9700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc4d006dea0 0x7fc4d0070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.811+0000 7fc4e8dc9700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc4d006dea0 0x7fc4d0070360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4197bf0 tx=0x7fc4e001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:03.816+0000 7fc4da7fc700 1 -- 172.21.15.70:0/1018353091 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc4d405aaf0 con 0x7fc4e4100ea0 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1699199324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: pgmap v29: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: Reconfiguring mds.cephfs.smithi179.mzfimf (monmap changed)... 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.mzfimf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: Reconfiguring daemon mds.cephfs.smithi179.mzfimf on smithi179 2024-04-02T04:53:04.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:03 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3327350653' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:53:04.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.090+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc4e4197d30 con 0x7fc4e4100ea0 2024-04-02T04:53:04.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.090+0000 7fc4da7fc700 1 -- 172.21.15.70:0/1018353091 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc4d405a680 con 0x7fc4e4100ea0 2024-04-02T04:53:04.094 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:53:04.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc4d006dea0 msgr2=0x7fc4d0070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc4d006dea0 0x7fc4d0070360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4197bf0 tx=0x7fc4e001a040 comp rx=0 tx=0).stop 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4100ea0 msgr2=0x7fc4e419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4100ea0 0x7fc4e419c700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400d8d0 tx=0x7fc4d400dbe0 comp rx=0 tx=0).stop 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 shutdown_connections 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc4d006dea0 0x7fc4d0070360 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4100ea0 0x7fc4e419c700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 --2- 172.21.15.70:0/1018353091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4e4101850 0x7fc4e419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 >> 172.21.15.70:0/1018353091 conn(0x7fc4e40fc750 msgr2=0x7fc4e40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 shutdown_connections 2024-04-02T04:53:04.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:04.092+0000 7fc4eb82e700 1 -- 172.21.15.70:0/1018353091 wait complete. 2024-04-02T04:53:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1699199324' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: pgmap v29: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: Reconfiguring mds.cephfs.smithi179.mzfimf (monmap changed)... 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.mzfimf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: Reconfiguring daemon mds.cephfs.smithi179.mzfimf on smithi179 2024-04-02T04:53:04.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:03 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3327350653' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:53:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:04 smithi179 ceph-mon[84288]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:04 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1018353091' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:53:05.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:04 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:05.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:04 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:05.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:04 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.atrmwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:53:05.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:04 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:04 smithi070 ceph-mon[113635]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:04 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1018353091' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:53:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:04 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:04 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:04 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.atrmwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:53:05.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:04 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:06.273 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:05 smithi179 ceph-mon[84288]: Reconfiguring mds.cephfs.smithi179.atrmwz (monmap changed)... 2024-04-02T04:53:06.273 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:05 smithi179 ceph-mon[84288]: Reconfiguring daemon mds.cephfs.smithi179.atrmwz on smithi179 2024-04-02T04:53:06.273 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:05 smithi179 ceph-mon[84288]: pgmap v30: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:06.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:05 smithi070 ceph-mon[113635]: Reconfiguring mds.cephfs.smithi179.atrmwz (monmap changed)... 2024-04-02T04:53:06.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:05 smithi070 ceph-mon[113635]: Reconfiguring daemon mds.cephfs.smithi179.atrmwz on smithi179 2024-04-02T04:53:06.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:05 smithi070 ceph-mon[113635]: pgmap v30: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all mon 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]: dispatch 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]': finished 2024-04-02T04:53:07.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi179"}]: dispatch 2024-04-02T04:53:07.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi179"}]': finished 2024-04-02T04:53:07.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:07 smithi070 ceph-mon[113635]: pgmap v31: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all mon 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]: dispatch 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]': finished 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi179"}]: dispatch 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi179"}]': finished 2024-04-02T04:53:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:07 smithi179 ceph-mon[84288]: pgmap v31: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:09.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:09 smithi070 ceph-mon[113635]: Upgrade: Updating crash.smithi070 (1/2) 2024-04-02T04:53:09.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:09 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:09.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:09 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:53:09.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:09 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:09.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:09 smithi070 ceph-mon[113635]: Deploying daemon crash.smithi070 on smithi070 2024-04-02T04:53:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:09 smithi179 ceph-mon[84288]: Upgrade: Updating crash.smithi070 (1/2) 2024-04-02T04:53:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:09 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:09 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:53:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:09 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:09 smithi179 ceph-mon[84288]: Deploying daemon crash.smithi070 on smithi070 2024-04-02T04:53:10.193 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:10 smithi070 ceph-mon[113635]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:10 smithi179 ceph-mon[84288]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:12.401 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:12 smithi179 ceph-mon[84288]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:12.401 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:12.401 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:12.401 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:12.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:12 smithi070 ceph-mon[113635]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:12.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:14.166 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:14 smithi179 ceph-mon[84288]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:14.166 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:14 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:14.167 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:14 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi179", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:53:14.167 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:14 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:14 smithi070 ceph-mon[113635]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:14 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:14 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi179", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:53:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:14 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:15.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:15 smithi070 ceph-mon[113635]: Upgrade: Updating crash.smithi179 (2/2) 2024-04-02T04:53:15.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:15 smithi070 ceph-mon[113635]: Deploying daemon crash.smithi179 on smithi179 2024-04-02T04:53:15.349 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:15 smithi179 ceph-mon[84288]: Upgrade: Updating crash.smithi179 (2/2) 2024-04-02T04:53:15.349 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:15 smithi179 ceph-mon[84288]: Deploying daemon crash.smithi179 on smithi179 2024-04-02T04:53:16.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:16 smithi179 ceph-mon[84288]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:16.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:16 smithi070 ceph-mon[113635]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:17.916 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:17.916 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:17.916 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:17.916 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:17 smithi070 ceph-mon[113635]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:17 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:17 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:17 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:17 smithi179 ceph-mon[84288]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:20.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:20 smithi070 ceph-mon[113635]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:20.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:20.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:20.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:20 smithi179 ceph-mon[84288]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:20.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:20.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:22.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:22.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:22.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:21 smithi070 ceph-mon[113635]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:21 smithi179 ceph-mon[84288]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:23.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:23 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:23.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:23 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:23 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:23 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:24 smithi070 ceph-mon[113635]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:24 smithi179 ceph-mon[84288]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:25.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:25 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:25.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:25 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:25.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:25 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:25.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:25 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:26 smithi070 ceph-mon[113635]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:26 smithi179 ceph-mon[84288]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:53:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all crash 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]: dispatch 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]': finished 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi179"}]: dispatch 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi179"}]': finished 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: Upgrade: osd.0 is safe to restart 2024-04-02T04:53:28.031 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:27 smithi070 ceph-mon[113635]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all crash 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]: dispatch 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]': finished 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi179"}]: dispatch 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi179"}]': finished 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: Upgrade: osd.0 is safe to restart 2024-04-02T04:53:28.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:27 smithi179 ceph-mon[84288]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:29 smithi179 ceph-mon[84288]: Upgrade: Updating osd.0 2024-04-02T04:53:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:29 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:29 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:53:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:29 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:29 smithi179 ceph-mon[84288]: Deploying daemon osd.0 on smithi070 2024-04-02T04:53:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:29 smithi179 ceph-mon[84288]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:29.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:29 smithi070 ceph-mon[113635]: Upgrade: Updating osd.0 2024-04-02T04:53:29.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:29 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:29.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:29 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:53:29.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:29 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:29.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:29 smithi070 ceph-mon[113635]: Deploying daemon osd.0 on smithi070 2024-04-02T04:53:29.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:29 smithi070 ceph-mon[113635]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:32.058 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:31 smithi070 systemd[1]: Stopping Ceph osd.0 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:53:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:32 smithi070 ceph-mon[113635]: pgmap v43: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:32 smithi070 ceph-mon[113635]: osd.0 marked itself down and dead 2024-04-02T04:53:32.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:32 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[47987]: 2024-04-02T04:53:32.054+0000 7fb4f865a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:53:32.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:32 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[47987]: 2024-04-02T04:53:32.054+0000 7fb4f865a700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-02T04:53:32.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:32 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[47987]: 2024-04-02T04:53:32.054+0000 7fb4f865a700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:53:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:32 smithi179 ceph-mon[84288]: pgmap v43: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:32 smithi179 ceph-mon[84288]: osd.0 marked itself down and dead 2024-04-02T04:53:33.009 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:32 smithi070 podman[125060]: 2024-04-02 04:53:32.696989218 +0000 UTC m=+0.769756277 container died 1a0a3ebd83019dfb60543b0deeb89dc0524e400573c4992ba682dcb2b13c6d9d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-02T04:53:33.279 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:33 smithi070 podman[125060]: 2024-04-02 04:53:33.204912736 +0000 UTC m=+1.277679784 container cleanup 1a0a3ebd83019dfb60543b0deeb89dc0524e400573c4992ba682dcb2b13c6d9d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, RELEASE=HEAD) 2024-04-02T04:53:33.280 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:33 smithi070 bash[125060]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T04:53:33.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:33 smithi070 ceph-mon[113635]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:53:33.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:33 smithi070 ceph-mon[113635]: osdmap e50: 6 total, 5 up, 6 in 2024-04-02T04:53:33.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:33 smithi070 ceph-mon[113635]: osdmap e51: 6 total, 5 up, 6 in 2024-04-02T04:53:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:33 smithi179 ceph-mon[84288]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:53:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:33 smithi179 ceph-mon[84288]: osdmap e50: 6 total, 5 up, 6 in 2024-04-02T04:53:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:33 smithi179 ceph-mon[84288]: osdmap e51: 6 total, 5 up, 6 in 2024-04-02T04:53:33.779 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:33 smithi070 podman[125078]: 2024-04-02 04:53:33.465538325 +0000 UTC m=+0.764953348 container remove 1a0a3ebd83019dfb60543b0deeb89dc0524e400573c4992ba682dcb2b13c6d9d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, ceph=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-02T04:53:34.105 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:33 smithi070 podman[125168]: 2024-04-02 04:53:33.709591308 +0000 UTC m=+0.018430325 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:53:34.105 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:33 smithi070 podman[125168]: 2024-04-02 04:53:33.807942582 +0000 UTC m=+0.116781584 container create bcb72b52a3236f309a20fca52431b410faab967afc09aaa44f2e3b9409c18f46 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:53:34.105 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:34 smithi070 podman[125168]: 2024-04-02 04:53:34.018765303 +0000 UTC m=+0.327604304 container init bcb72b52a3236f309a20fca52431b410faab967afc09aaa44f2e3b9409c18f46 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:53:34.105 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:34 smithi070 podman[125168]: 2024-04-02 04:53:34.021838602 +0000 UTC m=+0.330677603 container start bcb72b52a3236f309a20fca52431b410faab967afc09aaa44f2e3b9409c18f46 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-04-02T04:53:34.106 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:34 smithi070 podman[125168]: 2024-04-02 04:53:34.102160838 +0000 UTC m=+0.410999857 container attach bcb72b52a3236f309a20fca52431b410faab967afc09aaa44f2e3b9409c18f46 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-02T04:53:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.203+0000 7fd7b10fa700 1 -- 172.21.15.70:0/225337581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 msgr2=0x7fd7ac105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.203+0000 7fd7b10fa700 1 --2- 172.21.15.70:0/225337581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 0x7fd7ac105890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009910 tx=0x7fd7a0009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.203+0000 7fd7b10fa700 1 -- 172.21.15.70:0/225337581 shutdown_connections 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.203+0000 7fd7b10fa700 1 --2- 172.21.15.70:0/225337581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 0x7fd7ac105890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.203+0000 7fd7b10fa700 1 --2- 172.21.15.70:0/225337581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7ac100e90 0x7fd7ac101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.203+0000 7fd7b10fa700 1 -- 172.21.15.70:0/225337581 >> 172.21.15.70:0/225337581 conn(0x7fd7ac0fc740 msgr2=0x7fd7ac0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.203+0000 7fd7b10fa700 1 -- 172.21.15.70:0/225337581 shutdown_connections 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.203+0000 7fd7b10fa700 1 -- 172.21.15.70:0/225337581 wait complete. 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.204+0000 7fd7b10fa700 1 Processor -- start 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.204+0000 7fd7b10fa700 1 -- start start 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.204+0000 7fd7b10fa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7ac100e90 0x7fd7ac073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.204+0000 7fd7b10fa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 0x7fd7ac0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.204+0000 7fd7b10fa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac073b10 con 0x7fd7ac101840 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.204+0000 7fd7b10fa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac073c50 con 0x7fd7ac100e90 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.204+0000 7fd7aa59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 0x7fd7ac0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.204+0000 7fd7aa59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 0x7fd7ac0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50688/0 (socket says 172.21.15.70:50688) 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd7aa59c700 1 -- 172.21.15.70:0/4113891241 learned_addr learned my addr 172.21.15.70:0/4113891241 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd7aad9d700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7ac100e90 0x7fd7ac073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd7aa59c700 1 -- 172.21.15.70:0/4113891241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7ac100e90 msgr2=0x7fd7ac073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd7aa59c700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7ac100e90 0x7fd7ac073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd7aa59c700 1 -- 172.21.15.70:0/4113891241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00095f0 con 0x7fd7ac101840 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd7aa59c700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 0x7fd7ac0735d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00092e0 tx=0x7fd7a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd793fff700 1 -- 172.21.15.70:0/4113891241 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7a001d070 con 0x7fd7ac101840 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd793fff700 1 -- 172.21.15.70:0/4113891241 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7a000f460 con 0x7fd7ac101840 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac073ed0 con 0x7fd7ac101840 2024-04-02T04:53:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd793fff700 1 -- 172.21.15.70:0/4113891241 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7a00175d0 con 0x7fd7ac101840 2024-04-02T04:53:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.205+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac1a2da0 con 0x7fd7ac101840 2024-04-02T04:53:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.206+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac109290 con 0x7fd7ac101840 2024-04-02T04:53:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.207+0000 7fd793fff700 1 -- 172.21.15.70:0/4113891241 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fd7a0003a60 con 0x7fd7ac101840 2024-04-02T04:53:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.207+0000 7fd793fff700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd79406dea0 0x7fd794070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.207+0000 7fd7aad9d700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd79406dea0 0x7fd794070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.208+0000 7fd793fff700 1 -- 172.21.15.70:0/4113891241 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd7a008f110 con 0x7fd7ac101840 2024-04-02T04:53:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.208+0000 7fd7aad9d700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd79406dea0 0x7fd794070360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd79c005950 tx=0x7fd79c00a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.212+0000 7fd793fff700 1 -- 172.21.15.70:0/4113891241 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd7a00585b0 con 0x7fd7ac101840 2024-04-02T04:53:34.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.438+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7ac0611d0 con 0x7fd79406dea0 2024-04-02T04:53:34.442 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:34 smithi070 ceph-mon[113635]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:34.442 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:34 smithi070 podman[125168]: 2024-04-02 04:53:34.327781085 +0000 UTC m=+0.636620090 container died bcb72b52a3236f309a20fca52431b410faab967afc09aaa44f2e3b9409c18f46 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-02T04:53:34.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.439+0000 7fd793fff700 1 -- 172.21.15.70:0/4113891241 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd7ac0611d0 con 0x7fd79406dea0 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd79406dea0 msgr2=0x7fd794070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd79406dea0 0x7fd794070360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd79c005950 tx=0x7fd79c00a300 comp rx=0 tx=0).stop 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 msgr2=0x7fd7ac0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 0x7fd7ac0735d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00092e0 tx=0x7fd7a000bd70 comp rx=0 tx=0).stop 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 shutdown_connections 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd79406dea0 0x7fd794070360 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7ac100e90 0x7fd7ac073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 --2- 172.21.15.70:0/4113891241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7ac101840 0x7fd7ac0735d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.441+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 >> 172.21.15.70:0/4113891241 conn(0x7fd7ac0fc740 msgr2=0x7fd7ac0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:34.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.442+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 shutdown_connections 2024-04-02T04:53:34.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.442+0000 7fd7b10fa700 1 -- 172.21.15.70:0/4113891241 wait complete. 2024-04-02T04:53:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:53:34.473 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:34 smithi179 ceph-mon[84288]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.548+0000 7fdc12828700 1 -- 172.21.15.70:0/2795401478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 msgr2=0x7fdc0c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.548+0000 7fdc12828700 1 --2- 172.21.15.70:0/2795401478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 0x7fdc0c1058a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009910 tx=0x7fdbfc009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.548+0000 7fdc12828700 1 -- 172.21.15.70:0/2795401478 shutdown_connections 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.548+0000 7fdc12828700 1 --2- 172.21.15.70:0/2795401478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 0x7fdc0c1058a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.548+0000 7fdc12828700 1 --2- 172.21.15.70:0/2795401478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c100ea0 0x7fdc0c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.548+0000 7fdc12828700 1 -- 172.21.15.70:0/2795401478 >> 172.21.15.70:0/2795401478 conn(0x7fdc0c0fc750 msgr2=0x7fdc0c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.548+0000 7fdc12828700 1 -- 172.21.15.70:0/2795401478 shutdown_connections 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.548+0000 7fdc12828700 1 -- 172.21.15.70:0/2795401478 wait complete. 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc12828700 1 Processor -- start 2024-04-02T04:53:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc12828700 1 -- start start 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc12828700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c100ea0 0x7fdc0c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc12828700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 0x7fdc0c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc12828700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c19d220 con 0x7fdc0c101850 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc12828700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c1966d0 con 0x7fdc0c100ea0 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc0b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 0x7fdc0c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc0b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 0x7fdc0c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50716/0 (socket says 172.21.15.70:50716) 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc0b7fe700 1 -- 172.21.15.70:0/390511577 learned_addr learned my addr 172.21.15.70:0/390511577 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc0bfff700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c100ea0 0x7fdc0c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc0b7fe700 1 -- 172.21.15.70:0/390511577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c100ea0 msgr2=0x7fdc0c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc0b7fe700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c100ea0 0x7fdc0c19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.549+0000 7fdc0b7fe700 1 -- 172.21.15.70:0/390511577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0095f0 con 0x7fdc0c101850 2024-04-02T04:53:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.550+0000 7fdc0b7fe700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 0x7fdc0c19cb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0092e0 tx=0x7fdbfc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.550+0000 7fdc097fa700 1 -- 172.21.15.70:0/390511577 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbfc01d070 con 0x7fdc0c101850 2024-04-02T04:53:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.550+0000 7fdc097fa700 1 -- 172.21.15.70:0/390511577 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbfc005dc0 con 0x7fdc0c101850 2024-04-02T04:53:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.550+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0c196950 con 0x7fdc0c101850 2024-04-02T04:53:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.550+0000 7fdc097fa700 1 -- 172.21.15.70:0/390511577 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbfc017400 con 0x7fdc0c101850 2024-04-02T04:53:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.550+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c196e40 con 0x7fdc0c101850 2024-04-02T04:53:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.551+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0c1091b0 con 0x7fdc0c101850 2024-04-02T04:53:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.551+0000 7fdc097fa700 1 -- 172.21.15.70:0/390511577 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fdbfc00f6f0 con 0x7fdc0c101850 2024-04-02T04:53:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.552+0000 7fdc097fa700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdbf806dea0 0x7fdbf8070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.552+0000 7fdc097fa700 1 -- 172.21.15.70:0/390511577 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdbfc08f000 con 0x7fdc0c101850 2024-04-02T04:53:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.552+0000 7fdc0bfff700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdbf806dea0 0x7fdbf8070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.552+0000 7fdc0bfff700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdbf806dea0 0x7fdbf8070360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4004630 tx=0x7fdbf400b3c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.556+0000 7fdc097fa700 1 -- 172.21.15.70:0/390511577 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdbfc05bfd0 con 0x7fdc0c101850 2024-04-02T04:53:34.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.777+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc0c1970b0 con 0x7fdbf806dea0 2024-04-02T04:53:34.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.779+0000 7fdc097fa700 1 -- 172.21.15.70:0/390511577 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdc0c1970b0 con 0x7fdbf806dea0 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdbf806dea0 msgr2=0x7fdbf8070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdbf806dea0 0x7fdbf8070360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4004630 tx=0x7fdbf400b3c0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 msgr2=0x7fdc0c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 0x7fdc0c19cb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0092e0 tx=0x7fdbfc00bd70 comp rx=0 tx=0).stop 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 shutdown_connections 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdbf806dea0 0x7fdbf8070360 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c100ea0 0x7fdc0c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 --2- 172.21.15.70:0/390511577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc0c101850 0x7fdc0c19cb90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 >> 172.21.15.70:0/390511577 conn(0x7fdc0c0fc750 msgr2=0x7fdc0c105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.781+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 shutdown_connections 2024-04-02T04:53:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.782+0000 7fdc12828700 1 -- 172.21.15.70:0/390511577 wait complete. 2024-04-02T04:53:34.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 -- 172.21.15.70:0/3597220852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 msgr2=0x7fef60101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 --2- 172.21.15.70:0/3597220852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 0x7fef60101280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009910 tx=0x7fef5c009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 -- 172.21.15.70:0/3597220852 shutdown_connections 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 --2- 172.21.15.70:0/3597220852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef60101850 0x7fef601058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 --2- 172.21.15.70:0/3597220852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 0x7fef60101280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 -- 172.21.15.70:0/3597220852 >> 172.21.15.70:0/3597220852 conn(0x7fef600fc750 msgr2=0x7fef600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 -- 172.21.15.70:0/3597220852 shutdown_connections 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 -- 172.21.15.70:0/3597220852 wait complete. 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 Processor -- start 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 -- start start 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 0x7fef6019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef60101850 0x7fef6019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6019d2e0 con 0x7fef60100ea0 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.887+0000 7fef6764f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef60196790 con 0x7fef60101850 2024-04-02T04:53:34.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef653eb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 0x7fef6019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef653eb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 0x7fef6019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50730/0 (socket says 172.21.15.70:50730) 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef653eb700 1 -- 172.21.15.70:0/1763191517 learned_addr learned my addr 172.21.15.70:0/1763191517 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef64bea700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef60101850 0x7fef6019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef653eb700 1 -- 172.21.15.70:0/1763191517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef60101850 msgr2=0x7fef6019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef653eb700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef60101850 0x7fef6019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef653eb700 1 -- 172.21.15.70:0/1763191517 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c0095f0 con 0x7fef60100ea0 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef653eb700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 0x7fef6019c710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005f50 tx=0x7fef5c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef567fc700 1 -- 172.21.15.70:0/1763191517 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef5c01d070 con 0x7fef60100ea0 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef567fc700 1 -- 172.21.15.70:0/1763191517 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef5c00b610 con 0x7fef60100ea0 2024-04-02T04:53:34.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef60196a10 con 0x7fef60100ea0 2024-04-02T04:53:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef567fc700 1 -- 172.21.15.70:0/1763191517 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef5c022470 con 0x7fef60100ea0 2024-04-02T04:53:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.888+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef60196f00 con 0x7fef60100ea0 2024-04-02T04:53:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.889+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef60109240 con 0x7fef60100ea0 2024-04-02T04:53:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.890+0000 7fef567fc700 1 -- 172.21.15.70:0/1763191517 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fef5c003780 con 0x7fef60100ea0 2024-04-02T04:53:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.890+0000 7fef567fc700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fef4c06dea0 0x7fef4c070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.890+0000 7fef567fc700 1 -- 172.21.15.70:0/1763191517 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fef5c08f0a0 con 0x7fef60100ea0 2024-04-02T04:53:34.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.890+0000 7fef64bea700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fef4c06dea0 0x7fef4c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:34.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.891+0000 7fef64bea700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fef4c06dea0 0x7fef4c070360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef60197c00 tx=0x7fef50009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:34.895+0000 7fef567fc700 1 -- 172.21.15.70:0/1763191517 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fef5c05bff0 con 0x7fef60100ea0 2024-04-02T04:53:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.103+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef60197d40 con 0x7fef4c06dea0 2024-04-02T04:53:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.115+0000 7fef567fc700 1 -- 172.21.15.70:0/1763191517 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fef60197d40 con 0x7fef4c06dea0 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (3m) 12s ago 14m 26.4M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (14m) 12s ago 14m 8158k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (13m) 15s ago 13m 7818k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (23s) 12s ago 14m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (18s) 15s ago 13m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (2m) 12s ago 13m 76.3M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (8m) 12s ago 8m 152M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (8m) 12s ago 8m 19.5M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (8m) 15s ago 8m 17.8M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (8m) 15s ago 8m 19.0M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (5m) 12s ago 15m 495M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:53:35.119 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (4m) 15s ago 13m 449M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (92s) 12s ago 15m 44.0M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (67s) 15s ago 13m 41.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (3m) 12s ago 14m 15.7M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (3m) 15s ago 13m 16.8M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (12m) 12s ago 12m 229M 1216M 18.2.1 5be31c24972a 1a0a3ebd8301 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (11m) 12s ago 11m 264M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (11m) 12s ago 11m 213M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (11m) 15s ago 11m 298M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (10m) 15s ago 10m 275M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (10m) 15s ago 10m 228M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:53:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (3m) 12s ago 13m 62.9M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.117+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fef4c06dea0 msgr2=0x7fef4c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.117+0000 7fef6764f700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fef4c06dea0 0x7fef4c070360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef60197c00 tx=0x7fef50009260 comp rx=0 tx=0).stop 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 msgr2=0x7fef6019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 0x7fef6019c710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005f50 tx=0x7fef5c00f690 comp rx=0 tx=0).stop 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 shutdown_connections 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fef4c06dea0 0x7fef4c070360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60100ea0 0x7fef6019c710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 --2- 172.21.15.70:0/1763191517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef60101850 0x7fef6019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 >> 172.21.15.70:0/1763191517 conn(0x7fef600fc750 msgr2=0x7fef600fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:35.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 shutdown_connections 2024-04-02T04:53:35.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.118+0000 7fef6764f700 1 -- 172.21.15.70:0/1763191517 wait complete. 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 -- 172.21.15.70:0/4108143717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc068580 msgr2=0x7f05cc0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 --2- 172.21.15.70:0/4108143717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc068580 0x7f05cc0689e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009910 tx=0x7f05b8009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 -- 172.21.15.70:0/4108143717 shutdown_connections 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 --2- 172.21.15.70:0/4108143717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc068580 0x7f05cc0689e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 --2- 172.21.15.70:0/4108143717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05cc106cd0 0x7f05cc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 -- 172.21.15.70:0/4108143717 >> 172.21.15.70:0/4108143717 conn(0x7f05cc075670 msgr2=0x7f05cc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 -- 172.21.15.70:0/4108143717 shutdown_connections 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 -- 172.21.15.70:0/4108143717 wait complete. 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 Processor -- start 2024-04-02T04:53:35.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.232+0000 7f05d2aee700 1 -- start start 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05d2aee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05cc068580 0x7f05cc19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05d2aee700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc106cd0 0x7f05cc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05d2aee700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc19d120 con 0x7f05cc106cd0 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05d2aee700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc1966b0 con 0x7f05cc068580 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05cbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc106cd0 0x7f05cc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05cbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc106cd0 0x7f05cc19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50750/0 (socket says 172.21.15.70:50750) 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05cbfff700 1 -- 172.21.15.70:0/3522883447 learned_addr learned my addr 172.21.15.70:0/3522883447 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05d088a700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05cc068580 0x7f05cc19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05cbfff700 1 -- 172.21.15.70:0/3522883447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05cc068580 msgr2=0x7f05cc19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05cbfff700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05cc068580 0x7f05cc19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05cbfff700 1 -- 172.21.15.70:0/3522883447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b80095f0 con 0x7f05cc106cd0 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05d088a700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05cc068580 0x7f05cc19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:53:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05cbfff700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc106cd0 0x7f05cc19cb00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05b800f690 tx=0x7f05b800f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:35.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05c9ffb700 1 -- 172.21.15.70:0/3522883447 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b801d070 con 0x7f05cc106cd0 2024-04-02T04:53:35.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05c9ffb700 1 -- 172.21.15.70:0/3522883447 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05b80050d0 con 0x7f05cc106cd0 2024-04-02T04:53:35.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.233+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc196930 con 0x7f05cc106cd0 2024-04-02T04:53:35.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.234+0000 7f05c9ffb700 1 -- 172.21.15.70:0/3522883447 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b8022470 con 0x7f05cc106cd0 2024-04-02T04:53:35.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.234+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc196e20 con 0x7f05cc106cd0 2024-04-02T04:53:35.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.235+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc1091d0 con 0x7f05cc106cd0 2024-04-02T04:53:35.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.235+0000 7f05c9ffb700 1 -- 172.21.15.70:0/3522883447 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f05b8003780 con 0x7f05cc106cd0 2024-04-02T04:53:35.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.235+0000 7f05c9ffb700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f05bc06de90 0x7f05bc070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:35.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.236+0000 7f05c9ffb700 1 -- 172.21.15.70:0/3522883447 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f05b808f0f0 con 0x7f05cc106cd0 2024-04-02T04:53:35.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.236+0000 7f05d088a700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f05bc06de90 0x7f05bc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:35.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.236+0000 7f05d088a700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f05bc06de90 0x7f05bc070350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f05c0011490 tx=0x7f05c0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:35.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.240+0000 7f05c9ffb700 1 -- 172.21.15.70:0/3522883447 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f05b805c0f0 con 0x7f05cc106cd0 2024-04-02T04:53:35.337 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 podman[125217]: 2024-04-02 04:53:35.081585155 +0000 UTC m=+0.748592271 container remove bcb72b52a3236f309a20fca52431b410faab967afc09aaa44f2e3b9409c18f46 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-02T04:53:35.337 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.0.service: Deactivated successfully. 2024-04-02T04:53:35.337 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 systemd[1]: Stopped Ceph osd.0 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:53:35.337 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.0.service: Consumed 1min 1.486s CPU time. 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.552+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f05cc197bf0 con 0x7f05cc106cd0 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.552+0000 7f05c9ffb700 1 -- 172.21.15.70:0/3522883447 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+687 (secure 0 0 0) 0x7f05b805bc80 con 0x7f05cc106cd0 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:53:35.556 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:53:35.557 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:53:35.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f05bc06de90 msgr2=0x7f05bc070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f05bc06de90 0x7f05bc070350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f05c0011490 tx=0x7f05c0009260 comp rx=0 tx=0).stop 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc106cd0 msgr2=0x7f05cc19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc106cd0 0x7f05cc19cb00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05b800f690 tx=0x7f05b800f6c0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 shutdown_connections 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f05bc06de90 0x7f05bc070350 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05cc068580 0x7f05cc19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 --2- 172.21.15.70:0/3522883447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05cc106cd0 0x7f05cc19cb00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 >> 172.21.15.70:0/3522883447 conn(0x7f05cc075670 msgr2=0x7f05cc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 shutdown_connections 2024-04-02T04:53:35.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.555+0000 7f05d2aee700 1 -- 172.21.15.70:0/3522883447 wait complete. 2024-04-02T04:53:35.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.663+0000 7f67173a8700 1 -- 172.21.15.70:0/2618775291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 msgr2=0x7f6710102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.663+0000 7f67173a8700 1 --2- 172.21.15.70:0/2618775291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 0x7f6710102240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f670c009940 tx=0x7f670c009c50 comp rx=0 tx=0).stop 2024-04-02T04:53:35.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.663+0000 7f67173a8700 1 -- 172.21.15.70:0/2618775291 shutdown_connections 2024-04-02T04:53:35.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.663+0000 7f67173a8700 1 --2- 172.21.15.70:0/2618775291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6710102780 0x7f671010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.663+0000 7f67173a8700 1 --2- 172.21.15.70:0/2618775291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 0x7f6710102240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.663+0000 7f67173a8700 1 -- 172.21.15.70:0/2618775291 >> 172.21.15.70:0/2618775291 conn(0x7f67100fb610 msgr2=0x7f67100fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:35.667 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 systemd[1]: Starting Ceph osd.0 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:53:35.667 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 podman[125391]: 2024-04-02 04:53:35.46647432 +0000 UTC m=+0.021665082 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:53:35.667 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 podman[125391]: 2024-04-02 04:53:35.582732579 +0000 UTC m=+0.137923324 container create 9b9a1bd09d23b538db82460811f670ab67e4981701f94530a2306d26c5e41218 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.663+0000 7f67173a8700 1 -- 172.21.15.70:0/2618775291 shutdown_connections 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.663+0000 7f67173a8700 1 -- 172.21.15.70:0/2618775291 wait complete. 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f67173a8700 1 Processor -- start 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f67173a8700 1 -- start start 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f67173a8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 0x7f67100ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f67173a8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6710102780 0x7f6710100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f67173a8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710100540 con 0x7f6710101e60 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f67173a8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710100680 con 0x7f6710102780 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f6715144700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 0x7f67100ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f6715144700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 0x7f67100ffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50760/0 (socket says 172.21.15.70:50760) 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f6715144700 1 -- 172.21.15.70:0/2654841 learned_addr learned my addr 172.21.15.70:0/2654841 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f6714943700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6710102780 0x7f6710100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f6715144700 1 -- 172.21.15.70:0/2654841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6710102780 msgr2=0x7f6710100000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f6715144700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6710102780 0x7f6710100000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f6715144700 1 -- 172.21.15.70:0/2654841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0095f0 con 0x7f6710101e60 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f6715144700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 0x7f67100ffac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f670c0055f0 tx=0x7f670c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.664+0000 7f67067fc700 1 -- 172.21.15.70:0/2654841 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670c01d070 con 0x7f6710101e60 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.665+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6710100900 con 0x7f6710101e60 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.665+0000 7f67067fc700 1 -- 172.21.15.70:0/2654841 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f670c005dc0 con 0x7f6710101e60 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.665+0000 7f67067fc700 1 -- 172.21.15.70:0/2654841 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670c021410 con 0x7f6710101e60 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.665+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67101a2dd0 con 0x7f6710101e60 2024-04-02T04:53:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.665+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6710108370 con 0x7f6710101e60 2024-04-02T04:53:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.666+0000 7f67067fc700 1 -- 172.21.15.70:0/2654841 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f670c00f810 con 0x7f6710101e60 2024-04-02T04:53:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.666+0000 7f67067fc700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f66fc06de90 0x7f66fc070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.667+0000 7f67067fc700 1 -- 172.21.15.70:0/2654841 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f670c08ef60 con 0x7f6710101e60 2024-04-02T04:53:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.667+0000 7f6714943700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f66fc06de90 0x7f66fc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.667+0000 7f6714943700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f66fc06de90 0x7f66fc070350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67101012b0 tx=0x7f670000b3c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:35.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.672+0000 7f67067fc700 1 -- 172.21.15.70:0/2654841 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f670c0c3280 con 0x7f6710101e60 2024-04-02T04:53:35.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.941+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6710100ff0 con 0x7f6710101e60 2024-04-02T04:53:35.945 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 podman[125391]: 2024-04-02 04:53:35.794207806 +0000 UTC m=+0.349398564 container init 9b9a1bd09d23b538db82460811f670ab67e4981701f94530a2306d26c5e41218 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-6cec793) 2024-04-02T04:53:35.945 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 podman[125391]: 2024-04-02 04:53:35.798975828 +0000 UTC m=+0.354166580 container start 9b9a1bd09d23b538db82460811f670ab67e4981701f94530a2306d26c5e41218 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:53:35.945 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:35 smithi070 podman[125391]: 2024-04-02 04:53:35.885440256 +0000 UTC m=+0.440631018 container attach 9b9a1bd09d23b538db82460811f670ab67e4981701f94530a2306d26c5e41218 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.942+0000 7f67067fc700 1 -- 172.21.15.70:0/2654841 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f670c05beb0 con 0x7f6710101e60 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:53:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:53:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:53:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:53:35.949 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:53:35.949 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:53:35.949 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:53:35.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f66fc06de90 msgr2=0x7f66fc070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f66fc06de90 0x7f66fc070350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67101012b0 tx=0x7f670000b3c0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 msgr2=0x7f67100ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 0x7f67100ffac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f670c0055f0 tx=0x7f670c005620 comp rx=0 tx=0).stop 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 shutdown_connections 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f66fc06de90 0x7f66fc070350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6710101e60 0x7f67100ffac0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 --2- 172.21.15.70:0/2654841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6710102780 0x7f6710100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.944+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 >> 172.21.15.70:0/2654841 conn(0x7f67100fb610 msgr2=0x7f67101054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.945+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 shutdown_connections 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:35.945+0000 7f67173a8700 1 -- 172.21.15.70:0/2654841 wait complete. 2024-04-02T04:53:35.950 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:53:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.054+0000 7f180af97700 1 -- 172.21.15.70:0/3132773279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 msgr2=0x7f180410d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.054+0000 7f180af97700 1 --2- 172.21.15.70:0/3132773279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 0x7f180410d000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009910 tx=0x7f17f0009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.054+0000 7f180af97700 1 -- 172.21.15.70:0/3132773279 shutdown_connections 2024-04-02T04:53:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.054+0000 7f180af97700 1 --2- 172.21.15.70:0/3132773279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 0x7f180410d000 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.054+0000 7f180af97700 1 --2- 172.21.15.70:0/3132773279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18040ff990 0x7f18040ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.054+0000 7f180af97700 1 -- 172.21.15.70:0/3132773279 >> 172.21.15.70:0/3132773279 conn(0x7f18040fb650 msgr2=0x7f18040fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.054+0000 7f180af97700 1 -- 172.21.15.70:0/3132773279 shutdown_connections 2024-04-02T04:53:36.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.054+0000 7f180af97700 1 -- 172.21.15.70:0/3132773279 wait complete. 2024-04-02T04:53:36.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f180af97700 1 Processor -- start 2024-04-02T04:53:36.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f180af97700 1 -- start start 2024-04-02T04:53:36.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f180af97700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18040ff990 0x7f180419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:36.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f180af97700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 0x7f180419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:36.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f180af97700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180419f430 con 0x7f1804100340 2024-04-02T04:53:36.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f180af97700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041988e0 con 0x7f18040ff990 2024-04-02T04:53:36.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f1803fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 0x7f180419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:36.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f1803fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 0x7f180419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50782/0 (socket says 172.21.15.70:50782) 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f1803fff700 1 -- 172.21.15.70:0/3764754800 learned_addr learned my addr 172.21.15.70:0/3764754800 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.055+0000 7f1808d33700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18040ff990 0x7f180419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f1803fff700 1 -- 172.21.15.70:0/3764754800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18040ff990 msgr2=0x7f180419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f1803fff700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18040ff990 0x7f180419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f1803fff700 1 -- 172.21.15.70:0/3764754800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f00095f0 con 0x7f1804100340 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f1803fff700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 0x7f180419eda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f17f0000c00 tx=0x7f17f0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f1801ffb700 1 -- 172.21.15.70:0/3764754800 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17f001d070 con 0x7f1804100340 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f1801ffb700 1 -- 172.21.15.70:0/3764754800 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17f00052a0 con 0x7f1804100340 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1804198b60 con 0x7f1804100340 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f1801ffb700 1 -- 172.21.15.70:0/3764754800 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17f0022470 con 0x7f1804100340 2024-04-02T04:53:36.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.056+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1804199050 con 0x7f1804100340 2024-04-02T04:53:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.057+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180410a770 con 0x7f1804100340 2024-04-02T04:53:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.058+0000 7f1801ffb700 1 -- 172.21.15.70:0/3764754800 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f17f0003680 con 0x7f1804100340 2024-04-02T04:53:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.058+0000 7f1801ffb700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f17f406de90 0x7f17f4070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.058+0000 7f1801ffb700 1 -- 172.21.15.70:0/3764754800 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f17f008f180 con 0x7f1804100340 2024-04-02T04:53:36.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.058+0000 7f1808d33700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f17f406de90 0x7f17f4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:36.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.059+0000 7f1808d33700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f17f406de90 0x7f17f4070350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009c60 tx=0x7f17f8009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:36.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.063+0000 7f1801ffb700 1 -- 172.21.15.70:0/3764754800 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f17f005c070 con 0x7f1804100340 2024-04-02T04:53:36.255 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-mon[113635]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:36.255 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-mon[113635]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:36.255 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-mon[113635]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:36.255 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-mon[113635]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:36.255 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3522883447' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:36.255 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2654841' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:53:36.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.287+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1804199270 con 0x7f17f406de90 2024-04-02T04:53:36.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.289+0000 7f1801ffb700 1 -- 172.21.15.70:0/3764754800 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1804199270 con 0x7f17f406de90 2024-04-02T04:53:36.292 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "6/23 daemons upgraded", 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:53:36.293 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f17f406de90 msgr2=0x7f17f4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f17f406de90 0x7f17f4070350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009c60 tx=0x7f17f8009260 comp rx=0 tx=0).stop 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 msgr2=0x7f180419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 0x7f180419eda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f17f0000c00 tx=0x7f17f0005f00 comp rx=0 tx=0).stop 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 shutdown_connections 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f17f406de90 0x7f17f4070350 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18040ff990 0x7f180419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 --2- 172.21.15.70:0/3764754800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1804100340 0x7f180419eda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.291+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 >> 172.21.15.70:0/3764754800 conn(0x7f18040fb650 msgr2=0x7f18040fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.292+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 shutdown_connections 2024-04-02T04:53:36.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.292+0000 7f180af97700 1 -- 172.21.15.70:0/3764754800 wait complete. 2024-04-02T04:53:36.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 -- 172.21.15.70:0/1120471270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c101d50 msgr2=0x7f335c102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 --2- 172.21.15.70:0/1120471270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c101d50 0x7f335c102130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3350009910 tx=0x7f3350009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 -- 172.21.15.70:0/1120471270 shutdown_connections 2024-04-02T04:53:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 --2- 172.21.15.70:0/1120471270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f335c102700 0x7f335c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 --2- 172.21.15.70:0/1120471270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c101d50 0x7f335c102130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 -- 172.21.15.70:0/1120471270 >> 172.21.15.70:0/1120471270 conn(0x7f335c0fb610 msgr2=0x7f335c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 -- 172.21.15.70:0/1120471270 shutdown_connections 2024-04-02T04:53:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 -- 172.21.15.70:0/1120471270 wait complete. 2024-04-02T04:53:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.395+0000 7f3361a65700 1 Processor -- start 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f3361a65700 1 -- start start 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f3361a65700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f335c101d50 0x7f335c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f3361a65700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c102700 0x7f335c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f3361a65700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c19d1e0 con 0x7f335c102700 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f3361a65700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c196690 con 0x7f335c101d50 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c102700 0x7f335c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c102700 0x7f335c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50796/0 (socket says 172.21.15.70:50796) 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335affd700 1 -- 172.21.15.70:0/2530122659 learned_addr learned my addr 172.21.15.70:0/2530122659 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335b7fe700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f335c101d50 0x7f335c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:36.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335affd700 1 -- 172.21.15.70:0/2530122659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f335c101d50 msgr2=0x7f335c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335affd700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f335c101d50 0x7f335c19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335affd700 1 -- 172.21.15.70:0/2530122659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33500095f0 con 0x7f335c102700 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335b7fe700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f335c101d50 0x7f335c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.396+0000 7f335affd700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c102700 0x7f335c19cb50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f334c00d900 tx=0x7f334c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.397+0000 7f3358ff9700 1 -- 172.21.15.70:0/2530122659 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334c009900 con 0x7f335c102700 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.397+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c196970 con 0x7f335c102700 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.397+0000 7f3358ff9700 1 -- 172.21.15.70:0/2530122659 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f334c010460 con 0x7f335c102700 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.397+0000 7f3358ff9700 1 -- 172.21.15.70:0/2530122659 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334c00f5f0 con 0x7f335c102700 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.397+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c196ec0 con 0x7f335c102700 2024-04-02T04:53:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.398+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c1082f0 con 0x7f335c102700 2024-04-02T04:53:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.398+0000 7f3358ff9700 1 -- 172.21.15.70:0/2530122659 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f334c0105d0 con 0x7f335c102700 2024-04-02T04:53:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.398+0000 7f3358ff9700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f334406dea0 0x7f3344070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.399+0000 7f3358ff9700 1 -- 172.21.15.70:0/2530122659 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f334c08db80 con 0x7f335c102700 2024-04-02T04:53:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.399+0000 7f335b7fe700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f334406dea0 0x7f3344070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.399+0000 7f335b7fe700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f334406dea0 0x7f3344070360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f335000b5c0 tx=0x7f33500053a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:36.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.403+0000 7f3358ff9700 1 -- 172.21.15.70:0/2530122659 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f334c05aad0 con 0x7f335c102700 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate[125429]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate[125429]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 bash[125391]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 bash[125391]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 bash[125391]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 bash[125391]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 bash[125391]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 bash[125391]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 bash[125391]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-02T04:53:36.530 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate[125429]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-02T04:53:36.531 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate[125429]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-02T04:53:36.531 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate[125429]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-02T04:53:36.531 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate[125429]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T04:53:36.531 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate[125429]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-02T04:53:36.531 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 podman[125391]: 2024-04-02 04:53:36.277223697 +0000 UTC m=+0.832414444 container died 9b9a1bd09d23b538db82460811f670ab67e4981701f94530a2306d26c5e41218 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-02T04:53:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:36 smithi179 ceph-mon[84288]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:36 smithi179 ceph-mon[84288]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:36 smithi179 ceph-mon[84288]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:36 smithi179 ceph-mon[84288]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:36 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3522883447' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:36 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2654841' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:53:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.682+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f335c04f2e0 con 0x7f335c102700 2024-04-02T04:53:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.682+0000 7f3358ff9700 1 -- 172.21.15.70:0/2530122659 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f334c05a660 con 0x7f335c102700 2024-04-02T04:53:36.686 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 osds down 2024-04-02T04:53:36.686 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-02T04:53:36.686 INFO:teuthology.orchestra.run.smithi070.stdout: osd.0 (root=default,host=smithi070) is down 2024-04-02T04:53:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f334406dea0 msgr2=0x7f3344070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f334406dea0 0x7f3344070360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f335000b5c0 tx=0x7f33500053a0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c102700 msgr2=0x7f335c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c102700 0x7f335c19cb50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f334c00d900 tx=0x7f334c00dcc0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 shutdown_connections 2024-04-02T04:53:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f334406dea0 0x7f3344070360 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f335c101d50 0x7f335c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 --2- 172.21.15.70:0/2530122659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f335c102700 0x7f335c19cb50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 >> 172.21.15.70:0/2530122659 conn(0x7f335c0fb610 msgr2=0x7f335c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 shutdown_connections 2024-04-02T04:53:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:53:36.685+0000 7f3361a65700 1 -- 172.21.15.70:0/2530122659 wait complete. 2024-04-02T04:53:37.168 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:36 smithi070 podman[125592]: 2024-04-02 04:53:36.889500736 +0000 UTC m=+0.606526187 container remove 9b9a1bd09d23b538db82460811f670ab67e4981701f94530a2306d26c5e41218 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-activate, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326) 2024-04-02T04:53:37.169 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:37 smithi070 podman[125680]: 2024-04-02 04:53:37.090414173 +0000 UTC m=+0.105674041 container create 0dd09ba7bb691e68d699d3782f2c8b86ff46d17aa5dfe17dd11d10273f876587 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T04:53:37.169 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:37 smithi070 podman[125680]: 2024-04-02 04:53:37.002491478 +0000 UTC m=+0.017751348 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:53:37.421 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:37 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2530122659' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:53:37.421 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:37 smithi070 podman[125680]: 2024-04-02 04:53:37.318194384 +0000 UTC m=+0.333454256 container init 0dd09ba7bb691e68d699d3782f2c8b86ff46d17aa5dfe17dd11d10273f876587 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:53:37.421 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:37 smithi070 podman[125680]: 2024-04-02 04:53:37.321189188 +0000 UTC m=+0.336449057 container start 0dd09ba7bb691e68d699d3782f2c8b86ff46d17aa5dfe17dd11d10273f876587 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:53:37.421 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:37 smithi070 ceph-osd[125699]: -- 172.21.15.70:0/354010951 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55556d7e5680 con 0x55556d7af000 2024-04-02T04:53:37.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:37 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2530122659' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:53:37.780 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:37 smithi070 bash[125680]: 0dd09ba7bb691e68d699d3782f2c8b86ff46d17aa5dfe17dd11d10273f876587 2024-04-02T04:53:37.780 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:37 smithi070 systemd[1]: Started Ceph osd.0 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:53:38.170 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:37 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[125695]: 2024-04-02T04:53:37.861+0000 7fa2fe0437c0 -1 Falling back to public interface 2024-04-02T04:53:38.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:38 smithi070 ceph-mon[113635]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:38.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:38 smithi070 ceph-mon[113635]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:53:38.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:38 smithi070 ceph-mon[113635]: Health check failed: Degraded data redundancy: 40/228 objects degraded (17.544%), 18 pgs degraded (PG_DEGRADED) 2024-04-02T04:53:38.447 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:38 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:38.447 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:38 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:38.447 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:38 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:38 smithi179 ceph-mon[84288]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:38 smithi179 ceph-mon[84288]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:53:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:38 smithi179 ceph-mon[84288]: Health check failed: Degraded data redundancy: 40/228 objects degraded (17.544%), 18 pgs degraded (PG_DEGRADED) 2024-04-02T04:53:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:38 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:38 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:38 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:40.494 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:40 smithi070 ceph-mon[113635]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:53:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:40 smithi179 ceph-mon[84288]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:53:42.280 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:42 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[125695]: 2024-04-02T04:53:42.066+0000 7fa2fe0437c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-04-02T04:53:42.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:42 smithi070 ceph-mon[113635]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:53:42.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:42.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:42 smithi179 ceph-mon[84288]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:53:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:42.779 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:42 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[125695]: 2024-04-02T04:53:42.510+0000 7fa2fe0437c0 -1 osd.0 49 log_to_monitors true 2024-04-02T04:53:43.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:43 smithi070 ceph-mon[113635]: from='osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T04:53:43.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:43.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:43 smithi179 ceph-mon[84288]: from='osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T04:53:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:44.149 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 04:53:43 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[125695]: 2024-04-02T04:53:43.862+0000 7fa2eb493700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:53:44.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:44 smithi070 ceph-mon[113635]: pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 40/228 objects degraded (17.544%) 2024-04-02T04:53:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:44 smithi070 ceph-mon[113635]: from='osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T04:53:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:44 smithi070 ceph-mon[113635]: osdmap e52: 6 total, 5 up, 6 in 2024-04-02T04:53:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:44 smithi070 ceph-mon[113635]: from='osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:53:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:44 smithi070 ceph-mon[113635]: mgrmap e37: smithi070.igidvr(active, since 92s), standbys: smithi179.labmvs 2024-04-02T04:53:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:44 smithi070 ceph-mon[113635]: from='osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577]' entity='osd.0' 2024-04-02T04:53:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:44 smithi179 ceph-mon[84288]: pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 40/228 objects degraded (17.544%) 2024-04-02T04:53:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:44 smithi179 ceph-mon[84288]: from='osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T04:53:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:44 smithi179 ceph-mon[84288]: osdmap e52: 6 total, 5 up, 6 in 2024-04-02T04:53:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:44 smithi179 ceph-mon[84288]: from='osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:53:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:44 smithi179 ceph-mon[84288]: mgrmap e37: smithi070.igidvr(active, since 92s), standbys: smithi179.labmvs 2024-04-02T04:53:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:44 smithi179 ceph-mon[84288]: from='osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577]' entity='osd.0' 2024-04-02T04:53:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:45.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:45 smithi070 ceph-mon[113635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:53:45.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:45 smithi070 ceph-mon[113635]: osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577] boot 2024-04-02T04:53:45.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:45 smithi070 ceph-mon[113635]: osdmap e53: 6 total, 6 up, 6 in 2024-04-02T04:53:45.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:45 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:53:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:45 smithi179 ceph-mon[84288]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:53:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:45 smithi179 ceph-mon[84288]: osd.0 [v2:172.21.15.70:6802/4011619577,v1:172.21.15.70:6803/4011619577] boot 2024-04-02T04:53:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:45 smithi179 ceph-mon[84288]: osdmap e53: 6 total, 6 up, 6 in 2024-04-02T04:53:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:45 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:53:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:46 smithi070 ceph-mon[113635]: pgmap v54: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 40/228 objects degraded (17.544%) 2024-04-02T04:53:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:46 smithi070 ceph-mon[113635]: osdmap e54: 6 total, 6 up, 6 in 2024-04-02T04:53:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:46 smithi179 ceph-mon[84288]: pgmap v54: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 40/228 objects degraded (17.544%) 2024-04-02T04:53:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:46 smithi179 ceph-mon[84288]: osdmap e54: 6 total, 6 up, 6 in 2024-04-02T04:53:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:47 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 6/228 objects degraded (2.632%), 3 pgs degraded (PG_DEGRADED) 2024-04-02T04:53:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:47 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 6/228 objects degraded (2.632%), 3 pgs degraded (PG_DEGRADED) 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 6/228 objects degraded (2.632%); 46 B/s, 0 objects/s recovering 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:48.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:53:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 6/228 objects degraded (2.632%); 46 B/s, 0 objects/s recovering 2024-04-02T04:53:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:53:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:48.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:48.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:48.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:53:49.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:49 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:53:49.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:49 smithi070 ceph-mon[113635]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-04-02T04:53:49.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:49 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:53:49.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:49 smithi179 ceph-mon[84288]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-04-02T04:53:50.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:50 smithi070 ceph-mon[113635]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46 B/s, 0 objects/s recovering 2024-04-02T04:53:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:50 smithi070 ceph-mon[113635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/228 objects degraded (2.632%), 3 pgs degraded) 2024-04-02T04:53:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:50 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:53:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:50 smithi179 ceph-mon[84288]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46 B/s, 0 objects/s recovering 2024-04-02T04:53:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:50 smithi179 ceph-mon[84288]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/228 objects degraded (2.632%), 3 pgs degraded) 2024-04-02T04:53:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:50 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:53:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:52 smithi070 ceph-mon[113635]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-04-02T04:53:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:52 smithi179 ceph-mon[84288]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-04-02T04:53:54.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:53 smithi179 ceph-mon[84288]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34 B/s, 0 objects/s recovering 2024-04-02T04:53:54.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:53 smithi070 ceph-mon[113635]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34 B/s, 0 objects/s recovering 2024-04-02T04:53:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:56 smithi070 ceph-mon[113635]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-04-02T04:53:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:56 smithi179 ceph-mon[84288]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-04-02T04:53:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:53:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:53:58 smithi070 ceph-mon[113635]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 25 B/s, 0 objects/s recovering 2024-04-02T04:53:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:53:58 smithi179 ceph-mon[84288]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 25 B/s, 0 objects/s recovering 2024-04-02T04:54:00.501 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:00 smithi070 ceph-mon[113635]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:00 smithi179 ceph-mon[84288]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:02 smithi070 ceph-mon[113635]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:02.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:02 smithi179 ceph-mon[84288]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:03.413 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:54:03.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:54:04.473 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:04 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:54:04.473 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:04 smithi179 ceph-mon[84288]: Upgrade: osd.1 is safe to restart 2024-04-02T04:54:04.473 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:04 smithi179 ceph-mon[84288]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:04 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:54:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:04 smithi070 ceph-mon[113635]: Upgrade: osd.1 is safe to restart 2024-04-02T04:54:04.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:04 smithi070 ceph-mon[113635]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:05.491 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:05 smithi070 ceph-mon[113635]: Upgrade: Updating osd.1 2024-04-02T04:54:05.492 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:05 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:05.492 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:05 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:54:05.492 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:05 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:54:05.492 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:05 smithi070 ceph-mon[113635]: Deploying daemon osd.1 on smithi070 2024-04-02T04:54:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:05 smithi179 ceph-mon[84288]: Upgrade: Updating osd.1 2024-04-02T04:54:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:05 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:05 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:54:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:05 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:54:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:05 smithi179 ceph-mon[84288]: Deploying daemon osd.1 on smithi070 2024-04-02T04:54:06.507 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:06 smithi070 ceph-mon[113635]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:06 smithi179 ceph-mon[84288]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.794+0000 7f6e22d8b700 1 -- 172.21.15.70:0/1584816637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c101de0 msgr2=0x7f6e1c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.794+0000 7f6e22d8b700 1 --2- 172.21.15.70:0/1584816637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c101de0 0x7f6e1c1021c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400b0a0 tx=0x7f6e1400b3b0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 -- 172.21.15.70:0/1584816637 shutdown_connections 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 --2- 172.21.15.70:0/1584816637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e1c102700 0x7f6e1c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 --2- 172.21.15.70:0/1584816637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c101de0 0x7f6e1c1021c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 -- 172.21.15.70:0/1584816637 >> 172.21.15.70:0/1584816637 conn(0x7f6e1c0fb650 msgr2=0x7f6e1c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 -- 172.21.15.70:0/1584816637 shutdown_connections 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 -- 172.21.15.70:0/1584816637 wait complete. 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 Processor -- start 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 -- start start 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e1c101de0 0x7f6e1c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c102700 0x7f6e1c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c19d270 con 0x7f6e1c102700 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.795+0000 7f6e22d8b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c196720 con 0x7f6e1c101de0 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e1bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c102700 0x7f6e1c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e1bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c102700 0x7f6e1c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49086/0 (socket says 172.21.15.70:49086) 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e1bfff700 1 -- 172.21.15.70:0/2285740366 learned_addr learned my addr 172.21.15.70:0/2285740366 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e20b27700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e1c101de0 0x7f6e1c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e1bfff700 1 -- 172.21.15.70:0/2285740366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e1c101de0 msgr2=0x7f6e1c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e1bfff700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e1c101de0 0x7f6e1c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e1bfff700 1 -- 172.21.15.70:0/2285740366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14009d00 con 0x7f6e1c102700 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e1bfff700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c102700 0x7f6e1c19cbe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000d6d0 tx=0x7f6e1000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e19ffb700 1 -- 172.21.15.70:0/2285740366 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1000f860 con 0x7f6e1c102700 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e19ffb700 1 -- 172.21.15.70:0/2285740366 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e1000fe80 con 0x7f6e1c102700 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e19ffb700 1 -- 172.21.15.70:0/2285740366 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e10010ea0 con 0x7f6e1c102700 2024-04-02T04:54:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c196a00 con 0x7f6e1c102700 2024-04-02T04:54:06.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.796+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c196f50 con 0x7f6e1c102700 2024-04-02T04:54:06.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.798+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c1082f0 con 0x7f6e1c102700 2024-04-02T04:54:06.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.798+0000 7f6e19ffb700 1 -- 172.21.15.70:0/2285740366 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f6e10010460 con 0x7f6e1c102700 2024-04-02T04:54:06.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.798+0000 7f6e19ffb700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6e0406dd60 0x7f6e04070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.798+0000 7f6e19ffb700 1 -- 172.21.15.70:0/2285740366 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6e1008d770 con 0x7f6e1c102700 2024-04-02T04:54:06.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.798+0000 7f6e20b27700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6e0406dd60 0x7f6e04070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.799+0000 7f6e20b27700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6e0406dd60 0x7f6e04070220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e14000f80 tx=0x7f6e140060c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:06.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:06.803+0000 7f6e19ffb700 1 -- 172.21.15.70:0/2285740366 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6e1005a8e0 con 0x7f6e1c102700 2024-04-02T04:54:07.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.016+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e1c1971d0 con 0x7f6e0406dd60 2024-04-02T04:54:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.018+0000 7f6e19ffb700 1 -- 172.21.15.70:0/2285740366 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6e1c1971d0 con 0x7f6e0406dd60 2024-04-02T04:54:07.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6e0406dd60 msgr2=0x7f6e04070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6e0406dd60 0x7f6e04070220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e14000f80 tx=0x7f6e140060c0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c102700 msgr2=0x7f6e1c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c102700 0x7f6e1c19cbe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000d6d0 tx=0x7f6e1000d9e0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 shutdown_connections 2024-04-02T04:54:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6e0406dd60 0x7f6e04070220 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e1c101de0 0x7f6e1c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 --2- 172.21.15.70:0/2285740366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e1c102700 0x7f6e1c19cbe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 >> 172.21.15.70:0/2285740366 conn(0x7f6e1c0fb650 msgr2=0x7f6e1c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 shutdown_connections 2024-04-02T04:54:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.020+0000 7f6e22d8b700 1 -- 172.21.15.70:0/2285740366 wait complete. 2024-04-02T04:54:07.034 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:54:07.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.126+0000 7f840d81f700 1 -- 172.21.15.70:0/213939993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 msgr2=0x7f8408100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.126+0000 7f840d81f700 1 --2- 172.21.15.70:0/213939993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 0x7f8408100180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009910 tx=0x7f83f8009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.126+0000 7f840d81f700 1 -- 172.21.15.70:0/213939993 shutdown_connections 2024-04-02T04:54:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.126+0000 7f840d81f700 1 --2- 172.21.15.70:0/213939993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408100750 0x7f84081047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.126+0000 7f840d81f700 1 --2- 172.21.15.70:0/213939993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 0x7f8408100180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.126+0000 7f840d81f700 1 -- 172.21.15.70:0/213939993 >> 172.21.15.70:0/213939993 conn(0x7f84080fb650 msgr2=0x7f84080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.126+0000 7f840d81f700 1 -- 172.21.15.70:0/213939993 shutdown_connections 2024-04-02T04:54:07.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.126+0000 7f840d81f700 1 -- 172.21.15.70:0/213939993 wait complete. 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f840d81f700 1 Processor -- start 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f840d81f700 1 -- start start 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f840d81f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 0x7f840819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f840d81f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408100750 0x7f840819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f840d81f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840819d230 con 0x7f84080ffda0 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f840d81f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84081966e0 con 0x7f8408100750 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f8406ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 0x7f840819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f8406ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 0x7f840819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49100/0 (socket says 172.21.15.70:49100) 2024-04-02T04:54:07.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f8406ffd700 1 -- 172.21.15.70:0/3356820907 learned_addr learned my addr 172.21.15.70:0/3356820907 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.127+0000 7f84067fc700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408100750 0x7f840819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f8406ffd700 1 -- 172.21.15.70:0/3356820907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408100750 msgr2=0x7f840819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f8406ffd700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408100750 0x7f840819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f8406ffd700 1 -- 172.21.15.70:0/3356820907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80095f0 con 0x7f84080ffda0 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f8406ffd700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 0x7f840819c660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f83f8006010 tx=0x7f83f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f840c81d700 1 -- 172.21.15.70:0/3356820907 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83f801d070 con 0x7f84080ffda0 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f840c81d700 1 -- 172.21.15.70:0/3356820907 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83f800f460 con 0x7f84080ffda0 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8408196960 con 0x7f84080ffda0 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f840c81d700 1 -- 172.21.15.70:0/3356820907 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83f8017610 con 0x7f84080ffda0 2024-04-02T04:54:07.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.128+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8408196e50 con 0x7f84080ffda0 2024-04-02T04:54:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.129+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840804f2e0 con 0x7f84080ffda0 2024-04-02T04:54:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.130+0000 7f840c81d700 1 -- 172.21.15.70:0/3356820907 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f83f8003a60 con 0x7f84080ffda0 2024-04-02T04:54:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.130+0000 7f840c81d700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f83f006dd60 0x7f83f0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.130+0000 7f840c81d700 1 -- 172.21.15.70:0/3356820907 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f83f808f050 con 0x7f84080ffda0 2024-04-02T04:54:07.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.131+0000 7f84067fc700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f83f006dd60 0x7f83f0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.131+0000 7f84067fc700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f83f006dd60 0x7f83f0070220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8408197b50 tx=0x7f83fc00a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.135+0000 7f840c81d700 1 -- 172.21.15.70:0/3356820907 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f83f805c1c0 con 0x7f84080ffda0 2024-04-02T04:54:07.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.350+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8408197130 con 0x7f83f006dd60 2024-04-02T04:54:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.352+0000 7f840c81d700 1 -- 172.21.15.70:0/3356820907 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8408197130 con 0x7f83f006dd60 2024-04-02T04:54:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f83f006dd60 msgr2=0x7f83f0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f83f006dd60 0x7f83f0070220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8408197b50 tx=0x7f83fc00a300 comp rx=0 tx=0).stop 2024-04-02T04:54:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 msgr2=0x7f840819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 0x7f840819c660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f83f8006010 tx=0x7f83f800bd70 comp rx=0 tx=0).stop 2024-04-02T04:54:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 shutdown_connections 2024-04-02T04:54:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f83f006dd60 0x7f83f0070220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84080ffda0 0x7f840819c660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 --2- 172.21.15.70:0/3356820907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8408100750 0x7f840819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 >> 172.21.15.70:0/3356820907 conn(0x7f84080fb650 msgr2=0x7f8408103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 shutdown_connections 2024-04-02T04:54:07.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.354+0000 7f840d81f700 1 -- 172.21.15.70:0/3356820907 wait complete. 2024-04-02T04:54:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.460+0000 7ff959f42700 1 -- 172.21.15.70:0/468483147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 msgr2=0x7ff954105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.460+0000 7ff959f42700 1 --2- 172.21.15.70:0/468483147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 0x7ff954105890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff948009910 tx=0x7ff948009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.460+0000 7ff959f42700 1 -- 172.21.15.70:0/468483147 shutdown_connections 2024-04-02T04:54:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.460+0000 7ff959f42700 1 --2- 172.21.15.70:0/468483147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 0x7ff954105890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.460+0000 7ff959f42700 1 --2- 172.21.15.70:0/468483147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff954100e90 0x7ff954101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.460+0000 7ff959f42700 1 -- 172.21.15.70:0/468483147 >> 172.21.15.70:0/468483147 conn(0x7ff9540fc740 msgr2=0x7ff9540feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.460+0000 7ff959f42700 1 -- 172.21.15.70:0/468483147 shutdown_connections 2024-04-02T04:54:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.461+0000 7ff959f42700 1 -- 172.21.15.70:0/468483147 wait complete. 2024-04-02T04:54:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.461+0000 7ff959f42700 1 Processor -- start 2024-04-02T04:54:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.461+0000 7ff959f42700 1 -- start start 2024-04-02T04:54:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff959f42700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff954100e90 0x7ff95419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff959f42700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 0x7ff95419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff959f42700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95419d240 con 0x7ff954101840 2024-04-02T04:54:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff959f42700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9541966f0 con 0x7ff954100e90 2024-04-02T04:54:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff952ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 0x7ff95419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff952ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 0x7ff95419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49122/0 (socket says 172.21.15.70:49122) 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff952ffd700 1 -- 172.21.15.70:0/3695105265 learned_addr learned my addr 172.21.15.70:0/3695105265 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff9537fe700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff954100e90 0x7ff95419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff952ffd700 1 -- 172.21.15.70:0/3695105265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff954100e90 msgr2=0x7ff95419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff952ffd700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff954100e90 0x7ff95419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff952ffd700 1 -- 172.21.15.70:0/3695105265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480095f0 con 0x7ff954101840 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff952ffd700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 0x7ff95419cbb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff9480092e0 tx=0x7ff94800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff950ff9700 1 -- 172.21.15.70:0/3695105265 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94801d070 con 0x7ff954101840 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.462+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff954196970 con 0x7ff954101840 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.463+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff954196e60 con 0x7ff954101840 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.463+0000 7ff950ff9700 1 -- 172.21.15.70:0/3695105265 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff948005dc0 con 0x7ff954101840 2024-04-02T04:54:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.463+0000 7ff950ff9700 1 -- 172.21.15.70:0/3695105265 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff948017400 con 0x7ff954101840 2024-04-02T04:54:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.464+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff940005320 con 0x7ff954101840 2024-04-02T04:54:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.464+0000 7ff950ff9700 1 -- 172.21.15.70:0/3695105265 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff948003a60 con 0x7ff954101840 2024-04-02T04:54:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.465+0000 7ff950ff9700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff93c06dd60 0x7ff93c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.465+0000 7ff950ff9700 1 -- 172.21.15.70:0/3695105265 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff94808ec80 con 0x7ff954101840 2024-04-02T04:54:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.465+0000 7ff9537fe700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff93c06dd60 0x7ff93c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.465+0000 7ff9537fe700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff93c06dd60 0x7ff93c070220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff944005950 tx=0x7ff94400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.469+0000 7ff950ff9700 1 -- 172.21.15.70:0/3695105265 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff94805be70 con 0x7ff954101840 2024-04-02T04:54:07.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.700+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff940000bf0 con 0x7ff93c06dd60 2024-04-02T04:54:07.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.711+0000 7ff950ff9700 1 -- 172.21.15.70:0/3695105265 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff940000bf0 con 0x7ff93c06dd60 2024-04-02T04:54:07.714 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:54:07.714 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (3m) 24s ago 14m 26.4M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:54:07.714 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (15m) 24s ago 15m 8182k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:54:07.714 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (14m) 47s ago 13m 7818k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (56s) 24s ago 14m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (51s) 47s ago 13m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (3m) 24s ago 14m 79.4M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (9m) 24s ago 9m 152M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (9m) 24s ago 9m 19.7M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (9m) 47s ago 9m 17.8M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (9m) 47s ago 9m 19.0M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (5m) 24s ago 16m 497M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (4m) 47s ago 13m 449M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (2m) 24s ago 16m 45.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (100s) 47s ago 13m 41.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (4m) 24s ago 14m 15.8M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (4m) 47s ago 13m 16.8M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:54:07.715 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (30s) 24s ago 12m 14.9M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:54:07.716 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (12m) 24s ago 12m 264M 1216M 18.2.1 5be31c24972a 310a395963bf 2024-04-02T04:54:07.716 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (11m) 24s ago 11m 212M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:54:07.716 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (11m) 47s ago 11m 298M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:54:07.716 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (11m) 47s ago 11m 275M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:54:07.716 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (10m) 47s ago 10m 228M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:54:07.716 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (4m) 24s ago 14m 62.9M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:54:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff93c06dd60 msgr2=0x7ff93c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff93c06dd60 0x7ff93c070220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff944005950 tx=0x7ff94400b410 comp rx=0 tx=0).stop 2024-04-02T04:54:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 msgr2=0x7ff95419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 0x7ff95419cbb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff9480092e0 tx=0x7ff94800bd70 comp rx=0 tx=0).stop 2024-04-02T04:54:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 shutdown_connections 2024-04-02T04:54:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff93c06dd60 0x7ff93c070220 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff954100e90 0x7ff95419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 --2- 172.21.15.70:0/3695105265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff954101840 0x7ff95419cbb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 >> 172.21.15.70:0/3695105265 conn(0x7ff9540fc740 msgr2=0x7ff9540fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 shutdown_connections 2024-04-02T04:54:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.714+0000 7ff959f42700 1 -- 172.21.15.70:0/3695105265 wait complete. 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.822+0000 7fa67074e700 1 -- 172.21.15.70:0/891831816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 msgr2=0x7fa668101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.822+0000 7fa67074e700 1 --2- 172.21.15.70:0/891831816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 0x7fa668101280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa658009910 tx=0x7fa658009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 -- 172.21.15.70:0/891831816 shutdown_connections 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 --2- 172.21.15.70:0/891831816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa668101850 0x7fa6681058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 --2- 172.21.15.70:0/891831816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 0x7fa668101280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 -- 172.21.15.70:0/891831816 >> 172.21.15.70:0/891831816 conn(0x7fa6680fc750 msgr2=0x7fa6680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 -- 172.21.15.70:0/891831816 shutdown_connections 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 -- 172.21.15.70:0/891831816 wait complete. 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 Processor -- start 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 -- start start 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 0x7fa668073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa668101850 0x7fa6680735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668073b10 con 0x7fa668100ea0 2024-04-02T04:54:07.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.823+0000 7fa67074e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668073c50 con 0x7fa668101850 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa66e4ea700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 0x7fa668073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa66e4ea700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 0x7fa668073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49150/0 (socket says 172.21.15.70:49150) 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa66e4ea700 1 -- 172.21.15.70:0/1968933850 learned_addr learned my addr 172.21.15.70:0/1968933850 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa66dce9700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa668101850 0x7fa6680735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa66e4ea700 1 -- 172.21.15.70:0/1968933850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa668101850 msgr2=0x7fa6680735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa66e4ea700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa668101850 0x7fa6680735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa66e4ea700 1 -- 172.21.15.70:0/1968933850 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6580095f0 con 0x7fa668100ea0 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa66e4ea700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 0x7fa668073090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa658006010 tx=0x7fa65800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa65f7fe700 1 -- 172.21.15.70:0/1968933850 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa65801d070 con 0x7fa668100ea0 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa65f7fe700 1 -- 172.21.15.70:0/1968933850 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa658005dc0 con 0x7fa668100ea0 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa65f7fe700 1 -- 172.21.15.70:0/1968933850 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa658017400 con 0x7fa668100ea0 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa668073ed0 con 0x7fa668100ea0 2024-04-02T04:54:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.824+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6681a2e40 con 0x7fa668100ea0 2024-04-02T04:54:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.825+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa668109240 con 0x7fa668100ea0 2024-04-02T04:54:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.826+0000 7fa65f7fe700 1 -- 172.21.15.70:0/1968933850 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa658003a60 con 0x7fa668100ea0 2024-04-02T04:54:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.827+0000 7fa65f7fe700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dd60 0x7fa654070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.827+0000 7fa65f7fe700 1 -- 172.21.15.70:0/1968933850 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa65808ee20 con 0x7fa668100ea0 2024-04-02T04:54:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.827+0000 7fa66dce9700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dd60 0x7fa654070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.827+0000 7fa66dce9700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dd60 0x7fa654070220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa668074880 tx=0x7fa66400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:07.832+0000 7fa65f7fe700 1 -- 172.21.15.70:0/1968933850 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa6580584e0 con 0x7fa668100ea0 2024-04-02T04:54:08.029 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:07 smithi070 systemd[1]: Stopping Ceph osd.1 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:54:08.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.157+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa668108c20 con 0x7fa668100ea0 2024-04-02T04:54:08.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.158+0000 7fa65f7fe700 1 -- 172.21.15.70:0/1968933850 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fa65805bba0 con 0x7fa668100ea0 2024-04-02T04:54:08.161 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:54:08.161 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:54:08.161 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:54:08.162 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-04-02T04:54:08.163 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 5 2024-04-02T04:54:08.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:54:08.163 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:54:08.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.160+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dd60 msgr2=0x7fa654070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.160+0000 7fa67074e700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dd60 0x7fa654070220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa668074880 tx=0x7fa66400b410 comp rx=0 tx=0).stop 2024-04-02T04:54:08.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.160+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 msgr2=0x7fa668073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.160+0000 7fa67074e700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 0x7fa668073090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa658006010 tx=0x7fa65800bd70 comp rx=0 tx=0).stop 2024-04-02T04:54:08.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.161+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 shutdown_connections 2024-04-02T04:54:08.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.161+0000 7fa67074e700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dd60 0x7fa654070220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.161+0000 7fa67074e700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa668100ea0 0x7fa668073090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.161+0000 7fa67074e700 1 --2- 172.21.15.70:0/1968933850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa668101850 0x7fa6680735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.161+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 >> 172.21.15.70:0/1968933850 conn(0x7fa6680fc750 msgr2=0x7fa6680fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.161+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 shutdown_connections 2024-04-02T04:54:08.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.161+0000 7fa67074e700 1 -- 172.21.15.70:0/1968933850 wait complete. 2024-04-02T04:54:08.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.273+0000 7f35146f0700 1 -- 172.21.15.70:0/2804107474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 msgr2=0x7f350c101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.273+0000 7f35146f0700 1 --2- 172.21.15.70:0/2804107474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 0x7f350c101f90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3508009910 tx=0x7f3508009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:08.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.273+0000 7f35146f0700 1 -- 172.21.15.70:0/2804107474 shutdown_connections 2024-04-02T04:54:08.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.273+0000 7f35146f0700 1 --2- 172.21.15.70:0/2804107474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f350c0ff340 0x7f350c0ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.273+0000 7f35146f0700 1 --2- 172.21.15.70:0/2804107474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 0x7f350c101f90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.273+0000 7f35146f0700 1 -- 172.21.15.70:0/2804107474 >> 172.21.15.70:0/2804107474 conn(0x7f350c074e30 msgr2=0x7f350c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.273+0000 7f35146f0700 1 -- 172.21.15.70:0/2804107474 shutdown_connections 2024-04-02T04:54:08.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.273+0000 7f35146f0700 1 -- 172.21.15.70:0/2804107474 wait complete. 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f35146f0700 1 Processor -- start 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f35146f0700 1 -- start start 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f35146f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f350c0ff340 0x7f350c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f35146f0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 0x7f350c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f35146f0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19f420 con 0x7f350c101bb0 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f35146f0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c1988d0 con 0x7f350c0ff340 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f3511c8b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 0x7f350c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f3511c8b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 0x7f350c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49168/0 (socket says 172.21.15.70:49168) 2024-04-02T04:54:08.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f3511c8b700 1 -- 172.21.15.70:0/2302174430 learned_addr learned my addr 172.21.15.70:0/2302174430 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.274+0000 7f351248c700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f350c0ff340 0x7f350c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f3511c8b700 1 -- 172.21.15.70:0/2302174430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f350c0ff340 msgr2=0x7f350c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f3511c8b700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f350c0ff340 0x7f350c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f3511c8b700 1 -- 172.21.15.70:0/2302174430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080095f0 con 0x7f350c101bb0 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f3511c8b700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 0x7f350c19ed90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00d8d0 tx=0x7f34fc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f35037fe700 1 -- 172.21.15.70:0/2302174430 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34fc0098a0 con 0x7f350c101bb0 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f35037fe700 1 -- 172.21.15.70:0/2302174430 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34fc010460 con 0x7f350c101bb0 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c198bb0 con 0x7f350c101bb0 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f35037fe700 1 -- 172.21.15.70:0/2302174430 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34fc00f5f0 con 0x7f350c101bb0 2024-04-02T04:54:08.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.275+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c199100 con 0x7f350c101bb0 2024-04-02T04:54:08.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.276+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c10b230 con 0x7f350c101bb0 2024-04-02T04:54:08.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.276+0000 7f35037fe700 1 -- 172.21.15.70:0/2302174430 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f34fc00f750 con 0x7f350c101bb0 2024-04-02T04:54:08.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.277+0000 7f35037fe700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f34f806dd60 0x7f34f8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.277+0000 7f35037fe700 1 -- 172.21.15.70:0/2302174430 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f34fc08d8b0 con 0x7f350c101bb0 2024-04-02T04:54:08.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.277+0000 7f351248c700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f34f806dd60 0x7f34f8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.277+0000 7f351248c700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f34f806dd60 0x7f34f8070220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3508006010 tx=0x7f350801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:08.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.281+0000 7f35037fe700 1 -- 172.21.15.70:0/2302174430 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f34fc05aaa0 con 0x7f350c101bb0 2024-04-02T04:54:08.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.522+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f350c04f2e0 con 0x7f350c101bb0 2024-04-02T04:54:08.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:08 smithi070 ceph-mon[113635]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:08.525 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:08 smithi070 ceph-mon[113635]: pgmap v66: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:08.526 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:08 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1968933850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:08.526 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:08 smithi070 ceph-mon[113635]: osd.1 marked itself down and dead 2024-04-02T04:54:08.526 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:08 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[55485]: 2024-04-02T04:54:08.160+0000 7f0eec831700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:54:08.526 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:08 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[55485]: 2024-04-02T04:54:08.160+0000 7f0eec831700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-02T04:54:08.526 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:08 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[55485]: 2024-04-02T04:54:08.160+0000 7f0eec831700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:54:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.522+0000 7f35037fe700 1 -- 172.21.15.70:0/2302174430 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f34fc05a630 con 0x7f350c101bb0 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:54:08.527 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:54:08.528 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:54:08.529 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:54:08.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f34f806dd60 msgr2=0x7f34f8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f34f806dd60 0x7f34f8070220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3508006010 tx=0x7f350801a040 comp rx=0 tx=0).stop 2024-04-02T04:54:08.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 msgr2=0x7f350c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 0x7f350c19ed90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00d8d0 tx=0x7f34fc00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 shutdown_connections 2024-04-02T04:54:08.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f34f806dd60 0x7f34f8070220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f350c0ff340 0x7f350c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 --2- 172.21.15.70:0/2302174430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f350c101bb0 0x7f350c19ed90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 >> 172.21.15.70:0/2302174430 conn(0x7f350c074e30 msgr2=0x7f350c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 shutdown_connections 2024-04-02T04:54:08.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.525+0000 7f35146f0700 1 -- 172.21.15.70:0/2302174430 wait complete. 2024-04-02T04:54:08.531 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:54:08.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:08 smithi179 ceph-mon[84288]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:08.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:08 smithi179 ceph-mon[84288]: pgmap v66: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:08.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:08 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1968933850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:08.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:08 smithi179 ceph-mon[84288]: osd.1 marked itself down and dead 2024-04-02T04:54:08.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.638+0000 7fb55fb1e700 1 -- 172.21.15.70:0/2737395720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558100ea0 msgr2=0x7fb558101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.638+0000 7fb55fb1e700 1 --2- 172.21.15.70:0/2737395720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558100ea0 0x7fb558101280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb554009910 tx=0x7fb554009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:08.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.638+0000 7fb55fb1e700 1 -- 172.21.15.70:0/2737395720 shutdown_connections 2024-04-02T04:54:08.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.638+0000 7fb55fb1e700 1 --2- 172.21.15.70:0/2737395720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558101850 0x7fb5581058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.638+0000 7fb55fb1e700 1 --2- 172.21.15.70:0/2737395720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558100ea0 0x7fb558101280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.638+0000 7fb55fb1e700 1 -- 172.21.15.70:0/2737395720 >> 172.21.15.70:0/2737395720 conn(0x7fb5580fc750 msgr2=0x7fb5580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.638+0000 7fb55fb1e700 1 -- 172.21.15.70:0/2737395720 shutdown_connections 2024-04-02T04:54:08.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.638+0000 7fb55fb1e700 1 -- 172.21.15.70:0/2737395720 wait complete. 2024-04-02T04:54:08.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55fb1e700 1 Processor -- start 2024-04-02T04:54:08.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55fb1e700 1 -- start start 2024-04-02T04:54:08.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55fb1e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558100ea0 0x7fb55819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55fb1e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558101850 0x7fb55819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55fb1e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55819d2c0 con 0x7fb558101850 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55fb1e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb558196770 con 0x7fb558100ea0 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55d0b9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558101850 0x7fb55819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55d0b9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558101850 0x7fb55819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49186/0 (socket says 172.21.15.70:49186) 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55d0b9700 1 -- 172.21.15.70:0/1496082511 learned_addr learned my addr 172.21.15.70:0/1496082511 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55d8ba700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558100ea0 0x7fb55819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55d0b9700 1 -- 172.21.15.70:0/1496082511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558100ea0 msgr2=0x7fb55819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55d0b9700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558100ea0 0x7fb55819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55d0b9700 1 -- 172.21.15.70:0/1496082511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540095f0 con 0x7fb558101850 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.639+0000 7fb55d0b9700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558101850 0x7fb55819cc30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb54800d900 tx=0x7fb54800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.640+0000 7fb54effd700 1 -- 172.21.15.70:0/1496082511 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb548009900 con 0x7fb558101850 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.640+0000 7fb54effd700 1 -- 172.21.15.70:0/1496082511 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb548010460 con 0x7fb558101850 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.640+0000 7fb54effd700 1 -- 172.21.15.70:0/1496082511 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54800f5f0 con 0x7fb558101850 2024-04-02T04:54:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.640+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb558196a50 con 0x7fb558101850 2024-04-02T04:54:08.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.640+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb558196f50 con 0x7fb558101850 2024-04-02T04:54:08.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.641+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb558109240 con 0x7fb558101850 2024-04-02T04:54:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.641+0000 7fb54effd700 1 -- 172.21.15.70:0/1496082511 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb5480105d0 con 0x7fb558101850 2024-04-02T04:54:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.642+0000 7fb54effd700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb54406dd60 0x7fb544070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.642+0000 7fb54effd700 1 -- 172.21.15.70:0/1496082511 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb54808d650 con 0x7fb558101850 2024-04-02T04:54:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.642+0000 7fb55d8ba700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb54406dd60 0x7fb544070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.642+0000 7fb55d8ba700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb54406dd60 0x7fb544070220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b5c0 tx=0x7fb55401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.647+0000 7fb54effd700 1 -- 172.21.15.70:0/1496082511 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb54805a7c0 con 0x7fb558101850 2024-04-02T04:54:08.779 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:08 smithi070 podman[130592]: 2024-04-02 04:54:08.629630334 +0000 UTC m=+0.638052247 container died 310a395963bfb37348722509280ae0bcd76e83eb23339f21799c5c693b2da65c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD) 2024-04-02T04:54:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.865+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb558197d90 con 0x7fb54406dd60 2024-04-02T04:54:08.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.866+0000 7fb54effd700 1 -- 172.21.15.70:0/1496082511 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb558197d90 con 0x7fb54406dd60 2024-04-02T04:54:08.869 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "7/23 daemons upgraded", 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:54:08.870 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb54406dd60 msgr2=0x7fb544070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb54406dd60 0x7fb544070220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b5c0 tx=0x7fb55401a040 comp rx=0 tx=0).stop 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558101850 msgr2=0x7fb55819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558101850 0x7fb55819cc30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb54800d900 tx=0x7fb54800dc10 comp rx=0 tx=0).stop 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 shutdown_connections 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb54406dd60 0x7fb544070220 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558100ea0 0x7fb55819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 --2- 172.21.15.70:0/1496082511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb558101850 0x7fb55819cc30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 >> 172.21.15.70:0/1496082511 conn(0x7fb5580fc750 msgr2=0x7fb5580fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 shutdown_connections 2024-04-02T04:54:08.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.868+0000 7fb55fb1e700 1 -- 172.21.15.70:0/1496082511 wait complete. 2024-04-02T04:54:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.974+0000 7f2d14976700 1 -- 172.21.15.70:0/1968964800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d101006b0 msgr2=0x7f2d10100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.974+0000 7f2d14976700 1 --2- 172.21.15.70:0/1968964800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d101006b0 0x7f2d10100a90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009910 tx=0x7f2d00009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.974+0000 7f2d14976700 1 -- 172.21.15.70:0/1968964800 shutdown_connections 2024-04-02T04:54:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.974+0000 7f2d14976700 1 --2- 172.21.15.70:0/1968964800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d10101060 0x7f2d101047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.974+0000 7f2d14976700 1 --2- 172.21.15.70:0/1968964800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d101006b0 0x7f2d10100a90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.974+0000 7f2d14976700 1 -- 172.21.15.70:0/1968964800 >> 172.21.15.70:0/1968964800 conn(0x7f2d10074e30 msgr2=0x7f2d10075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.975+0000 7f2d14976700 1 -- 172.21.15.70:0/1968964800 shutdown_connections 2024-04-02T04:54:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.975+0000 7f2d14976700 1 -- 172.21.15.70:0/1968964800 wait complete. 2024-04-02T04:54:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.975+0000 7f2d14976700 1 Processor -- start 2024-04-02T04:54:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.975+0000 7f2d14976700 1 -- start start 2024-04-02T04:54:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.975+0000 7f2d14976700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d101006b0 0x7f2d1019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.975+0000 7f2d14976700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d10101060 0x7f2d1019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.975+0000 7f2d14976700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1019d3b0 con 0x7f2d10101060 2024-04-02T04:54:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.975+0000 7f2d14976700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10196860 con 0x7f2d101006b0 2024-04-02T04:54:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d0dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d10101060 0x7f2d1019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d0dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d10101060 0x7f2d1019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49204/0 (socket says 172.21.15.70:49204) 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d0dd9b700 1 -- 172.21.15.70:0/3947808647 learned_addr learned my addr 172.21.15.70:0/3947808647 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d0e59c700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d101006b0 0x7f2d1019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d0dd9b700 1 -- 172.21.15.70:0/3947808647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d101006b0 msgr2=0x7f2d1019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d0dd9b700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d101006b0 0x7f2d1019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d0dd9b700 1 -- 172.21.15.70:0/3947808647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000095f0 con 0x7f2d10101060 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d0dd9b700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d10101060 0x7f2d1019cd20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b480 tx=0x7f2d0400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2cff7fe700 1 -- 172.21.15.70:0/3947808647 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d040118c0 con 0x7f2d10101060 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2cff7fe700 1 -- 172.21.15.70:0/3947808647 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d04010380 con 0x7f2d10101060 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d10196b40 con 0x7f2d10101060 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2cff7fe700 1 -- 172.21.15.70:0/3947808647 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0400f4f0 con 0x7f2d10101060 2024-04-02T04:54:08.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.976+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d10197090 con 0x7f2d10101060 2024-04-02T04:54:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.977+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1004f2e0 con 0x7f2d10101060 2024-04-02T04:54:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.978+0000 7f2cff7fe700 1 -- 172.21.15.70:0/3947808647 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2d040104f0 con 0x7f2d10101060 2024-04-02T04:54:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.978+0000 7f2cff7fe700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2cf806dd60 0x7f2cf8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.978+0000 7f2cff7fe700 1 -- 172.21.15.70:0/3947808647 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2d0408d6d0 con 0x7f2d10101060 2024-04-02T04:54:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.978+0000 7f2d0e59c700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2cf806dd60 0x7f2cf8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.979+0000 7f2d0e59c700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2cf806dd60 0x7f2cf8070220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000b5c0 tx=0x7f2d0001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:08.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:08.983+0000 7f2cff7fe700 1 -- 172.21.15.70:0/3947808647 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2d0405a840 con 0x7f2d10101060 2024-04-02T04:54:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.294+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2d10197310 con 0x7f2d10101060 2024-04-02T04:54:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.294+0000 7f2cff7fe700 1 -- 172.21.15.70:0/3947808647 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f2d0405a3d0 con 0x7f2d10101060 2024-04-02T04:54:09.298 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 osds down 2024-04-02T04:54:09.298 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-02T04:54:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: osd.1 (root=default,host=smithi070) is down 2024-04-02T04:54:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.296+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2cf806dd60 msgr2=0x7f2cf8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2cf806dd60 0x7f2cf8070220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000b5c0 tx=0x7f2d0001a040 comp rx=0 tx=0).stop 2024-04-02T04:54:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d10101060 msgr2=0x7f2d1019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d10101060 0x7f2d1019cd20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b480 tx=0x7f2d0400b840 comp rx=0 tx=0).stop 2024-04-02T04:54:09.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 shutdown_connections 2024-04-02T04:54:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2cf806dd60 0x7f2cf8070220 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d101006b0 0x7f2d1019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 --2- 172.21.15.70:0/3947808647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d10101060 0x7f2d1019cd20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 >> 172.21.15.70:0/3947808647 conn(0x7f2d10074e30 msgr2=0x7f2d100fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 shutdown_connections 2024-04-02T04:54:09.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:09.297+0000 7f2d14976700 1 -- 172.21.15.70:0/3947808647 wait complete. 2024-04-02T04:54:09.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:09 smithi070 ceph-mon[113635]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:09 smithi070 ceph-mon[113635]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:09 smithi070 ceph-mon[113635]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:54:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:09 smithi070 ceph-mon[113635]: osdmap e55: 6 total, 5 up, 6 in 2024-04-02T04:54:09.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:09 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2302174430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:54:09.530 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:09 smithi070 podman[130592]: 2024-04-02 04:54:09.136254806 +0000 UTC m=+1.144676717 container cleanup 310a395963bfb37348722509280ae0bcd76e83eb23339f21799c5c693b2da65c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_CLEAN=True) 2024-04-02T04:54:09.530 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:09 smithi070 bash[130592]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T04:54:09.530 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:09 smithi070 podman[130636]: 2024-04-02 04:54:09.371524554 +0000 UTC m=+0.740877750 container remove 310a395963bfb37348722509280ae0bcd76e83eb23339f21799c5c693b2da65c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-02T04:54:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:09 smithi179 ceph-mon[84288]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:09 smithi179 ceph-mon[84288]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:09 smithi179 ceph-mon[84288]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:54:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:09 smithi179 ceph-mon[84288]: osdmap e55: 6 total, 5 up, 6 in 2024-04-02T04:54:09.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:09 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2302174430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:54:09.995 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:09 smithi070 podman[130774]: 2024-04-02 04:54:09.570945809 +0000 UTC m=+0.018114017 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:54:09.995 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:09 smithi070 podman[130774]: 2024-04-02 04:54:09.672356636 +0000 UTC m=+0.119524844 container create ce7efa2125586795814177e47b535d5b0bfbb7c9b4161a11737aca1e42baf5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T04:54:09.995 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:09 smithi070 podman[130774]: 2024-04-02 04:54:09.90827532 +0000 UTC m=+0.355443542 container init ce7efa2125586795814177e47b535d5b0bfbb7c9b4161a11737aca1e42baf5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-02T04:54:09.995 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:09 smithi070 podman[130774]: 2024-04-02 04:54:09.911447943 +0000 UTC m=+0.358616153 container start ce7efa2125586795814177e47b535d5b0bfbb7c9b4161a11737aca1e42baf5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-02T04:54:09.995 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:09 smithi070 podman[130774]: 2024-04-02 04:54:09.99161339 +0000 UTC m=+0.438781610 container attach ce7efa2125586795814177e47b535d5b0bfbb7c9b4161a11737aca1e42baf5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326) 2024-04-02T04:54:10.266 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:10 smithi070 podman[130774]: 2024-04-02 04:54:10.209565794 +0000 UTC m=+0.656734011 container died ce7efa2125586795814177e47b535d5b0bfbb7c9b4161a11737aca1e42baf5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-02T04:54:10.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:10 smithi070 ceph-mon[113635]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:10.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:10 smithi070 ceph-mon[113635]: pgmap v68: 65 pgs: 12 stale+active+clean, 53 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:10.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:10 smithi070 ceph-mon[113635]: osdmap e56: 6 total, 5 up, 6 in 2024-04-02T04:54:10.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:10 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3947808647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:54:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:10 smithi179 ceph-mon[84288]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:10 smithi179 ceph-mon[84288]: pgmap v68: 65 pgs: 12 stale+active+clean, 53 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:10 smithi179 ceph-mon[84288]: osdmap e56: 6 total, 5 up, 6 in 2024-04-02T04:54:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:10 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3947808647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:54:10.994 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:10 smithi070 podman[130800]: 2024-04-02 04:54:10.762368884 +0000 UTC m=+0.544249110 container remove ce7efa2125586795814177e47b535d5b0bfbb7c9b4161a11737aca1e42baf5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-04-02T04:54:10.994 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:10 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.1.service: Deactivated successfully. 2024-04-02T04:54:10.994 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:10 smithi070 systemd[1]: Stopped Ceph osd.1 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:54:10.995 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:10 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.1.service: Consumed 1min 10.823s CPU time. 2024-04-02T04:54:11.250 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:10 smithi070 systemd[1]: Starting Ceph osd.1 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:54:11.250 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 podman[130904]: 2024-04-02 04:54:11.112742676 +0000 UTC m=+0.019736309 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:54:11.250 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 podman[130904]: 2024-04-02 04:54:11.246673666 +0000 UTC m=+0.153667267 container create 26b7905772ecd6f9cd1c32cdad5df90f22afa604f896926b3b9c3c6c3058604b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-04-02T04:54:11.529 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 podman[130904]: 2024-04-02 04:54:11.491220712 +0000 UTC m=+0.398214316 container init 26b7905772ecd6f9cd1c32cdad5df90f22afa604f896926b3b9c3c6c3058604b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:54:11.530 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 podman[130904]: 2024-04-02 04:54:11.494437837 +0000 UTC m=+0.401431440 container start 26b7905772ecd6f9cd1c32cdad5df90f22afa604f896926b3b9c3c6c3058604b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 5]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 1]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1, 3, 2]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 1]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [3, 1]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]: dispatch 2024-04-02T04:54:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:11.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 5]}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 1]}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1, 3, 2]}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 1]}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [3, 1]}]: dispatch 2024-04-02T04:54:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]: dispatch 2024-04-02T04:54:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:11.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:11.939 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 podman[130904]: 2024-04-02 04:54:11.566185865 +0000 UTC m=+0.473179471 container attach 26b7905772ecd6f9cd1c32cdad5df90f22afa604f896926b3b9c3c6c3058604b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T04:54:12.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate[130919]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T04:54:12.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate[130919]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-02T04:54:12.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 bash[130904]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T04:54:12.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 bash[130904]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 bash[130904]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 bash[130904]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 bash[130904]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 bash[130904]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 bash[130904]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate[130919]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate[130919]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate[130919]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate[130919]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate[130919]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-02T04:54:12.281 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:11 smithi070 podman[130904]: 2024-04-02 04:54:11.960389177 +0000 UTC m=+0.867382785 container died 26b7905772ecd6f9cd1c32cdad5df90f22afa604f896926b3b9c3c6c3058604b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-02T04:54:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]': finished 2024-04-02T04:54:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 5]}]': finished 2024-04-02T04:54:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 1]}]': finished 2024-04-02T04:54:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-04-02T04:54:12.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1, 3, 2]}]': finished 2024-04-02T04:54:12.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-04-02T04:54:12.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 1]}]': finished 2024-04-02T04:54:12.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [3, 1]}]': finished 2024-04-02T04:54:12.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]': finished 2024-04-02T04:54:12.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:12 smithi179 ceph-mon[84288]: osdmap e57: 6 total, 5 up, 6 in 2024-04-02T04:54:12.599 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:12.599 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]': finished 2024-04-02T04:54:12.599 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 5]}]': finished 2024-04-02T04:54:12.599 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 1]}]': finished 2024-04-02T04:54:12.599 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-04-02T04:54:12.600 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1, 3, 2]}]': finished 2024-04-02T04:54:12.600 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 1]}]': finished 2024-04-02T04:54:12.600 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 1]}]': finished 2024-04-02T04:54:12.600 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [3, 1]}]': finished 2024-04-02T04:54:12.600 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]': finished 2024-04-02T04:54:12.600 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:12 smithi070 ceph-mon[113635]: osdmap e57: 6 total, 5 up, 6 in 2024-04-02T04:54:12.955 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:12 smithi070 podman[131059]: 2024-04-02 04:54:12.595799794 +0000 UTC m=+0.629997022 container remove 26b7905772ecd6f9cd1c32cdad5df90f22afa604f896926b3b9c3c6c3058604b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T04:54:12.955 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:12 smithi070 podman[131126]: 2024-04-02 04:54:12.712420831 +0000 UTC m=+0.017815182 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:54:12.955 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:12 smithi070 podman[131126]: 2024-04-02 04:54:12.813146867 +0000 UTC m=+0.118541175 container create 10ef3364d9403246f75c580a8c30970d6d520b36b1c6cadee2af0fd73ea6d27c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, RELEASE=squid-6cec793, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:54:13.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:13 smithi070 podman[131126]: 2024-04-02 04:54:13.090886493 +0000 UTC m=+0.396280831 container init 10ef3364d9403246f75c580a8c30970d6d520b36b1c6cadee2af0fd73ea6d27c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T04:54:13.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:13 smithi070 podman[131126]: 2024-04-02 04:54:13.093948018 +0000 UTC m=+0.399342327 container start 10ef3364d9403246f75c580a8c30970d6d520b36b1c6cadee2af0fd73ea6d27c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.build-date=20240326) 2024-04-02T04:54:13.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:13 smithi070 ceph-osd[131145]: -- 172.21.15.70:0/2896370236 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55959c48f680 con 0x55959c458c00 2024-04-02T04:54:13.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:13 smithi070 bash[131126]: 10ef3364d9403246f75c580a8c30970d6d520b36b1c6cadee2af0fd73ea6d27c 2024-04-02T04:54:13.280 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:13 smithi070 systemd[1]: Started Ceph osd.1 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:54:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:13 smithi179 ceph-mon[84288]: osdmap e58: 6 total, 5 up, 6 in 2024-04-02T04:54:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:13 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:13 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:13 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:54:13.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:13 smithi179 ceph-mon[84288]: osdmap e59: 6 total, 5 up, 6 in 2024-04-02T04:54:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:13 smithi070 ceph-mon[113635]: osdmap e58: 6 total, 5 up, 6 in 2024-04-02T04:54:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:13 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:13 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:13 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:54:13.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:13 smithi070 ceph-mon[113635]: osdmap e59: 6 total, 5 up, 6 in 2024-04-02T04:54:14.029 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:13 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[131141]: 2024-04-02T04:54:13.635+0000 7f4f126a97c0 -1 Falling back to public interface 2024-04-02T04:54:14.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:14 smithi070 ceph-mon[113635]: pgmap v73: 65 pgs: 5 unknown, 2 peering, 10 active+undersized+degraded, 3 stale+active+clean, 17 active+undersized, 28 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21/219 objects degraded (9.589%) 2024-04-02T04:54:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:14 smithi070 ceph-mon[113635]: Health check failed: Degraded data redundancy: 21/219 objects degraded (9.589%), 10 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:14 smithi179 ceph-mon[84288]: pgmap v73: 65 pgs: 5 unknown, 2 peering, 10 active+undersized+degraded, 3 stale+active+clean, 17 active+undersized, 28 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21/219 objects degraded (9.589%) 2024-04-02T04:54:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:14 smithi179 ceph-mon[84288]: Health check failed: Degraded data redundancy: 21/219 objects degraded (9.589%), 10 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:15.570 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:15 smithi070 ceph-mon[113635]: osdmap e60: 6 total, 5 up, 6 in 2024-04-02T04:54:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:15 smithi179 ceph-mon[84288]: osdmap e60: 6 total, 5 up, 6 in 2024-04-02T04:54:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:16 smithi070 ceph-mon[113635]: pgmap v76: 65 pgs: 1 remapped+peering, 25 active+clean, 21 active+undersized, 11 active+undersized+degraded, 2 peering, 5 unknown; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/219 objects degraded (10.046%) 2024-04-02T04:54:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:16 smithi070 ceph-mon[113635]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-02T04:54:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:16 smithi070 ceph-mon[113635]: osdmap e61: 6 total, 5 up, 6 in 2024-04-02T04:54:16.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:16 smithi179 ceph-mon[84288]: pgmap v76: 65 pgs: 1 remapped+peering, 25 active+clean, 21 active+undersized, 11 active+undersized+degraded, 2 peering, 5 unknown; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/219 objects degraded (10.046%) 2024-04-02T04:54:16.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:16 smithi179 ceph-mon[84288]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-02T04:54:16.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:16 smithi179 ceph-mon[84288]: osdmap e61: 6 total, 5 up, 6 in 2024-04-02T04:54:17.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:17 smithi070 ceph-mon[113635]: osdmap e62: 6 total, 5 up, 6 in 2024-04-02T04:54:17.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:17 smithi070 ceph-mon[113635]: pgmap v79: 65 pgs: 7 active+clean+remapped, 1 remapped+peering, 25 active+clean, 21 active+undersized, 11 active+undersized+degraded; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s wr, 115 op/s; 22/228 objects degraded (9.649%); 20/228 objects misplaced (8.772%); 4.5 MiB/s, 11 objects/s recovering 2024-04-02T04:54:17.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:17 smithi179 ceph-mon[84288]: osdmap e62: 6 total, 5 up, 6 in 2024-04-02T04:54:17.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:17 smithi179 ceph-mon[84288]: pgmap v79: 65 pgs: 7 active+clean+remapped, 1 remapped+peering, 25 active+clean, 21 active+undersized, 11 active+undersized+degraded; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s wr, 115 op/s; 22/228 objects degraded (9.649%); 20/228 objects misplaced (8.772%); 4.5 MiB/s, 11 objects/s recovering 2024-04-02T04:54:18.529 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:18 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[131141]: 2024-04-02T04:54:18.165+0000 7f4f126a97c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-04-02T04:54:19.029 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:18 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[131141]: 2024-04-02T04:54:18.693+0000 7f4f126a97c0 -1 osd.1 54 log_to_monitors true 2024-04-02T04:54:20.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:19 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:20.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:19 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:20.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:19 smithi070 ceph-mon[113635]: from='osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T04:54:20.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:19 smithi070 ceph-mon[113635]: pgmap v80: 65 pgs: 9 active+clean+remapped, 24 active+clean, 21 active+undersized, 11 active+undersized+degraded; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s wr, 79 op/s; 22/228 objects degraded (9.649%); 20/228 objects misplaced (8.772%); 3.1 MiB/s, 127 objects/s recovering 2024-04-02T04:54:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:19 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:19 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:19 smithi179 ceph-mon[84288]: from='osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T04:54:20.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:19 smithi179 ceph-mon[84288]: pgmap v80: 65 pgs: 9 active+clean+remapped, 24 active+clean, 21 active+undersized, 11 active+undersized+degraded; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s wr, 79 op/s; 22/228 objects degraded (9.649%); 20/228 objects misplaced (8.772%); 3.1 MiB/s, 127 objects/s recovering 2024-04-02T04:54:20.664 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 04:54:20 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[131141]: 2024-04-02T04:54:20.408+0000 7f4effaf9700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:54:21.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:20 smithi070 ceph-mon[113635]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-02T04:54:21.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:20 smithi070 ceph-mon[113635]: from='osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T04:54:21.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:20 smithi070 ceph-mon[113635]: osdmap e63: 6 total, 5 up, 6 in 2024-04-02T04:54:21.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:20 smithi070 ceph-mon[113635]: from='osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:54:21.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:21.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:21.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:20 smithi070 ceph-mon[113635]: from='osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655]' entity='osd.1' 2024-04-02T04:54:21.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:20 smithi179 ceph-mon[84288]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-02T04:54:21.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:20 smithi179 ceph-mon[84288]: from='osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T04:54:21.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:20 smithi179 ceph-mon[84288]: osdmap e63: 6 total, 5 up, 6 in 2024-04-02T04:54:21.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:20 smithi179 ceph-mon[84288]: from='osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:54:21.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:21.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:21.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:20 smithi179 ceph-mon[84288]: from='osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655]' entity='osd.1' 2024-04-02T04:54:21.864 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:21 smithi070 ceph-mon[113635]: pgmap v82: 65 pgs: 9 active+clean+remapped, 24 active+clean, 21 active+undersized, 11 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s wr, 77 op/s; 22/228 objects degraded (9.649%); 20/228 objects misplaced (8.772%); 3.0 MiB/s, 124 objects/s recovering 2024-04-02T04:54:21.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:21 smithi070 ceph-mon[113635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:54:21.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:21 smithi070 ceph-mon[113635]: osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655] boot 2024-04-02T04:54:21.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:21 smithi070 ceph-mon[113635]: osdmap e64: 6 total, 6 up, 6 in 2024-04-02T04:54:21.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:54:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:21 smithi179 ceph-mon[84288]: pgmap v82: 65 pgs: 9 active+clean+remapped, 24 active+clean, 21 active+undersized, 11 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s wr, 77 op/s; 22/228 objects degraded (9.649%); 20/228 objects misplaced (8.772%); 3.0 MiB/s, 124 objects/s recovering 2024-04-02T04:54:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:21 smithi179 ceph-mon[84288]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:54:22.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:21 smithi179 ceph-mon[84288]: osd.1 [v2:172.21.15.70:6810/2318524655,v1:172.21.15.70:6811/2318524655] boot 2024-04-02T04:54:22.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:21 smithi179 ceph-mon[84288]: osdmap e64: 6 total, 6 up, 6 in 2024-04-02T04:54:22.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:54:23.612 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:23 smithi070 ceph-mon[113635]: osdmap e65: 6 total, 6 up, 6 in 2024-04-02T04:54:23.612 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:23 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 22/228 objects degraded (9.649%), 11 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:23.612 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:23 smithi070 ceph-mon[113635]: pgmap v85: 65 pgs: 7 remapped+peering, 19 peering, 2 active+clean+remapped, 24 active+clean, 8 active+undersized, 5 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 9/228 objects degraded (3.947%); 0 B/s, 116 objects/s recovering 2024-04-02T04:54:23.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:23 smithi179 ceph-mon[84288]: osdmap e65: 6 total, 6 up, 6 in 2024-04-02T04:54:23.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:23 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 22/228 objects degraded (9.649%), 11 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:23.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:23 smithi179 ceph-mon[84288]: pgmap v85: 65 pgs: 7 remapped+peering, 19 peering, 2 active+clean+remapped, 24 active+clean, 8 active+undersized, 5 active+undersized+degraded; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 9/228 objects degraded (3.947%); 0 B/s, 116 objects/s recovering 2024-04-02T04:54:24.771 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: osdmap e66: 6 total, 6 up, 6 in 2024-04-02T04:54:24.771 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:24.771 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:24.771 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:54:24.771 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:54:24.772 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:24.772 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:54:24.772 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:24.772 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:24.772 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:24.772 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:54:24.772 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:54:24.773 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:24 smithi179 ceph-mon[84288]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-04-02T04:54:24.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: osdmap e66: 6 total, 6 up, 6 in 2024-04-02T04:54:24.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:24.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:54:24.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:24 smithi070 ceph-mon[113635]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-04-02T04:54:25.744 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:25 smithi070 ceph-mon[113635]: osdmap e67: 6 total, 6 up, 6 in 2024-04-02T04:54:25.744 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:25 smithi070 ceph-mon[113635]: pgmap v88: 65 pgs: 5 remapped+peering, 22 peering, 38 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s, 3 objects/s recovering 2024-04-02T04:54:25.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:25 smithi179 ceph-mon[84288]: osdmap e67: 6 total, 6 up, 6 in 2024-04-02T04:54:25.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:25 smithi179 ceph-mon[84288]: pgmap v88: 65 pgs: 5 remapped+peering, 22 peering, 38 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s, 3 objects/s recovering 2024-04-02T04:54:26.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:26 smithi070 ceph-mon[113635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/228 objects degraded (3.947%), 5 pgs degraded) 2024-04-02T04:54:26.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:26 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:54:26.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:26 smithi070 ceph-mon[113635]: osdmap e68: 6 total, 6 up, 6 in 2024-04-02T04:54:26.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:26.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:26.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:26 smithi179 ceph-mon[84288]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/228 objects degraded (3.947%), 5 pgs degraded) 2024-04-02T04:54:26.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:26 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:54:26.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:26 smithi179 ceph-mon[84288]: osdmap e68: 6 total, 6 up, 6 in 2024-04-02T04:54:26.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:26.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:27.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:27 smithi070 ceph-mon[113635]: osdmap e69: 6 total, 6 up, 6 in 2024-04-02T04:54:27.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:27 smithi070 ceph-mon[113635]: pgmap v91: 65 pgs: 1 active+undersized+remapped, 2 remapped+peering, 3 peering, 59 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.5 MiB/s, 197 objects/s recovering 2024-04-02T04:54:27.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:27 smithi179 ceph-mon[84288]: osdmap e69: 6 total, 6 up, 6 in 2024-04-02T04:54:27.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:27 smithi179 ceph-mon[84288]: pgmap v91: 65 pgs: 1 active+undersized+remapped, 2 remapped+peering, 3 peering, 59 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.5 MiB/s, 197 objects/s recovering 2024-04-02T04:54:28.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:28 smithi070 ceph-mon[113635]: osdmap e70: 6 total, 6 up, 6 in 2024-04-02T04:54:28.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:28 smithi179 ceph-mon[84288]: osdmap e70: 6 total, 6 up, 6 in 2024-04-02T04:54:29.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:29 smithi070 ceph-mon[113635]: osdmap e71: 6 total, 6 up, 6 in 2024-04-02T04:54:29.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:29 smithi070 ceph-mon[113635]: pgmap v94: 65 pgs: 1 active+undersized+remapped, 2 remapped+peering, 2 peering, 60 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.0 MiB/s, 193 objects/s recovering 2024-04-02T04:54:29.818 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:29 smithi179 ceph-mon[84288]: osdmap e71: 6 total, 6 up, 6 in 2024-04-02T04:54:29.818 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:29 smithi179 ceph-mon[84288]: pgmap v94: 65 pgs: 1 active+undersized+remapped, 2 remapped+peering, 2 peering, 60 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.0 MiB/s, 193 objects/s recovering 2024-04-02T04:54:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:32 smithi070 ceph-mon[113635]: pgmap v95: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.2 MiB/s, 134 objects/s recovering 2024-04-02T04:54:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:32 smithi179 ceph-mon[84288]: pgmap v95: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.2 MiB/s, 134 objects/s recovering 2024-04-02T04:54:34.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:34 smithi070 ceph-mon[113635]: pgmap v96: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s, 114 objects/s recovering 2024-04-02T04:54:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:34 smithi179 ceph-mon[84288]: pgmap v96: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s, 114 objects/s recovering 2024-04-02T04:54:36.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:36 smithi070 ceph-mon[113635]: pgmap v97: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:36 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:36.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:36 smithi179 ceph-mon[84288]: pgmap v97: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:36.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:36 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:38.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:38 smithi070 ceph-mon[113635]: pgmap v98: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:38 smithi179 ceph-mon[84288]: pgmap v98: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:39.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 -- 172.21.15.70:0/3311905559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 msgr2=0x7f9844107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:39.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 --2- 172.21.15.70:0/3311905559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 0x7f9844107880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9834009910 tx=0x7f9834009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:39.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 -- 172.21.15.70:0/3311905559 shutdown_connections 2024-04-02T04:54:39.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 --2- 172.21.15.70:0/3311905559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844107e50 0x7f984410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 --2- 172.21.15.70:0/3311905559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 0x7f9844107880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 -- 172.21.15.70:0/3311905559 >> 172.21.15.70:0/3311905559 conn(0x7f98440769f0 msgr2=0x7f9844078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 -- 172.21.15.70:0/3311905559 shutdown_connections 2024-04-02T04:54:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 -- 172.21.15.70:0/3311905559 wait complete. 2024-04-02T04:54:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.407+0000 7f984a15a700 1 Processor -- start 2024-04-02T04:54:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f984a15a700 1 -- start start 2024-04-02T04:54:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f984a15a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 0x7f984419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f984a15a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844107e50 0x7f984419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f984a15a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984419dbb0 con 0x7f98441074a0 2024-04-02T04:54:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f984a15a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98441a1940 con 0x7f9844107e50 2024-04-02T04:54:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f98437fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 0x7f984419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f98437fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 0x7f984419cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34726/0 (socket says 172.21.15.70:34726) 2024-04-02T04:54:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f98437fe700 1 -- 172.21.15.70:0/530272686 learned_addr learned my addr 172.21.15.70:0/530272686 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f9842ffd700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844107e50 0x7f984419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f98437fe700 1 -- 172.21.15.70:0/530272686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844107e50 msgr2=0x7f984419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f98437fe700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844107e50 0x7f984419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f98437fe700 1 -- 172.21.15.70:0/530272686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340095f0 con 0x7f98441074a0 2024-04-02T04:54:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f9842ffd700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844107e50 0x7f984419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:54:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f98437fe700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 0x7f984419cf90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9834006010 tx=0x7f983400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.408+0000 7f9840ff9700 1 -- 172.21.15.70:0/530272686 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983401d070 con 0x7f98441074a0 2024-04-02T04:54:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.409+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98441a1bc0 con 0x7f98441074a0 2024-04-02T04:54:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.409+0000 7f9840ff9700 1 -- 172.21.15.70:0/530272686 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9834004fb0 con 0x7f98441074a0 2024-04-02T04:54:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.409+0000 7f9840ff9700 1 -- 172.21.15.70:0/530272686 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983400f690 con 0x7f98441074a0 2024-04-02T04:54:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.409+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98441a20b0 con 0x7f98441074a0 2024-04-02T04:54:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.410+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984410f840 con 0x7f98441074a0 2024-04-02T04:54:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.410+0000 7f9840ff9700 1 -- 172.21.15.70:0/530272686 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f983400f7f0 con 0x7f98441074a0 2024-04-02T04:54:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.411+0000 7f9840ff9700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f982c06dd60 0x7f982c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.411+0000 7f9840ff9700 1 -- 172.21.15.70:0/530272686 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f983408f410 con 0x7f98441074a0 2024-04-02T04:54:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.411+0000 7f9842ffd700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f982c06dd60 0x7f982c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.411+0000 7f9842ffd700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f982c06dd60 0x7f982c070220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f984419e5b0 tx=0x7f9838008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.415+0000 7f9840ff9700 1 -- 172.21.15.70:0/530272686 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9834058940 con 0x7f98441074a0 2024-04-02T04:54:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:39 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:54:39.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:39 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:54:39.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.638+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98441a22d0 con 0x7f982c06dd60 2024-04-02T04:54:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.639+0000 7f9840ff9700 1 -- 172.21.15.70:0/530272686 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f98441a22d0 con 0x7f982c06dd60 2024-04-02T04:54:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f982c06dd60 msgr2=0x7f982c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f982c06dd60 0x7f982c070220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f984419e5b0 tx=0x7f9838008040 comp rx=0 tx=0).stop 2024-04-02T04:54:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 msgr2=0x7f984419cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 0x7f984419cf90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9834006010 tx=0x7f983400be20 comp rx=0 tx=0).stop 2024-04-02T04:54:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 shutdown_connections 2024-04-02T04:54:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f982c06dd60 0x7f982c070220 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98441074a0 0x7f984419cf90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 --2- 172.21.15.70:0/530272686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844107e50 0x7f984419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 >> 172.21.15.70:0/530272686 conn(0x7f98440769f0 msgr2=0x7f9844078280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 shutdown_connections 2024-04-02T04:54:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.642+0000 7f984a15a700 1 -- 172.21.15.70:0/530272686 wait complete. 2024-04-02T04:54:39.656 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:54:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.752+0000 7f79b3efc700 1 -- 172.21.15.70:0/2652054796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 msgr2=0x7f79ac1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.752+0000 7f79b3efc700 1 --2- 172.21.15.70:0/2652054796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 0x7f79ac1047a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f799c009910 tx=0x7f799c009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.752+0000 7f79b3efc700 1 -- 172.21.15.70:0/2652054796 shutdown_connections 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.752+0000 7f79b3efc700 1 --2- 172.21.15.70:0/2652054796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 0x7f79ac1047a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.752+0000 7f79b3efc700 1 --2- 172.21.15.70:0/2652054796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79ac0ffda0 0x7f79ac100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.752+0000 7f79b3efc700 1 -- 172.21.15.70:0/2652054796 >> 172.21.15.70:0/2652054796 conn(0x7f79ac0fb650 msgr2=0x7f79ac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.752+0000 7f79b3efc700 1 -- 172.21.15.70:0/2652054796 shutdown_connections 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.752+0000 7f79b3efc700 1 -- 172.21.15.70:0/2652054796 wait complete. 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.753+0000 7f79b3efc700 1 Processor -- start 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.753+0000 7f79b3efc700 1 -- start start 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.753+0000 7f79b3efc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79ac0ffda0 0x7f79ac1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.753+0000 7f79b3efc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 0x7f79ac197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.753+0000 7f79b3efc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac198320 con 0x7f79ac100750 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.753+0000 7f79b3efc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac198460 con 0x7f79ac0ffda0 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.753+0000 7f79b1497700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 0x7f79ac197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b1497700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 0x7f79ac197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34738/0 (socket says 172.21.15.70:34738) 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b1497700 1 -- 172.21.15.70:0/213143077 learned_addr learned my addr 172.21.15.70:0/213143077 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b1c98700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79ac0ffda0 0x7f79ac1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b1497700 1 -- 172.21.15.70:0/213143077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79ac0ffda0 msgr2=0x7f79ac1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b1497700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79ac0ffda0 0x7f79ac1977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b1497700 1 -- 172.21.15.70:0/213143077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c0095f0 con 0x7f79ac100750 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b1497700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 0x7f79ac197d00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f799c0092e0 tx=0x7f799c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79a2ffd700 1 -- 172.21.15.70:0/213143077 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f799c01d070 con 0x7f79ac100750 2024-04-02T04:54:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac1a0830 con 0x7f79ac100750 2024-04-02T04:54:39.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79a2ffd700 1 -- 172.21.15.70:0/213143077 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f799c005dc0 con 0x7f79ac100750 2024-04-02T04:54:39.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79a2ffd700 1 -- 172.21.15.70:0/213143077 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f799c017400 con 0x7f79ac100750 2024-04-02T04:54:39.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.754+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac1a0c40 con 0x7f79ac100750 2024-04-02T04:54:39.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.755+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac1081a0 con 0x7f79ac100750 2024-04-02T04:54:39.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.756+0000 7f79a2ffd700 1 -- 172.21.15.70:0/213143077 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f799c00f460 con 0x7f79ac100750 2024-04-02T04:54:39.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.756+0000 7f79a2ffd700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f799806dd60 0x7f7998070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:39.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.756+0000 7f79a2ffd700 1 -- 172.21.15.70:0/213143077 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f799c08f0a0 con 0x7f79ac100750 2024-04-02T04:54:39.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.756+0000 7f79b1c98700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f799806dd60 0x7f7998070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:39.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.757+0000 7f79b1c98700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f799806dd60 0x7f7998070220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005950 tx=0x7f79a80058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:39.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.761+0000 7f79a2ffd700 1 -- 172.21.15.70:0/213143077 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f799c0585d0 con 0x7f79ac100750 2024-04-02T04:54:39.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.984+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79ac0611d0 con 0x7f799806dd60 2024-04-02T04:54:39.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.985+0000 7f79a2ffd700 1 -- 172.21.15.70:0/213143077 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f79ac0611d0 con 0x7f799806dd60 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f799806dd60 msgr2=0x7f7998070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f799806dd60 0x7f7998070220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005950 tx=0x7f79a80058e0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 msgr2=0x7f79ac197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 0x7f79ac197d00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f799c0092e0 tx=0x7f799c00be20 comp rx=0 tx=0).stop 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 shutdown_connections 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f799806dd60 0x7f7998070220 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79ac0ffda0 0x7f79ac1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 --2- 172.21.15.70:0/213143077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ac100750 0x7f79ac197d00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 >> 172.21.15.70:0/213143077 conn(0x7f79ac0fb650 msgr2=0x7f79ac0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:39.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 shutdown_connections 2024-04-02T04:54:39.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:39.987+0000 7f79b3efc700 1 -- 172.21.15.70:0/213143077 wait complete. 2024-04-02T04:54:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.091+0000 7fd50553a700 1 -- 172.21.15.70:0/38666810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000ffb40 msgr2=0x7fd5000fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.091+0000 7fd50553a700 1 --2- 172.21.15.70:0/38666810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000ffb40 0x7fd5000fffc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009910 tx=0x7fd4e8009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.091+0000 7fd50553a700 1 -- 172.21.15.70:0/38666810 shutdown_connections 2024-04-02T04:54:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.091+0000 7fd50553a700 1 --2- 172.21.15.70:0/38666810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000ffb40 0x7fd5000fffc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.091+0000 7fd50553a700 1 --2- 172.21.15.70:0/38666810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd500105bf0 0x7fd500105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.091+0000 7fd50553a700 1 -- 172.21.15.70:0/38666810 >> 172.21.15.70:0/38666810 conn(0x7fd5000fb610 msgr2=0x7fd5000fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.091+0000 7fd50553a700 1 -- 172.21.15.70:0/38666810 shutdown_connections 2024-04-02T04:54:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.091+0000 7fd50553a700 1 -- 172.21.15.70:0/38666810 wait complete. 2024-04-02T04:54:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd50553a700 1 Processor -- start 2024-04-02T04:54:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd50553a700 1 -- start start 2024-04-02T04:54:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd50553a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5000ffb40 0x7fd5001017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd50553a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd500105bf0 0x7fd500101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd50553a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500102230 con 0x7fd500105bf0 2024-04-02T04:54:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd50553a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500102370 con 0x7fd5000ffb40 2024-04-02T04:54:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd4f67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd500105bf0 0x7fd500101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd4f67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd500105bf0 0x7fd500101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34746/0 (socket says 172.21.15.70:34746) 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.092+0000 7fd4f67fc700 1 -- 172.21.15.70:0/1824995230 learned_addr learned my addr 172.21.15.70:0/1824995230 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd4feffd700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5000ffb40 0x7fd5001017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd4f67fc700 1 -- 172.21.15.70:0/1824995230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5000ffb40 msgr2=0x7fd5001017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd4f67fc700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5000ffb40 0x7fd5001017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd4f67fc700 1 -- 172.21.15.70:0/1824995230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80095f0 con 0x7fd500105bf0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd4f67fc700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd500105bf0 0x7fd500101cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8006010 tx=0x7fd4e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd4fcff9700 1 -- 172.21.15.70:0/1824995230 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e801d070 con 0x7fd500105bf0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd4fcff9700 1 -- 172.21.15.70:0/1824995230 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4e8004fb0 con 0x7fd500105bf0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5001025f0 con 0x7fd500105bf0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5001a2e60 con 0x7fd500105bf0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.093+0000 7fd4fcff9700 1 -- 172.21.15.70:0/1824995230 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e8022ea0 con 0x7fd500105bf0 2024-04-02T04:54:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.094+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50004f2e0 con 0x7fd500105bf0 2024-04-02T04:54:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.094+0000 7fd4fcff9700 1 -- 172.21.15.70:0/1824995230 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd4e8022470 con 0x7fd500105bf0 2024-04-02T04:54:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.095+0000 7fd4fcff9700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd4ec06dd60 0x7fd4ec070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.095+0000 7fd4fcff9700 1 -- 172.21.15.70:0/1824995230 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd4e808f110 con 0x7fd500105bf0 2024-04-02T04:54:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.095+0000 7fd4feffd700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd4ec06dd60 0x7fd4ec070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.095+0000 7fd4feffd700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd4ec06dd60 0x7fd4ec070220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0006fd0 tx=0x7fd4f0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:40.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.099+0000 7fd4fcff9700 1 -- 172.21.15.70:0/1824995230 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd4e8058810 con 0x7fd500105bf0 2024-04-02T04:54:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.312+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd500069600 con 0x7fd4ec06dd60 2024-04-02T04:54:40.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.323+0000 7fd4fcff9700 1 -- 172.21.15.70:0/1824995230 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd500069600 con 0x7fd4ec06dd60 2024-04-02T04:54:40.328 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:54:40.328 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (4m) 21s ago 15m 26.4M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:54:40.328 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (15m) 21s ago 15m 8372k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:54:40.328 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (14m) 80s ago 14m 7818k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:54:40.328 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (88s) 21s ago 15m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:54:40.328 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (83s) 80s ago 14m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:54:40.328 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (3m) 21s ago 14m 79.5M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:54:40.328 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (9m) 21s ago 9m 152M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (9m) 21s ago 9m 19.9M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (9m) 80s ago 9m 17.8M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (9m) 80s ago 9m 19.0M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (6m) 21s ago 16m 502M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (5m) 80s ago 14m 449M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (2m) 21s ago 16m 47.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (2m) 80s ago 14m 41.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (4m) 21s ago 15m 15.5M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (4m) 80s ago 14m 16.8M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (63s) 21s ago 13m 122M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (27s) 21s ago 13m 14.3M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (12m) 21s ago 12m 227M 1216M 18.2.1 5be31c24972a f1bc321ec6c3 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (12m) 80s ago 12m 298M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:54:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (11m) 80s ago 11m 275M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:54:40.330 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (11m) 80s ago 11m 228M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:54:40.330 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (4m) 21s ago 14m 62.9M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:54:40.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd4ec06dd60 msgr2=0x7fd4ec070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd4ec06dd60 0x7fd4ec070220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0006fd0 tx=0x7fd4f0008040 comp rx=0 tx=0).stop 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd500105bf0 msgr2=0x7fd500101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd500105bf0 0x7fd500101cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8006010 tx=0x7fd4e800be20 comp rx=0 tx=0).stop 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 shutdown_connections 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd4ec06dd60 0x7fd4ec070220 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5000ffb40 0x7fd5001017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 --2- 172.21.15.70:0/1824995230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd500105bf0 0x7fd500101cf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 >> 172.21.15.70:0/1824995230 conn(0x7fd5000fb610 msgr2=0x7fd500068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 shutdown_connections 2024-04-02T04:54:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.325+0000 7fd50553a700 1 -- 172.21.15.70:0/1824995230 wait complete. 2024-04-02T04:54:40.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.428+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1527497554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 msgr2=0x7fa944101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.428+0000 7fa94c0a6700 1 --2- 172.21.15.70:0/1527497554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 0x7fa944101280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00b0a0 tx=0x7fa93c00b3b0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.428+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1527497554 shutdown_connections 2024-04-02T04:54:40.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.428+0000 7fa94c0a6700 1 --2- 172.21.15.70:0/1527497554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944101850 0x7fa9441058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.428+0000 7fa94c0a6700 1 --2- 172.21.15.70:0/1527497554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 0x7fa944101280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.428+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1527497554 >> 172.21.15.70:0/1527497554 conn(0x7fa9440fc750 msgr2=0x7fa9440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:40.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:40 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:54:40.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:40 smithi070 ceph-mon[113635]: Upgrade: osd.2 is safe to restart 2024-04-02T04:54:40.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:40 smithi070 ceph-mon[113635]: pgmap v99: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.428+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1527497554 shutdown_connections 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.428+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1527497554 wait complete. 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa94c0a6700 1 Processor -- start 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa94c0a6700 1 -- start start 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa94c0a6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 0x7fa944196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa94c0a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944101850 0x7fa944196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa94c0a6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa944197560 con 0x7fa944100ea0 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa94c0a6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94419b370 con 0x7fa944101850 2024-04-02T04:54:40.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa949e42700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 0x7fa944196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa949e42700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 0x7fa944196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34770/0 (socket says 172.21.15.70:34770) 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.429+0000 7fa949e42700 1 -- 172.21.15.70:0/1855040733 learned_addr learned my addr 172.21.15.70:0/1855040733 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa949641700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944101850 0x7fa944196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa949e42700 1 -- 172.21.15.70:0/1855040733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944101850 msgr2=0x7fa944196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa949e42700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944101850 0x7fa944196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa949e42700 1 -- 172.21.15.70:0/1855040733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c009d00 con 0x7fa944100ea0 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa949641700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944101850 0x7fa944196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa949e42700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 0x7fa944196990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa93c000f80 tx=0x7fa93c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa93affd700 1 -- 172.21.15.70:0/1855040733 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa93c00e040 con 0x7fa944100ea0 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa93affd700 1 -- 172.21.15.70:0/1855040733 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa93c003e60 con 0x7fa944100ea0 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94419b5f0 con 0x7fa944100ea0 2024-04-02T04:54:40.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa93affd700 1 -- 172.21.15.70:0/1855040733 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa93c01d910 con 0x7fa944100ea0 2024-04-02T04:54:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.430+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94419bae0 con 0x7fa944100ea0 2024-04-02T04:54:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.431+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa944109240 con 0x7fa944100ea0 2024-04-02T04:54:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.432+0000 7fa93affd700 1 -- 172.21.15.70:0/1855040733 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa93c004820 con 0x7fa944100ea0 2024-04-02T04:54:40.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.432+0000 7fa93affd700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa93006dd60 0x7fa930070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.432+0000 7fa93affd700 1 -- 172.21.15.70:0/1855040733 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa93c08f4d0 con 0x7fa944100ea0 2024-04-02T04:54:40.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.432+0000 7fa949641700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa93006dd60 0x7fa930070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.432+0000 7fa949641700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa93006dd60 0x7fa930070220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa944197fb0 tx=0x7fa934008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:40.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.437+0000 7fa93affd700 1 -- 172.21.15.70:0/1855040733 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa93c05c530 con 0x7fa944100ea0 2024-04-02T04:54:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:40 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:54:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:40 smithi179 ceph-mon[84288]: Upgrade: osd.2 is safe to restart 2024-04-02T04:54:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:40 smithi179 ceph-mon[84288]: pgmap v99: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:40.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.721+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa94404f2e0 con 0x7fa944100ea0 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.722+0000 7fa93affd700 1 -- 172.21.15.70:0/1855040733 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fa93c05c0c0 con 0x7fa944100ea0 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:54:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:54:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:54:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:54:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:54:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:54:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-04-02T04:54:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:54:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:54:40.728 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa93006dd60 msgr2=0x7fa930070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa93006dd60 0x7fa930070220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa944197fb0 tx=0x7fa934008040 comp rx=0 tx=0).stop 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 msgr2=0x7fa944196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 0x7fa944196990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa93c000f80 tx=0x7fa93c00cdd0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 shutdown_connections 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa93006dd60 0x7fa930070220 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa944100ea0 0x7fa944196990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 --2- 172.21.15.70:0/1855040733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa944101850 0x7fa944196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.724+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 >> 172.21.15.70:0/1855040733 conn(0x7fa9440fc750 msgr2=0x7fa9440fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.725+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 shutdown_connections 2024-04-02T04:54:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.725+0000 7fa94c0a6700 1 -- 172.21.15.70:0/1855040733 wait complete. 2024-04-02T04:54:40.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.833+0000 7f8990809700 1 -- 172.21.15.70:0/1844657522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 msgr2=0x7f898c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.833+0000 7f8990809700 1 --2- 172.21.15.70:0/1844657522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 0x7f898c105840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f897c009910 tx=0x7f897c009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:40.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.833+0000 7f8990809700 1 -- 172.21.15.70:0/1844657522 shutdown_connections 2024-04-02T04:54:40.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.833+0000 7f8990809700 1 --2- 172.21.15.70:0/1844657522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 0x7f898c105840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.833+0000 7f8990809700 1 --2- 172.21.15.70:0/1844657522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f898c0ff5a0 0x7f898c0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.833+0000 7f8990809700 1 -- 172.21.15.70:0/1844657522 >> 172.21.15.70:0/1844657522 conn(0x7f898c074e30 msgr2=0x7f898c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:40.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.833+0000 7f8990809700 1 -- 172.21.15.70:0/1844657522 shutdown_connections 2024-04-02T04:54:40.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.833+0000 7f8990809700 1 -- 172.21.15.70:0/1844657522 wait complete. 2024-04-02T04:54:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8990809700 1 Processor -- start 2024-04-02T04:54:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8990809700 1 -- start start 2024-04-02T04:54:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8990809700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f898c0ff5a0 0x7f898c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8990809700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 0x7f898c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8990809700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c19d260 con 0x7f898c0fff50 2024-04-02T04:54:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8990809700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c196710 con 0x7f898c0ff5a0 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8989d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 0x7f898c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8989d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 0x7f898c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34788/0 (socket says 172.21.15.70:34788) 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.834+0000 7f8989d9b700 1 -- 172.21.15.70:0/3638370675 learned_addr learned my addr 172.21.15.70:0/3638370675 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f898a59c700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f898c0ff5a0 0x7f898c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f8989d9b700 1 -- 172.21.15.70:0/3638370675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f898c0ff5a0 msgr2=0x7f898c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f8989d9b700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f898c0ff5a0 0x7f898c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f8989d9b700 1 -- 172.21.15.70:0/3638370675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0095f0 con 0x7f898c0fff50 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f898a59c700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f898c0ff5a0 0x7f898c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:54:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f8989d9b700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 0x7f898c19cbd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f897c0092e0 tx=0x7f897c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f897b7fe700 1 -- 172.21.15.70:0/3638370675 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897c01d070 con 0x7f898c0fff50 2024-04-02T04:54:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f897b7fe700 1 -- 172.21.15.70:0/3638370675 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f897c00f460 con 0x7f898c0fff50 2024-04-02T04:54:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c196990 con 0x7f898c0fff50 2024-04-02T04:54:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f897b7fe700 1 -- 172.21.15.70:0/3638370675 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897c0175f0 con 0x7f898c0fff50 2024-04-02T04:54:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.835+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c196e80 con 0x7f898c0fff50 2024-04-02T04:54:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.836+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c109200 con 0x7f898c0fff50 2024-04-02T04:54:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.836+0000 7f897b7fe700 1 -- 172.21.15.70:0/3638370675 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f897c00f5d0 con 0x7f898c0fff50 2024-04-02T04:54:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.837+0000 7f897b7fe700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f897406dd60 0x7f8974070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.837+0000 7f898a59c700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f897406dd60 0x7f8974070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.837+0000 7f897b7fe700 1 -- 172.21.15.70:0/3638370675 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f897c08f130 con 0x7f898c0fff50 2024-04-02T04:54:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.838+0000 7f898a59c700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f897406dd60 0x7f8974070220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8980005fd0 tx=0x7f8980005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:40.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:40.842+0000 7f897b7fe700 1 -- 172.21.15.70:0/3638370675 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f897c05c190 con 0x7f898c0fff50 2024-04-02T04:54:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.117+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f898c197cc0 con 0x7f898c0fff50 2024-04-02T04:54:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.117+0000 7f897b7fe700 1 -- 172.21.15.70:0/3638370675 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f897c05bd20 con 0x7f898c0fff50 2024-04-02T04:54:41.122 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:54:41.122 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:54:41.123 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:54:41.123 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:54:41.123 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:54:41.123 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:54:41.123 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:54:41.123 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:54:41.124 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:54:41.124 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:54:41.124 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:54:41.124 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:54:41.124 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:54:41.124 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:54:41.125 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:54:41.125 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:54:41.125 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:54:41.125 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:54:41.125 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:54:41.125 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:54:41.126 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:54:41.126 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:54:41.126 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:54:41.126 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:54:41.126 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:54:41.126 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:54:41.127 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:54:41.127 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:54:41.127 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:54:41.127 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:54:41.127 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:54:41.127 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:54:41.128 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:54:41.128 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:54:41.128 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:54:41.128 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:54:41.128 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:54:41.128 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:54:41.129 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:54:41.129 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:54:41.129 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:54:41.129 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:54:41.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f897406dd60 msgr2=0x7f8974070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f897406dd60 0x7f8974070220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8980005fd0 tx=0x7f8980005dc0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 msgr2=0x7f898c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 0x7f898c19cbd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f897c0092e0 tx=0x7f897c00be20 comp rx=0 tx=0).stop 2024-04-02T04:54:41.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 shutdown_connections 2024-04-02T04:54:41.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f897406dd60 0x7f8974070220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f898c0ff5a0 0x7f898c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 --2- 172.21.15.70:0/3638370675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f898c0fff50 0x7f898c19cbd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 >> 172.21.15.70:0/3638370675 conn(0x7f898c074e30 msgr2=0x7f898c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:41.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 shutdown_connections 2024-04-02T04:54:41.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.119+0000 7f8990809700 1 -- 172.21.15.70:0/3638370675 wait complete. 2024-04-02T04:54:41.133 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:54:41.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.230+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3763537367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 msgr2=0x7efdec0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.230+0000 7efdf2a1f700 1 --2- 172.21.15.70:0/3763537367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 0x7efdec0ff980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efddc009910 tx=0x7efddc009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:41.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.230+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3763537367 shutdown_connections 2024-04-02T04:54:41.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.230+0000 7efdf2a1f700 1 --2- 172.21.15.70:0/3763537367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0fff50 0x7efdec105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.230+0000 7efdf2a1f700 1 --2- 172.21.15.70:0/3763537367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 0x7efdec0ff980 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.230+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3763537367 >> 172.21.15.70:0/3763537367 conn(0x7efdec074e30 msgr2=0x7efdec075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.230+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3763537367 shutdown_connections 2024-04-02T04:54:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.230+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3763537367 wait complete. 2024-04-02T04:54:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdf2a1f700 1 Processor -- start 2024-04-02T04:54:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdf2a1f700 1 -- start start 2024-04-02T04:54:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdf2a1f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 0x7efdec100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdf2a1f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0fff50 0x7efdec100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdf2a1f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec100fa0 con 0x7efdec0ff5a0 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdf2a1f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec1010e0 con 0x7efdec0fff50 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 0x7efdec100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 0x7efdec100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34798/0 (socket says 172.21.15.70:34798) 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.231+0000 7efdebfff700 1 -- 172.21.15.70:0/3672369499 learned_addr learned my addr 172.21.15.70:0/3672369499 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efdeb7fe700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0fff50 0x7efdec100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efdebfff700 1 -- 172.21.15.70:0/3672369499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0fff50 msgr2=0x7efdec100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efdebfff700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0fff50 0x7efdec100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efdebfff700 1 -- 172.21.15.70:0/3672369499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0095f0 con 0x7efdec0ff5a0 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efdebfff700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 0x7efdec100520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efddc006010 tx=0x7efddc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efde97fa700 1 -- 172.21.15.70:0/3672369499 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efddc01d070 con 0x7efdec0ff5a0 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efde97fa700 1 -- 172.21.15.70:0/3672369499 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efddc005dc0 con 0x7efdec0ff5a0 2024-04-02T04:54:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efde97fa700 1 -- 172.21.15.70:0/3672369499 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efddc017400 con 0x7efdec0ff5a0 2024-04-02T04:54:41.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec101360 con 0x7efdec0ff5a0 2024-04-02T04:54:41.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.232+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec1a2e30 con 0x7efdec0ff5a0 2024-04-02T04:54:41.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.233+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec109270 con 0x7efdec0ff5a0 2024-04-02T04:54:41.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.234+0000 7efde97fa700 1 -- 172.21.15.70:0/3672369499 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7efddc003a60 con 0x7efdec0ff5a0 2024-04-02T04:54:41.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.234+0000 7efde97fa700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efdd406dd60 0x7efdd4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:41.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.234+0000 7efde97fa700 1 -- 172.21.15.70:0/3672369499 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efddc08efb0 con 0x7efdec0ff5a0 2024-04-02T04:54:41.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.234+0000 7efdeb7fe700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efdd406dd60 0x7efdd4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:41.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.235+0000 7efdeb7fe700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efdd406dd60 0x7efdd4070220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efdec101d10 tx=0x7efde000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:41.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.239+0000 7efde97fa700 1 -- 172.21.15.70:0/3672369499 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efddc0584e0 con 0x7efdec0ff5a0 2024-04-02T04:54:41.385 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:41 smithi070 ceph-mon[113635]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:41.385 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:41 smithi070 ceph-mon[113635]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:41.385 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:41 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1855040733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:41.385 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:41.385 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:54:41.385 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:54:41.386 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:41 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3638370675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:54:41.386 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:41.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.464+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efdec0611d0 con 0x7efdd406dd60 2024-04-02T04:54:41.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.465+0000 7efde97fa700 1 -- 172.21.15.70:0/3672369499 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7efdec0611d0 con 0x7efdd406dd60 2024-04-02T04:54:41.469 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "8/23 daemons upgraded", 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:54:41.470 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.467+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efdd406dd60 msgr2=0x7efdd4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.467+0000 7efdf2a1f700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efdd406dd60 0x7efdd4070220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efdec101d10 tx=0x7efde000b410 comp rx=0 tx=0).stop 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.467+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 msgr2=0x7efdec100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.467+0000 7efdf2a1f700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 0x7efdec100520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efddc006010 tx=0x7efddc00bd70 comp rx=0 tx=0).stop 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.468+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 shutdown_connections 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.468+0000 7efdf2a1f700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efdd406dd60 0x7efdd4070220 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.468+0000 7efdf2a1f700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdec0ff5a0 0x7efdec100520 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.468+0000 7efdf2a1f700 1 --2- 172.21.15.70:0/3672369499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdec0fff50 0x7efdec100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.468+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 >> 172.21.15.70:0/3672369499 conn(0x7efdec074e30 msgr2=0x7efdec0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:41.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.468+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 shutdown_connections 2024-04-02T04:54:41.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.468+0000 7efdf2a1f700 1 -- 172.21.15.70:0/3672369499 wait complete. 2024-04-02T04:54:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:41 smithi179 ceph-mon[84288]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:41 smithi179 ceph-mon[84288]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:41 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1855040733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:54:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:54:41.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:54:41.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:41 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3638370675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:54:41.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:41.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 -- 172.21.15.70:0/3235163628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 msgr2=0x7f76941070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 --2- 172.21.15.70:0/3235163628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 0x7f76941070b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7690009910 tx=0x7f7690009c20 comp rx=0 tx=0).stop 2024-04-02T04:54:41.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 -- 172.21.15.70:0/3235163628 shutdown_connections 2024-04-02T04:54:41.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 --2- 172.21.15.70:0/3235163628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7694068580 0x7f76940689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 --2- 172.21.15.70:0/3235163628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 0x7f76941070b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 -- 172.21.15.70:0/3235163628 >> 172.21.15.70:0/3235163628 conn(0x7f7694075670 msgr2=0x7f7694075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:41.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 -- 172.21.15.70:0/3235163628 shutdown_connections 2024-04-02T04:54:41.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 -- 172.21.15.70:0/3235163628 wait complete. 2024-04-02T04:54:41.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 Processor -- start 2024-04-02T04:54:41.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.575+0000 7f769c0f9700 1 -- start start 2024-04-02T04:54:41.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f769c0f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7694068580 0x7f769419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:41.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f769c0f9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 0x7f769419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:41.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f769c0f9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769419d1a0 con 0x7f7694106cd0 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f769c0f9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941966e0 con 0x7f7694068580 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f7699694700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 0x7f769419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f7699694700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 0x7f769419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34824/0 (socket says 172.21.15.70:34824) 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f7699694700 1 -- 172.21.15.70:0/2527446682 learned_addr learned my addr 172.21.15.70:0/2527446682 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f7699e95700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7694068580 0x7f769419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f7699694700 1 -- 172.21.15.70:0/2527446682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7694068580 msgr2=0x7f769419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f7699694700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7694068580 0x7f769419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f7699694700 1 -- 172.21.15.70:0/2527446682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76900095f0 con 0x7f7694106cd0 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f7699694700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 0x7f769419cba0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f768400c930 tx=0x7f768400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f768affd700 1 -- 172.21.15.70:0/2527446682 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7684007ad0 con 0x7f7694106cd0 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76941969c0 con 0x7f7694106cd0 2024-04-02T04:54:41.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f768affd700 1 -- 172.21.15.70:0/2527446682 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7684007c30 con 0x7f7694106cd0 2024-04-02T04:54:41.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f768affd700 1 -- 172.21.15.70:0/2527446682 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7684018610 con 0x7f7694106cd0 2024-04-02T04:54:41.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.576+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7694196f10 con 0x7f7694106cd0 2024-04-02T04:54:41.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.577+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76941091d0 con 0x7f7694106cd0 2024-04-02T04:54:41.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.578+0000 7f768affd700 1 -- 172.21.15.70:0/2527446682 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f768400f450 con 0x7f7694106cd0 2024-04-02T04:54:41.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.578+0000 7f768affd700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f768006dd60 0x7f7680070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:41.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.578+0000 7f768affd700 1 -- 172.21.15.70:0/2527446682 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f768408dd80 con 0x7f7694106cd0 2024-04-02T04:54:41.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.578+0000 7f7699e95700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f768006dd60 0x7f7680070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:41.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.579+0000 7f7699e95700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f768006dd60 0x7f7680070220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f769000b5c0 tx=0x7f7690005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:41.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.583+0000 7f768affd700 1 -- 172.21.15.70:0/2527446682 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f768405ade0 con 0x7f7694106cd0 2024-04-02T04:54:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.864+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7694197c20 con 0x7f7694106cd0 2024-04-02T04:54:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.864+0000 7f768affd700 1 -- 172.21.15.70:0/2527446682 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f768405a970 con 0x7f7694106cd0 2024-04-02T04:54:41.868 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:54:41.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f768006dd60 msgr2=0x7f7680070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f768006dd60 0x7f7680070220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f769000b5c0 tx=0x7f7690005960 comp rx=0 tx=0).stop 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 msgr2=0x7f769419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 0x7f769419cba0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f768400c930 tx=0x7f768400ccf0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 shutdown_connections 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f768006dd60 0x7f7680070220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7694068580 0x7f769419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 --2- 172.21.15.70:0/2527446682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7694106cd0 0x7f769419cba0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 >> 172.21.15.70:0/2527446682 conn(0x7f7694075670 msgr2=0x7f76940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 shutdown_connections 2024-04-02T04:54:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:54:41.866+0000 7f769c0f9700 1 -- 172.21.15.70:0/2527446682 wait complete. 2024-04-02T04:54:42.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:42 smithi070 ceph-mon[113635]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:42.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:42 smithi070 ceph-mon[113635]: Upgrade: Updating osd.2 2024-04-02T04:54:42.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:42 smithi070 ceph-mon[113635]: Deploying daemon osd.2 on smithi070 2024-04-02T04:54:42.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:42 smithi070 ceph-mon[113635]: pgmap v100: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:42.493 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:42 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2527446682' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:54:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:42 smithi179 ceph-mon[84288]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:42 smithi179 ceph-mon[84288]: Upgrade: Updating osd.2 2024-04-02T04:54:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:42 smithi179 ceph-mon[84288]: Deploying daemon osd.2 on smithi070 2024-04-02T04:54:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:42 smithi179 ceph-mon[84288]: pgmap v100: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:42 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2527446682' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:54:43.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:43 smithi070 ceph-mon[113635]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:43 smithi179 ceph-mon[84288]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:54:44.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:44 smithi070 ceph-mon[113635]: pgmap v101: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:44.280 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:44 smithi070 systemd[1]: Stopping Ceph osd.2 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:54:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:44 smithi179 ceph-mon[84288]: pgmap v101: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:44.780 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:44 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[63325]: 2024-04-02T04:54:44.427+0000 7f370342f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:54:44.780 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:44 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[63325]: 2024-04-02T04:54:44.427+0000 7f370342f700 -1 osd.2 71 *** Got signal Terminated *** 2024-04-02T04:54:44.780 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:44 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[63325]: 2024-04-02T04:54:44.427+0000 7f370342f700 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:54:45.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:45 smithi179 ceph-mon[84288]: osd.2 marked itself down and dead 2024-04-02T04:54:45.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:45 smithi070 ceph-mon[113635]: osd.2 marked itself down and dead 2024-04-02T04:54:45.882 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:45 smithi070 podman[136248]: 2024-04-02 04:54:45.59166494 +0000 UTC m=+1.277727605 container died f1bc321ec6c36680c322939858cfc818bd09a11d57322455cf4bef3ae606a07e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1) 2024-04-02T04:54:46.188 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:45 smithi070 podman[136248]: 2024-04-02 04:54:45.951830935 +0000 UTC m=+1.637893599 container cleanup f1bc321ec6c36680c322939858cfc818bd09a11d57322455cf4bef3ae606a07e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-02T04:54:46.188 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:45 smithi070 bash[136248]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T04:54:46.529 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:46 smithi070 podman[136265]: 2024-04-02 04:54:46.228236288 +0000 UTC m=+0.637380430 container remove f1bc321ec6c36680c322939858cfc818bd09a11d57322455cf4bef3ae606a07e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T04:54:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:46 smithi070 ceph-mon[113635]: pgmap v102: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:46 smithi070 ceph-mon[113635]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:54:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:46 smithi070 ceph-mon[113635]: osdmap e72: 6 total, 5 up, 6 in 2024-04-02T04:54:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:46 smithi179 ceph-mon[84288]: pgmap v102: 65 pgs: 65 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:46 smithi179 ceph-mon[84288]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:54:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:46 smithi179 ceph-mon[84288]: osdmap e72: 6 total, 5 up, 6 in 2024-04-02T04:54:46.886 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:46 smithi070 podman[136357]: 2024-04-02 04:54:46.473184364 +0000 UTC m=+0.017923229 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:54:46.886 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:46 smithi070 podman[136357]: 2024-04-02 04:54:46.612961876 +0000 UTC m=+0.157700719 container create 8917f39b8e09085faec89498653cc6e2f691815c77d4576b434e9ce1b0ca965b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T04:54:46.886 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:46 smithi070 podman[136357]: 2024-04-02 04:54:46.882738586 +0000 UTC m=+0.427477440 container init 8917f39b8e09085faec89498653cc6e2f691815c77d4576b434e9ce1b0ca965b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T04:54:47.175 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:46 smithi070 podman[136357]: 2024-04-02 04:54:46.886161714 +0000 UTC m=+0.430900565 container start 8917f39b8e09085faec89498653cc6e2f691815c77d4576b434e9ce1b0ca965b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, ceph=True, GIT_CLEAN=True) 2024-04-02T04:54:47.176 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:46 smithi070 podman[136357]: 2024-04-02 04:54:46.965559375 +0000 UTC m=+0.510298232 container attach 8917f39b8e09085faec89498653cc6e2f691815c77d4576b434e9ce1b0ca965b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T04:54:47.514 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:47 smithi070 ceph-mon[113635]: osdmap e73: 6 total, 5 up, 6 in 2024-04-02T04:54:47.515 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:47 smithi070 podman[136357]: 2024-04-02 04:54:47.172708847 +0000 UTC m=+0.717447696 container died 8917f39b8e09085faec89498653cc6e2f691815c77d4576b434e9ce1b0ca965b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:54:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:47 smithi179 ceph-mon[84288]: osdmap e73: 6 total, 5 up, 6 in 2024-04-02T04:54:47.767 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:47 smithi070 podman[136383]: 2024-04-02 04:54:47.761025806 +0000 UTC m=+0.582835161 container remove 8917f39b8e09085faec89498653cc6e2f691815c77d4576b434e9ce1b0ca965b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793) 2024-04-02T04:54:48.197 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:47 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.2.service: Deactivated successfully. 2024-04-02T04:54:48.198 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:47 smithi070 systemd[1]: Stopped Ceph osd.2 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:54:48.198 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:47 smithi070 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.2.service: Consumed 1min 61ms CPU time. 2024-04-02T04:54:48.198 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:47 smithi070 systemd[1]: Starting Ceph osd.2 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:54:48.510 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-mon[113635]: pgmap v105: 65 pgs: 5 active+undersized, 5 active+undersized+degraded, 7 stale+active+clean, 48 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 10/228 objects degraded (4.386%) 2024-04-02T04:54:48.510 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-mon[113635]: Health check failed: Degraded data redundancy: 10/228 objects degraded (4.386%), 5 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:48.511 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 podman[136486]: 2024-04-02 04:54:48.109576855 +0000 UTC m=+0.018265592 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:54:48.511 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 podman[136486]: 2024-04-02 04:54:48.246653412 +0000 UTC m=+0.155342117 container create 0d925f7b514763435b6ef84ac47f758e041a8a77e67dd69f2518b78d0b47dfb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-02T04:54:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:48 smithi179 ceph-mon[84288]: pgmap v105: 65 pgs: 5 active+undersized, 5 active+undersized+degraded, 7 stale+active+clean, 48 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 10/228 objects degraded (4.386%) 2024-04-02T04:54:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:48 smithi179 ceph-mon[84288]: Health check failed: Degraded data redundancy: 10/228 objects degraded (4.386%), 5 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:48.779 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 podman[136486]: 2024-04-02 04:54:48.507306558 +0000 UTC m=+0.415995283 container init 0d925f7b514763435b6ef84ac47f758e041a8a77e67dd69f2518b78d0b47dfb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate, io.buildah.version=1.33.5, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T04:54:48.780 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 podman[136486]: 2024-04-02 04:54:48.510610008 +0000 UTC m=+0.419298716 container start 0d925f7b514763435b6ef84ac47f758e041a8a77e67dd69f2518b78d0b47dfb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=squid-6cec793) 2024-04-02T04:54:48.780 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 podman[136486]: 2024-04-02 04:54:48.590336335 +0000 UTC m=+0.499025047 container attach 0d925f7b514763435b6ef84ac47f758e041a8a77e67dd69f2518b78d0b47dfb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-02T04:54:49.280 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate[136500]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T04:54:49.280 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate[136500]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-02T04:54:49.280 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 bash[136486]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T04:54:49.280 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 bash[136486]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-02T04:54:49.280 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 bash[136486]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 bash[136486]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 bash[136486]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 bash[136486]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 bash[136486]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate[136500]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate[136500]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate[136500]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate[136500]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate[136500]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-02T04:54:49.281 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:48 smithi070 podman[136486]: 2024-04-02 04:54:48.98180965 +0000 UTC m=+0.890498368 container died 0d925f7b514763435b6ef84ac47f758e041a8a77e67dd69f2518b78d0b47dfb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-02T04:54:49.946 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:49 smithi070 podman[136641]: 2024-04-02 04:54:49.585757356 +0000 UTC m=+0.595311627 container remove 0d925f7b514763435b6ef84ac47f758e041a8a77e67dd69f2518b78d0b47dfb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-02T04:54:49.946 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:49 smithi070 podman[136708]: 2024-04-02 04:54:49.702642878 +0000 UTC m=+0.017259815 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:54:49.947 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:49 smithi070 podman[136708]: 2024-04-02 04:54:49.811974109 +0000 UTC m=+0.126591092 container create 87cd16d907e6098ce8795c506b3ac88c5f8b6ab8ef32daee6cd204c81674e796 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.build-date=20240326) 2024-04-02T04:54:50.200 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:50 smithi070 podman[136708]: 2024-04-02 04:54:50.064694104 +0000 UTC m=+0.379311041 container init 87cd16d907e6098ce8795c506b3ac88c5f8b6ab8ef32daee6cd204c81674e796 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T04:54:50.200 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:50 smithi070 podman[136708]: 2024-04-02 04:54:50.067828698 +0000 UTC m=+0.382445638 container start 87cd16d907e6098ce8795c506b3ac88c5f8b6ab8ef32daee6cd204c81674e796 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T04:54:50.201 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:50 smithi070 ceph-osd[136727]: -- 172.21.15.70:0/3049381566 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576806ff680 con 0x5576806c8c00 2024-04-02T04:54:50.201 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:50 smithi070 bash[136708]: 87cd16d907e6098ce8795c506b3ac88c5f8b6ab8ef32daee6cd204c81674e796 2024-04-02T04:54:50.201 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:50 smithi070 systemd[1]: Started Ceph osd.2 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:54:50.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:50 smithi179 ceph-mon[84288]: pgmap v106: 65 pgs: 10 active+undersized, 8 active+undersized+degraded, 5 stale+active+clean, 42 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 15/228 objects degraded (6.579%) 2024-04-02T04:54:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:50 smithi070 ceph-mon[113635]: pgmap v106: 65 pgs: 10 active+undersized, 8 active+undersized+degraded, 5 stale+active+clean, 42 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 15/228 objects degraded (6.579%) 2024-04-02T04:54:51.029 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:50 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[136723]: 2024-04-02T04:54:50.611+0000 7f3438b097c0 -1 Falling back to public interface 2024-04-02T04:54:51.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:51.305 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:51.305 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:54:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:54:52.311 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:52 smithi070 ceph-mon[113635]: pgmap v107: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/228 objects degraded (14.035%) 2024-04-02T04:54:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:52 smithi179 ceph-mon[84288]: pgmap v107: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/228 objects degraded (14.035%) 2024-04-02T04:54:53.516 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:53 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 32/228 objects degraded (14.035%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:53 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 32/228 objects degraded (14.035%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:54.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:54 smithi070 ceph-mon[113635]: pgmap v108: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/228 objects degraded (14.035%) 2024-04-02T04:54:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:54 smithi179 ceph-mon[84288]: pgmap v108: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/228 objects degraded (14.035%) 2024-04-02T04:54:55.029 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:54 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[136723]: 2024-04-02T04:54:54.806+0000 7f3438b097c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-04-02T04:54:55.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:55 smithi179 ceph-mon[84288]: from='osd.2 [v2:172.21.15.70:6818/347493648,v1:172.21.15.70:6819/347493648]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T04:54:55.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:55 smithi070 ceph-mon[113635]: from='osd.2 [v2:172.21.15.70:6818/347493648,v1:172.21.15.70:6819/347493648]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T04:54:55.530 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:55 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[136723]: 2024-04-02T04:54:55.226+0000 7f3438b097c0 -1 osd.2 71 log_to_monitors true 2024-04-02T04:54:55.530 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 04:54:55 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[136723]: 2024-04-02T04:54:55.300+0000 7f3429f61700 -1 osd.2 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:54:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:56 smithi070 ceph-mon[113635]: pgmap v109: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/228 objects degraded (14.035%) 2024-04-02T04:54:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:56 smithi070 ceph-mon[113635]: from='osd.2 [v2:172.21.15.70:6818/347493648,v1:172.21.15.70:6819/347493648]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T04:54:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:56 smithi070 ceph-mon[113635]: osdmap e74: 6 total, 5 up, 6 in 2024-04-02T04:54:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:56 smithi070 ceph-mon[113635]: from='osd.2 [v2:172.21.15.70:6818/347493648,v1:172.21.15.70:6819/347493648]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:54:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:56 smithi179 ceph-mon[84288]: pgmap v109: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 205 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/228 objects degraded (14.035%) 2024-04-02T04:54:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:56 smithi179 ceph-mon[84288]: from='osd.2 [v2:172.21.15.70:6818/347493648,v1:172.21.15.70:6819/347493648]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T04:54:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:56 smithi179 ceph-mon[84288]: osdmap e74: 6 total, 5 up, 6 in 2024-04-02T04:54:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:56 smithi179 ceph-mon[84288]: from='osd.2 [v2:172.21.15.70:6818/347493648,v1:172.21.15.70:6819/347493648]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-04-02T04:54:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:57.461 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:57 smithi070 ceph-mon[113635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:54:57.461 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:57 smithi070 ceph-mon[113635]: osd.2 [v2:172.21.15.70:6818/347493648,v1:172.21.15.70:6819/347493648] boot 2024-04-02T04:54:57.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:57 smithi070 ceph-mon[113635]: osdmap e75: 6 total, 6 up, 6 in 2024-04-02T04:54:57.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:54:57.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:57.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:57 smithi179 ceph-mon[84288]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:54:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:57 smithi179 ceph-mon[84288]: osd.2 [v2:172.21.15.70:6818/347493648,v1:172.21.15.70:6819/347493648] boot 2024-04-02T04:54:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:57 smithi179 ceph-mon[84288]: osdmap e75: 6 total, 6 up, 6 in 2024-04-02T04:54:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:54:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:57.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:54:58.494 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:58 smithi070 ceph-mon[113635]: pgmap v112: 65 pgs: 8 peering, 11 active+undersized, 9 active+undersized+degraded, 37 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23/228 objects degraded (10.088%) 2024-04-02T04:54:58.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:58 smithi070 ceph-mon[113635]: osdmap e76: 6 total, 6 up, 6 in 2024-04-02T04:54:58.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:54:58 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 23/228 objects degraded (10.088%), 9 pgs degraded (PG_DEGRADED) 2024-04-02T04:54:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:58 smithi179 ceph-mon[84288]: pgmap v112: 65 pgs: 8 peering, 11 active+undersized, 9 active+undersized+degraded, 37 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23/228 objects degraded (10.088%) 2024-04-02T04:54:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:58 smithi179 ceph-mon[84288]: osdmap e76: 6 total, 6 up, 6 in 2024-04-02T04:54:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:54:58 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 23/228 objects degraded (10.088%), 9 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:00 smithi070 ceph-mon[113635]: pgmap v114: 65 pgs: 8 peering, 7 active+undersized, 7 active+undersized+degraded, 43 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20/228 objects degraded (8.772%) 2024-04-02T04:55:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:00 smithi179 ceph-mon[84288]: pgmap v114: 65 pgs: 8 peering, 7 active+undersized, 7 active+undersized+degraded, 43 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20/228 objects degraded (8.772%) 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:55:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:55:01.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-04-02T04:55:01.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:01 smithi070 ceph-mon[113635]: pgmap v115: 65 pgs: 8 peering, 57 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:01.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:01.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:01.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:01.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-04-02T04:55:01.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:01 smithi179 ceph-mon[84288]: pgmap v115: 65 pgs: 8 peering, 57 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:02.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:02 smithi070 ceph-mon[113635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/228 objects degraded (8.772%), 7 pgs degraded) 2024-04-02T04:55:02.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:02 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:55:02.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:02 smithi179 ceph-mon[84288]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/228 objects degraded (8.772%), 7 pgs degraded) 2024-04-02T04:55:02.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:02 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:55:03.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:03 smithi070 ceph-mon[113635]: pgmap v116: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:03.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:03 smithi179 ceph-mon[84288]: pgmap v116: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:06.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:06 smithi070 ceph-mon[113635]: pgmap v117: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:06 smithi179 ceph-mon[84288]: pgmap v117: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:08.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:08 smithi070 ceph-mon[113635]: pgmap v118: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:08.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:08 smithi179 ceph-mon[84288]: pgmap v118: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:10.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:10 smithi070 ceph-mon[113635]: pgmap v119: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:10 smithi179 ceph-mon[84288]: pgmap v119: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:11.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 -- 172.21.15.70:0/3929110952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64100750 msgr2=0x7fbe641047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 --2- 172.21.15.70:0/3929110952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64100750 0x7fbe641047a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009910 tx=0x7fbe54009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 -- 172.21.15.70:0/3929110952 shutdown_connections 2024-04-02T04:55:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 --2- 172.21.15.70:0/3929110952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe64100750 0x7fbe641047a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 --2- 172.21.15.70:0/3929110952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe640ffda0 0x7fbe64100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 -- 172.21.15.70:0/3929110952 >> 172.21.15.70:0/3929110952 conn(0x7fbe640fb650 msgr2=0x7fbe640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 -- 172.21.15.70:0/3929110952 shutdown_connections 2024-04-02T04:55:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 -- 172.21.15.70:0/3929110952 wait complete. 2024-04-02T04:55:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.979+0000 7fbe6a12c700 1 Processor -- start 2024-04-02T04:55:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe6a12c700 1 -- start start 2024-04-02T04:55:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe6a12c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe640ffda0 0x7fbe64101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe6a12c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe64100750 0x7fbe64101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe6a12c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe641022c0 con 0x7fbe640ffda0 2024-04-02T04:55:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe6a12c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64102400 con 0x7fbe64100750 2024-04-02T04:55:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe637fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe640ffda0 0x7fbe64101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe637fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe640ffda0 0x7fbe64101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54908/0 (socket says 172.21.15.70:54908) 2024-04-02T04:55:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe637fe700 1 -- 172.21.15.70:0/2775325841 learned_addr learned my addr 172.21.15.70:0/2775325841 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe62ffd700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe64100750 0x7fbe64101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe637fe700 1 -- 172.21.15.70:0/2775325841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe64100750 msgr2=0x7fbe64101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe637fe700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe64100750 0x7fbe64101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe637fe700 1 -- 172.21.15.70:0/2775325841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe540095f0 con 0x7fbe640ffda0 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.980+0000 7fbe637fe700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe640ffda0 0x7fbe64101840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d8d0 tx=0x7fbe5800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.981+0000 7fbe60ff9700 1 -- 172.21.15.70:0/2775325841 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe580098a0 con 0x7fbe640ffda0 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.981+0000 7fbe60ff9700 1 -- 172.21.15.70:0/2775325841 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe58010460 con 0x7fbe640ffda0 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.981+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe641a2a70 con 0x7fbe640ffda0 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.981+0000 7fbe60ff9700 1 -- 172.21.15.70:0/2775325841 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5800f5f0 con 0x7fbe640ffda0 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.981+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe641a2e80 con 0x7fbe640ffda0 2024-04-02T04:55:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.982+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe641091b0 con 0x7fbe640ffda0 2024-04-02T04:55:11.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.982+0000 7fbe60ff9700 1 -- 172.21.15.70:0/2775325841 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbe58009a00 con 0x7fbe640ffda0 2024-04-02T04:55:11.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.982+0000 7fbe60ff9700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06dd60 0x7fbe4c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:11.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.983+0000 7fbe60ff9700 1 -- 172.21.15.70:0/2775325841 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbe5808d990 con 0x7fbe640ffda0 2024-04-02T04:55:11.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.983+0000 7fbe62ffd700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06dd60 0x7fbe4c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:11.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.983+0000 7fbe62ffd700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06dd60 0x7fbe4c070220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe64103030 tx=0x7fbe5401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:11.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:11.987+0000 7fbe60ff9700 1 -- 172.21.15.70:0/2775325841 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe58056f40 con 0x7fbe640ffda0 2024-04-02T04:55:12.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.200+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe6402cce0 con 0x7fbe4c06dd60 2024-04-02T04:55:12.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.201+0000 7fbe60ff9700 1 -- 172.21.15.70:0/2775325841 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbe6402cce0 con 0x7fbe4c06dd60 2024-04-02T04:55:12.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06dd60 msgr2=0x7fbe4c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06dd60 0x7fbe4c070220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe64103030 tx=0x7fbe5401a040 comp rx=0 tx=0).stop 2024-04-02T04:55:12.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe640ffda0 msgr2=0x7fbe64101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe640ffda0 0x7fbe64101840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d8d0 tx=0x7fbe5800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 shutdown_connections 2024-04-02T04:55:12.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbe4c06dd60 0x7fbe4c070220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe640ffda0 0x7fbe64101840 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 --2- 172.21.15.70:0/2775325841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe64100750 0x7fbe64101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 >> 172.21.15.70:0/2775325841 conn(0x7fbe640fb650 msgr2=0x7fbe64069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:12.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 shutdown_connections 2024-04-02T04:55:12.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.204+0000 7fbe6a12c700 1 -- 172.21.15.70:0/2775325841 wait complete. 2024-04-02T04:55:12.221 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:55:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:12 smithi070 ceph-mon[113635]: pgmap v120: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.310+0000 7f9a52f3d700 1 -- 172.21.15.70:0/2778815744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c105350 msgr2=0x7f9a4c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.310+0000 7f9a52f3d700 1 --2- 172.21.15.70:0/2778815744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c105350 0x7f9a4c105730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400aa20 tx=0x7f9a4400ad30 comp rx=0 tx=0).stop 2024-04-02T04:55:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.310+0000 7f9a52f3d700 1 -- 172.21.15.70:0/2778815744 shutdown_connections 2024-04-02T04:55:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.310+0000 7f9a52f3d700 1 --2- 172.21.15.70:0/2778815744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c0ff340 0x7f9a4c0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.310+0000 7f9a52f3d700 1 --2- 172.21.15.70:0/2778815744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c105350 0x7f9a4c105730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.310+0000 7f9a52f3d700 1 -- 172.21.15.70:0/2778815744 >> 172.21.15.70:0/2778815744 conn(0x7f9a4c074e30 msgr2=0x7f9a4c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.310+0000 7f9a52f3d700 1 -- 172.21.15.70:0/2778815744 shutdown_connections 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.310+0000 7f9a52f3d700 1 -- 172.21.15.70:0/2778815744 wait complete. 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a52f3d700 1 Processor -- start 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a52f3d700 1 -- start start 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a52f3d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c0ff340 0x7f9a4c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a52f3d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c105350 0x7f9a4c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a52f3d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c19d260 con 0x7f9a4c0ff340 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a52f3d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c196710 con 0x7f9a4c105350 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a50cd9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c0ff340 0x7f9a4c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a50cd9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c0ff340 0x7f9a4c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54932/0 (socket says 172.21.15.70:54932) 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a50cd9700 1 -- 172.21.15.70:0/20825757 learned_addr learned my addr 172.21.15.70:0/20825757 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a4bfff700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c105350 0x7f9a4c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a50cd9700 1 -- 172.21.15.70:0/20825757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c105350 msgr2=0x7f9a4c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a50cd9700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c105350 0x7f9a4c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a50cd9700 1 -- 172.21.15.70:0/20825757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4400a700 con 0x7f9a4c0ff340 2024-04-02T04:55:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.311+0000 7f9a50cd9700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c0ff340 0x7f9a4c19c690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9a44006010 tx=0x7f9a4400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.312+0000 7f9a49ffb700 1 -- 172.21.15.70:0/20825757 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a44004b70 con 0x7f9a4c0ff340 2024-04-02T04:55:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.312+0000 7f9a49ffb700 1 -- 172.21.15.70:0/20825757 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a44004cd0 con 0x7f9a4c0ff340 2024-04-02T04:55:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.312+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c196990 con 0x7f9a4c0ff340 2024-04-02T04:55:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.312+0000 7f9a49ffb700 1 -- 172.21.15.70:0/20825757 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a44016820 con 0x7f9a4c0ff340 2024-04-02T04:55:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.312+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c196e80 con 0x7f9a4c0ff340 2024-04-02T04:55:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.313+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c107850 con 0x7f9a4c0ff340 2024-04-02T04:55:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.313+0000 7f9a49ffb700 1 -- 172.21.15.70:0/20825757 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f9a4400ec30 con 0x7f9a4c0ff340 2024-04-02T04:55:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.314+0000 7f9a49ffb700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9a3806dd60 0x7f9a38070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.314+0000 7f9a49ffb700 1 -- 172.21.15.70:0/20825757 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f9a44012070 con 0x7f9a4c0ff340 2024-04-02T04:55:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.314+0000 7f9a4bfff700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9a3806dd60 0x7f9a38070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.314+0000 7f9a4bfff700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9a3806dd60 0x7f9a38070220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c197b80 tx=0x7f9a3c00c040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.318+0000 7f9a49ffb700 1 -- 172.21.15.70:0/20825757 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9a4405cc80 con 0x7f9a4c0ff340 2024-04-02T04:55:12.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.532+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a4c1970f0 con 0x7f9a3806dd60 2024-04-02T04:55:12.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.534+0000 7f9a49ffb700 1 -- 172.21.15.70:0/20825757 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9a4c1970f0 con 0x7f9a3806dd60 2024-04-02T04:55:12.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9a3806dd60 msgr2=0x7f9a38070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9a3806dd60 0x7f9a38070220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c197b80 tx=0x7f9a3c00c040 comp rx=0 tx=0).stop 2024-04-02T04:55:12.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c0ff340 msgr2=0x7f9a4c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c0ff340 0x7f9a4c19c690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9a44006010 tx=0x7f9a4400cd20 comp rx=0 tx=0).stop 2024-04-02T04:55:12.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 shutdown_connections 2024-04-02T04:55:12.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9a3806dd60 0x7f9a38070220 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a4c0ff340 0x7f9a4c19c690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 --2- 172.21.15.70:0/20825757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c105350 0x7f9a4c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 >> 172.21.15.70:0/20825757 conn(0x7f9a4c074e30 msgr2=0x7f9a4c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:12.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 shutdown_connections 2024-04-02T04:55:12.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.536+0000 7f9a52f3d700 1 -- 172.21.15.70:0/20825757 wait complete. 2024-04-02T04:55:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:12 smithi179 ceph-mon[84288]: pgmap v120: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:12.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 -- 172.21.15.70:0/1083703131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4100750 msgr2=0x7f6ff41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 --2- 172.21.15.70:0/1083703131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4100750 0x7f6ff41047a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009910 tx=0x7f6fe4009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:12.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 -- 172.21.15.70:0/1083703131 shutdown_connections 2024-04-02T04:55:12.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 --2- 172.21.15.70:0/1083703131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff4100750 0x7f6ff41047a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 --2- 172.21.15.70:0/1083703131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff40ffda0 0x7f6ff4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 -- 172.21.15.70:0/1083703131 >> 172.21.15.70:0/1083703131 conn(0x7f6ff40fb650 msgr2=0x7f6ff40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:12.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 -- 172.21.15.70:0/1083703131 shutdown_connections 2024-04-02T04:55:12.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 -- 172.21.15.70:0/1083703131 wait complete. 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.640+0000 7f6ffc465700 1 Processor -- start 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffc465700 1 -- start start 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffc465700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff40ffda0 0x7f6ff4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffc465700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff4100750 0x7f6ff4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffc465700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff41022c0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffc465700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4102400 con 0x7f6ff4100750 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffa201700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff40ffda0 0x7f6ff4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffa201700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff40ffda0 0x7f6ff4101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54954/0 (socket says 172.21.15.70:54954) 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffa201700 1 -- 172.21.15.70:0/373198402 learned_addr learned my addr 172.21.15.70:0/373198402 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ff9a00700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff4100750 0x7f6ff4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffa201700 1 -- 172.21.15.70:0/373198402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff4100750 msgr2=0x7f6ff4101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffa201700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff4100750 0x7f6ff4101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffa201700 1 -- 172.21.15.70:0/373198402 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40095f0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ff9a00700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff4100750 0x7f6ff4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.641+0000 7f6ffa201700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff40ffda0 0x7f6ff4101840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000c8a0 tx=0x7f6ff000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.642+0000 7f6feb7fe700 1 -- 172.21.15.70:0/373198402 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff000cea0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.642+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff41a2ab0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.642+0000 7f6feb7fe700 1 -- 172.21.15.70:0/373198402 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ff0004830 con 0x7f6ff40ffda0 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.642+0000 7f6feb7fe700 1 -- 172.21.15.70:0/373198402 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff00055b0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.642+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff41a2ec0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.643+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff404f2e0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.644+0000 7f6feb7fe700 1 -- 172.21.15.70:0/373198402 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f6ff0007ad0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.644+0000 7f6feb7fe700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6fe006dd60 0x7f6fe0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.644+0000 7f6feb7fe700 1 -- 172.21.15.70:0/373198402 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6ff008d8d0 con 0x7f6ff40ffda0 2024-04-02T04:55:12.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.644+0000 7f6ff9a00700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6fe006dd60 0x7f6fe0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.645+0000 7f6ff9a00700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6fe006dd60 0x7f6fe0070220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4103030 tx=0x7f6fe401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:12.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.649+0000 7f6feb7fe700 1 -- 172.21.15.70:0/373198402 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6ff0056e00 con 0x7f6ff40ffda0 2024-04-02T04:55:12.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.858+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ff4103960 con 0x7f6fe006dd60 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.868+0000 7f6feb7fe700 1 -- 172.21.15.70:0/373198402 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6ff4103960 con 0x7f6fe006dd60 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (4m) 17s ago 15m 26.2M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (16m) 17s ago 16m 8678k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (15m) 113s ago 15m 7818k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (2m) 17s ago 16m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (116s) 113s ago 15m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (4m) 17s ago 15m 76.8M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:55:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (10m) 17s ago 10m 151M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (10m) 17s ago 10m 20.2M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (10m) 113s ago 10m 17.8M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (10m) 113s ago 10m 19.0M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (6m) 17s ago 17m 504M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (5m) 113s ago 14m 449M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (3m) 17s ago 17m 49.2M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (2m) 113s ago 14m 41.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (5m) 17s ago 15m 15.5M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (5m) 113s ago 14m 16.8M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (95s) 17s ago 14m 141M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (60s) 17s ago 13m 157M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (23s) 17s ago 13m 11.8M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (12m) 113s ago 12m 298M 2240M 18.2.1 5be31c24972a 3b51065ff69f 2024-04-02T04:55:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (12m) 113s ago 12m 275M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:55:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (11m) 113s ago 11m 228M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:55:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (5m) 17s ago 15m 62.9M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.870+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6fe006dd60 msgr2=0x7f6fe0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.870+0000 7f6ffc465700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6fe006dd60 0x7f6fe0070220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4103030 tx=0x7f6fe401a040 comp rx=0 tx=0).stop 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.870+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff40ffda0 msgr2=0x7f6ff4101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.870+0000 7f6ffc465700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff40ffda0 0x7f6ff4101840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000c8a0 tx=0x7f6ff000cc60 comp rx=0 tx=0).stop 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.871+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 shutdown_connections 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.871+0000 7f6ffc465700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f6fe006dd60 0x7f6fe0070220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.871+0000 7f6ffc465700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ff40ffda0 0x7f6ff4101840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.871+0000 7f6ffc465700 1 --2- 172.21.15.70:0/373198402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff4100750 0x7f6ff4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.871+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 >> 172.21.15.70:0/373198402 conn(0x7f6ff40fb650 msgr2=0x7f6ff40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:12.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.871+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 shutdown_connections 2024-04-02T04:55:12.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.871+0000 7f6ffc465700 1 -- 172.21.15.70:0/373198402 wait complete. 2024-04-02T04:55:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.972+0000 7f34411d9700 1 -- 172.21.15.70:0/544027789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c100750 msgr2=0x7f343c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.972+0000 7f34411d9700 1 --2- 172.21.15.70:0/544027789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c100750 0x7f343c1047a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3430009910 tx=0x7f3430009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.972+0000 7f34411d9700 1 -- 172.21.15.70:0/544027789 shutdown_connections 2024-04-02T04:55:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.972+0000 7f34411d9700 1 --2- 172.21.15.70:0/544027789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c100750 0x7f343c1047a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.972+0000 7f34411d9700 1 --2- 172.21.15.70:0/544027789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f343c0ffda0 0x7f343c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.972+0000 7f34411d9700 1 -- 172.21.15.70:0/544027789 >> 172.21.15.70:0/544027789 conn(0x7f343c0fb650 msgr2=0x7f343c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.973+0000 7f34411d9700 1 -- 172.21.15.70:0/544027789 shutdown_connections 2024-04-02T04:55:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.973+0000 7f34411d9700 1 -- 172.21.15.70:0/544027789 wait complete. 2024-04-02T04:55:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.973+0000 7f34411d9700 1 Processor -- start 2024-04-02T04:55:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f34411d9700 1 -- start start 2024-04-02T04:55:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f34411d9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c0ffda0 0x7f343c1955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f34411d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f343c100750 0x7f343c195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f34411d9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c196030 con 0x7f343c0ffda0 2024-04-02T04:55:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f34411d9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c1961a0 con 0x7f343c100750 2024-04-02T04:55:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f343ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c0ffda0 0x7f343c1955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f343ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c0ffda0 0x7f343c1955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54966/0 (socket says 172.21.15.70:54966) 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f343ad9d700 1 -- 172.21.15.70:0/2698881485 learned_addr learned my addr 172.21.15.70:0/2698881485 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f343a59c700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f343c100750 0x7f343c195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f343ad9d700 1 -- 172.21.15.70:0/2698881485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f343c100750 msgr2=0x7f343c195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f343ad9d700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f343c100750 0x7f343c195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f343ad9d700 1 -- 172.21.15.70:0/2698881485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34300095f0 con 0x7f343c0ffda0 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.974+0000 7f343a59c700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f343c100750 0x7f343c195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.975+0000 7f343ad9d700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c0ffda0 0x7f343c1955b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d8d0 tx=0x7f342c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.975+0000 7f3423fff700 1 -- 172.21.15.70:0/2698881485 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342c0098a0 con 0x7f343c0ffda0 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.975+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c1a2c20 con 0x7f343c0ffda0 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.975+0000 7f3423fff700 1 -- 172.21.15.70:0/2698881485 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f342c010460 con 0x7f343c0ffda0 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.975+0000 7f3423fff700 1 -- 172.21.15.70:0/2698881485 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342c00f5f0 con 0x7f343c0ffda0 2024-04-02T04:55:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.975+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c1a2f80 con 0x7f343c0ffda0 2024-04-02T04:55:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.976+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c1081a0 con 0x7f343c0ffda0 2024-04-02T04:55:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.976+0000 7f3423fff700 1 -- 172.21.15.70:0/2698881485 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f342c009a00 con 0x7f343c0ffda0 2024-04-02T04:55:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.977+0000 7f3423fff700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f342406dd60 0x7f3424070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.977+0000 7f3423fff700 1 -- 172.21.15.70:0/2698881485 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f342c08da30 con 0x7f343c0ffda0 2024-04-02T04:55:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.977+0000 7f343a59c700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f342406dd60 0x7f3424070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:12.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.977+0000 7f343a59c700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f342406dd60 0x7f3424070220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f343c069800 tx=0x7f343001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:12.981+0000 7f3423fff700 1 -- 172.21.15.70:0/2698881485 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f342c05aa90 con 0x7f343c0ffda0 2024-04-02T04:55:13.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.258+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f343c04f2e0 con 0x7f343c0ffda0 2024-04-02T04:55:13.263 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:13 smithi070 ceph-mon[113635]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:13.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.258+0000 7f3423fff700 1 -- 172.21.15.70:0/2698881485 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f342c05a620 con 0x7f343c0ffda0 2024-04-02T04:55:13.263 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:55:13.263 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:55:13.263 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:55:13.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 3 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:55:13.264 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-04-02T04:55:13.265 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 7 2024-04-02T04:55:13.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:55:13.265 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:55:13.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f342406dd60 msgr2=0x7f3424070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f342406dd60 0x7f3424070220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f343c069800 tx=0x7f343001a040 comp rx=0 tx=0).stop 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c0ffda0 msgr2=0x7f343c1955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c0ffda0 0x7f343c1955b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d8d0 tx=0x7f342c00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 shutdown_connections 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f342406dd60 0x7f3424070220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c0ffda0 0x7f343c1955b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 --2- 172.21.15.70:0/2698881485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f343c100750 0x7f343c195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.260+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 >> 172.21.15.70:0/2698881485 conn(0x7f343c0fb650 msgr2=0x7f343c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.261+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 shutdown_connections 2024-04-02T04:55:13.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.261+0000 7f34411d9700 1 -- 172.21.15.70:0/2698881485 wait complete. 2024-04-02T04:55:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.368+0000 7f2734c67700 1 -- 172.21.15.70:0/3635394795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 msgr2=0x7f2730105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.368+0000 7f2734c67700 1 --2- 172.21.15.70:0/3635394795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 0x7f2730105890 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2724009910 tx=0x7f2724009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.368+0000 7f2734c67700 1 -- 172.21.15.70:0/3635394795 shutdown_connections 2024-04-02T04:55:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.368+0000 7f2734c67700 1 --2- 172.21.15.70:0/3635394795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 0x7f2730105890 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.368+0000 7f2734c67700 1 --2- 172.21.15.70:0/3635394795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2730100e90 0x7f2730101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.368+0000 7f2734c67700 1 -- 172.21.15.70:0/3635394795 >> 172.21.15.70:0/3635394795 conn(0x7f27300fc740 msgr2=0x7f27300feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.368+0000 7f2734c67700 1 -- 172.21.15.70:0/3635394795 shutdown_connections 2024-04-02T04:55:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.368+0000 7f2734c67700 1 -- 172.21.15.70:0/3635394795 wait complete. 2024-04-02T04:55:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f2734c67700 1 Processor -- start 2024-04-02T04:55:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f2734c67700 1 -- start start 2024-04-02T04:55:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f2734c67700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2730100e90 0x7f2730073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f2734c67700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 0x7f27300735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:13.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f2734c67700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2730073b10 con 0x7f2730101840 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f2734c67700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2730073c50 con 0x7f2730100e90 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f272dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 0x7f27300735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f272dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 0x7f27300735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54974/0 (socket says 172.21.15.70:54974) 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f272dd9b700 1 -- 172.21.15.70:0/1822118062 learned_addr learned my addr 172.21.15.70:0/1822118062 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f272e59c700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2730100e90 0x7f2730073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f272dd9b700 1 -- 172.21.15.70:0/1822118062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2730100e90 msgr2=0x7f2730073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f272dd9b700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2730100e90 0x7f2730073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.369+0000 7f272dd9b700 1 -- 172.21.15.70:0/1822118062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27240095f0 con 0x7f2730101840 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.370+0000 7f272dd9b700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 0x7f27300735d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2724005f50 tx=0x7f272400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.370+0000 7f271f7fe700 1 -- 172.21.15.70:0/1822118062 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f272401d070 con 0x7f2730101840 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.370+0000 7f271f7fe700 1 -- 172.21.15.70:0/1822118062 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f272400f460 con 0x7f2730101840 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.370+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2730073ed0 con 0x7f2730101840 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.370+0000 7f271f7fe700 1 -- 172.21.15.70:0/1822118062 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2724017630 con 0x7f2730101840 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.370+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27301a2eb0 con 0x7f2730101840 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.371+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27301091e0 con 0x7f2730101840 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.371+0000 7f271f7fe700 1 -- 172.21.15.70:0/1822118062 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f272400f5d0 con 0x7f2730101840 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.371+0000 7f271f7fe700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f271806dd60 0x7f2718070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.372+0000 7f271f7fe700 1 -- 172.21.15.70:0/1822118062 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f272408f0b0 con 0x7f2730101840 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.372+0000 7f272e59c700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f271806dd60 0x7f2718070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.372+0000 7f272e59c700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f271806dd60 0x7f2718070220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2720009a10 tx=0x7f2720009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.376+0000 7f271f7fe700 1 -- 172.21.15.70:0/1822118062 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f27240585e0 con 0x7f2730101840 2024-04-02T04:55:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:13 smithi179 ceph-mon[84288]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:13.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.617+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f273004f2e0 con 0x7f2730101840 2024-04-02T04:55:13.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.617+0000 7f271f7fe700 1 -- 172.21.15.70:0/1822118062 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f272405bca0 con 0x7f2730101840 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:55:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:55:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:55:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:55:13.625 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:55:13.625 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:55:13.625 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:55:13.625 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:55:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f271806dd60 msgr2=0x7f2718070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f271806dd60 0x7f2718070220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2720009a10 tx=0x7f2720009190 comp rx=0 tx=0).stop 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 msgr2=0x7f27300735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 0x7f27300735d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2724005f50 tx=0x7f272400bd70 comp rx=0 tx=0).stop 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 shutdown_connections 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f271806dd60 0x7f2718070220 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2730100e90 0x7f2730073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 --2- 172.21.15.70:0/1822118062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2730101840 0x7f27300735d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 >> 172.21.15.70:0/1822118062 conn(0x7f27300fc740 msgr2=0x7f27300fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 shutdown_connections 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.619+0000 7f2734c67700 1 -- 172.21.15.70:0/1822118062 wait complete. 2024-04-02T04:55:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:55:13.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.724+0000 7f5a06535700 1 -- 172.21.15.70:0/640064561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 msgr2=0x7f5a000689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.724+0000 7f5a06535700 1 --2- 172.21.15.70:0/640064561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 0x7f5a000689e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009910 tx=0x7f59e8009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:13.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.724+0000 7f5a06535700 1 -- 172.21.15.70:0/640064561 shutdown_connections 2024-04-02T04:55:13.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.724+0000 7f5a06535700 1 --2- 172.21.15.70:0/640064561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 0x7f5a000689e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.724+0000 7f5a06535700 1 --2- 172.21.15.70:0/640064561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a00106cd0 0x7f5a001070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.724+0000 7f5a06535700 1 -- 172.21.15.70:0/640064561 >> 172.21.15.70:0/640064561 conn(0x7f5a00075670 msgr2=0x7f5a00075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.724+0000 7f5a06535700 1 -- 172.21.15.70:0/640064561 shutdown_connections 2024-04-02T04:55:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.724+0000 7f5a06535700 1 -- 172.21.15.70:0/640064561 wait complete. 2024-04-02T04:55:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f5a06535700 1 Processor -- start 2024-04-02T04:55:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f5a06535700 1 -- start start 2024-04-02T04:55:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f5a06535700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 0x7f5a0019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f5a06535700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a00106cd0 0x7f5a0019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f5a06535700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0019d200 con 0x7f5a00068580 2024-04-02T04:55:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f5a06535700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a001966b0 con 0x7f5a00106cd0 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f59fffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 0x7f5a0019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f59fffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 0x7f5a0019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54996/0 (socket says 172.21.15.70:54996) 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f59fffff700 1 -- 172.21.15.70:0/181295249 learned_addr learned my addr 172.21.15.70:0/181295249 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f59ff7fe700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a00106cd0 0x7f5a0019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.725+0000 7f59fffff700 1 -- 172.21.15.70:0/181295249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a00106cd0 msgr2=0x7f5a0019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.726+0000 7f59fffff700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a00106cd0 0x7f5a0019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.726+0000 7f59fffff700 1 -- 172.21.15.70:0/181295249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e80095f0 con 0x7f5a00068580 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.726+0000 7f59fffff700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 0x7f5a0019c630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f59f000d900 tx=0x7f59f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.726+0000 7f59fd7fa700 1 -- 172.21.15.70:0/181295249 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59f00041f0 con 0x7f5a00068580 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.726+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a00196990 con 0x7f5a00068580 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.726+0000 7f59fd7fa700 1 -- 172.21.15.70:0/181295249 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59f0004cf0 con 0x7f5a00068580 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.726+0000 7f59fd7fa700 1 -- 172.21.15.70:0/181295249 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59f000b750 con 0x7f5a00068580 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.726+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a00196ee0 con 0x7f5a00068580 2024-04-02T04:55:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.727+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a001091d0 con 0x7f5a00068580 2024-04-02T04:55:13.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.727+0000 7f59fd7fa700 1 -- 172.21.15.70:0/181295249 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f59f0004350 con 0x7f5a00068580 2024-04-02T04:55:13.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.728+0000 7f59fd7fa700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f59ec06dd60 0x7f59ec070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:13.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.728+0000 7f59ff7fe700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f59ec06dd60 0x7f59ec070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:13.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.728+0000 7f59fd7fa700 1 -- 172.21.15.70:0/181295249 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f59f008da90 con 0x7f5a00068580 2024-04-02T04:55:13.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.728+0000 7f59ff7fe700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f59ec06dd60 0x7f59ec070220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59e8005b40 tx=0x7f59e801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:13.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.732+0000 7f59fd7fa700 1 -- 172.21.15.70:0/181295249 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f59f0092070 con 0x7f5a00068580 2024-04-02T04:55:13.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.948+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a00197c60 con 0x7f59ec06dd60 2024-04-02T04:55:13.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.950+0000 7f59fd7fa700 1 -- 172.21.15.70:0/181295249 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5a00197c60 con 0x7f59ec06dd60 2024-04-02T04:55:13.954 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "9/23 daemons upgraded", 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:55:13.955 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:55:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f59ec06dd60 msgr2=0x7f59ec070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f59ec06dd60 0x7f59ec070220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59e8005b40 tx=0x7f59e801a040 comp rx=0 tx=0).stop 2024-04-02T04:55:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 msgr2=0x7f5a0019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 0x7f5a0019c630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f59f000d900 tx=0x7f59f000dcc0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 shutdown_connections 2024-04-02T04:55:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f59ec06dd60 0x7f59ec070220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a00068580 0x7f5a0019c630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 --2- 172.21.15.70:0/181295249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a00106cd0 0x7f5a0019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 >> 172.21.15.70:0/181295249 conn(0x7f5a00075670 msgr2=0x7f5a000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 shutdown_connections 2024-04-02T04:55:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:13.953+0000 7f5a06535700 1 -- 172.21.15.70:0/181295249 wait complete. 2024-04-02T04:55:14.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.056+0000 7f5f30539700 1 -- 172.21.15.70:0/88741040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 msgr2=0x7f5f28100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:14.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.056+0000 7f5f30539700 1 --2- 172.21.15.70:0/88741040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 0x7f5f28100180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009910 tx=0x7f5f18009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:14.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.056+0000 7f5f30539700 1 -- 172.21.15.70:0/88741040 shutdown_connections 2024-04-02T04:55:14.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.056+0000 7f5f30539700 1 --2- 172.21.15.70:0/88741040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f28100750 0x7f5f281047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.056+0000 7f5f30539700 1 --2- 172.21.15.70:0/88741040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 0x7f5f28100180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.056+0000 7f5f30539700 1 -- 172.21.15.70:0/88741040 >> 172.21.15.70:0/88741040 conn(0x7f5f280fb650 msgr2=0x7f5f280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.056+0000 7f5f30539700 1 -- 172.21.15.70:0/88741040 shutdown_connections 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.056+0000 7f5f30539700 1 -- 172.21.15.70:0/88741040 wait complete. 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f30539700 1 Processor -- start 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f30539700 1 -- start start 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f30539700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 0x7f5f2819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f30539700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f28100750 0x7f5f2819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f30539700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2819c120 con 0x7f5f280ffda0 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f30539700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281955d0 con 0x7f5f28100750 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f2e2d5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 0x7f5f2819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f2e2d5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 0x7f5f2819b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55014/0 (socket says 172.21.15.70:55014) 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f2e2d5700 1 -- 172.21.15.70:0/2787977397 learned_addr learned my addr 172.21.15.70:0/2787977397 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f2dad4700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f28100750 0x7f5f2819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f2e2d5700 1 -- 172.21.15.70:0/2787977397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f28100750 msgr2=0x7f5f2819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f2e2d5700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f28100750 0x7f5f2819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.057+0000 7f5f2e2d5700 1 -- 172.21.15.70:0/2787977397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180095f0 con 0x7f5f280ffda0 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.058+0000 7f5f2e2d5700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 0x7f5f2819b550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f18006010 tx=0x7f5f1800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.058+0000 7f5f1f7fe700 1 -- 172.21.15.70:0/2787977397 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f1801d070 con 0x7f5f280ffda0 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.058+0000 7f5f1f7fe700 1 -- 172.21.15.70:0/2787977397 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f18005dc0 con 0x7f5f280ffda0 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.058+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f28195850 con 0x7f5f280ffda0 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.058+0000 7f5f1f7fe700 1 -- 172.21.15.70:0/2787977397 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f18017400 con 0x7f5f280ffda0 2024-04-02T04:55:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.058+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f28195d40 con 0x7f5f280ffda0 2024-04-02T04:55:14.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.059+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2804f2e0 con 0x7f5f280ffda0 2024-04-02T04:55:14.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.059+0000 7f5f1f7fe700 1 -- 172.21.15.70:0/2787977397 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5f18003a60 con 0x7f5f280ffda0 2024-04-02T04:55:14.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.060+0000 7f5f1f7fe700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f1406dd60 0x7f5f14070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:14.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.060+0000 7f5f1f7fe700 1 -- 172.21.15.70:0/2787977397 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5f1808efe0 con 0x7f5f280ffda0 2024-04-02T04:55:14.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.060+0000 7f5f2dad4700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f1406dd60 0x7f5f14070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:14.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.060+0000 7f5f2dad4700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f1406dd60 0x7f5f14070220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f28104a20 tx=0x7f5f2400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:14.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.064+0000 7f5f1f7fe700 1 -- 172.21.15.70:0/2787977397 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f1805c040 con 0x7f5f280ffda0 2024-04-02T04:55:14.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:14 smithi070 ceph-mon[113635]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:14.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:14 smithi070 ceph-mon[113635]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:14.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:14 smithi070 ceph-mon[113635]: pgmap v121: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:14.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:14 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2698881485' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:14.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:14 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1822118062' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:55:14.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.353+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f28195f60 con 0x7f5f280ffda0 2024-04-02T04:55:14.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.353+0000 7f5f1f7fe700 1 -- 172.21.15.70:0/2787977397 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5f1805bbd0 con 0x7f5f280ffda0 2024-04-02T04:55:14.358 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:55:14.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f1406dd60 msgr2=0x7f5f14070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:14.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f1406dd60 0x7f5f14070220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f28104a20 tx=0x7f5f2400b410 comp rx=0 tx=0).stop 2024-04-02T04:55:14.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 msgr2=0x7f5f2819b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:14.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 0x7f5f2819b550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f18006010 tx=0x7f5f1800bd70 comp rx=0 tx=0).stop 2024-04-02T04:55:14.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 shutdown_connections 2024-04-02T04:55:14.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f1406dd60 0x7f5f14070220 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:14.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f280ffda0 0x7f5f2819b550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:14.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 --2- 172.21.15.70:0/2787977397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f28100750 0x7f5f2819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:14.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 >> 172.21.15.70:0/2787977397 conn(0x7f5f280fb650 msgr2=0x7f5f280fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:14.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 shutdown_connections 2024-04-02T04:55:14.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:14.355+0000 7f5f30539700 1 -- 172.21.15.70:0/2787977397 wait complete. 2024-04-02T04:55:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:14 smithi179 ceph-mon[84288]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:14 smithi179 ceph-mon[84288]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:14 smithi179 ceph-mon[84288]: pgmap v121: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:14 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2698881485' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:14 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1822118062' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:55:15.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:15 smithi070 ceph-mon[113635]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:15.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:15 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2787977397' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:55:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:15 smithi179 ceph-mon[84288]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:15.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:15 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2787977397' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:55:16.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:16 smithi179 ceph-mon[84288]: pgmap v122: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:16.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:16 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:55:16.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:16 smithi070 ceph-mon[113635]: pgmap v122: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:16 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:55:17.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:17 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:55:17.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:17 smithi179 ceph-mon[84288]: Upgrade: osd.3 is safe to restart 2024-04-02T04:55:17.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:17 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:17.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:17 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:55:17.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:17 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:17.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:17 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:55:17.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:17 smithi070 ceph-mon[113635]: Upgrade: osd.3 is safe to restart 2024-04-02T04:55:17.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:17.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:55:17.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:18.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:18 smithi070 ceph-mon[113635]: Upgrade: Updating osd.3 2024-04-02T04:55:18.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:18 smithi070 ceph-mon[113635]: Deploying daemon osd.3 on smithi179 2024-04-02T04:55:18.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:18 smithi070 ceph-mon[113635]: pgmap v123: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:18.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:18 smithi179 ceph-mon[84288]: Upgrade: Updating osd.3 2024-04-02T04:55:18.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:18 smithi179 ceph-mon[84288]: Deploying daemon osd.3 on smithi179 2024-04-02T04:55:18.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:18 smithi179 ceph-mon[84288]: pgmap v123: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:20.281 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:20 smithi179 ceph-mon[84288]: pgmap v124: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:20.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:20 smithi070 ceph-mon[113635]: pgmap v124: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:20.542 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:20 smithi179 systemd[1]: Stopping Ceph osd.3 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:55:20.823 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:20 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[42695]: 2024-04-02T04:55:20.670+0000 7f49c85cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:55:20.823 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:20 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[42695]: 2024-04-02T04:55:20.670+0000 7f49c85cd700 -1 osd.3 76 *** Got signal Terminated *** 2024-04-02T04:55:20.823 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:20 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[42695]: 2024-04-02T04:55:20.670+0000 7f49c85cd700 -1 osd.3 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:55:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:21 smithi070 ceph-mon[113635]: osd.3 marked itself down and dead 2024-04-02T04:55:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:21 smithi179 ceph-mon[84288]: osd.3 marked itself down and dead 2024-04-02T04:55:21.902 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:21 smithi179 podman[93625]: 2024-04-02 04:55:21.631624368 +0000 UTC m=+1.077633583 container died 3b51065ff69f21b19253cbcfee97316c2c131af9b25c65ce82654bce08842c74 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=HEAD) 2024-04-02T04:55:22.178 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:21 smithi179 podman[93625]: 2024-04-02 04:55:21.987505947 +0000 UTC m=+1.433515153 container cleanup 3b51065ff69f21b19253cbcfee97316c2c131af9b25c65ce82654bce08842c74 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.build-date=20240222, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, ceph=True, CEPH_POINT_RELEASE=-18.2.1) 2024-04-02T04:55:22.178 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:21 smithi179 bash[93625]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T04:55:22.431 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:22 smithi070 ceph-mon[113635]: pgmap v125: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:22.431 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:22 smithi070 ceph-mon[113635]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:55:22.431 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:22 smithi070 ceph-mon[113635]: osdmap e77: 6 total, 5 up, 6 in 2024-04-02T04:55:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:22 smithi179 ceph-mon[84288]: pgmap v125: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:22 smithi179 ceph-mon[84288]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:55:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:22 smithi179 ceph-mon[84288]: osdmap e77: 6 total, 5 up, 6 in 2024-04-02T04:55:22.573 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:22 smithi179 podman[93644]: 2024-04-02 04:55:22.289360923 +0000 UTC m=+0.653992310 container remove 3b51065ff69f21b19253cbcfee97316c2c131af9b25c65ce82654bce08842c74 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-04-02T04:55:22.937 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:22 smithi179 podman[93737]: 2024-04-02 04:55:22.509345414 +0000 UTC m=+0.018156472 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:55:22.937 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:22 smithi179 podman[93737]: 2024-04-02 04:55:22.606636517 +0000 UTC m=+0.115447576 container create d4a582876aef23b701de241bdabeff96acf305057b1ee6c7ecf95fcc2305dc10 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:55:22.937 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:22 smithi179 podman[93737]: 2024-04-02 04:55:22.850972323 +0000 UTC m=+0.359783398 container init d4a582876aef23b701de241bdabeff96acf305057b1ee6c7ecf95fcc2305dc10 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=) 2024-04-02T04:55:22.937 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:22 smithi179 podman[93737]: 2024-04-02 04:55:22.854098644 +0000 UTC m=+0.362909713 container start d4a582876aef23b701de241bdabeff96acf305057b1ee6c7ecf95fcc2305dc10 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T04:55:22.937 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:22 smithi179 podman[93737]: 2024-04-02 04:55:22.934132434 +0000 UTC m=+0.442943506 container attach d4a582876aef23b701de241bdabeff96acf305057b1ee6c7ecf95fcc2305dc10 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-02T04:55:23.189 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:23 smithi179 podman[93737]: 2024-04-02 04:55:23.143057972 +0000 UTC m=+0.651869021 container died d4a582876aef23b701de241bdabeff96acf305057b1ee6c7ecf95fcc2305dc10 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-02T04:55:23.443 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:23 smithi070 ceph-mon[113635]: osdmap e78: 6 total, 5 up, 6 in 2024-04-02T04:55:23.458 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:23 smithi179 ceph-mon[84288]: osdmap e78: 6 total, 5 up, 6 in 2024-04-02T04:55:24.072 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:23 smithi179 podman[93762]: 2024-04-02 04:55:23.746442613 +0000 UTC m=+0.597903387 container remove d4a582876aef23b701de241bdabeff96acf305057b1ee6c7ecf95fcc2305dc10 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:55:24.072 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:23 smithi179 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.3.service: Deactivated successfully. 2024-04-02T04:55:24.072 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:23 smithi179 systemd[1]: Stopped Ceph osd.3 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:55:24.072 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:23 smithi179 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.3.service: Consumed 1min 20.438s CPU time. 2024-04-02T04:55:24.072 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:23 smithi179 systemd[1]: Starting Ceph osd.3 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:55:24.478 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-mon[84288]: pgmap v128: 65 pgs: 13 peering, 6 stale+active+clean, 46 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:24.478 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-mon[84288]: Health check failed: Reduced data availability: 7 pgs peering (PG_AVAILABILITY) 2024-04-02T04:55:24.478 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 podman[93864]: 2024-04-02 04:55:24.16414131 +0000 UTC m=+0.101626703 container create 46aba6e1e9740d6ca9be41755bea7efbc3bb61a075f242626c6f3566da262f90 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:55:24.478 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 podman[93864]: 2024-04-02 04:55:24.080348535 +0000 UTC m=+0.017833942 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:55:24.478 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 podman[93864]: 2024-04-02 04:55:24.392226815 +0000 UTC m=+0.329712228 container init 46aba6e1e9740d6ca9be41755bea7efbc3bb61a075f242626c6f3566da262f90 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T04:55:24.478 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 podman[93864]: 2024-04-02 04:55:24.395522804 +0000 UTC m=+0.333008209 container start 46aba6e1e9740d6ca9be41755bea7efbc3bb61a075f242626c6f3566da262f90 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326) 2024-04-02T04:55:24.478 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 podman[93864]: 2024-04-02 04:55:24.474920378 +0000 UTC m=+0.412405784 container attach 46aba6e1e9740d6ca9be41755bea7efbc3bb61a075f242626c6f3566da262f90 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:55:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:24 smithi070 ceph-mon[113635]: pgmap v128: 65 pgs: 13 peering, 6 stale+active+clean, 46 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:24 smithi070 ceph-mon[113635]: Health check failed: Reduced data availability: 7 pgs peering (PG_AVAILABILITY) 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate[93879]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate[93879]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 bash[93864]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 bash[93864]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 bash[93864]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 bash[93864]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 bash[93864]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 bash[93864]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T04:55:25.165 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 bash[93864]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-02T04:55:25.166 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate[93879]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-02T04:55:25.166 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate[93879]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-02T04:55:25.166 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate[93879]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-02T04:55:25.166 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate[93879]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T04:55:25.166 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate[93879]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-02T04:55:25.166 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:24 smithi179 podman[93864]: 2024-04-02 04:55:24.866242006 +0000 UTC m=+0.803727416 container died 46aba6e1e9740d6ca9be41755bea7efbc3bb61a075f242626c6f3566da262f90 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:55:25.481 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:25 smithi179 ceph-mon[84288]: Health check failed: Degraded data redundancy: 18/228 objects degraded (7.895%), 7 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:25.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:25 smithi070 ceph-mon[113635]: Health check failed: Degraded data redundancy: 18/228 objects degraded (7.895%), 7 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:25.814 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:25 smithi179 podman[94004]: 2024-04-02 04:55:25.478664404 +0000 UTC m=+0.607285014 container remove 46aba6e1e9740d6ca9be41755bea7efbc3bb61a075f242626c6f3566da262f90 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-02T04:55:25.815 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:25 smithi179 podman[94066]: 2024-04-02 04:55:25.592971038 +0000 UTC m=+0.017122440 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:55:25.815 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:25 smithi179 podman[94066]: 2024-04-02 04:55:25.692400525 +0000 UTC m=+0.116551972 container create f717fd6c83061d4da3cf9e26f8d49bae2ab603cfb1353eb8cfb8a50fc8dc676c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-02T04:55:26.241 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:25 smithi179 podman[94066]: 2024-04-02 04:55:25.915691665 +0000 UTC m=+0.339843069 container init f717fd6c83061d4da3cf9e26f8d49bae2ab603cfb1353eb8cfb8a50fc8dc676c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, GIT_CLEAN=True, ceph=True, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T04:55:26.241 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:25 smithi179 podman[94066]: 2024-04-02 04:55:25.91884377 +0000 UTC m=+0.342995172 container start f717fd6c83061d4da3cf9e26f8d49bae2ab603cfb1353eb8cfb8a50fc8dc676c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:55:26.241 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:25 smithi179 ceph-osd[94085]: -- 172.21.15.179:0/3599842456 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644cf49d680 con 0x5644cf466c00 2024-04-02T04:55:26.241 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:25 smithi179 bash[94066]: f717fd6c83061d4da3cf9e26f8d49bae2ab603cfb1353eb8cfb8a50fc8dc676c 2024-04-02T04:55:26.241 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:25 smithi179 systemd[1]: Started Ceph osd.3 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:55:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:26 smithi070 ceph-mon[113635]: pgmap v129: 65 pgs: 9 active+undersized, 7 active+undersized+degraded, 13 peering, 1 stale+active+clean, 35 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18/228 objects degraded (7.895%) 2024-04-02T04:55:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:26.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:55:26.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:26.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:26.575 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T04:55:26.458+0000 7fdc3e2727c0 -1 Falling back to public interface 2024-04-02T04:55:26.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:26 smithi179 ceph-mon[84288]: pgmap v129: 65 pgs: 9 active+undersized, 7 active+undersized+degraded, 13 peering, 1 stale+active+clean, 35 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18/228 objects degraded (7.895%) 2024-04-02T04:55:26.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:26.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:26.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:55:26.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:26.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:27 smithi179 ceph-mon[84288]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs peering) 2024-04-02T04:55:27.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:27 smithi070 ceph-mon[113635]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs peering) 2024-04-02T04:55:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:28 smithi179 ceph-mon[84288]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/228 objects degraded (19.298%) 2024-04-02T04:55:28.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:28 smithi070 ceph-mon[113635]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/228 objects degraded (19.298%) 2024-04-02T04:55:30.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:29 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:30.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:29 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:30.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:29 smithi070 ceph-mon[113635]: pgmap v131: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/228 objects degraded (19.298%) 2024-04-02T04:55:30.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:29 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:30.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:29 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:30.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:29 smithi179 ceph-mon[84288]: pgmap v131: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/228 objects degraded (19.298%) 2024-04-02T04:55:31.292 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:31 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:31.292 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:31 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:31.489 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:31 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:31.489 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:31 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:31.572 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:31 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T04:55:31.286+0000 7fdc3e2727c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-04-02T04:55:32.029 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:31 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T04:55:31.778+0000 7fdc3e2727c0 -1 osd.3 76 log_to_monitors true 2024-04-02T04:55:32.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:32 smithi179 ceph-mon[84288]: pgmap v132: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/228 objects degraded (19.298%) 2024-04-02T04:55:32.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:32 smithi179 ceph-mon[84288]: from='osd.3 [v2:172.21.15.179:6800/657192449,v1:172.21.15.179:6801/657192449]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:55:32.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:32 smithi179 ceph-mon[84288]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:55:32.322 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 04:55:32 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T04:55:32.161+0000 7fdc2f6ca700 -1 osd.3 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:55:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:32 smithi070 ceph-mon[113635]: pgmap v132: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/228 objects degraded (19.298%) 2024-04-02T04:55:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:32 smithi070 ceph-mon[113635]: from='osd.3 [v2:172.21.15.179:6800/657192449,v1:172.21.15.179:6801/657192449]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:55:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:32 smithi070 ceph-mon[113635]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:55:33.309 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:33 smithi179 ceph-mon[84288]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T04:55:33.310 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:33 smithi179 ceph-mon[84288]: from='osd.3 [v2:172.21.15.179:6800/657192449,v1:172.21.15.179:6801/657192449]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:55:33.310 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:33 smithi179 ceph-mon[84288]: osdmap e79: 6 total, 5 up, 6 in 2024-04-02T04:55:33.310 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:33 smithi179 ceph-mon[84288]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:55:33.310 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:33 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 44/228 objects degraded (19.298%), 15 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:33 smithi070 ceph-mon[113635]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T04:55:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:33 smithi070 ceph-mon[113635]: from='osd.3 [v2:172.21.15.179:6800/657192449,v1:172.21.15.179:6801/657192449]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:55:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:33 smithi070 ceph-mon[113635]: osdmap e79: 6 total, 5 up, 6 in 2024-04-02T04:55:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:33 smithi070 ceph-mon[113635]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:55:33.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:33 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 44/228 objects degraded (19.298%), 15 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: pgmap v134: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 44/228 objects degraded (19.298%) 2024-04-02T04:55:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:55:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: osd.3 [v2:172.21.15.179:6800/657192449,v1:172.21.15.179:6801/657192449] boot 2024-04-02T04:55:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: osdmap e80: 6 total, 6 up, 6 in 2024-04-02T04:55:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:55:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:34.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:55:34.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:34.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:55:34.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:34.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:34.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:34.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: pgmap v134: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 44/228 objects degraded (19.298%) 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: osd.3 [v2:172.21.15.179:6800/657192449,v1:172.21.15.179:6801/657192449] boot 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: osdmap e80: 6 total, 6 up, 6 in 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:34.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:55:34.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:34.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:55:34.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:34.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:34.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:34.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:55:35.420 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:35 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:55:35.420 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:35 smithi179 ceph-mon[84288]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-04-02T04:55:35.420 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:35 smithi179 ceph-mon[84288]: osdmap e81: 6 total, 6 up, 6 in 2024-04-02T04:55:35.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:35 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:55:35.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:35 smithi070 ceph-mon[113635]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-04-02T04:55:35.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:35 smithi070 ceph-mon[113635]: osdmap e81: 6 total, 6 up, 6 in 2024-04-02T04:55:36.432 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:36 smithi179 ceph-mon[84288]: pgmap v137: 65 pgs: 3 peering, 11 active+undersized, 11 active+undersized+degraded, 40 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31/228 objects degraded (13.596%) 2024-04-02T04:55:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:36 smithi070 ceph-mon[113635]: pgmap v137: 65 pgs: 3 peering, 11 active+undersized, 11 active+undersized+degraded, 40 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31/228 objects degraded (13.596%) 2024-04-02T04:55:38.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:38 smithi070 ceph-mon[113635]: pgmap v138: 65 pgs: 3 peering, 5 active+undersized, 8 active+undersized+degraded, 49 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21/228 objects degraded (9.211%) 2024-04-02T04:55:38.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:38 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 21/228 objects degraded (9.211%), 8 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:38 smithi179 ceph-mon[84288]: pgmap v138: 65 pgs: 3 peering, 5 active+undersized, 8 active+undersized+degraded, 49 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21/228 objects degraded (9.211%) 2024-04-02T04:55:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:38 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 21/228 objects degraded (9.211%), 8 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:39 smithi070 ceph-mon[113635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/228 objects degraded (9.211%), 8 pgs degraded) 2024-04-02T04:55:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:39 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:55:39.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:39 smithi179 ceph-mon[84288]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/228 objects degraded (9.211%), 8 pgs degraded) 2024-04-02T04:55:39.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:39 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:55:40.469 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:40 smithi179 ceph-mon[84288]: pgmap v139: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:40.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:40 smithi070 ceph-mon[113635]: pgmap v139: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:42 smithi070 ceph-mon[113635]: pgmap v140: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:42 smithi179 ceph-mon[84288]: pgmap v140: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:44.479 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:44 smithi070 ceph-mon[113635]: pgmap v141: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.474+0000 7f5b6b024700 1 -- 172.21.15.70:0/515292066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 msgr2=0x7f5b6410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.474+0000 7f5b6b024700 1 --2- 172.21.15.70:0/515292066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 0x7f5b6410acd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009940 tx=0x7f5b54009c50 comp rx=0 tx=0).stop 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.474+0000 7f5b6b024700 1 -- 172.21.15.70:0/515292066 shutdown_connections 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.474+0000 7f5b6b024700 1 --2- 172.21.15.70:0/515292066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 0x7f5b6410acd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.474+0000 7f5b6b024700 1 --2- 172.21.15.70:0/515292066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b64101ec0 0x7f5b641022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.474+0000 7f5b6b024700 1 -- 172.21.15.70:0/515292066 >> 172.21.15.70:0/515292066 conn(0x7f5b640fb650 msgr2=0x7f5b640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.474+0000 7f5b6b024700 1 -- 172.21.15.70:0/515292066 shutdown_connections 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.474+0000 7f5b6b024700 1 -- 172.21.15.70:0/515292066 wait complete. 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b6b024700 1 Processor -- start 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b6b024700 1 -- start start 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b6b024700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b64101ec0 0x7f5b6419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:44.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b6b024700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 0x7f5b6419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b6b024700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6419d230 con 0x7f5b641027e0 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b6b024700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b641966e0 con 0x7f5b64101ec0 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b5bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 0x7f5b6419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b5bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 0x7f5b6419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56928/0 (socket says 172.21.15.70:56928) 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.475+0000 7f5b5bfff700 1 -- 172.21.15.70:0/92908059 learned_addr learned my addr 172.21.15.70:0/92908059 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b68dc0700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b64101ec0 0x7f5b6419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b5bfff700 1 -- 172.21.15.70:0/92908059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b64101ec0 msgr2=0x7f5b6419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b5bfff700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b64101ec0 0x7f5b6419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b5bfff700 1 -- 172.21.15.70:0/92908059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b540095f0 con 0x7f5b641027e0 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b5bfff700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 0x7f5b6419cba0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400b580 tx=0x7f5b540056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b59ffb700 1 -- 172.21.15.70:0/92908059 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b5401d070 con 0x7f5b641027e0 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b64196960 con 0x7f5b641027e0 2024-04-02T04:55:44.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b59ffb700 1 -- 172.21.15.70:0/92908059 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b54005dc0 con 0x7f5b641027e0 2024-04-02T04:55:44.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b59ffb700 1 -- 172.21.15.70:0/92908059 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b54021410 con 0x7f5b641027e0 2024-04-02T04:55:44.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.476+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b64196e50 con 0x7f5b641027e0 2024-04-02T04:55:44.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.477+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b64108360 con 0x7f5b641027e0 2024-04-02T04:55:44.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.478+0000 7f5b59ffb700 1 -- 172.21.15.70:0/92908059 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5b5400f7f0 con 0x7f5b641027e0 2024-04-02T04:55:44.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.478+0000 7f5b59ffb700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5b4c06dd60 0x7f5b4c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:44.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.478+0000 7f5b59ffb700 1 -- 172.21.15.70:0/92908059 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f5b5408edd0 con 0x7f5b641027e0 2024-04-02T04:55:44.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.478+0000 7f5b68dc0700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5b4c06dd60 0x7f5b4c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:44.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.479+0000 7f5b68dc0700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5b4c06dd60 0x7f5b4c070220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005950 tx=0x7f5b600058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:44.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.483+0000 7f5b59ffb700 1 -- 172.21.15.70:0/92908059 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5b5405be30 con 0x7f5b641027e0 2024-04-02T04:55:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:44 smithi179 ceph-mon[84288]: pgmap v141: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:44.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.693+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b64197070 con 0x7f5b4c06dd60 2024-04-02T04:55:44.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.694+0000 7f5b59ffb700 1 -- 172.21.15.70:0/92908059 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5b64197070 con 0x7f5b4c06dd60 2024-04-02T04:55:44.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5b4c06dd60 msgr2=0x7f5b4c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:44.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5b4c06dd60 0x7f5b4c070220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005950 tx=0x7f5b600058e0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 msgr2=0x7f5b6419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 0x7f5b6419cba0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400b580 tx=0x7f5b540056a0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 shutdown_connections 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5b4c06dd60 0x7f5b4c070220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b64101ec0 0x7f5b6419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 --2- 172.21.15.70:0/92908059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b641027e0 0x7f5b6419cba0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 >> 172.21.15.70:0/92908059 conn(0x7f5b640fb650 msgr2=0x7f5b641054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 shutdown_connections 2024-04-02T04:55:44.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.696+0000 7f5b6b024700 1 -- 172.21.15.70:0/92908059 wait complete. 2024-04-02T04:55:44.711 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:55:44.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.801+0000 7ffa8187c700 1 -- 172.21.15.70:0/352789225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c101850 msgr2=0x7ffa7c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:44.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.801+0000 7ffa8187c700 1 --2- 172.21.15.70:0/352789225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c101850 0x7ffa7c1058a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009910 tx=0x7ffa70009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:44.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.801+0000 7ffa8187c700 1 -- 172.21.15.70:0/352789225 shutdown_connections 2024-04-02T04:55:44.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.801+0000 7ffa8187c700 1 --2- 172.21.15.70:0/352789225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c101850 0x7ffa7c1058a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.801+0000 7ffa8187c700 1 --2- 172.21.15.70:0/352789225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c100ea0 0x7ffa7c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.801+0000 7ffa8187c700 1 -- 172.21.15.70:0/352789225 >> 172.21.15.70:0/352789225 conn(0x7ffa7c0fc750 msgr2=0x7ffa7c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.801+0000 7ffa8187c700 1 -- 172.21.15.70:0/352789225 shutdown_connections 2024-04-02T04:55:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.801+0000 7ffa8187c700 1 -- 172.21.15.70:0/352789225 wait complete. 2024-04-02T04:55:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa8187c700 1 Processor -- start 2024-04-02T04:55:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa8187c700 1 -- start start 2024-04-02T04:55:44.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa8187c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c100ea0 0x7ffa7c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:44.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa8187c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101850 0x7ffa7c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:44.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa8187c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c19d2d0 con 0x7ffa7c100ea0 2024-04-02T04:55:44.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa8187c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c196780 con 0x7ffa7c101850 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa7affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c100ea0 0x7ffa7c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa7affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c100ea0 0x7ffa7c19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56950/0 (socket says 172.21.15.70:56950) 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.802+0000 7ffa7affd700 1 -- 172.21.15.70:0/1002891728 learned_addr learned my addr 172.21.15.70:0/1002891728 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa7affd700 1 -- 172.21.15.70:0/1002891728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101850 msgr2=0x7ffa7c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa7affd700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101850 0x7ffa7c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa7affd700 1 -- 172.21.15.70:0/1002891728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700095f0 con 0x7ffa7c100ea0 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa7affd700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c100ea0 0x7ffa7c19c700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00c8a0 tx=0x7ffa6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa8087a700 1 -- 172.21.15.70:0/1002891728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa6c00cea0 con 0x7ffa7c100ea0 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa8087a700 1 -- 172.21.15.70:0/1002891728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa6c004830 con 0x7ffa7c100ea0 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7c196a60 con 0x7ffa7c100ea0 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa8087a700 1 -- 172.21.15.70:0/1002891728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa6c0055b0 con 0x7ffa7c100ea0 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.803+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c196fb0 con 0x7ffa7c100ea0 2024-04-02T04:55:44.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.804+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c1092a0 con 0x7ffa7c100ea0 2024-04-02T04:55:44.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.805+0000 7ffa8087a700 1 -- 172.21.15.70:0/1002891728 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ffa6c007ad0 con 0x7ffa7c100ea0 2024-04-02T04:55:44.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.805+0000 7ffa8087a700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ffa6406dd60 0x7ffa64070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:44.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.805+0000 7ffa8087a700 1 -- 172.21.15.70:0/1002891728 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ffa6c08d8d0 con 0x7ffa7c100ea0 2024-04-02T04:55:44.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.805+0000 7ffa7a7fc700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ffa6406dd60 0x7ffa64070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:44.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.805+0000 7ffa7a7fc700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ffa6406dd60 0x7ffa64070220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa70006010 tx=0x7ffa7000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:44.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:44.810+0000 7ffa8087a700 1 -- 172.21.15.70:0/1002891728 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffa6c05a930 con 0x7ffa7c100ea0 2024-04-02T04:55:45.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.023+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa7c197290 con 0x7ffa6406dd60 2024-04-02T04:55:45.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.024+0000 7ffa8087a700 1 -- 172.21.15.70:0/1002891728 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ffa7c197290 con 0x7ffa6406dd60 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ffa6406dd60 msgr2=0x7ffa64070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ffa6406dd60 0x7ffa64070220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa70006010 tx=0x7ffa7000b540 comp rx=0 tx=0).stop 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c100ea0 msgr2=0x7ffa7c19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c100ea0 0x7ffa7c19c700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00c8a0 tx=0x7ffa6c00cc60 comp rx=0 tx=0).stop 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 shutdown_connections 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ffa6406dd60 0x7ffa64070220 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa7c100ea0 0x7ffa7c19c700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 --2- 172.21.15.70:0/1002891728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7c101850 0x7ffa7c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 >> 172.21.15.70:0/1002891728 conn(0x7ffa7c0fc750 msgr2=0x7ffa7c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 shutdown_connections 2024-04-02T04:55:45.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.026+0000 7ffa8187c700 1 -- 172.21.15.70:0/1002891728 wait complete. 2024-04-02T04:55:45.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.130+0000 7fba77c51700 1 -- 172.21.15.70:0/2933836922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70106cd0 msgr2=0x7fba701070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.130+0000 7fba77c51700 1 --2- 172.21.15.70:0/2933836922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70106cd0 0x7fba701070b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b0a0 tx=0x7fba6c00b3b0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.130+0000 7fba77c51700 1 -- 172.21.15.70:0/2933836922 shutdown_connections 2024-04-02T04:55:45.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.130+0000 7fba77c51700 1 --2- 172.21.15.70:0/2933836922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70068580 0x7fba700689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.130+0000 7fba77c51700 1 --2- 172.21.15.70:0/2933836922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70106cd0 0x7fba701070b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.130+0000 7fba77c51700 1 -- 172.21.15.70:0/2933836922 >> 172.21.15.70:0/2933836922 conn(0x7fba70075670 msgr2=0x7fba70075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:45.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.130+0000 7fba77c51700 1 -- 172.21.15.70:0/2933836922 shutdown_connections 2024-04-02T04:55:45.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.130+0000 7fba77c51700 1 -- 172.21.15.70:0/2933836922 wait complete. 2024-04-02T04:55:45.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba77c51700 1 Processor -- start 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba77c51700 1 -- start start 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba77c51700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70068580 0x7fba70196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba77c51700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70106cd0 0x7fba70196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba77c51700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba70197550 con 0x7fba70068580 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba77c51700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7019b360 con 0x7fba70106cd0 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba759ed700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70068580 0x7fba70196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba759ed700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70068580 0x7fba70196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56976/0 (socket says 172.21.15.70:56976) 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba759ed700 1 -- 172.21.15.70:0/2343596550 learned_addr learned my addr 172.21.15.70:0/2343596550 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba751ec700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70106cd0 0x7fba70196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba759ed700 1 -- 172.21.15.70:0/2343596550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70106cd0 msgr2=0x7fba70196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.131+0000 7fba759ed700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70106cd0 0x7fba70196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.132+0000 7fba759ed700 1 -- 172.21.15.70:0/2343596550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c009d00 con 0x7fba70068580 2024-04-02T04:55:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.132+0000 7fba759ed700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70068580 0x7fba70196980 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000f80 tx=0x7fba6c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.132+0000 7fba66ffd700 1 -- 172.21.15.70:0/2343596550 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba6c00e040 con 0x7fba70068580 2024-04-02T04:55:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.132+0000 7fba66ffd700 1 -- 172.21.15.70:0/2343596550 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba6c003de0 con 0x7fba70068580 2024-04-02T04:55:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.132+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7019b5e0 con 0x7fba70068580 2024-04-02T04:55:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.132+0000 7fba66ffd700 1 -- 172.21.15.70:0/2343596550 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba6c01d7d0 con 0x7fba70068580 2024-04-02T04:55:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.132+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7019bad0 con 0x7fba70068580 2024-04-02T04:55:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.133+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba701091f0 con 0x7fba70068580 2024-04-02T04:55:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.133+0000 7fba66ffd700 1 -- 172.21.15.70:0/2343596550 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fba6c004040 con 0x7fba70068580 2024-04-02T04:55:45.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.134+0000 7fba66ffd700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fba5c06dd60 0x7fba5c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.134+0000 7fba66ffd700 1 -- 172.21.15.70:0/2343596550 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fba6c08f4c0 con 0x7fba70068580 2024-04-02T04:55:45.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.134+0000 7fba751ec700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fba5c06dd60 0x7fba5c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.134+0000 7fba751ec700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fba5c06dd60 0x7fba5c070220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fba70197fa0 tx=0x7fba60009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:45.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.139+0000 7fba66ffd700 1 -- 172.21.15.70:0/2343596550 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fba6c05c520 con 0x7fba70068580 2024-04-02T04:55:45.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.343+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fba70197df0 con 0x7fba5c06dd60 2024-04-02T04:55:45.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.354+0000 7fba66ffd700 1 -- 172.21.15.70:0/2343596550 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fba70197df0 con 0x7fba5c06dd60 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (5m) 49s ago 16m 26.2M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (16m) 49s ago 16m 8678k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (15m) 16s ago 15m 7998k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (2m) 49s ago 16m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (2m) 16s ago 15m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (4m) 49s ago 16m 76.8M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (10m) 49s ago 10m 151M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (10m) 49s ago 10m 20.2M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (10m) 16s ago 10m 18.7M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (10m) 16s ago 10m 19.9M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (7m) 49s ago 17m 504M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:55:45.359 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (6m) 16s ago 15m 450M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (3m) 49s ago 17m 49.2M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (3m) 16s ago 15m 42.6M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (6m) 49s ago 16m 15.5M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (5m) 16s ago 15m 16.9M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (2m) 49s ago 14m 141M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (92s) 49s ago 14m 157M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (55s) 49s ago 13m 11.8M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (19s) 16s ago 13m 12.2M 2240M 19.0.0-1979-g6cec793d 78f830190839 f717fd6c8306 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (12m) 16s ago 12m 309M 2240M 18.2.1 5be31c24972a 61db8cd8a24f 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (12m) 16s ago 12m 255M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:55:45.360 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (5m) 49s ago 15m 62.9M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fba5c06dd60 msgr2=0x7fba5c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fba5c06dd60 0x7fba5c070220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fba70197fa0 tx=0x7fba60009190 comp rx=0 tx=0).stop 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70068580 msgr2=0x7fba70196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70068580 0x7fba70196980 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000f80 tx=0x7fba6c00cd20 comp rx=0 tx=0).stop 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 shutdown_connections 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fba5c06dd60 0x7fba5c070220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba70068580 0x7fba70196980 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 --2- 172.21.15.70:0/2343596550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70106cd0 0x7fba70196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 >> 172.21.15.70:0/2343596550 conn(0x7fba70075670 msgr2=0x7fba700fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:45.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 shutdown_connections 2024-04-02T04:55:45.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.356+0000 7fba77c51700 1 -- 172.21.15.70:0/2343596550 wait complete. 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.459+0000 7f4053b7d700 1 -- 172.21.15.70:0/488596018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 msgr2=0x7f404c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.459+0000 7f4053b7d700 1 --2- 172.21.15.70:0/488596018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 0x7f404c101260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4048009910 tx=0x7f4048009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.459+0000 7f4053b7d700 1 -- 172.21.15.70:0/488596018 shutdown_connections 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.459+0000 7f4053b7d700 1 --2- 172.21.15.70:0/488596018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404c101830 0x7f404c105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.459+0000 7f4053b7d700 1 --2- 172.21.15.70:0/488596018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 0x7f404c101260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.459+0000 7f4053b7d700 1 -- 172.21.15.70:0/488596018 >> 172.21.15.70:0/488596018 conn(0x7f404c0fc710 msgr2=0x7f404c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.459+0000 7f4053b7d700 1 -- 172.21.15.70:0/488596018 shutdown_connections 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.459+0000 7f4053b7d700 1 -- 172.21.15.70:0/488596018 wait complete. 2024-04-02T04:55:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4053b7d700 1 Processor -- start 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4053b7d700 1 -- start start 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4053b7d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 0x7f404c196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4053b7d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404c101830 0x7f404c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4053b7d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c197560 con 0x7f404c100e80 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4053b7d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c19b370 con 0x7f404c101830 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4051919700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 0x7f404c196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4051919700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 0x7f404c196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57006/0 (socket says 172.21.15.70:57006) 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.460+0000 7f4051919700 1 -- 172.21.15.70:0/3994334392 learned_addr learned my addr 172.21.15.70:0/3994334392 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4051118700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404c101830 0x7f404c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4051919700 1 -- 172.21.15.70:0/3994334392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404c101830 msgr2=0x7f404c196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4051919700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404c101830 0x7f404c196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4051919700 1 -- 172.21.15.70:0/3994334392 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480095f0 con 0x7f404c100e80 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4051919700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 0x7f404c196990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4048006010 tx=0x7f404800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4042ffd700 1 -- 172.21.15.70:0/3994334392 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f404801d070 con 0x7f404c100e80 2024-04-02T04:55:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c19b5f0 con 0x7f404c100e80 2024-04-02T04:55:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4042ffd700 1 -- 172.21.15.70:0/3994334392 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4048004fb0 con 0x7f404c100e80 2024-04-02T04:55:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4042ffd700 1 -- 172.21.15.70:0/3994334392 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4048022ea0 con 0x7f404c100e80 2024-04-02T04:55:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.461+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c19bae0 con 0x7f404c100e80 2024-04-02T04:55:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.462+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c109220 con 0x7f404c100e80 2024-04-02T04:55:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.463+0000 7f4042ffd700 1 -- 172.21.15.70:0/3994334392 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f4048022470 con 0x7f404c100e80 2024-04-02T04:55:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.463+0000 7f4042ffd700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403806dd60 0x7f4038070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.463+0000 7f4042ffd700 1 -- 172.21.15.70:0/3994334392 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f404808f090 con 0x7f404c100e80 2024-04-02T04:55:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.463+0000 7f4051118700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403806dd60 0x7f4038070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.463+0000 7f4051118700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403806dd60 0x7f4038070220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:45.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.469+0000 7f4042ffd700 1 -- 172.21.15.70:0/3994334392 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4048094060 con 0x7f404c100e80 2024-04-02T04:55:45.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.746+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f404c04f2e0 con 0x7f404c100e80 2024-04-02T04:55:45.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.747+0000 7f4042ffd700 1 -- 172.21.15.70:0/3994334392 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f404805c060 con 0x7f404c100e80 2024-04-02T04:55:45.751 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:55:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:55:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:55:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:55:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:55:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 8 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:55:45.752 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:55:45.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403806dd60 msgr2=0x7f4038070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403806dd60 0x7f4038070220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c008040 comp rx=0 tx=0).stop 2024-04-02T04:55:45.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 msgr2=0x7f404c196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 0x7f404c196990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4048006010 tx=0x7f404800be20 comp rx=0 tx=0).stop 2024-04-02T04:55:45.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 shutdown_connections 2024-04-02T04:55:45.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403806dd60 0x7f4038070220 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f404c100e80 0x7f404c196990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 --2- 172.21.15.70:0/3994334392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f404c101830 0x7f404c196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 >> 172.21.15.70:0/3994334392 conn(0x7f404c0fc710 msgr2=0x7f404c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:45.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 shutdown_connections 2024-04-02T04:55:45.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.749+0000 7f4053b7d700 1 -- 172.21.15.70:0/3994334392 wait complete. 2024-04-02T04:55:45.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.852+0000 7f723df94700 1 -- 172.21.15.70:0/3689745651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238100e90 msgr2=0x7f7238101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.852+0000 7f723df94700 1 --2- 172.21.15.70:0/3689745651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238100e90 0x7f7238101270 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7228009910 tx=0x7f7228009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:45.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.852+0000 7f723df94700 1 -- 172.21.15.70:0/3689745651 shutdown_connections 2024-04-02T04:55:45.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.852+0000 7f723df94700 1 --2- 172.21.15.70:0/3689745651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7238101840 0x7f7238105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.852+0000 7f723df94700 1 --2- 172.21.15.70:0/3689745651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238100e90 0x7f7238101270 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.852+0000 7f723df94700 1 -- 172.21.15.70:0/3689745651 >> 172.21.15.70:0/3689745651 conn(0x7f72380fc740 msgr2=0x7f72380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:45.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.852+0000 7f723df94700 1 -- 172.21.15.70:0/3689745651 shutdown_connections 2024-04-02T04:55:45.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.852+0000 7f723df94700 1 -- 172.21.15.70:0/3689745651 wait complete. 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f723df94700 1 Processor -- start 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f723df94700 1 -- start start 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f723df94700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7238100e90 0x7f723819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f723df94700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238101840 0x7f723819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f723df94700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723819d360 con 0x7f7238101840 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f723df94700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7238196810 con 0x7f7238100e90 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f722effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238101840 0x7f723819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f722effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238101840 0x7f723819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57026/0 (socket says 172.21.15.70:57026) 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f722effd700 1 -- 172.21.15.70:0/3518725333 learned_addr learned my addr 172.21.15.70:0/3518725333 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f72377fe700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7238100e90 0x7f723819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f722effd700 1 -- 172.21.15.70:0/3518725333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7238100e90 msgr2=0x7f723819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f722effd700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7238100e90 0x7f723819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.853+0000 7f722effd700 1 -- 172.21.15.70:0/3518725333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72280095f0 con 0x7f7238101840 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.854+0000 7f72377fe700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7238100e90 0x7f723819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.854+0000 7f722effd700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238101840 0x7f723819ccd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f722000b480 tx=0x7f722000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.854+0000 7f72357fa700 1 -- 172.21.15.70:0/3518725333 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72200118c0 con 0x7f7238101840 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.854+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7238196af0 con 0x7f7238101840 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.854+0000 7f72357fa700 1 -- 172.21.15.70:0/3518725333 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7220010380 con 0x7f7238101840 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.854+0000 7f72357fa700 1 -- 172.21.15.70:0/3518725333 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f722000f4f0 con 0x7f7238101840 2024-04-02T04:55:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.854+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7238197040 con 0x7f7238101840 2024-04-02T04:55:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.855+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7238109230 con 0x7f7238101840 2024-04-02T04:55:45.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.855+0000 7f72357fa700 1 -- 172.21.15.70:0/3518725333 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f72200104f0 con 0x7f7238101840 2024-04-02T04:55:45.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.856+0000 7f72357fa700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f722406dd60 0x7f7224070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:45.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.856+0000 7f72357fa700 1 -- 172.21.15.70:0/3518725333 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f722008d6f0 con 0x7f7238101840 2024-04-02T04:55:45.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.856+0000 7f72377fe700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f722406dd60 0x7f7224070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:45.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.856+0000 7f72377fe700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f722406dd60 0x7f7224070220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72280059d0 tx=0x7f722801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:45.860+0000 7f72357fa700 1 -- 172.21.15.70:0/3518725333 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f722005a7d0 con 0x7f7238101840 2024-04-02T04:55:46.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.098+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7238197310 con 0x7f7238101840 2024-04-02T04:55:46.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.099+0000 7f72357fa700 1 -- 172.21.15.70:0/3518725333 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f722005a360 con 0x7f7238101840 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:55:46.104 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:55:46.105 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:55:46.106 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:55:46.107 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:55:46.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f722406dd60 msgr2=0x7f7224070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f722406dd60 0x7f7224070220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f72280059d0 tx=0x7f722801a040 comp rx=0 tx=0).stop 2024-04-02T04:55:46.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238101840 msgr2=0x7f723819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238101840 0x7f723819ccd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f722000b480 tx=0x7f722000b840 comp rx=0 tx=0).stop 2024-04-02T04:55:46.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 shutdown_connections 2024-04-02T04:55:46.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f722406dd60 0x7f7224070220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7238100e90 0x7f723819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 --2- 172.21.15.70:0/3518725333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7238101840 0x7f723819ccd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 >> 172.21.15.70:0/3518725333 conn(0x7f72380fc740 msgr2=0x7f72380fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:46.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 shutdown_connections 2024-04-02T04:55:46.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.101+0000 7f723df94700 1 -- 172.21.15.70:0/3518725333 wait complete. 2024-04-02T04:55:46.108 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.208+0000 7f9bfd87d700 1 -- 172.21.15.70:0/3439725888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf80ffda0 msgr2=0x7f9bf8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.208+0000 7f9bfd87d700 1 --2- 172.21.15.70:0/3439725888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf80ffda0 0x7f9bf8100180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009910 tx=0x7f9be8009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.208+0000 7f9bfd87d700 1 -- 172.21.15.70:0/3439725888 shutdown_connections 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.208+0000 7f9bfd87d700 1 --2- 172.21.15.70:0/3439725888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bf8100750 0x7f9bf81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.208+0000 7f9bfd87d700 1 --2- 172.21.15.70:0/3439725888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf80ffda0 0x7f9bf8100180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.208+0000 7f9bfd87d700 1 -- 172.21.15.70:0/3439725888 >> 172.21.15.70:0/3439725888 conn(0x7f9bf80fb650 msgr2=0x7f9bf80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.208+0000 7f9bfd87d700 1 -- 172.21.15.70:0/3439725888 shutdown_connections 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.208+0000 7f9bfd87d700 1 -- 172.21.15.70:0/3439725888 wait complete. 2024-04-02T04:55:46.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bfd87d700 1 Processor -- start 2024-04-02T04:55:46.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bfd87d700 1 -- start start 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bfd87d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bf80ffda0 0x7f9bf819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bfd87d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf8100750 0x7f9bf819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bfd87d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819d230 con 0x7f9bf8100750 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bfd87d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf81966b0 con 0x7f9bf80ffda0 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bf67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf8100750 0x7f9bf819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bf67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf8100750 0x7f9bf819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57044/0 (socket says 172.21.15.70:57044) 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bf67fc700 1 -- 172.21.15.70:0/2591108425 learned_addr learned my addr 172.21.15.70:0/2591108425 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.209+0000 7f9bf6ffd700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bf80ffda0 0x7f9bf819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bf67fc700 1 -- 172.21.15.70:0/2591108425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bf80ffda0 msgr2=0x7f9bf819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bf67fc700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bf80ffda0 0x7f9bf819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bf67fc700 1 -- 172.21.15.70:0/2591108425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80095f0 con 0x7f9bf8100750 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bf6ffd700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bf80ffda0 0x7f9bf819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bf67fc700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf8100750 0x7f9bf819cb50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00d900 tx=0x7f9bec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bfc87b700 1 -- 172.21.15.70:0/2591108425 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bec009900 con 0x7f9bf8100750 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bfc87b700 1 -- 172.21.15.70:0/2591108425 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bec010460 con 0x7f9bf8100750 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf8196990 con 0x7f9bf8100750 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bfc87b700 1 -- 172.21.15.70:0/2591108425 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bec00f5f0 con 0x7f9bf8100750 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.210+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf8196ee0 con 0x7f9bf8100750 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.211+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf804f2e0 con 0x7f9bf8100750 2024-04-02T04:55:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.211+0000 7f9bfc87b700 1 -- 172.21.15.70:0/2591108425 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f9bec0105d0 con 0x7f9bf8100750 2024-04-02T04:55:46.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.212+0000 7f9bfc87b700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9be006dd60 0x7f9be0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:46.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.212+0000 7f9bfc87b700 1 -- 172.21.15.70:0/2591108425 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f9bec08da30 con 0x7f9bf8100750 2024-04-02T04:55:46.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.212+0000 7f9bf6ffd700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9be006dd60 0x7f9be0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:46.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.212+0000 7f9bf6ffd700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9be006dd60 0x7f9be0070220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006010 tx=0x7f9be800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:46.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.216+0000 7f9bfc87b700 1 -- 172.21.15.70:0/2591108425 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9bec05aa90 con 0x7f9bf8100750 2024-04-02T04:55:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.429+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bf8197260 con 0x7f9be006dd60 2024-04-02T04:55:46.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:46 smithi070 ceph-mon[113635]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:46.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:46 smithi070 ceph-mon[113635]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:46.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:46 smithi070 ceph-mon[113635]: pgmap v142: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:46.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:46 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3994334392' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:46.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:46 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3518725333' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:55:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.431+0000 7f9bfc87b700 1 -- 172.21.15.70:0/2591108425 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9bf8197260 con 0x7f9be006dd60 2024-04-02T04:55:46.435 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:55:46.435 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:55:46.435 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "10/23 daemons upgraded", 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:55:46.436 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:55:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9be006dd60 msgr2=0x7f9be0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9be006dd60 0x7f9be0070220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006010 tx=0x7f9be800b540 comp rx=0 tx=0).stop 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf8100750 msgr2=0x7f9bf819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf8100750 0x7f9bf819cb50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00d900 tx=0x7f9bec00dcc0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 shutdown_connections 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9be006dd60 0x7f9be0070220 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bf80ffda0 0x7f9bf819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 --2- 172.21.15.70:0/2591108425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf8100750 0x7f9bf819cb50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 >> 172.21.15.70:0/2591108425 conn(0x7f9bf80fb650 msgr2=0x7f9bf80fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 shutdown_connections 2024-04-02T04:55:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.433+0000 7f9bfd87d700 1 -- 172.21.15.70:0/2591108425 wait complete. 2024-04-02T04:55:46.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 -- 172.21.15.70:0/3072624459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 msgr2=0x7f62e4105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 --2- 172.21.15.70:0/3072624459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 0x7f62e4105730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009910 tx=0x7f62d4009c20 comp rx=0 tx=0).stop 2024-04-02T04:55:46.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 -- 172.21.15.70:0/3072624459 shutdown_connections 2024-04-02T04:55:46.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 --2- 172.21.15.70:0/3072624459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62e40ff340 0x7f62e40ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 --2- 172.21.15.70:0/3072624459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 0x7f62e4105730 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 -- 172.21.15.70:0/3072624459 >> 172.21.15.70:0/3072624459 conn(0x7f62e4074e30 msgr2=0x7f62e4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:46.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 -- 172.21.15.70:0/3072624459 shutdown_connections 2024-04-02T04:55:46.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 -- 172.21.15.70:0/3072624459 wait complete. 2024-04-02T04:55:46.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.537+0000 7f62e8ffa700 1 Processor -- start 2024-04-02T04:55:46.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62e8ffa700 1 -- start start 2024-04-02T04:55:46.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62e8ffa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62e40ff340 0x7f62e41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:46.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62e8ffa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 0x7f62e4197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62e8ffa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e4198350 con 0x7f62e4105350 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62e8ffa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e4198490 con 0x7f62e40ff340 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62d9d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 0x7f62e4197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62d9d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 0x7f62e4197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57060/0 (socket says 172.21.15.70:57060) 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62d9d9b700 1 -- 172.21.15.70:0/2228652962 learned_addr learned my addr 172.21.15.70:0/2228652962 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62e259c700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62e40ff340 0x7f62e41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62d9d9b700 1 -- 172.21.15.70:0/2228652962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62e40ff340 msgr2=0x7f62e41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62d9d9b700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62e40ff340 0x7f62e41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62d9d9b700 1 -- 172.21.15.70:0/2228652962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d40095f0 con 0x7f62e4105350 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.538+0000 7f62d9d9b700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 0x7f62e4197d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00b4b0 tx=0x7f62cc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.539+0000 7f62dbfff700 1 -- 172.21.15.70:0/2228652962 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62cc011800 con 0x7f62e4105350 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.539+0000 7f62dbfff700 1 -- 172.21.15.70:0/2228652962 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62cc011e20 con 0x7f62e4105350 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.539+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e41a0860 con 0x7f62e4105350 2024-04-02T04:55:46.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.539+0000 7f62dbfff700 1 -- 172.21.15.70:0/2228652962 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62cc00f2e0 con 0x7f62e4105350 2024-04-02T04:55:46.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.539+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e41a0d20 con 0x7f62e4105350 2024-04-02T04:55:46.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.540+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e404f2e0 con 0x7f62e4105350 2024-04-02T04:55:46.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.540+0000 7f62dbfff700 1 -- 172.21.15.70:0/2228652962 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f62cc010380 con 0x7f62e4105350 2024-04-02T04:55:46.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.541+0000 7f62dbfff700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f62d006dd60 0x7f62d0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:55:46.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.541+0000 7f62dbfff700 1 -- 172.21.15.70:0/2228652962 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f62cc08d730 con 0x7f62e4105350 2024-04-02T04:55:46.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.541+0000 7f62e259c700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f62d006dd60 0x7f62d0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:55:46.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.541+0000 7f62e259c700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f62d006dd60 0x7f62d0070220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005f50 tx=0x7f62d401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:55:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.545+0000 7f62dbfff700 1 -- 172.21.15.70:0/2228652962 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f62cc056c60 con 0x7f62e4105350 2024-04-02T04:55:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:46 smithi179 ceph-mon[84288]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:46 smithi179 ceph-mon[84288]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:46 smithi179 ceph-mon[84288]: pgmap v142: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:46 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3994334392' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:55:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:46 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3518725333' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:55:46.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.822+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f62e41a0fa0 con 0x7f62e4105350 2024-04-02T04:55:46.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.823+0000 7f62dbfff700 1 -- 172.21.15.70:0/2228652962 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f62cc05a320 con 0x7f62e4105350 2024-04-02T04:55:46.827 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:55:46.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f62d006dd60 msgr2=0x7f62d0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f62d006dd60 0x7f62d0070220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005f50 tx=0x7f62d401a040 comp rx=0 tx=0).stop 2024-04-02T04:55:46.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 msgr2=0x7f62e4197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:55:46.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 0x7f62e4197d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00b4b0 tx=0x7f62cc00b7c0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 shutdown_connections 2024-04-02T04:55:46.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f62d006dd60 0x7f62d0070220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62e40ff340 0x7f62e41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 --2- 172.21.15.70:0/2228652962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62e4105350 0x7f62e4197d30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:55:46.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 >> 172.21.15.70:0/2228652962 conn(0x7f62e4074e30 msgr2=0x7f62e4103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:55:46.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 shutdown_connections 2024-04-02T04:55:46.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:55:46.825+0000 7f62e8ffa700 1 -- 172.21.15.70:0/2228652962 wait complete. 2024-04-02T04:55:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:47 smithi070 ceph-mon[113635]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:47.550 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:47.550 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:47 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2228652962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:55:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:47 smithi179 ceph-mon[84288]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:47 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2228652962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:55:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:48 smithi070 ceph-mon[113635]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:48 smithi070 ceph-mon[113635]: pgmap v143: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:48 smithi179 ceph-mon[84288]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:55:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:48 smithi179 ceph-mon[84288]: pgmap v143: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:49.525 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:49 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:55:49.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:49 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:55:50.283 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:50 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:55:50.283 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:50 smithi179 ceph-mon[84288]: Upgrade: osd.4 is safe to restart 2024-04-02T04:55:50.283 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:50 smithi179 ceph-mon[84288]: pgmap v144: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:50 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:55:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:50 smithi070 ceph-mon[113635]: Upgrade: osd.4 is safe to restart 2024-04-02T04:55:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:50 smithi070 ceph-mon[113635]: pgmap v144: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:51.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:51 smithi179 ceph-mon[84288]: Upgrade: Updating osd.4 2024-04-02T04:55:51.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:51.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:55:51.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:51.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:51 smithi179 ceph-mon[84288]: Deploying daemon osd.4 on smithi179 2024-04-02T04:55:51.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:51 smithi179 ceph-mon[84288]: pgmap v145: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:51.713 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:51 smithi070 ceph-mon[113635]: Upgrade: Updating osd.4 2024-04-02T04:55:51.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:51.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:55:51.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:51.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:51 smithi070 ceph-mon[113635]: Deploying daemon osd.4 on smithi179 2024-04-02T04:55:51.714 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:51 smithi070 ceph-mon[113635]: pgmap v145: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:53.572 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:53 smithi179 systemd[1]: Stopping Ceph osd.4 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:55:54.072 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:53 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[49174]: 2024-04-02T04:55:53.731+0000 7f0a785e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:55:54.072 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:53 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[49174]: 2024-04-02T04:55:53.731+0000 7f0a785e7700 -1 osd.4 81 *** Got signal Terminated *** 2024-04-02T04:55:54.072 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:53 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[49174]: 2024-04-02T04:55:53.731+0000 7f0a785e7700 -1 osd.4 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:55:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:54 smithi070 ceph-mon[113635]: pgmap v146: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:54 smithi070 ceph-mon[113635]: osd.4 marked itself down and dead 2024-04-02T04:55:54.569 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:54 smithi179 ceph-mon[84288]: pgmap v146: 65 pgs: 65 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:54.569 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:54 smithi179 ceph-mon[84288]: osd.4 marked itself down and dead 2024-04-02T04:55:54.822 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:54 smithi179 podman[98987]: 2024-04-02 04:55:54.565766251 +0000 UTC m=+0.940572757 container died 61db8cd8a24f5db42eeeccc723ded1f3762017ff15fb68be02c8c025e2ea1a01 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, ceph=True) 2024-04-02T04:55:55.194 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:54 smithi179 podman[98987]: 2024-04-02 04:55:54.931484489 +0000 UTC m=+1.306290995 container cleanup 61db8cd8a24f5db42eeeccc723ded1f3762017ff15fb68be02c8c025e2ea1a01 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.license=GPLv2, ceph=True, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0) 2024-04-02T04:55:55.195 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:54 smithi179 bash[98987]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T04:55:55.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:55 smithi070 ceph-mon[113635]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:55:55.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:55 smithi070 ceph-mon[113635]: osdmap e82: 6 total, 5 up, 6 in 2024-04-02T04:55:55.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:55 smithi070 ceph-mon[113635]: osdmap e83: 6 total, 5 up, 6 in 2024-04-02T04:55:55.536 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:55 smithi179 ceph-mon[84288]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:55:55.536 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:55 smithi179 ceph-mon[84288]: osdmap e82: 6 total, 5 up, 6 in 2024-04-02T04:55:55.537 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:55 smithi179 ceph-mon[84288]: osdmap e83: 6 total, 5 up, 6 in 2024-04-02T04:55:55.537 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:55 smithi179 podman[99014]: 2024-04-02 04:55:55.207780408 +0000 UTC m=+0.640363074 container remove 61db8cd8a24f5db42eeeccc723ded1f3762017ff15fb68be02c8c025e2ea1a01 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, RELEASE=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:55:55.822 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:55 smithi179 podman[99107]: 2024-04-02 04:55:55.533791814 +0000 UTC m=+0.112264176 container create 360be5a2c4d4fa7073e0e56275a56b68e5f836c7ed1bfc9c3dc4ff9c38cc5afa (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, RELEASE=squid-6cec793, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-04-02T04:55:55.822 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:55 smithi179 podman[99107]: 2024-04-02 04:55:55.439199976 +0000 UTC m=+0.017672365 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:55:55.823 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:55 smithi179 podman[99107]: 2024-04-02 04:55:55.778498892 +0000 UTC m=+0.356971265 container init 360be5a2c4d4fa7073e0e56275a56b68e5f836c7ed1bfc9c3dc4ff9c38cc5afa (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-02T04:55:55.823 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:55 smithi179 podman[99107]: 2024-04-02 04:55:55.782259364 +0000 UTC m=+0.360731729 container start 360be5a2c4d4fa7073e0e56275a56b68e5f836c7ed1bfc9c3dc4ff9c38cc5afa (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T04:55:56.195 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:55 smithi179 podman[99107]: 2024-04-02 04:55:55.853274934 +0000 UTC m=+0.431747296 container attach 360be5a2c4d4fa7073e0e56275a56b68e5f836c7ed1bfc9c3dc4ff9c38cc5afa (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793) 2024-04-02T04:55:56.196 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:56 smithi179 podman[99107]: 2024-04-02 04:55:56.073613219 +0000 UTC m=+0.652085589 container died 360be5a2c4d4fa7073e0e56275a56b68e5f836c7ed1bfc9c3dc4ff9c38cc5afa (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:55:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:56 smithi070 ceph-mon[113635]: pgmap v148: 65 pgs: 10 peering, 4 stale+active+clean, 51 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:56 smithi179 ceph-mon[84288]: pgmap v148: 65 pgs: 10 peering, 4 stale+active+clean, 51 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:55:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:57.036 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:56 smithi179 podman[99133]: 2024-04-02 04:55:56.598359522 +0000 UTC m=+0.514623885 container remove 360be5a2c4d4fa7073e0e56275a56b68e5f836c7ed1bfc9c3dc4ff9c38cc5afa (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-02T04:55:57.036 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:56 smithi179 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.4.service: Deactivated successfully. 2024-04-02T04:55:57.037 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:56 smithi179 systemd[1]: Stopped Ceph osd.4 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:55:57.037 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:56 smithi179 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.4.service: Consumed 1min 9.588s CPU time. 2024-04-02T04:55:57.037 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:56 smithi179 systemd[1]: Starting Ceph osd.4 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:55:57.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:57 smithi179 ceph-mon[84288]: Health check failed: Degraded data redundancy: 8/228 objects degraded (3.509%), 2 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:57.322 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 podman[99235]: 2024-04-02 04:55:57.032939325 +0000 UTC m=+0.108584660 container create c4876a50f4de6032dd42831e1b561195cf25754f8537bf91c897f236a090f0f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-04-02T04:55:57.322 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 podman[99235]: 2024-04-02 04:55:56.942368199 +0000 UTC m=+0.018013513 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:55:57.322 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 podman[99235]: 2024-04-02 04:55:57.244484004 +0000 UTC m=+0.320129314 container init c4876a50f4de6032dd42831e1b561195cf25754f8537bf91c897f236a090f0f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T04:55:57.323 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 podman[99235]: 2024-04-02 04:55:57.248333174 +0000 UTC m=+0.323978467 container start c4876a50f4de6032dd42831e1b561195cf25754f8537bf91c897f236a090f0f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-02T04:55:57.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:57 smithi070 ceph-mon[113635]: Health check failed: Degraded data redundancy: 8/228 objects degraded (3.509%), 2 pgs degraded (PG_DEGRADED) 2024-04-02T04:55:57.694 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 podman[99235]: 2024-04-02 04:55:57.32729735 +0000 UTC m=+0.402942650 container attach c4876a50f4de6032dd42831e1b561195cf25754f8537bf91c897f236a090f0f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-02T04:55:58.039 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate[99250]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T04:55:58.039 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate[99250]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 bash[99235]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 bash[99235]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 bash[99235]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 bash[99235]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 bash[99235]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 bash[99235]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 bash[99235]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate[99250]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate[99250]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate[99250]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate[99250]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate[99250]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-02T04:55:58.040 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:57 smithi179 podman[99235]: 2024-04-02 04:55:57.715997364 +0000 UTC m=+0.791642659 container died c4876a50f4de6032dd42831e1b561195cf25754f8537bf91c897f236a090f0f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:55:58.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:58 smithi179 ceph-mon[84288]: pgmap v150: 65 pgs: 5 active+undersized, 2 active+undersized+degraded, 10 peering, 2 stale+active+clean, 46 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8/228 objects degraded (3.509%) 2024-04-02T04:55:58.322 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:58 smithi179 podman[99385]: 2024-04-02 04:55:58.272844148 +0000 UTC m=+0.551630386 container remove c4876a50f4de6032dd42831e1b561195cf25754f8537bf91c897f236a090f0f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-activate, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-6cec793) 2024-04-02T04:55:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:58 smithi070 ceph-mon[113635]: pgmap v150: 65 pgs: 5 active+undersized, 2 active+undersized+degraded, 10 peering, 2 stale+active+clean, 46 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8/228 objects degraded (3.509%) 2024-04-02T04:55:58.735 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:58 smithi179 podman[99451]: 2024-04-02 04:55:58.384512108 +0000 UTC m=+0.017787741 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:55:58.735 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:58 smithi179 podman[99451]: 2024-04-02 04:55:58.48206151 +0000 UTC m=+0.115337127 container create 2e5bbf3235ff1eae6e2b736bdd3055d96e699eb8b70113f77acf331680b85666 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793) 2024-04-02T04:55:58.735 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:58 smithi179 podman[99451]: 2024-04-02 04:55:58.709246365 +0000 UTC m=+0.342521995 container init 2e5bbf3235ff1eae6e2b736bdd3055d96e699eb8b70113f77acf331680b85666 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:55:58.736 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:58 smithi179 podman[99451]: 2024-04-02 04:55:58.712210665 +0000 UTC m=+0.345486284 container start 2e5bbf3235ff1eae6e2b736bdd3055d96e699eb8b70113f77acf331680b85666 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-02T04:55:59.072 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:58 smithi179 ceph-osd[99469]: -- 172.21.15.179:0/3017154458 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c7c96f680 con 0x555c7c939000 2024-04-02T04:55:59.072 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:58 smithi179 bash[99451]: 2e5bbf3235ff1eae6e2b736bdd3055d96e699eb8b70113f77acf331680b85666 2024-04-02T04:55:59.072 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:58 smithi179 systemd[1]: Started Ceph osd.4 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:55:59.572 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:55:59 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T04:55:59.256+0000 7fd1490677c0 -1 Falling back to public interface 2024-04-02T04:56:00.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:59 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:00.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:59 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:00.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:59 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:00.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:55:59 smithi179 ceph-mon[84288]: pgmap v151: 65 pgs: 13 active+undersized, 9 active+undersized+degraded, 10 peering, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27/228 objects degraded (11.842%) 2024-04-02T04:56:00.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:59 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:00.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:59 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:00.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:59 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:00.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:55:59 smithi070 ceph-mon[113635]: pgmap v151: 65 pgs: 13 active+undersized, 9 active+undersized+degraded, 10 peering, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27/228 objects degraded (11.842%) 2024-04-02T04:56:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:02 smithi070 ceph-mon[113635]: pgmap v152: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:56:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:02 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:02 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:02.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:02 smithi179 ceph-mon[84288]: pgmap v152: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:56:02.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:02 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:02.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:02 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:03.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:03 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 40/228 objects degraded (17.544%), 15 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:03.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:03.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:03.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:03 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 40/228 objects degraded (17.544%), 15 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:03.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:03.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:04.169 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:56:03 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T04:56:03.867+0000 7fd1490677c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-04-02T04:56:04.470 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:04 smithi179 ceph-mon[84288]: pgmap v153: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:56:04.470 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:56:04 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T04:56:04.373+0000 7fd1490677c0 -1 osd.4 81 log_to_monitors true 2024-04-02T04:56:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:04 smithi070 ceph-mon[113635]: pgmap v153: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:56:05.364 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:05 smithi179 ceph-mon[84288]: from='osd.4 [v2:172.21.15.179:6808/229347770,v1:172.21.15.179:6809/229347770]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:56:05.365 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:05 smithi179 ceph-mon[84288]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:56:05.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:05 smithi070 ceph-mon[113635]: from='osd.4 [v2:172.21.15.179:6808/229347770,v1:172.21.15.179:6809/229347770]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:56:05.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:05 smithi070 ceph-mon[113635]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:56:06.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:06 smithi179 ceph-mon[84288]: pgmap v154: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:56:06.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:06 smithi179 ceph-mon[84288]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T04:56:06.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:06 smithi179 ceph-mon[84288]: osdmap e84: 6 total, 5 up, 6 in 2024-04-02T04:56:06.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:06 smithi179 ceph-mon[84288]: from='osd.4 [v2:172.21.15.179:6808/229347770,v1:172.21.15.179:6809/229347770]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:56:06.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:06 smithi179 ceph-mon[84288]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:56:06.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:06 smithi179 ceph-mon[84288]: from='osd.4 ' entity='osd.4' 2024-04-02T04:56:06.323 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 04:56:06 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T04:56:06.086+0000 7fd1364b7700 -1 osd.4 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:56:06.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:06 smithi070 ceph-mon[113635]: pgmap v154: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 205 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 40/228 objects degraded (17.544%) 2024-04-02T04:56:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:06 smithi070 ceph-mon[113635]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T04:56:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:06 smithi070 ceph-mon[113635]: osdmap e84: 6 total, 5 up, 6 in 2024-04-02T04:56:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:06 smithi070 ceph-mon[113635]: from='osd.4 [v2:172.21.15.179:6808/229347770,v1:172.21.15.179:6809/229347770]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:56:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:06 smithi070 ceph-mon[113635]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:56:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:06 smithi070 ceph-mon[113635]: from='osd.4 ' entity='osd.4' 2024-04-02T04:56:07.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:07.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:07.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:07.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: osd.4 [v2:172.21.15.179:6808/229347770,v1:172.21.15.179:6809/229347770] boot 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: osdmap e85: 6 total, 6 up, 6 in 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:56:07.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:07 smithi070 ceph-mon[113635]: pgmap v157: 65 pgs: 4 peering, 14 active+undersized, 14 active+undersized+degraded, 33 active+clean; 205 MiB data, 935 MiB used, 536 GiB / 536 GiB avail; 34/228 objects degraded (14.912%) 2024-04-02T04:56:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:07.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: osd.4 [v2:172.21.15.179:6808/229347770,v1:172.21.15.179:6809/229347770] boot 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: osdmap e85: 6 total, 6 up, 6 in 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:56:07.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:07 smithi179 ceph-mon[84288]: pgmap v157: 65 pgs: 4 peering, 14 active+undersized, 14 active+undersized+degraded, 33 active+clean; 205 MiB data, 935 MiB used, 536 GiB / 536 GiB avail; 34/228 objects degraded (14.912%) 2024-04-02T04:56:09.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:08 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 34/228 objects degraded (14.912%), 14 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:09.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:08 smithi070 ceph-mon[113635]: osdmap e86: 6 total, 6 up, 6 in 2024-04-02T04:56:09.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:08 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 34/228 objects degraded (14.912%), 14 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:09.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:08 smithi179 ceph-mon[84288]: osdmap e86: 6 total, 6 up, 6 in 2024-04-02T04:56:10.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:09 smithi070 ceph-mon[113635]: pgmap v159: 65 pgs: 12 peering, 7 active+undersized, 8 active+undersized+degraded, 38 active+clean; 205 MiB data, 935 MiB used, 536 GiB / 536 GiB avail; 17/228 objects degraded (7.456%) 2024-04-02T04:56:10.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:09 smithi179 ceph-mon[84288]: pgmap v159: 65 pgs: 12 peering, 7 active+undersized, 8 active+undersized+degraded, 38 active+clean; 205 MiB data, 935 MiB used, 536 GiB / 536 GiB avail; 17/228 objects degraded (7.456%) 2024-04-02T04:56:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:12 smithi070 ceph-mon[113635]: pgmap v160: 65 pgs: 12 peering, 53 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:12 smithi070 ceph-mon[113635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/228 objects degraded (7.456%), 8 pgs degraded) 2024-04-02T04:56:12.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:12 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:56:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:12 smithi179 ceph-mon[84288]: pgmap v160: 65 pgs: 12 peering, 53 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:12 smithi179 ceph-mon[84288]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/228 objects degraded (7.456%), 8 pgs degraded) 2024-04-02T04:56:12.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:12 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:56:14.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:14 smithi070 ceph-mon[113635]: pgmap v161: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:14 smithi179 ceph-mon[84288]: pgmap v161: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:16 smithi070 ceph-mon[113635]: pgmap v162: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:16.539 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:16 smithi179 ceph-mon[84288]: pgmap v162: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.942+0000 7fe2fb659700 1 -- 172.21.15.70:0/931641993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073130 msgr2=0x7fe2f4073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.942+0000 7fe2fb659700 1 --2- 172.21.15.70:0/931641993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073130 0x7fe2f4073510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009910 tx=0x7fe2f0009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.942+0000 7fe2fb659700 1 -- 172.21.15.70:0/931641993 shutdown_connections 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.942+0000 7fe2fb659700 1 --2- 172.21.15.70:0/931641993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2f4073a50 0x7fe2f4111710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.942+0000 7fe2fb659700 1 --2- 172.21.15.70:0/931641993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073130 0x7fe2f4073510 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.942+0000 7fe2fb659700 1 -- 172.21.15.70:0/931641993 >> 172.21.15.70:0/931641993 conn(0x7fe2f40fc750 msgr2=0x7fe2f40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.942+0000 7fe2fb659700 1 -- 172.21.15.70:0/931641993 shutdown_connections 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.942+0000 7fe2fb659700 1 -- 172.21.15.70:0/931641993 wait complete. 2024-04-02T04:56:16.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2fb659700 1 Processor -- start 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2fb659700 1 -- start start 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2fb659700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2f4073130 0x7fe2f419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2fb659700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073a50 0x7fe2f419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2fb659700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f419db70 con 0x7fe2f4073a50 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2fb659700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f41a1900 con 0x7fe2f4073130 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2f8bf4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073a50 0x7fe2f419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2f8bf4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073a50 0x7fe2f419d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38990/0 (socket says 172.21.15.70:38990) 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.943+0000 7fe2f8bf4700 1 -- 172.21.15.70:0/1674988383 learned_addr learned my addr 172.21.15.70:0/1674988383 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2f93f5700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2f4073130 0x7fe2f419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2f8bf4700 1 -- 172.21.15.70:0/1674988383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2f4073130 msgr2=0x7fe2f419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2f8bf4700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2f4073130 0x7fe2f419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2f8bf4700 1 -- 172.21.15.70:0/1674988383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f00095f0 con 0x7fe2f4073a50 2024-04-02T04:56:16.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2f93f5700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2f4073130 0x7fe2f419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:56:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2f8bf4700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073a50 0x7fe2f419d490 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400b4b0 tx=0x7fe2e400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2ea7fc700 1 -- 172.21.15.70:0/1674988383 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e4011800 con 0x7fe2f4073a50 2024-04-02T04:56:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f41a1be0 con 0x7fe2f4073a50 2024-04-02T04:56:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2ea7fc700 1 -- 172.21.15.70:0/1674988383 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2e4011e20 con 0x7fe2f4073a50 2024-04-02T04:56:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2ea7fc700 1 -- 172.21.15.70:0/1674988383 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e400f2e0 con 0x7fe2f4073a50 2024-04-02T04:56:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.944+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f41a2130 con 0x7fe2f4073a50 2024-04-02T04:56:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.945+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f410ee90 con 0x7fe2f4073a50 2024-04-02T04:56:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.946+0000 7fe2ea7fc700 1 -- 172.21.15.70:0/1674988383 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fe2e4010380 con 0x7fe2f4073a50 2024-04-02T04:56:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.946+0000 7fe2ea7fc700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe2e006dd60 0x7fe2e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.946+0000 7fe2ea7fc700 1 -- 172.21.15.70:0/1674988383 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe2e408d7a0 con 0x7fe2f4073a50 2024-04-02T04:56:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.946+0000 7fe2f93f5700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe2e006dd60 0x7fe2e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.947+0000 7fe2f93f5700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe2e006dd60 0x7fe2e0070220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000b5c0 tx=0x7fe2f0005480 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:16.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:16.951+0000 7fe2ea7fc700 1 -- 172.21.15.70:0/1674988383 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe2e4056cd0 con 0x7fe2f4073a50 2024-04-02T04:56:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.181+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2f41a23b0 con 0x7fe2e006dd60 2024-04-02T04:56:17.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.182+0000 7fe2ea7fc700 1 -- 172.21.15.70:0/1674988383 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe2f41a23b0 con 0x7fe2e006dd60 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe2e006dd60 msgr2=0x7fe2e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe2e006dd60 0x7fe2e0070220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000b5c0 tx=0x7fe2f0005480 comp rx=0 tx=0).stop 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073a50 msgr2=0x7fe2f419d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073a50 0x7fe2f419d490 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400b4b0 tx=0x7fe2e400b870 comp rx=0 tx=0).stop 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 shutdown_connections 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe2e006dd60 0x7fe2e0070220 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2f4073130 0x7fe2f419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 --2- 172.21.15.70:0/1674988383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f4073a50 0x7fe2f419d490 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 >> 172.21.15.70:0/1674988383 conn(0x7fe2f40fc750 msgr2=0x7fe2f4103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:17.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 shutdown_connections 2024-04-02T04:56:17.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.184+0000 7fe2fb659700 1 -- 172.21.15.70:0/1674988383 wait complete. 2024-04-02T04:56:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:56:17.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 -- 172.21.15.70:0/1996913678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c105c00 msgr2=0x7feb9c105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 --2- 172.21.15.70:0/1996913678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c105c00 0x7feb9c105fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb98009910 tx=0x7feb98009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:17.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 -- 172.21.15.70:0/1996913678 shutdown_connections 2024-04-02T04:56:17.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 --2- 172.21.15.70:0/1996913678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb9c0ffb50 0x7feb9c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 --2- 172.21.15.70:0/1996913678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c105c00 0x7feb9c105fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 -- 172.21.15.70:0/1996913678 >> 172.21.15.70:0/1996913678 conn(0x7feb9c0fb620 msgr2=0x7feb9c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:17.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 -- 172.21.15.70:0/1996913678 shutdown_connections 2024-04-02T04:56:17.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 -- 172.21.15.70:0/1996913678 wait complete. 2024-04-02T04:56:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 Processor -- start 2024-04-02T04:56:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.292+0000 7feba371e700 1 -- start start 2024-04-02T04:56:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba371e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c0ffb50 0x7feb9c195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba371e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb9c105c00 0x7feb9c195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba371e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c195ed0 con 0x7feb9c0ffb50 2024-04-02T04:56:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba371e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c196040 con 0x7feb9c105c00 2024-04-02T04:56:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba14ba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c0ffb50 0x7feb9c195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba14ba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c0ffb50 0x7feb9c195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39014/0 (socket says 172.21.15.70:39014) 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba14ba700 1 -- 172.21.15.70:0/3708229672 learned_addr learned my addr 172.21.15.70:0/3708229672 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba0cb9700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb9c105c00 0x7feb9c195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba14ba700 1 -- 172.21.15.70:0/3708229672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb9c105c00 msgr2=0x7feb9c195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba14ba700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb9c105c00 0x7feb9c195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba14ba700 1 -- 172.21.15.70:0/3708229672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb980095f0 con 0x7feb9c0ffb50 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba14ba700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c0ffb50 0x7feb9c195450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb98006010 tx=0x7feb9800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feb927fc700 1 -- 172.21.15.70:0/3708229672 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9801d070 con 0x7feb9c0ffb50 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feb927fc700 1 -- 172.21.15.70:0/3708229672 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb98005dc0 con 0x7feb9c0ffb50 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9c1962c0 con 0x7feb9c0ffb50 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feb927fc700 1 -- 172.21.15.70:0/3708229672 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb98017400 con 0x7feb9c0ffb50 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.293+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9c1a2d40 con 0x7feb9c0ffb50 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.294+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9c108030 con 0x7feb9c0ffb50 2024-04-02T04:56:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.295+0000 7feb927fc700 1 -- 172.21.15.70:0/3708229672 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7feb98003a60 con 0x7feb9c0ffb50 2024-04-02T04:56:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.295+0000 7feb927fc700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7feb8806dd60 0x7feb88070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.295+0000 7feb927fc700 1 -- 172.21.15.70:0/3708229672 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6597+0+0 (secure 0 0 0) 0x7feb9808ee90 con 0x7feb9c0ffb50 2024-04-02T04:56:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.295+0000 7feba0cb9700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7feb8806dd60 0x7feb88070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.296+0000 7feba0cb9700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7feb8806dd60 0x7feb88070220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0697f0 tx=0x7feb8c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:17.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.300+0000 7feb927fc700 1 -- 172.21.15.70:0/3708229672 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feb9805bef0 con 0x7feb9c0ffb50 2024-04-02T04:56:17.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.512+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb9c1962c0 con 0x7feb8806dd60 2024-04-02T04:56:17.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.513+0000 7feb927fc700 1 -- 172.21.15.70:0/3708229672 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7feb9c1962c0 con 0x7feb8806dd60 2024-04-02T04:56:17.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.515+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7feb8806dd60 msgr2=0x7feb88070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.515+0000 7feba371e700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7feb8806dd60 0x7feb88070220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0697f0 tx=0x7feb8c00b410 comp rx=0 tx=0).stop 2024-04-02T04:56:17.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.515+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c0ffb50 msgr2=0x7feb9c195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.515+0000 7feba371e700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c0ffb50 0x7feb9c195450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb98006010 tx=0x7feb9800bd70 comp rx=0 tx=0).stop 2024-04-02T04:56:17.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.516+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 shutdown_connections 2024-04-02T04:56:17.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.516+0000 7feba371e700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7feb8806dd60 0x7feb88070220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.516+0000 7feba371e700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9c0ffb50 0x7feb9c195450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.516+0000 7feba371e700 1 --2- 172.21.15.70:0/3708229672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb9c105c00 0x7feb9c195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.516+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 >> 172.21.15.70:0/3708229672 conn(0x7feb9c0fb620 msgr2=0x7feb9c0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:17.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.516+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 shutdown_connections 2024-04-02T04:56:17.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.516+0000 7feba371e700 1 -- 172.21.15.70:0/3708229672 wait complete. 2024-04-02T04:56:17.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.619+0000 7fe12276b700 1 -- 172.21.15.70:0/3036444220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 msgr2=0x7fe11c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.619+0000 7fe12276b700 1 --2- 172.21.15.70:0/3036444220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 0x7fe11c0ff980 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe110009910 tx=0x7fe110009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:17.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.619+0000 7fe12276b700 1 -- 172.21.15.70:0/3036444220 shutdown_connections 2024-04-02T04:56:17.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.619+0000 7fe12276b700 1 --2- 172.21.15.70:0/3036444220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c0fff50 0x7fe11c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.619+0000 7fe12276b700 1 --2- 172.21.15.70:0/3036444220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 0x7fe11c0ff980 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.619+0000 7fe12276b700 1 -- 172.21.15.70:0/3036444220 >> 172.21.15.70:0/3036444220 conn(0x7fe11c074e30 msgr2=0x7fe11c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.619+0000 7fe12276b700 1 -- 172.21.15.70:0/3036444220 shutdown_connections 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.619+0000 7fe12276b700 1 -- 172.21.15.70:0/3036444220 wait complete. 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe12276b700 1 Processor -- start 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe12276b700 1 -- start start 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe12276b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 0x7fe11c100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe12276b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c0fff50 0x7fe11c100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe12276b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c100fa0 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe12276b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c1010e0 con 0x7fe11c0fff50 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe11bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 0x7fe11c100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe11bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 0x7fe11c100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39034/0 (socket says 172.21.15.70:39034) 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.620+0000 7fe11bfff700 1 -- 172.21.15.70:0/4012976531 learned_addr learned my addr 172.21.15.70:0/4012976531 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe11bfff700 1 -- 172.21.15.70:0/4012976531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c0fff50 msgr2=0x7fe11c100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe11bfff700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c0fff50 0x7fe11c100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe11bfff700 1 -- 172.21.15.70:0/4012976531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1100095f0 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe11bfff700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 0x7fe11c100520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1100092e0 tx=0x7fe11000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe1197fa700 1 -- 172.21.15.70:0/4012976531 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11001d070 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c101360 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe1197fa700 1 -- 172.21.15.70:0/4012976531 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe110005e00 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe1197fa700 1 -- 172.21.15.70:0/4012976531 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe110017400 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.621+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c1a2db0 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.622+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c109270 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.622+0000 7fe1197fa700 1 -- 172.21.15.70:0/4012976531 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fe11000f460 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.623+0000 7fe1197fa700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe10406dd60 0x7fe104070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.623+0000 7fe11b7fe700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe10406dd60 0x7fe104070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:17.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.623+0000 7fe1197fa700 1 -- 172.21.15.70:0/4012976531 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe11008f0b0 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.623+0000 7fe11b7fe700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe10406dd60 0x7fe104070220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005950 tx=0x7fe10c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:17.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.628+0000 7fe1197fa700 1 -- 172.21.15.70:0/4012976531 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe1100585e0 con 0x7fe11c0ff5a0 2024-04-02T04:56:17.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.835+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe11c0611d0 con 0x7fe10406dd60 2024-04-02T04:56:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.845+0000 7fe1197fa700 1 -- 172.21.15.70:0/4012976531 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe11c0611d0 con 0x7fe10406dd60 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (5m) 82s ago 16m 26.2M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (17m) 82s ago 17m 8678k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (16m) 16s ago 16m 8258k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (3m) 82s ago 17m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (3m) 16s ago 16m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (5m) 82s ago 16m 76.8M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (11m) 82s ago 11m 151M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:56:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (11m) 82s ago 11m 20.2M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (11m) 16s ago 11m 18.9M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (11m) 16s ago 11m 20.2M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (7m) 82s ago 18m 504M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (7m) 16s ago 15m 451M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (4m) 82s ago 18m 49.2M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (3m) 16s ago 15m 45.4M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (6m) 82s ago 17m 15.5M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (6m) 16s ago 16m 16.9M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (2m) 82s ago 15m 141M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (2m) 82s ago 14m 157M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (88s) 82s ago 14m 11.8M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (52s) 16s ago 13m 97.2M 2240M 19.0.0-1979-g6cec793d 78f830190839 f717fd6c8306 2024-04-02T04:56:17.851 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (19s) 16s ago 13m 12.6M 2240M 19.0.0-1979-g6cec793d 78f830190839 2e5bbf3235ff 2024-04-02T04:56:17.852 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (12m) 16s ago 12m 256M 2240M 18.2.1 5be31c24972a 99d6490f2b0d 2024-04-02T04:56:17.852 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (6m) 82s ago 16m 62.9M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:56:17.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe10406dd60 msgr2=0x7fe104070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe10406dd60 0x7fe104070220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005950 tx=0x7fe10c0058e0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 msgr2=0x7fe11c100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 0x7fe11c100520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1100092e0 tx=0x7fe11000be20 comp rx=0 tx=0).stop 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 shutdown_connections 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fe10406dd60 0x7fe104070220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c0ff5a0 0x7fe11c100520 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 --2- 172.21.15.70:0/4012976531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe11c0fff50 0x7fe11c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 >> 172.21.15.70:0/4012976531 conn(0x7fe11c074e30 msgr2=0x7fe11c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 shutdown_connections 2024-04-02T04:56:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.848+0000 7fe12276b700 1 -- 172.21.15.70:0/4012976531 wait complete. 2024-04-02T04:56:17.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 -- 172.21.15.70:0/665701804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 msgr2=0x7fdd2c10d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 --2- 172.21.15.70:0/665701804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 0x7fdd2c10d290 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009910 tx=0x7fdd1c009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 -- 172.21.15.70:0/665701804 shutdown_connections 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 --2- 172.21.15.70:0/665701804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 0x7fdd2c10d290 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 --2- 172.21.15.70:0/665701804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd2c068580 0x7fdd2c068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 -- 172.21.15.70:0/665701804 >> 172.21.15.70:0/665701804 conn(0x7fdd2c075670 msgr2=0x7fdd2c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 -- 172.21.15.70:0/665701804 shutdown_connections 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 -- 172.21.15.70:0/665701804 wait complete. 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 Processor -- start 2024-04-02T04:56:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.951+0000 7fdd32493700 1 -- start start 2024-04-02T04:56:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd32493700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd2c068580 0x7fdd2c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd32493700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 0x7fdd2c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd32493700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c19f4b0 con 0x7fdd2c068ea0 2024-04-02T04:56:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd32493700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c198960 con 0x7fdd2c068580 2024-04-02T04:56:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd2b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 0x7fdd2c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd2b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 0x7fdd2c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39052/0 (socket says 172.21.15.70:39052) 2024-04-02T04:56:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd2b7fe700 1 -- 172.21.15.70:0/1162178719 learned_addr learned my addr 172.21.15.70:0/1162178719 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd2bfff700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd2c068580 0x7fdd2c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd2b7fe700 1 -- 172.21.15.70:0/1162178719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd2c068580 msgr2=0x7fdd2c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd2b7fe700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd2c068580 0x7fdd2c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd2b7fe700 1 -- 172.21.15.70:0/1162178719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0095f0 con 0x7fdd2c068ea0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.952+0000 7fdd2b7fe700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 0x7fdd2c19ee20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c006010 tx=0x7fdd1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.953+0000 7fdd297fa700 1 -- 172.21.15.70:0/1162178719 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1c01d070 con 0x7fdd2c068ea0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.953+0000 7fdd297fa700 1 -- 172.21.15.70:0/1162178719 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd1c00f460 con 0x7fdd2c068ea0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.953+0000 7fdd297fa700 1 -- 172.21.15.70:0/1162178719 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1c017630 con 0x7fdd2c068ea0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.953+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c198be0 con 0x7fdd2c068ea0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.953+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c1990d0 con 0x7fdd2c068ea0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.954+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c10aa00 con 0x7fdd2c068ea0 2024-04-02T04:56:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.954+0000 7fdd297fa700 1 -- 172.21.15.70:0/1162178719 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fdd1c00f5d0 con 0x7fdd2c068ea0 2024-04-02T04:56:17.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.954+0000 7fdd297fa700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdd1406dd60 0x7fdd14070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:17.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.955+0000 7fdd2bfff700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdd1406dd60 0x7fdd14070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:17.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.955+0000 7fdd297fa700 1 -- 172.21.15.70:0/1162178719 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fdd1c08efb0 con 0x7fdd2c068ea0 2024-04-02T04:56:17.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.955+0000 7fdd2bfff700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdd1406dd60 0x7fdd14070220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd20005950 tx=0x7fdd200058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:17.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:17.959+0000 7fdd297fa700 1 -- 172.21.15.70:0/1162178719 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdd1c05c090 con 0x7fdd2c068ea0 2024-04-02T04:56:18.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.235+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdd2c199e70 con 0x7fdd2c068ea0 2024-04-02T04:56:18.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.236+0000 7fdd297fa700 1 -- 172.21.15.70:0/1162178719 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fdd1c05bc20 con 0x7fdd2c068ea0 2024-04-02T04:56:18.240 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:56:18.240 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:56:18.240 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 5 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:56:18.241 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-04-02T04:56:18.242 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 9 2024-04-02T04:56:18.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:56:18.242 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:56:18.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdd1406dd60 msgr2=0x7fdd14070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdd1406dd60 0x7fdd14070220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd20005950 tx=0x7fdd200058e0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 msgr2=0x7fdd2c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 0x7fdd2c19ee20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c006010 tx=0x7fdd1c00be20 comp rx=0 tx=0).stop 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 shutdown_connections 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdd1406dd60 0x7fdd14070220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd2c068580 0x7fdd2c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 --2- 172.21.15.70:0/1162178719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd2c068ea0 0x7fdd2c19ee20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 >> 172.21.15.70:0/1162178719 conn(0x7fdd2c075670 msgr2=0x7fdd2c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 shutdown_connections 2024-04-02T04:56:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.238+0000 7fdd32493700 1 -- 172.21.15.70:0/1162178719 wait complete. 2024-04-02T04:56:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.341+0000 7f8e74ca7700 1 -- 172.21.15.70:0/154987915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 msgr2=0x7f8e70105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.341+0000 7f8e74ca7700 1 --2- 172.21.15.70:0/154987915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 0x7f8e70105840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009910 tx=0x7f8e64009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.341+0000 7f8e74ca7700 1 -- 172.21.15.70:0/154987915 shutdown_connections 2024-04-02T04:56:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.341+0000 7f8e74ca7700 1 --2- 172.21.15.70:0/154987915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 0x7f8e70105840 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.341+0000 7f8e74ca7700 1 --2- 172.21.15.70:0/154987915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e700ff5a0 0x7f8e700ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.341+0000 7f8e74ca7700 1 -- 172.21.15.70:0/154987915 >> 172.21.15.70:0/154987915 conn(0x7f8e70074e30 msgr2=0x7f8e70075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.341+0000 7f8e74ca7700 1 -- 172.21.15.70:0/154987915 shutdown_connections 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.342+0000 7f8e74ca7700 1 -- 172.21.15.70:0/154987915 wait complete. 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.342+0000 7f8e74ca7700 1 Processor -- start 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.342+0000 7f8e74ca7700 1 -- start start 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.342+0000 7f8e74ca7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e700ff5a0 0x7f8e70196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.342+0000 7f8e74ca7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 0x7f8e70196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.342+0000 7f8e74ca7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e70197560 con 0x7f8e700fff50 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.342+0000 7f8e74ca7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7019b370 con 0x7f8e700ff5a0 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 0x7f8e70196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 0x7f8e70196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39082/0 (socket says 172.21.15.70:39082) 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6dd9b700 1 -- 172.21.15.70:0/1346562951 learned_addr learned my addr 172.21.15.70:0/1346562951 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6e59c700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e700ff5a0 0x7f8e70196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6dd9b700 1 -- 172.21.15.70:0/1346562951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e700ff5a0 msgr2=0x7f8e70196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6dd9b700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e700ff5a0 0x7f8e70196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6dd9b700 1 -- 172.21.15.70:0/1346562951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e640095f0 con 0x7f8e700fff50 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6e59c700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e700ff5a0 0x7f8e70196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e6dd9b700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 0x7f8e70196ed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005f50 tx=0x7f8e6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e5f7fe700 1 -- 172.21.15.70:0/1346562951 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e6401d070 con 0x7f8e700fff50 2024-04-02T04:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7019b5f0 con 0x7f8e700fff50 2024-04-02T04:56:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e5f7fe700 1 -- 172.21.15.70:0/1346562951 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e6400f460 con 0x7f8e700fff50 2024-04-02T04:56:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e5f7fe700 1 -- 172.21.15.70:0/1346562951 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e64017630 con 0x7f8e700fff50 2024-04-02T04:56:18.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.343+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7019bae0 con 0x7f8e700fff50 2024-04-02T04:56:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.344+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e70109220 con 0x7f8e700fff50 2024-04-02T04:56:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.345+0000 7f8e5f7fe700 1 -- 172.21.15.70:0/1346562951 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f8e6400f5d0 con 0x7f8e700fff50 2024-04-02T04:56:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.345+0000 7f8e5f7fe700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8e5806dd60 0x7f8e58070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.345+0000 7f8e5f7fe700 1 -- 172.21.15.70:0/1346562951 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f8e6408f1a0 con 0x7f8e700fff50 2024-04-02T04:56:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.345+0000 7f8e6e59c700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8e5806dd60 0x7f8e58070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.345+0000 7f8e6e59c700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8e5806dd60 0x7f8e58070220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8e60007900 tx=0x7f8e60008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.350+0000 7f8e5f7fe700 1 -- 172.21.15.70:0/1346562951 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8e6405c200 con 0x7f8e700fff50 2024-04-02T04:56:18.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:18 smithi070 ceph-mon[113635]: pgmap v163: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:18.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:18 smithi070 ceph-mon[113635]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:18.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:18 smithi179 ceph-mon[84288]: pgmap v163: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:18.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:18 smithi179 ceph-mon[84288]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:18.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.589+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8e7004f2e0 con 0x7f8e700fff50 2024-04-02T04:56:18.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.590+0000 7f8e5f7fe700 1 -- 172.21.15.70:0/1346562951 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f8e6405bd90 con 0x7f8e700fff50 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:e14 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 13 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:45:08.643957+0000 2024-04-02T04:56:18.595 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14468} 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:56:18.596 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6826/1351191990,v1:172.21.15.70:6827/1351191990] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:14480} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:56:18.597 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:56:18.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8e5806dd60 msgr2=0x7f8e58070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8e5806dd60 0x7f8e58070220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8e60007900 tx=0x7f8e60008040 comp rx=0 tx=0).stop 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 msgr2=0x7f8e70196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 0x7f8e70196ed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005f50 tx=0x7f8e6400be20 comp rx=0 tx=0).stop 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 shutdown_connections 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8e5806dd60 0x7f8e58070220 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e700ff5a0 0x7f8e70196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 --2- 172.21.15.70:0/1346562951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e700fff50 0x7f8e70196ed0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 >> 172.21.15.70:0/1346562951 conn(0x7f8e70074e30 msgr2=0x7f8e700fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 shutdown_connections 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.592+0000 7f8e74ca7700 1 -- 172.21.15.70:0/1346562951 wait complete. 2024-04-02T04:56:18.599 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T04:56:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.697+0000 7f22f7a9b700 1 -- 172.21.15.70:0/1214393558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 msgr2=0x7f22f01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.697+0000 7f22f7a9b700 1 --2- 172.21.15.70:0/1214393558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 0x7f22f01058a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009910 tx=0x7f22e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.697+0000 7f22f7a9b700 1 -- 172.21.15.70:0/1214393558 shutdown_connections 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.697+0000 7f22f7a9b700 1 --2- 172.21.15.70:0/1214393558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 0x7f22f01058a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.697+0000 7f22f7a9b700 1 --2- 172.21.15.70:0/1214393558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22f0100ea0 0x7f22f0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.697+0000 7f22f7a9b700 1 -- 172.21.15.70:0/1214393558 >> 172.21.15.70:0/1214393558 conn(0x7f22f00fc750 msgr2=0x7f22f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.697+0000 7f22f7a9b700 1 -- 172.21.15.70:0/1214393558 shutdown_connections 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.697+0000 7f22f7a9b700 1 -- 172.21.15.70:0/1214393558 wait complete. 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f7a9b700 1 Processor -- start 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f7a9b700 1 -- start start 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f7a9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22f0100ea0 0x7f22f019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f7a9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 0x7f22f019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f7a9b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f019d360 con 0x7f22f0101850 2024-04-02T04:56:18.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f7a9b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f0196810 con 0x7f22f0100ea0 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f5036700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 0x7f22f019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f5036700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 0x7f22f019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39096/0 (socket says 172.21.15.70:39096) 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f5036700 1 -- 172.21.15.70:0/2363994685 learned_addr learned my addr 172.21.15.70:0/2363994685 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f5837700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22f0100ea0 0x7f22f019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.698+0000 7f22f5036700 1 -- 172.21.15.70:0/2363994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22f0100ea0 msgr2=0x7f22f019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22f5036700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22f0100ea0 0x7f22f019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22f5036700 1 -- 172.21.15.70:0/2363994685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00095f0 con 0x7f22f0101850 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22f5837700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22f0100ea0 0x7f22f019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22f5036700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 0x7f22f019ccd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f22e00092e0 tx=0x7f22e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22e6ffd700 1 -- 172.21.15.70:0/2363994685 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e001d070 con 0x7f22f0101850 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22e6ffd700 1 -- 172.21.15.70:0/2363994685 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22e000f460 con 0x7f22f0101850 2024-04-02T04:56:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f0196a90 con 0x7f22f0101850 2024-04-02T04:56:18.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f0196f80 con 0x7f22f0101850 2024-04-02T04:56:18.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.699+0000 7f22e6ffd700 1 -- 172.21.15.70:0/2363994685 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e002b610 con 0x7f22f0101850 2024-04-02T04:56:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.700+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f01091f0 con 0x7f22f0101850 2024-04-02T04:56:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.700+0000 7f22e6ffd700 1 -- 172.21.15.70:0/2363994685 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f22e000f5d0 con 0x7f22f0101850 2024-04-02T04:56:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.701+0000 7f22e6ffd700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f22dc06dd60 0x7f22dc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.701+0000 7f22e6ffd700 1 -- 172.21.15.70:0/2363994685 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f22e008f110 con 0x7f22f0101850 2024-04-02T04:56:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.701+0000 7f22f5837700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f22dc06dd60 0x7f22dc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:18.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.701+0000 7f22f5837700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f22dc06dd60 0x7f22dc070220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f22ec007900 tx=0x7f22ec008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.706+0000 7f22e6ffd700 1 -- 172.21.15.70:0/2363994685 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f22e005c110 con 0x7f22f0101850 2024-04-02T04:56:18.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.920+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22f0197e30 con 0x7f22dc06dd60 2024-04-02T04:56:18.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.921+0000 7f22e6ffd700 1 -- 172.21.15.70:0/2363994685 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f22f0197e30 con 0x7f22dc06dd60 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "11/23 daemons upgraded", 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:56:18.926 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.923+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f22dc06dd60 msgr2=0x7f22dc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.923+0000 7f22f7a9b700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f22dc06dd60 0x7f22dc070220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f22ec007900 tx=0x7f22ec008040 comp rx=0 tx=0).stop 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.923+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 msgr2=0x7f22f019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.923+0000 7f22f7a9b700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 0x7f22f019ccd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f22e00092e0 tx=0x7f22e000be20 comp rx=0 tx=0).stop 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.923+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 shutdown_connections 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.923+0000 7f22f7a9b700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f22dc06dd60 0x7f22dc070220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.923+0000 7f22f7a9b700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22f0100ea0 0x7f22f019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.923+0000 7f22f7a9b700 1 --2- 172.21.15.70:0/2363994685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f0101850 0x7f22f019ccd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.924+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 >> 172.21.15.70:0/2363994685 conn(0x7f22f00fc750 msgr2=0x7f22f00fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:18.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.924+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 shutdown_connections 2024-04-02T04:56:18.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:18.924+0000 7f22f7a9b700 1 -- 172.21.15.70:0/2363994685 wait complete. 2024-04-02T04:56:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.029+0000 7efc90b3c700 1 -- 172.21.15.70:0/1142514460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 msgr2=0x7efc8c10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.029+0000 7efc90b3c700 1 --2- 172.21.15.70:0/1142514460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 0x7efc8c10ac70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc80009910 tx=0x7efc80009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 -- 172.21.15.70:0/1142514460 shutdown_connections 2024-04-02T04:56:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 --2- 172.21.15.70:0/1142514460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 0x7efc8c10ac70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 --2- 172.21.15.70:0/1142514460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c101e60 0x7efc8c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 -- 172.21.15.70:0/1142514460 >> 172.21.15.70:0/1142514460 conn(0x7efc8c0fb650 msgr2=0x7efc8c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:19.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 -- 172.21.15.70:0/1142514460 shutdown_connections 2024-04-02T04:56:19.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 -- 172.21.15.70:0/1142514460 wait complete. 2024-04-02T04:56:19.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 Processor -- start 2024-04-02T04:56:19.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 -- start start 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.030+0000 7efc90b3c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c101e60 0x7efc8c19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc90b3c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 0x7efc8c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc90b3c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c19d130 con 0x7efc8c102780 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc90b3c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c1966c0 con 0x7efc8c101e60 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc89d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 0x7efc8c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc89d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 0x7efc8c19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39120/0 (socket says 172.21.15.70:39120) 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc89d9b700 1 -- 172.21.15.70:0/1255272103 learned_addr learned my addr 172.21.15.70:0/1255272103 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc8a59c700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c101e60 0x7efc8c19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc89d9b700 1 -- 172.21.15.70:0/1255272103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c101e60 msgr2=0x7efc8c19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc89d9b700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c101e60 0x7efc8c19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:19.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc89d9b700 1 -- 172.21.15.70:0/1255272103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800095f0 con 0x7efc8c102780 2024-04-02T04:56:19.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc89d9b700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 0x7efc8c19cb10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc800092e0 tx=0x7efc8000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:19.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc7b7fe700 1 -- 172.21.15.70:0/1255272103 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc8001d070 con 0x7efc8c102780 2024-04-02T04:56:19.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc7b7fe700 1 -- 172.21.15.70:0/1255272103 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc8000f460 con 0x7efc8c102780 2024-04-02T04:56:19.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc7b7fe700 1 -- 172.21.15.70:0/1255272103 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc800175d0 con 0x7efc8c102780 2024-04-02T04:56:19.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c196940 con 0x7efc8c102780 2024-04-02T04:56:19.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.031+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8c196e30 con 0x7efc8c102780 2024-04-02T04:56:19.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.032+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8c108300 con 0x7efc8c102780 2024-04-02T04:56:19.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.033+0000 7efc7b7fe700 1 -- 172.21.15.70:0/1255272103 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7efc8000f5d0 con 0x7efc8c102780 2024-04-02T04:56:19.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.033+0000 7efc7b7fe700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efc7406dd60 0x7efc74070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:19.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.033+0000 7efc7b7fe700 1 -- 172.21.15.70:0/1255272103 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efc8008f040 con 0x7efc8c102780 2024-04-02T04:56:19.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.033+0000 7efc8a59c700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efc7406dd60 0x7efc74070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:19.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.034+0000 7efc8a59c700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efc7406dd60 0x7efc74070220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc7c005950 tx=0x7efc7c00a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:19.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.038+0000 7efc7b7fe700 1 -- 172.21.15.70:0/1255272103 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efc8005c0a0 con 0x7efc8c102780 2024-04-02T04:56:19.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:19 smithi070 ceph-mon[113635]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:19.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:19 smithi070 ceph-mon[113635]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:19.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:19 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1162178719' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:19.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:19 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1346562951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:56:19.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.318+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efc8c107ce0 con 0x7efc8c102780 2024-04-02T04:56:19.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.319+0000 7efc7b7fe700 1 -- 172.21.15.70:0/1255272103 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7efc8005bc30 con 0x7efc8c102780 2024-04-02T04:56:19.323 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:56:19.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efc7406dd60 msgr2=0x7efc74070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:19.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efc7406dd60 0x7efc74070220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc7c005950 tx=0x7efc7c00a300 comp rx=0 tx=0).stop 2024-04-02T04:56:19.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 msgr2=0x7efc8c19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:19.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 0x7efc8c19cb10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc800092e0 tx=0x7efc8000bd70 comp rx=0 tx=0).stop 2024-04-02T04:56:19.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 shutdown_connections 2024-04-02T04:56:19.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7efc7406dd60 0x7efc74070220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:19.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c101e60 0x7efc8c19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:19.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 --2- 172.21.15.70:0/1255272103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc8c102780 0x7efc8c19cb10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:19.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 >> 172.21.15.70:0/1255272103 conn(0x7efc8c0fb650 msgr2=0x7efc8c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:19.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 shutdown_connections 2024-04-02T04:56:19.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:19.321+0000 7efc90b3c700 1 -- 172.21.15.70:0/1255272103 wait complete. 2024-04-02T04:56:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:19 smithi179 ceph-mon[84288]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:19 smithi179 ceph-mon[84288]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:19 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1162178719' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:19 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1346562951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:56:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:20 smithi070 ceph-mon[113635]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:20 smithi070 ceph-mon[113635]: pgmap v164: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:20 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1255272103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:56:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:20 smithi179 ceph-mon[84288]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:20 smithi179 ceph-mon[84288]: pgmap v164: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:20 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1255272103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:56:22.332 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:22 smithi070 ceph-mon[113635]: pgmap v165: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:22.332 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:22.332 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:56:22.525 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:22 smithi179 ceph-mon[84288]: pgmap v165: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:22.525 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:22.525 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:56:23.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:23 smithi070 ceph-mon[113635]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:56:23.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:23 smithi070 ceph-mon[113635]: Upgrade: osd.5 is safe to restart 2024-04-02T04:56:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:23 smithi179 ceph-mon[84288]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:56:23.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:23 smithi179 ceph-mon[84288]: Upgrade: osd.5 is safe to restart 2024-04-02T04:56:24.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:24 smithi179 ceph-mon[84288]: pgmap v166: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:24.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:24 smithi179 ceph-mon[84288]: Upgrade: Updating osd.5 2024-04-02T04:56:24.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:24.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:56:24.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:24 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:24.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:24 smithi179 ceph-mon[84288]: Deploying daemon osd.5 on smithi179 2024-04-02T04:56:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:24 smithi070 ceph-mon[113635]: pgmap v166: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:24 smithi070 ceph-mon[113635]: Upgrade: Updating osd.5 2024-04-02T04:56:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:56:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:24 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:24.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:24 smithi070 ceph-mon[113635]: Deploying daemon osd.5 on smithi179 2024-04-02T04:56:26.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:26 smithi179 ceph-mon[84288]: pgmap v167: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:26.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:26.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:26.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:26 smithi070 ceph-mon[113635]: pgmap v167: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:26.627 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:26 smithi179 systemd[1]: Stopping Ceph osd.5 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:56:27.072 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[55862]: 2024-04-02T04:56:26.622+0000 7fc0762f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:56:27.072 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[55862]: 2024-04-02T04:56:26.622+0000 7fc0762f3700 -1 osd.5 86 *** Got signal Terminated *** 2024-04-02T04:56:27.072 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[55862]: 2024-04-02T04:56:26.622+0000 7fc0762f3700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:56:27.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:27 smithi070 ceph-mon[113635]: osd.5 marked itself down and dead 2024-04-02T04:56:27.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:27 smithi179 ceph-mon[84288]: osd.5 marked itself down and dead 2024-04-02T04:56:28.061 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:27 smithi179 podman[103962]: 2024-04-02 04:56:27.777313364 +0000 UTC m=+1.280558823 container died 99d6490f2b0d1b04a019d7ed973229b2491a796ee19caf99f3818b5643a1104c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T04:56:28.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:28 smithi179 podman[103962]: 2024-04-02 04:56:28.148428453 +0000 UTC m=+1.651673902 container cleanup 99d6490f2b0d1b04a019d7ed973229b2491a796ee19caf99f3818b5643a1104c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1) 2024-04-02T04:56:28.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:28 smithi179 bash[103962]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T04:56:28.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:28 smithi179 ceph-mon[84288]: pgmap v168: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:28.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:28 smithi179 ceph-mon[84288]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:56:28.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:28 smithi179 ceph-mon[84288]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-02T04:56:28.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:28 smithi179 ceph-mon[84288]: osdmap e87: 6 total, 5 up, 6 in 2024-04-02T04:56:28.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:28 smithi070 ceph-mon[113635]: pgmap v168: 65 pgs: 65 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:28.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:28 smithi070 ceph-mon[113635]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:56:28.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:28 smithi070 ceph-mon[113635]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-02T04:56:28.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:28 smithi070 ceph-mon[113635]: osdmap e87: 6 total, 5 up, 6 in 2024-04-02T04:56:28.762 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:28 smithi179 podman[103980]: 2024-04-02 04:56:28.458621524 +0000 UTC m=+0.684281184 container remove 99d6490f2b0d1b04a019d7ed973229b2491a796ee19caf99f3818b5643a1104c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-04-02T04:56:29.065 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:28 smithi179 podman[104074]: 2024-04-02 04:56:28.759536066 +0000 UTC m=+0.114601052 container create d597668aeaabe11a7683744b6ac29f41b21fce72d43ebc9cd4662aff694cc30e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:56:29.066 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:28 smithi179 podman[104074]: 2024-04-02 04:56:28.663183721 +0000 UTC m=+0.018248715 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:56:29.066 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:28 smithi179 podman[104074]: 2024-04-02 04:56:28.98753815 +0000 UTC m=+0.342603135 container init d597668aeaabe11a7683744b6ac29f41b21fce72d43ebc9cd4662aff694cc30e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, RELEASE=squid-6cec793, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-02T04:56:29.066 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:28 smithi179 podman[104074]: 2024-04-02 04:56:28.990607868 +0000 UTC m=+0.345672857 container start d597668aeaabe11a7683744b6ac29f41b21fce72d43ebc9cd4662aff694cc30e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-02T04:56:29.066 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:29 smithi179 podman[104074]: 2024-04-02 04:56:29.062603507 +0000 UTC m=+0.417668504 container attach d597668aeaabe11a7683744b6ac29f41b21fce72d43ebc9cd4662aff694cc30e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-04-02T04:56:29.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:29 smithi179 ceph-mon[84288]: osdmap e88: 6 total, 5 up, 6 in 2024-04-02T04:56:29.323 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:29 smithi179 podman[104074]: 2024-04-02 04:56:29.283874449 +0000 UTC m=+0.638939437 container died d597668aeaabe11a7683744b6ac29f41b21fce72d43ebc9cd4662aff694cc30e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, RELEASE=squid-6cec793, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T04:56:29.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:29 smithi070 ceph-mon[113635]: osdmap e88: 6 total, 5 up, 6 in 2024-04-02T04:56:29.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:29 smithi179 podman[104100]: 2024-04-02 04:56:29.766785391 +0000 UTC m=+0.477797655 container remove d597668aeaabe11a7683744b6ac29f41b21fce72d43ebc9cd4662aff694cc30e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793) 2024-04-02T04:56:29.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:29 smithi179 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.5.service: Deactivated successfully. 2024-04-02T04:56:29.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:29 smithi179 systemd[1]: Stopped Ceph osd.5 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:56:29.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:29 smithi179 systemd[1]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.5.service: Consumed 1min 8.565s CPU time. 2024-04-02T04:56:30.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-mon[84288]: pgmap v171: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:30.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-mon[84288]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-02T04:56:30.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:29 smithi179 systemd[1]: Starting Ceph osd.5 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T04:56:30.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 podman[104209]: 2024-04-02 04:56:30.116776355 +0000 UTC m=+0.018061032 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:56:30.323 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 podman[104209]: 2024-04-02 04:56:30.243729238 +0000 UTC m=+0.145013913 container create 89bc5316add7c6403427fa931a293c53ac633bd6319f5acc80cb92a740f3daf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-04-02T04:56:30.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:30 smithi070 ceph-mon[113635]: pgmap v171: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:30.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:30 smithi070 ceph-mon[113635]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-02T04:56:30.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 podman[104209]: 2024-04-02 04:56:30.48002671 +0000 UTC m=+0.381311398 container init 89bc5316add7c6403427fa931a293c53ac633bd6319f5acc80cb92a740f3daf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-02T04:56:30.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 podman[104209]: 2024-04-02 04:56:30.483552505 +0000 UTC m=+0.384837183 container start 89bc5316add7c6403427fa931a293c53ac633bd6319f5acc80cb92a740f3daf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:56:30.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 podman[104209]: 2024-04-02 04:56:30.579995836 +0000 UTC m=+0.481280513 container attach 89bc5316add7c6403427fa931a293c53ac633bd6319f5acc80cb92a740f3daf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-02T04:56:31.262 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate[104224]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate[104224]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 bash[104209]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 bash[104209]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 bash[104209]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 bash[104209]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 bash[104209]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 bash[104209]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 bash[104209]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate[104224]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate[104224]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate[104224]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate[104224]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T04:56:31.263 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate[104224]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-02T04:56:31.264 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:30 smithi179 podman[104209]: 2024-04-02 04:56:30.946392815 +0000 UTC m=+0.847677499 container died 89bc5316add7c6403427fa931a293c53ac633bd6319f5acc80cb92a740f3daf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T04:56:31.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:31 smithi070 ceph-mon[113635]: Health check failed: Degraded data redundancy: 28/228 objects degraded (12.281%), 9 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:31.537 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:31 smithi179 ceph-mon[84288]: Health check failed: Degraded data redundancy: 28/228 objects degraded (12.281%), 9 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:31.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:31 smithi179 podman[104344]: 2024-04-02 04:56:31.534765507 +0000 UTC m=+0.583159939 container remove 89bc5316add7c6403427fa931a293c53ac633bd6319f5acc80cb92a740f3daf0 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T04:56:31.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:31 smithi179 podman[104407]: 2024-04-02 04:56:31.652297497 +0000 UTC m=+0.018122122 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:56:31.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:31 smithi179 podman[104407]: 2024-04-02 04:56:31.76127145 +0000 UTC m=+0.127096094 container create 71351459ac7073c4bdf8f97116cfac721b73aa45863a15a87542877d57ab8229 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793) 2024-04-02T04:56:32.272 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:31 smithi179 podman[104407]: 2024-04-02 04:56:31.99721228 +0000 UTC m=+0.363036888 container init 71351459ac7073c4bdf8f97116cfac721b73aa45863a15a87542877d57ab8229 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-6cec793) 2024-04-02T04:56:32.272 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:32 smithi179 podman[104407]: 2024-04-02 04:56:32.000317669 +0000 UTC m=+0.366142282 container start 71351459ac7073c4bdf8f97116cfac721b73aa45863a15a87542877d57ab8229 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:56:32.272 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:32 smithi179 ceph-osd[104426]: -- 172.21.15.179:0/3754675915 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a609633680 con 0x55a6095fd000 2024-04-02T04:56:32.272 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:32 smithi179 bash[104407]: 71351459ac7073c4bdf8f97116cfac721b73aa45863a15a87542877d57ab8229 2024-04-02T04:56:32.272 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:32 smithi179 systemd[1]: Started Ceph osd.5 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4. 2024-04-02T04:56:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:32 smithi070 ceph-mon[113635]: pgmap v172: 65 pgs: 10 active+undersized, 9 active+undersized+degraded, 3 peering, 2 stale+active+clean, 41 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail; 28/228 objects degraded (12.281%) 2024-04-02T04:56:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:32.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:32.542 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:32 smithi179 ceph-mon[84288]: pgmap v172: 65 pgs: 10 active+undersized, 9 active+undersized+degraded, 3 peering, 2 stale+active+clean, 41 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail; 28/228 objects degraded (12.281%) 2024-04-02T04:56:32.542 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:32.542 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:32.542 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:32.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:32 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T04:56:32.538+0000 7f0bbd3ea7c0 -1 Falling back to public interface 2024-04-02T04:56:34.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:33 smithi179 ceph-mon[84288]: pgmap v173: 65 pgs: 13 active+undersized, 13 active+undersized+degraded, 3 peering, 36 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail; 42/228 objects degraded (18.421%) 2024-04-02T04:56:34.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:33 smithi070 ceph-mon[113635]: pgmap v173: 65 pgs: 13 active+undersized, 13 active+undersized+degraded, 3 peering, 36 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail; 42/228 objects degraded (18.421%) 2024-04-02T04:56:36.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:36 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:36.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:36 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:36.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:36 smithi070 ceph-mon[113635]: pgmap v174: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail; 42/228 objects degraded (18.421%) 2024-04-02T04:56:36.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:36 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:36.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:36 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:36.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:36 smithi179 ceph-mon[84288]: pgmap v174: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail; 42/228 objects degraded (18.421%) 2024-04-02T04:56:37.048 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:37 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T04:56:37.000+0000 7f0bbd3ea7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-04-02T04:56:37.049 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:37 smithi179 ceph-mon[84288]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-02T04:56:37.049 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:37.049 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:37.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:37 smithi070 ceph-mon[113635]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-02T04:56:37.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:37.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:37.779 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:37 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T04:56:37.443+0000 7f0bbd3ea7c0 -1 osd.5 86 log_to_monitors true 2024-04-02T04:56:38.044 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:38 smithi179 ceph-mon[84288]: pgmap v175: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail; 42/228 objects degraded (18.421%) 2024-04-02T04:56:38.044 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:38 smithi179 ceph-mon[84288]: from='osd.5 [v2:172.21.15.179:6816/2533703533,v1:172.21.15.179:6817/2533703533]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:56:38.044 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:38 smithi179 ceph-mon[84288]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:56:38.045 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:38 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 42/228 objects degraded (18.421%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:38.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:38 smithi070 ceph-mon[113635]: pgmap v175: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 205 MiB data, 939 MiB used, 535 GiB / 536 GiB avail; 42/228 objects degraded (18.421%) 2024-04-02T04:56:38.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:38 smithi070 ceph-mon[113635]: from='osd.5 [v2:172.21.15.179:6816/2533703533,v1:172.21.15.179:6817/2533703533]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:56:38.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:38 smithi070 ceph-mon[113635]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:56:38.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:38 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 42/228 objects degraded (18.421%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:38.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 04:56:38 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T04:56:38.040+0000 7f0bae842700 -1 osd.5 86 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:56:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:39 smithi179 ceph-mon[84288]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T04:56:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:39 smithi179 ceph-mon[84288]: osdmap e89: 6 total, 5 up, 6 in 2024-04-02T04:56:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:39 smithi179 ceph-mon[84288]: from='osd.5 [v2:172.21.15.179:6816/2533703533,v1:172.21.15.179:6817/2533703533]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:56:39.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:39 smithi179 ceph-mon[84288]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:56:39.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:39 smithi070 ceph-mon[113635]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T04:56:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:39 smithi070 ceph-mon[113635]: osdmap e89: 6 total, 5 up, 6 in 2024-04-02T04:56:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:39 smithi070 ceph-mon[113635]: from='osd.5 [v2:172.21.15.179:6816/2533703533,v1:172.21.15.179:6817/2533703533]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:56:39.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:39 smithi070 ceph-mon[113635]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: osd.5 [v2:172.21.15.179:6816/2533703533,v1:172.21.15.179:6817/2533703533] boot 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: osdmap e90: 6 total, 6 up, 6 in 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: pgmap v178: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 42/228 objects degraded (18.421%) 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:40.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-02T04:56:40.324 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-02T04:56:40.325 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-02T04:56:40.325 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-02T04:56:40.325 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-02T04:56:40.325 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-02T04:56:40.325 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-02T04:56:40.325 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-02T04:56:40.325 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-02T04:56:40.325 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:40 smithi179 ceph-mon[84288]: osdmap e91: 6 total, 6 up, 6 in 2024-04-02T04:56:40.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:56:40.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: osd.5 [v2:172.21.15.179:6816/2533703533,v1:172.21.15.179:6817/2533703533] boot 2024-04-02T04:56:40.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: osdmap e90: 6 total, 6 up, 6 in 2024-04-02T04:56:40.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: pgmap v178: 65 pgs: 16 active+undersized, 13 active+undersized+degraded, 36 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 42/228 objects degraded (18.421%) 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-02T04:56:40.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-02T04:56:40.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:40 smithi070 ceph-mon[113635]: osdmap e91: 6 total, 6 up, 6 in 2024-04-02T04:56:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:41 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all osd 2024-04-02T04:56:41.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:41 smithi070 ceph-mon[113635]: Upgrade: Setting require_osd_release to 19 squid 2024-04-02T04:56:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:41 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all osd 2024-04-02T04:56:41.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:41 smithi179 ceph-mon[84288]: Upgrade: Setting require_osd_release to 19 squid 2024-04-02T04:56:42.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:42 smithi070 ceph-mon[113635]: pgmap v180: 65 pgs: 5 peering, 11 active+undersized, 8 active+undersized+degraded, 41 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 26/228 objects degraded (11.404%) 2024-04-02T04:56:42.403 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:42.403 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:42.403 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:42.403 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.mnzrwt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:56:42.403 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:42 smithi179 ceph-mon[84288]: pgmap v180: 65 pgs: 5 peering, 11 active+undersized, 8 active+undersized+degraded, 41 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 26/228 objects degraded (11.404%) 2024-04-02T04:56:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.mnzrwt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:56:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:43.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:43 smithi070 ceph-mon[113635]: Upgrade: Updating mds.cephfs.smithi070.mnzrwt 2024-04-02T04:56:43.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:43 smithi070 ceph-mon[113635]: Deploying daemon mds.cephfs.smithi070.mnzrwt on smithi070 2024-04-02T04:56:43.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:43 smithi070 ceph-mon[113635]: Health check update: Degraded data redundancy: 26/228 objects degraded (11.404%), 8 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:43 smithi179 ceph-mon[84288]: Upgrade: Updating mds.cephfs.smithi070.mnzrwt 2024-04-02T04:56:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:43 smithi179 ceph-mon[84288]: Deploying daemon mds.cephfs.smithi070.mnzrwt on smithi070 2024-04-02T04:56:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:43 smithi179 ceph-mon[84288]: Health check update: Degraded data redundancy: 26/228 objects degraded (11.404%), 8 pgs degraded (PG_DEGRADED) 2024-04-02T04:56:44.157 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:43 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[113631]: 2024-04-02T04:56:43.850+0000 7f6fcc9da700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: pgmap v181: 65 pgs: 5 peering, 1 active+undersized, 59 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 26/228 objects degraded (11.404%), 8 pgs degraded) 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: osdmap e92: 6 total, 6 up, 6 in 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: Standby daemon mds.cephfs.smithi070.zvraae assigned to filesystem cephfs as rank 0 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:56:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:44 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi070.zvraae=up:replay} 2 up:standby 2024-04-02T04:56:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: pgmap v181: 65 pgs: 5 peering, 1 active+undersized, 59 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:56:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 26/228 objects degraded (11.404%), 8 pgs degraded) 2024-04-02T04:56:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:56:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:56:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:56:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: osdmap e92: 6 total, 6 up, 6 in 2024-04-02T04:56:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: Standby daemon mds.cephfs.smithi070.zvraae assigned to filesystem cephfs as rank 0 2024-04-02T04:56:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:56:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:56:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:44 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi070.zvraae=up:replay} 2 up:standby 2024-04-02T04:56:46.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:46 smithi070 ceph-mon[113635]: pgmap v183: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 1 op/s 2024-04-02T04:56:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:46 smithi179 ceph-mon[84288]: pgmap v183: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 1 op/s 2024-04-02T04:56:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:48 smithi070 ceph-mon[113635]: pgmap v184: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-04-02T04:56:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:48 smithi179 ceph-mon[84288]: pgmap v184: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-04-02T04:56:49.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 -- 172.21.15.70:0/639216700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 msgr2=0x7fd438105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:49.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 --2- 172.21.15.70:0/639216700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 0x7fd438105890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009910 tx=0x7fd42c009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 -- 172.21.15.70:0/639216700 shutdown_connections 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 --2- 172.21.15.70:0/639216700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 0x7fd438105890 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 --2- 172.21.15.70:0/639216700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd438100e90 0x7fd438101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 -- 172.21.15.70:0/639216700 >> 172.21.15.70:0/639216700 conn(0x7fd4380fc740 msgr2=0x7fd4380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 -- 172.21.15.70:0/639216700 shutdown_connections 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 -- 172.21.15.70:0/639216700 wait complete. 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.434+0000 7fd43dfa6700 1 Processor -- start 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd43dfa6700 1 -- start start 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd43dfa6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd438100e90 0x7fd43819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd43dfa6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 0x7fd43819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd43dfa6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43819d350 con 0x7fd438101840 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd43dfa6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd438196800 con 0x7fd438100e90 2024-04-02T04:56:49.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd436ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 0x7fd43819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd436ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 0x7fd43819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55130/0 (socket says 172.21.15.70:55130) 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd436ffd700 1 -- 172.21.15.70:0/2147253176 learned_addr learned my addr 172.21.15.70:0/2147253176 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.435+0000 7fd4377fe700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd438100e90 0x7fd43819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd436ffd700 1 -- 172.21.15.70:0/2147253176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd438100e90 msgr2=0x7fd43819c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd436ffd700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd438100e90 0x7fd43819c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd436ffd700 1 -- 172.21.15.70:0/2147253176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c0095f0 con 0x7fd438101840 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd436ffd700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 0x7fd43819ccc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0092e0 tx=0x7fd42c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd434ff9700 1 -- 172.21.15.70:0/2147253176 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd42c01d070 con 0x7fd438101840 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd434ff9700 1 -- 172.21.15.70:0/2147253176 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd42c00f460 con 0x7fd438101840 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd438196a80 con 0x7fd438101840 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd434ff9700 1 -- 172.21.15.70:0/2147253176 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd42c0175d0 con 0x7fd438101840 2024-04-02T04:56:49.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.436+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd438196f70 con 0x7fd438101840 2024-04-02T04:56:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.437+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4381091e0 con 0x7fd438101840 2024-04-02T04:56:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.438+0000 7fd434ff9700 1 -- 172.21.15.70:0/2147253176 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd42c00f5d0 con 0x7fd438101840 2024-04-02T04:56:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.438+0000 7fd434ff9700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd42006dd60 0x7fd420070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.438+0000 7fd434ff9700 1 -- 172.21.15.70:0/2147253176 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fd42c08f090 con 0x7fd438101840 2024-04-02T04:56:49.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.438+0000 7fd4377fe700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd42006dd60 0x7fd420070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:49.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.439+0000 7fd4377fe700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd42006dd60 0x7fd420070220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd428009a10 tx=0x7fd428009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.443+0000 7fd434ff9700 1 -- 172.21.15.70:0/2147253176 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd42c05c0a0 con 0x7fd438101840 2024-04-02T04:56:49.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.747+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd438197190 con 0x7fd42006dd60 2024-04-02T04:56:49.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.748+0000 7fd434ff9700 1 -- 172.21.15.70:0/2147253176 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd438197190 con 0x7fd42006dd60 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd42006dd60 msgr2=0x7fd420070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd42006dd60 0x7fd420070220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd428009a10 tx=0x7fd428009190 comp rx=0 tx=0).stop 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 msgr2=0x7fd43819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 0x7fd43819ccc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0092e0 tx=0x7fd42c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 shutdown_connections 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd42006dd60 0x7fd420070220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd438100e90 0x7fd43819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 --2- 172.21.15.70:0/2147253176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd438101840 0x7fd43819ccc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 >> 172.21.15.70:0/2147253176 conn(0x7fd4380fc740 msgr2=0x7fd4380fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 shutdown_connections 2024-04-02T04:56:49.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.752+0000 7fd43dfa6700 1 -- 172.21.15.70:0/2147253176 wait complete. 2024-04-02T04:56:49.773 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:56:49.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.874+0000 7f18620cd700 1 -- 172.21.15.70:0/2487824350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 msgr2=0x7f185c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:49.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.874+0000 7f18620cd700 1 --2- 172.21.15.70:0/2487824350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 0x7f185c0ff980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f184c009910 tx=0x7f184c009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:49.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.874+0000 7f18620cd700 1 -- 172.21.15.70:0/2487824350 shutdown_connections 2024-04-02T04:56:49.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.874+0000 7f18620cd700 1 --2- 172.21.15.70:0/2487824350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f185c0fff50 0x7f185c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.874+0000 7f18620cd700 1 --2- 172.21.15.70:0/2487824350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 0x7f185c0ff980 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.875+0000 7f18620cd700 1 -- 172.21.15.70:0/2487824350 >> 172.21.15.70:0/2487824350 conn(0x7f185c074e30 msgr2=0x7f185c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:49.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.875+0000 7f18620cd700 1 -- 172.21.15.70:0/2487824350 shutdown_connections 2024-04-02T04:56:49.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.875+0000 7f18620cd700 1 -- 172.21.15.70:0/2487824350 wait complete. 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.876+0000 7f18620cd700 1 Processor -- start 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.877+0000 7f18620cd700 1 -- start start 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.877+0000 7f18620cd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 0x7f185c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.877+0000 7f18620cd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f185c0fff50 0x7f185c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.878+0000 7f18620cd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c19d250 con 0x7f185c0ff5a0 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.878+0000 7f18620cd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c196700 con 0x7f185c0fff50 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.878+0000 7f185affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f185c0fff50 0x7f185c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.878+0000 7f185affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f185c0fff50 0x7f185c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.70:59192/0 (socket says 172.21.15.70:59192) 2024-04-02T04:56:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.878+0000 7f185affd700 1 -- 172.21.15.70:0/2174900960 learned_addr learned my addr 172.21.15.70:0/2174900960 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:49.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.879+0000 7f185b7fe700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 0x7f185c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:49.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.881+0000 7f185b7fe700 1 -- 172.21.15.70:0/2174900960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f185c0fff50 msgr2=0x7f185c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:49.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.881+0000 7f185b7fe700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f185c0fff50 0x7f185c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:49.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.881+0000 7f185b7fe700 1 -- 172.21.15.70:0/2174900960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c0095f0 con 0x7f185c0ff5a0 2024-04-02T04:56:49.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.883+0000 7f185b7fe700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 0x7f185c19c680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f184c000c00 tx=0x7f184c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:49.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.884+0000 7f1858ff9700 1 -- 172.21.15.70:0/2174900960 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184c01d070 con 0x7f185c0ff5a0 2024-04-02T04:56:49.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.884+0000 7f1858ff9700 1 -- 172.21.15.70:0/2174900960 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f184c005dc0 con 0x7f185c0ff5a0 2024-04-02T04:56:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.884+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c196980 con 0x7f185c0ff5a0 2024-04-02T04:56:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.884+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c196e70 con 0x7f185c0ff5a0 2024-04-02T04:56:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.884+0000 7f1858ff9700 1 -- 172.21.15.70:0/2174900960 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184c00fe60 con 0x7f185c0ff5a0 2024-04-02T04:56:49.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.886+0000 7f1858ff9700 1 -- 172.21.15.70:0/2174900960 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f184c02b440 con 0x7f185c0ff5a0 2024-04-02T04:56:49.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.887+0000 7f1858ff9700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f184406de80 0x7f1844070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:49.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.887+0000 7f1858ff9700 1 -- 172.21.15.70:0/2174900960 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f184c08eed0 con 0x7f185c0ff5a0 2024-04-02T04:56:49.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.888+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1848005320 con 0x7f185c0ff5a0 2024-04-02T04:56:49.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.890+0000 7f185affd700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f184406de80 0x7f1844070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:49.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.890+0000 7f185affd700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f184406de80 0x7f1844070340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f185c197b70 tx=0x7f185000b480 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:49.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:49.895+0000 7f1858ff9700 1 -- 172.21.15.70:0/2174900960 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f184c05cfc0 con 0x7f185c0ff5a0 2024-04-02T04:56:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.230+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1848000bf0 con 0x7f184406de80 2024-04-02T04:56:50.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.232+0000 7f1858ff9700 1 -- 172.21.15.70:0/2174900960 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1848000bf0 con 0x7f184406de80 2024-04-02T04:56:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f184406de80 msgr2=0x7f1844070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f184406de80 0x7f1844070340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f185c197b70 tx=0x7f185000b480 comp rx=0 tx=0).stop 2024-04-02T04:56:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 msgr2=0x7f185c19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 0x7f185c19c680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f184c000c00 tx=0x7f184c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:56:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 shutdown_connections 2024-04-02T04:56:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f184406de80 0x7f1844070340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c0ff5a0 0x7f185c19c680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 --2- 172.21.15.70:0/2174900960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f185c0fff50 0x7f185c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 >> 172.21.15.70:0/2174900960 conn(0x7f185c074e30 msgr2=0x7f185c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 shutdown_connections 2024-04-02T04:56:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.234+0000 7f18620cd700 1 -- 172.21.15.70:0/2174900960 wait complete. 2024-04-02T04:56:50.332 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:50 smithi070 ceph-mon[113635]: pgmap v185: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-02T04:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.342+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/1170345790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 msgr2=0x7ff4c81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.342+0000 7ff4cd4ef700 1 --2- 172.21.15.70:0/1170345790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 0x7ff4c81070b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009910 tx=0x7ff4b8009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.342+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/1170345790 shutdown_connections 2024-04-02T04:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.342+0000 7ff4cd4ef700 1 --2- 172.21.15.70:0/1170345790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8068580 0x7ff4c80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.342+0000 7ff4cd4ef700 1 --2- 172.21.15.70:0/1170345790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 0x7ff4c81070b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.342+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/1170345790 >> 172.21.15.70:0/1170345790 conn(0x7ff4c8075670 msgr2=0x7ff4c8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.343+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/1170345790 shutdown_connections 2024-04-02T04:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.343+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/1170345790 wait complete. 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.343+0000 7ff4cd4ef700 1 Processor -- start 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4cd4ef700 1 -- start start 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4cd4ef700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8068580 0x7ff4c819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4cd4ef700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 0x7ff4c819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4cd4ef700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c819d240 con 0x7ff4c8106cd0 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4cd4ef700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c81966f0 con 0x7ff4c8068580 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4c67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 0x7ff4c819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4c67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 0x7ff4c819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55144/0 (socket says 172.21.15.70:55144) 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4c67fc700 1 -- 172.21.15.70:0/3069374018 learned_addr learned my addr 172.21.15.70:0/3069374018 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4c6ffd700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8068580 0x7ff4c819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4c67fc700 1 -- 172.21.15.70:0/3069374018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8068580 msgr2=0x7ff4c819c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4c67fc700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8068580 0x7ff4c819c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.344+0000 7ff4c67fc700 1 -- 172.21.15.70:0/3069374018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80095f0 con 0x7ff4c8106cd0 2024-04-02T04:56:50.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.345+0000 7ff4c67fc700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 0x7ff4c819cbb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00d900 tx=0x7ff4bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:50.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.345+0000 7ff4affff700 1 -- 172.21.15.70:0/3069374018 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc004a00 con 0x7ff4c8106cd0 2024-04-02T04:56:50.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.345+0000 7ff4affff700 1 -- 172.21.15.70:0/3069374018 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4bc0054e0 con 0x7ff4c8106cd0 2024-04-02T04:56:50.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.345+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c81969d0 con 0x7ff4c8106cd0 2024-04-02T04:56:50.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.345+0000 7ff4affff700 1 -- 172.21.15.70:0/3069374018 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc009d60 con 0x7ff4c8106cd0 2024-04-02T04:56:50.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.345+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c8196f20 con 0x7ff4c8106cd0 2024-04-02T04:56:50.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.346+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c81091d0 con 0x7ff4c8106cd0 2024-04-02T04:56:50.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.347+0000 7ff4affff700 1 -- 172.21.15.70:0/3069374018 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff4bc005020 con 0x7ff4c8106cd0 2024-04-02T04:56:50.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.347+0000 7ff4affff700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff4b006dd60 0x7ff4b0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:50.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.347+0000 7ff4affff700 1 -- 172.21.15.70:0/3069374018 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6683+0+0 (secure 0 0 0) 0x7ff4bc08e0d0 con 0x7ff4c8106cd0 2024-04-02T04:56:50.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.347+0000 7ff4c6ffd700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff4b006dd60 0x7ff4b0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:50.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.348+0000 7ff4c6ffd700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff4b006dd60 0x7ff4b0070220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800b5c0 tx=0x7ff4b801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:50.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.351+0000 7ff4affff700 1 -- 172.21.15.70:0/3069374018 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff4bc05b200 con 0x7ff4c8106cd0 2024-04-02T04:56:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:50 smithi179 ceph-mon[84288]: pgmap v185: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-02T04:56:50.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.633+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4c8197190 con 0x7ff4b006dd60 2024-04-02T04:56:50.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.651+0000 7ff4affff700 1 -- 172.21.15.70:0/3069374018 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff4c8197190 con 0x7ff4b006dd60 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (6m) 115s ago 17m 26.2M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (17m) 115s ago 17m 8678k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (16m) 15s ago 16m 8493k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (3m) 115s ago 17m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (3m) 15s ago 16m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (6m) 115s ago 17m 76.8M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (11m) 115s ago 11m 151M - 18.2.1 5be31c24972a 8f9abb26a8fc 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (11m) 115s ago 11m 20.2M - 18.2.1 5be31c24972a 33e75a0368cc 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (11m) 15s ago 11m 19.1M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (11m) 15s ago 11m 20.4M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (8m) 115s ago 18m 504M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:56:50.656 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (7m) 15s ago 16m 451M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (4m) 115s ago 19m 49.2M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (4m) 15s ago 16m 44.4M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 115s ago 17m 15.5M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (7m) 15s ago 16m 17.3M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (3m) 115s ago 15m 141M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (2m) 115s ago 15m 157M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (2m) 115s ago 14m 11.8M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (85s) 15s ago 14m 97.7M 2240M 19.0.0-1979-g6cec793d 78f830190839 f717fd6c8306 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (52s) 15s ago 13m 109M 2240M 19.0.0-1979-g6cec793d 78f830190839 2e5bbf3235ff 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (18s) 15s ago 13m 15.1M 2240M 19.0.0-1979-g6cec793d 78f830190839 71351459ac70 2024-04-02T04:56:50.657 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (6m) 115s ago 17m 62.9M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:56:50.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff4b006dd60 msgr2=0x7ff4b0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:50.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff4b006dd60 0x7ff4b0070220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800b5c0 tx=0x7ff4b801a040 comp rx=0 tx=0).stop 2024-04-02T04:56:50.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 msgr2=0x7ff4c819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:50.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 0x7ff4c819cbb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00d900 tx=0x7ff4bc00dcc0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 shutdown_connections 2024-04-02T04:56:50.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff4b006dd60 0x7ff4b0070220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4c8068580 0x7ff4c819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 --2- 172.21.15.70:0/3069374018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c8106cd0 0x7ff4c819cbb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 >> 172.21.15.70:0/3069374018 conn(0x7ff4c8075670 msgr2=0x7ff4c80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:50.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 shutdown_connections 2024-04-02T04:56:50.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.653+0000 7ff4cd4ef700 1 -- 172.21.15.70:0/3069374018 wait complete. 2024-04-02T04:56:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.777+0000 7ff2f6248700 1 -- 172.21.15.70:0/1279098876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 msgr2=0x7ff2f0105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.777+0000 7ff2f6248700 1 --2- 172.21.15.70:0/1279098876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 0x7ff2f0105730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009910 tx=0x7ff2e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.778+0000 7ff2f6248700 1 -- 172.21.15.70:0/1279098876 shutdown_connections 2024-04-02T04:56:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.778+0000 7ff2f6248700 1 --2- 172.21.15.70:0/1279098876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2f00ff340 0x7ff2f00ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.778+0000 7ff2f6248700 1 --2- 172.21.15.70:0/1279098876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 0x7ff2f0105730 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.778+0000 7ff2f6248700 1 -- 172.21.15.70:0/1279098876 >> 172.21.15.70:0/1279098876 conn(0x7ff2f0074e30 msgr2=0x7ff2f0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.778+0000 7ff2f6248700 1 -- 172.21.15.70:0/1279098876 shutdown_connections 2024-04-02T04:56:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.778+0000 7ff2f6248700 1 -- 172.21.15.70:0/1279098876 wait complete. 2024-04-02T04:56:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.778+0000 7ff2f6248700 1 Processor -- start 2024-04-02T04:56:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2f6248700 1 -- start start 2024-04-02T04:56:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2f6248700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2f00ff340 0x7ff2f019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2f6248700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 0x7ff2f019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2f6248700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f019f420 con 0x7ff2f0105350 2024-04-02T04:56:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2f6248700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01988d0 con 0x7ff2f00ff340 2024-04-02T04:56:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2eeffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 0x7ff2f019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2eeffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 0x7ff2f019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55168/0 (socket says 172.21.15.70:55168) 2024-04-02T04:56:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2eeffd700 1 -- 172.21.15.70:0/182084021 learned_addr learned my addr 172.21.15.70:0/182084021 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2ef7fe700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2f00ff340 0x7ff2f019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2eeffd700 1 -- 172.21.15.70:0/182084021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2f00ff340 msgr2=0x7ff2f019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2eeffd700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2f00ff340 0x7ff2f019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.779+0000 7ff2eeffd700 1 -- 172.21.15.70:0/182084021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00095f0 con 0x7ff2f0105350 2024-04-02T04:56:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.780+0000 7ff2eeffd700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 0x7ff2f019ed90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400e940 tx=0x7ff2e400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.780+0000 7ff2ecff9700 1 -- 172.21.15.70:0/182084021 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e400cba0 con 0x7ff2f0105350 2024-04-02T04:56:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.780+0000 7ff2ecff9700 1 -- 172.21.15.70:0/182084021 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2e400ee90 con 0x7ff2f0105350 2024-04-02T04:56:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.780+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f0198bb0 con 0x7ff2f0105350 2024-04-02T04:56:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.780+0000 7ff2ecff9700 1 -- 172.21.15.70:0/182084021 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e40055d0 con 0x7ff2f0105350 2024-04-02T04:56:50.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.780+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f01990b0 con 0x7ff2f0105350 2024-04-02T04:56:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.781+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f0107850 con 0x7ff2f0105350 2024-04-02T04:56:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.782+0000 7ff2ecff9700 1 -- 172.21.15.70:0/182084021 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff2e4004750 con 0x7ff2f0105350 2024-04-02T04:56:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.782+0000 7ff2ecff9700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff2d806dd60 0x7ff2d8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.782+0000 7ff2ecff9700 1 -- 172.21.15.70:0/182084021 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6683+0+0 (secure 0 0 0) 0x7ff2e4014070 con 0x7ff2f0105350 2024-04-02T04:56:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.782+0000 7ff2ef7fe700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff2d806dd60 0x7ff2d8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.783+0000 7ff2ef7fe700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff2d806dd60 0x7ff2d8070220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000b760 tx=0x7ff2e0005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:50.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:50.790+0000 7ff2ecff9700 1 -- 172.21.15.70:0/182084021 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff2e405b150 con 0x7ff2f0105350 2024-04-02T04:56:51.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.119+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff2f004f2e0 con 0x7ff2f0105350 2024-04-02T04:56:51.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.120+0000 7ff2ecff9700 1 -- 172.21.15.70:0/182084021 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7ff2e405ace0 con 0x7ff2f0105350 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:56:51.125 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-02T04:56:51.126 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:56:51.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:56:51.126 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:56:51.126 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-02T04:56:51.126 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 11 2024-04-02T04:56:51.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:56:51.126 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:56:51.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff2d806dd60 msgr2=0x7ff2d8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff2d806dd60 0x7ff2d8070220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000b760 tx=0x7ff2e0005a90 comp rx=0 tx=0).stop 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 msgr2=0x7ff2f019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 0x7ff2f019ed90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400e940 tx=0x7ff2e400ec50 comp rx=0 tx=0).stop 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 shutdown_connections 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff2d806dd60 0x7ff2d8070220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2f00ff340 0x7ff2f019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 --2- 172.21.15.70:0/182084021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f0105350 0x7ff2f019ed90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.123+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 >> 172.21.15.70:0/182084021 conn(0x7ff2f0074e30 msgr2=0x7ff2f00fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.124+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 shutdown_connections 2024-04-02T04:56:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.124+0000 7ff2f6248700 1 -- 172.21.15.70:0/182084021 wait complete. 2024-04-02T04:56:51.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.239+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/2556652741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 msgr2=0x7fa6ac101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.239+0000 7fa6b2cd6700 1 --2- 172.21.15.70:0/2556652741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 0x7fa6ac101270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009910 tx=0x7fa6a0009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:51.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.239+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/2556652741 shutdown_connections 2024-04-02T04:56:51.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.239+0000 7fa6b2cd6700 1 --2- 172.21.15.70:0/2556652741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac101840 0x7fa6ac105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.239+0000 7fa6b2cd6700 1 --2- 172.21.15.70:0/2556652741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 0x7fa6ac101270 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.239+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/2556652741 >> 172.21.15.70:0/2556652741 conn(0x7fa6ac0fc740 msgr2=0x7fa6ac0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:51.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.239+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/2556652741 shutdown_connections 2024-04-02T04:56:51.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.239+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/2556652741 wait complete. 2024-04-02T04:56:51.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.240+0000 7fa6b2cd6700 1 Processor -- start 2024-04-02T04:56:51.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.240+0000 7fa6b2cd6700 1 -- start start 2024-04-02T04:56:51.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.240+0000 7fa6b2cd6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 0x7fa6ac19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.240+0000 7fa6b2cd6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac101840 0x7fa6ac19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.240+0000 7fa6b2cd6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac19d230 con 0x7fa6ac100e90 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.240+0000 7fa6b2cd6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac1966e0 con 0x7fa6ac101840 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.240+0000 7fa6b0a72700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 0x7fa6ac19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6b0a72700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 0x7fa6ac19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55176/0 (socket says 172.21.15.70:55176) 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6b0a72700 1 -- 172.21.15.70:0/3945580027 learned_addr learned my addr 172.21.15.70:0/3945580027 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6abfff700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac101840 0x7fa6ac19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6b0a72700 1 -- 172.21.15.70:0/3945580027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac101840 msgr2=0x7fa6ac19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6b0a72700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac101840 0x7fa6ac19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6b0a72700 1 -- 172.21.15.70:0/3945580027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a00095f0 con 0x7fa6ac100e90 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6b0a72700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 0x7fa6ac19c660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006010 tx=0x7fa6a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6a9ffb700 1 -- 172.21.15.70:0/3945580027 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6a001d070 con 0x7fa6ac100e90 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6a9ffb700 1 -- 172.21.15.70:0/3945580027 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6a0005dc0 con 0x7fa6ac100e90 2024-04-02T04:56:51.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6a9ffb700 1 -- 172.21.15.70:0/3945580027 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6a0017400 con 0x7fa6ac100e90 2024-04-02T04:56:51.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.241+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac196960 con 0x7fa6ac100e90 2024-04-02T04:56:51.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.242+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac196e00 con 0x7fa6ac100e90 2024-04-02T04:56:51.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.243+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac109230 con 0x7fa6ac100e90 2024-04-02T04:56:51.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.244+0000 7fa6a9ffb700 1 -- 172.21.15.70:0/3945580027 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa6a0003a60 con 0x7fa6ac100e90 2024-04-02T04:56:51.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.244+0000 7fa6a9ffb700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa69c06dd60 0x7fa69c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:51.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.244+0000 7fa6a9ffb700 1 -- 172.21.15.70:0/3945580027 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6683+0+0 (secure 0 0 0) 0x7fa6a008f040 con 0x7fa6ac100e90 2024-04-02T04:56:51.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.244+0000 7fa6abfff700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa69c06dd60 0x7fa69c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:51.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.245+0000 7fa6abfff700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa69c06dd60 0x7fa69c070220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac197b50 tx=0x7fa6980073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:51.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.253+0000 7fa6a9ffb700 1 -- 172.21.15.70:0/3945580027 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa6a005c050 con 0x7fa6ac100e90 2024-04-02T04:56:51.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:51 smithi070 ceph-mon[113635]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:51 smithi070 ceph-mon[113635]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:51 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/182084021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:51 smithi179 ceph-mon[84288]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:51 smithi179 ceph-mon[84288]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:51.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:51.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:51 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/182084021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:56:51.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.573+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa6ac197c90 con 0x7fa6ac100e90 2024-04-02T04:56:51.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.574+0000 7fa6a9ffb700 1 -- 172.21.15.70:0/3945580027 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fa6a005bbe0 con 0x7fa6ac100e90 2024-04-02T04:56:51.578 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-04-02T04:56:51.578 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:56:51.578 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:56:51.578 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:56:51.578 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:56:51.578 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 16 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:56:43.855740+0000 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:56:51.579 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 92 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14480} 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:56:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{0:14480} state up:replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:56:51.581 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:56:51.581 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:56:51.581 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:56:51.581 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:56:51.581 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:24253} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:56:51.581 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:56:51.581 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa69c06dd60 msgr2=0x7fa69c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa69c06dd60 0x7fa69c070220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac197b50 tx=0x7fa6980073d0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 msgr2=0x7fa6ac19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 0x7fa6ac19c660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006010 tx=0x7fa6a000bd70 comp rx=0 tx=0).stop 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 shutdown_connections 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa69c06dd60 0x7fa69c070220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ac100e90 0x7fa6ac19c660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 --2- 172.21.15.70:0/3945580027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac101840 0x7fa6ac19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 >> 172.21.15.70:0/3945580027 conn(0x7fa6ac0fc740 msgr2=0x7fa6ac0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:51.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 shutdown_connections 2024-04-02T04:56:51.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.576+0000 7fa6b2cd6700 1 -- 172.21.15.70:0/3945580027 wait complete. 2024-04-02T04:56:51.583 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-04-02T04:56:51.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 -- 172.21.15.70:0/940315919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 msgr2=0x7f06a80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 --2- 172.21.15.70:0/940315919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 0x7f06a80689e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f069c009910 tx=0x7f069c009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:51.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 -- 172.21.15.70:0/940315919 shutdown_connections 2024-04-02T04:56:51.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 --2- 172.21.15.70:0/940315919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 0x7f06a80689e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 --2- 172.21.15.70:0/940315919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a8106cd0 0x7f06a81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 -- 172.21.15.70:0/940315919 >> 172.21.15.70:0/940315919 conn(0x7f06a8075670 msgr2=0x7f06a8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:51.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 -- 172.21.15.70:0/940315919 shutdown_connections 2024-04-02T04:56:51.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 -- 172.21.15.70:0/940315919 wait complete. 2024-04-02T04:56:51.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.684+0000 7f06adb6c700 1 Processor -- start 2024-04-02T04:56:51.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06adb6c700 1 -- start start 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06adb6c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 0x7f06a80ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06adb6c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a8106cd0 0x7f06a8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06adb6c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8100770 con 0x7f06a8068580 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06adb6c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a81008b0 con 0x7f06a8106cd0 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06a77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 0x7f06a80ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06a77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 0x7f06a80ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55202/0 (socket says 172.21.15.70:55202) 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06a77fe700 1 -- 172.21.15.70:0/3499228457 learned_addr learned my addr 172.21.15.70:0/3499228457 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06a6ffd700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a8106cd0 0x7f06a8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06a77fe700 1 -- 172.21.15.70:0/3499228457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a8106cd0 msgr2=0x7f06a8100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06a77fe700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a8106cd0 0x7f06a8100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.685+0000 7f06a77fe700 1 -- 172.21.15.70:0/3499228457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0095f0 con 0x7f06a8068580 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.686+0000 7f06a77fe700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 0x7f06a80ffcf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f069800d8d0 tx=0x7f069800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:51.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.686+0000 7f06a4ff9700 1 -- 172.21.15.70:0/3499228457 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06980098a0 con 0x7f06a8068580 2024-04-02T04:56:51.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.686+0000 7f06a4ff9700 1 -- 172.21.15.70:0/3499228457 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0698010460 con 0x7f06a8068580 2024-04-02T04:56:51.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.686+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a81a2a30 con 0x7f06a8068580 2024-04-02T04:56:51.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.686+0000 7f06a4ff9700 1 -- 172.21.15.70:0/3499228457 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f069800f5f0 con 0x7f06a8068580 2024-04-02T04:56:51.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.686+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a81a2e40 con 0x7f06a8068580 2024-04-02T04:56:51.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.687+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a81091d0 con 0x7f06a8068580 2024-04-02T04:56:51.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.688+0000 7f06a4ff9700 1 -- 172.21.15.70:0/3499228457 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f0698009a00 con 0x7f06a8068580 2024-04-02T04:56:51.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.688+0000 7f06a4ff9700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f069006dd60 0x7f0690070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:51.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.689+0000 7f06a4ff9700 1 -- 172.21.15.70:0/3499228457 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f069808d9b0 con 0x7f06a8068580 2024-04-02T04:56:51.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.689+0000 7f06a6ffd700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f069006dd60 0x7f0690070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:51.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.689+0000 7f06a6ffd700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f069006dd60 0x7f0690070220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f06a81014e0 tx=0x7f069c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:51.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.693+0000 7f06a4ff9700 1 -- 172.21.15.70:0/3499228457 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0698056f10 con 0x7f06a8068580 2024-04-02T04:56:51.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.919+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06a802cce0 con 0x7f069006dd60 2024-04-02T04:56:51.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.921+0000 7f06a4ff9700 1 -- 172.21.15.70:0/3499228457 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f06a802cce0 con 0x7f069006dd60 2024-04-02T04:56:51.925 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:56:51.925 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:56:51.925 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:56:51.925 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:56:51.925 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd", 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "12/23 daemons upgraded", 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mds daemons", 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:56:51.926 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:56:51.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.923+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f069006dd60 msgr2=0x7f0690070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.923+0000 7f06adb6c700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f069006dd60 0x7f0690070220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f06a81014e0 tx=0x7f069c01a040 comp rx=0 tx=0).stop 2024-04-02T04:56:51.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.923+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 msgr2=0x7f06a80ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:51.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.923+0000 7f06adb6c700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 0x7f06a80ffcf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f069800d8d0 tx=0x7f069800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.924+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 shutdown_connections 2024-04-02T04:56:51.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.924+0000 7f06adb6c700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f069006dd60 0x7f0690070220 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.924+0000 7f06adb6c700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a8068580 0x7f06a80ffcf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.924+0000 7f06adb6c700 1 --2- 172.21.15.70:0/3499228457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a8106cd0 0x7f06a8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:51.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.924+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 >> 172.21.15.70:0/3499228457 conn(0x7f06a8075670 msgr2=0x7f06a80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:51.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.924+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 shutdown_connections 2024-04-02T04:56:51.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:51.924+0000 7f06adb6c700 1 -- 172.21.15.70:0/3499228457 wait complete. 2024-04-02T04:56:52.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 -- 172.21.15.70:0/386087198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 msgr2=0x7f8a2c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 --2- 172.21.15.70:0/386087198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 0x7f8a2c100a90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8a28009910 tx=0x7f8a28009c20 comp rx=0 tx=0).stop 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 -- 172.21.15.70:0/386087198 shutdown_connections 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 --2- 172.21.15.70:0/386087198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a2c101060 0x7f8a2c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 --2- 172.21.15.70:0/386087198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 0x7f8a2c100a90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 -- 172.21.15.70:0/386087198 >> 172.21.15.70:0/386087198 conn(0x7f8a2c074e30 msgr2=0x7f8a2c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 -- 172.21.15.70:0/386087198 shutdown_connections 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 -- 172.21.15.70:0/386087198 wait complete. 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 Processor -- start 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.031+0000 7f8a34254700 1 -- start start 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a34254700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 0x7f8a2c101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a34254700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a2c101060 0x7f8a2c101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a34254700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c1022d0 con 0x7f8a2c1006b0 2024-04-02T04:56:52.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a34254700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c102410 con 0x7f8a2c101060 2024-04-02T04:56:52.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a31ff0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 0x7f8a2c101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:52.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a31ff0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 0x7f8a2c101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55218/0 (socket says 172.21.15.70:55218) 2024-04-02T04:56:52.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a31ff0700 1 -- 172.21.15.70:0/2328117884 learned_addr learned my addr 172.21.15.70:0/2328117884 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:56:52.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a317ef700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a2c101060 0x7f8a2c101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:52.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a31ff0700 1 -- 172.21.15.70:0/2328117884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a2c101060 msgr2=0x7f8a2c101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:52.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a31ff0700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a2c101060 0x7f8a2c101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:52.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a31ff0700 1 -- 172.21.15.70:0/2328117884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280095f0 con 0x7f8a2c1006b0 2024-04-02T04:56:52.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.032+0000 7f8a31ff0700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 0x7f8a2c101850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005f50 tx=0x7f8a2800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:52.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.033+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2328117884 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a2801d070 con 0x7f8a2c1006b0 2024-04-02T04:56:52.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.033+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2328117884 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a2800b610 con 0x7f8a2c1006b0 2024-04-02T04:56:52.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.033+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2328117884 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a28022470 con 0x7f8a2c1006b0 2024-04-02T04:56:52.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.033+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c102690 con 0x7f8a2c1006b0 2024-04-02T04:56:52.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.033+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2c1a2d50 con 0x7f8a2c1006b0 2024-04-02T04:56:52.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.034+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2c109130 con 0x7f8a2c1006b0 2024-04-02T04:56:52.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.035+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2328117884 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f8a28003780 con 0x7f8a2c1006b0 2024-04-02T04:56:52.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.035+0000 7f8a22ffd700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8a1806dd60 0x7f8a18070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:56:52.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.036+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2328117884 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f8a2808ef90 con 0x7f8a2c1006b0 2024-04-02T04:56:52.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.036+0000 7f8a317ef700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8a1806dd60 0x7f8a18070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:56:52.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.036+0000 7f8a317ef700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8a1806dd60 0x7f8a18070220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c103040 tx=0x7f8a1c009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:56:52.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.040+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2328117884 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8a28058470 con 0x7f8a2c1006b0 2024-04-02T04:56:52.429 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:52 smithi070 ceph-mon[113635]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:52.429 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:52 smithi070 ceph-mon[113635]: pgmap v186: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-02T04:56:52.429 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:52 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] up:boot 2024-04-02T04:56:52.429 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:52 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi070.zvraae=up:replay} 3 up:standby 2024-04-02T04:56:52.429 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:52 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:56:52.429 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:52 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:52.429 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:52 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3945580027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:56:52.430 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 filesystem is degraded 2024-04-02T04:56:52.430 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-02T04:56:52.430 INFO:teuthology.orchestra.run.smithi070.stdout: fs cephfs is degraded 2024-04-02T04:56:52.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.423+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8a2c102e20 con 0x7f8a2c1006b0 2024-04-02T04:56:52.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.424+0000 7f8a22ffd700 1 -- 172.21.15.70:0/2328117884 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7f8a2805bb30 con 0x7f8a2c1006b0 2024-04-02T04:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8a1806dd60 msgr2=0x7f8a18070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8a1806dd60 0x7f8a18070220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c103040 tx=0x7f8a1c009260 comp rx=0 tx=0).stop 2024-04-02T04:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 msgr2=0x7f8a2c101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 0x7f8a2c101850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005f50 tx=0x7f8a2800f690 comp rx=0 tx=0).stop 2024-04-02T04:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 shutdown_connections 2024-04-02T04:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8a1806dd60 0x7f8a18070220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a2c1006b0 0x7f8a2c101850 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:52.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 --2- 172.21.15.70:0/2328117884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a2c101060 0x7f8a2c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:56:52.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 >> 172.21.15.70:0/2328117884 conn(0x7f8a2c074e30 msgr2=0x7f8a2c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:56:52.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 shutdown_connections 2024-04-02T04:56:52.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:56:52.427+0000 7f8a34254700 1 -- 172.21.15.70:0/2328117884 wait complete. 2024-04-02T04:56:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:52 smithi179 ceph-mon[84288]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:52 smithi179 ceph-mon[84288]: pgmap v186: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-02T04:56:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:52 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] up:boot 2024-04-02T04:56:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:52 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi070.zvraae=up:replay} 3 up:standby 2024-04-02T04:56:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:52 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:56:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:52 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:52 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3945580027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:56:53.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:53 smithi070 ceph-mon[113635]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:53.446 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:53 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2328117884' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:56:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:53 smithi179 ceph-mon[84288]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:56:53.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:53 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2328117884' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:56:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:54 smithi070 ceph-mon[113635]: pgmap v187: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-02T04:56:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:54 smithi070 ceph-mon[113635]: reconnect by client.14532 192.168.0.1:0/3290270869 after 0 2024-04-02T04:56:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:54 smithi070 ceph-mon[113635]: reconnect by client.24309 192.168.0.1:0/2957635722 after 0.000999972 2024-04-02T04:56:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:54 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:reconnect 2024-04-02T04:56:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:54 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi070.zvraae=up:reconnect} 3 up:standby 2024-04-02T04:56:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:54 smithi179 ceph-mon[84288]: pgmap v187: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-02T04:56:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:54 smithi179 ceph-mon[84288]: reconnect by client.14532 192.168.0.1:0/3290270869 after 0 2024-04-02T04:56:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:54 smithi179 ceph-mon[84288]: reconnect by client.24309 192.168.0.1:0/2957635722 after 0.000999972 2024-04-02T04:56:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:54 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:reconnect 2024-04-02T04:56:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:54 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi070.zvraae=up:reconnect} 3 up:standby 2024-04-02T04:56:55.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:55 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:rejoin 2024-04-02T04:56:55.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:55 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi070.zvraae=up:rejoin} 3 up:standby 2024-04-02T04:56:55.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:55 smithi070 ceph-mon[113635]: daemon mds.cephfs.smithi070.zvraae is now active in filesystem cephfs as rank 0 2024-04-02T04:56:55.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:55 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:rejoin 2024-04-02T04:56:55.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:55 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi070.zvraae=up:rejoin} 3 up:standby 2024-04-02T04:56:55.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:55 smithi179 ceph-mon[84288]: daemon mds.cephfs.smithi070.zvraae is now active in filesystem cephfs as rank 0 2024-04-02T04:56:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:56 smithi070 ceph-mon[113635]: pgmap v188: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 90 B/s wr, 5 op/s 2024-04-02T04:56:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:56 smithi070 ceph-mon[113635]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:56:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:56 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:56:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:56 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:active 2024-04-02T04:56:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:56 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi070.zvraae=up:active} 3 up:standby 2024-04-02T04:56:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:56 smithi179 ceph-mon[84288]: pgmap v188: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 90 B/s wr, 5 op/s 2024-04-02T04:56:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:56 smithi179 ceph-mon[84288]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:56:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:56 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:56:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:56 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.70:6828/2586361462,v1:172.21.15.70:6829/2586361462] up:active 2024-04-02T04:56:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:56 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi070.zvraae=up:active} 3 up:standby 2024-04-02T04:56:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:56:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:56:58 smithi070 ceph-mon[113635]: pgmap v189: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:56:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:56:58 smithi179 ceph-mon[84288]: pgmap v189: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:57:00.479 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:00 smithi070 ceph-mon[113635]: pgmap v190: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 255 B/s wr, 6 op/s 2024-04-02T04:57:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:00 smithi179 ceph-mon[84288]: pgmap v190: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 255 B/s wr, 6 op/s 2024-04-02T04:57:02.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:02 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:02.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:02 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:02.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:02 smithi179 ceph-mon[84288]: pgmap v191: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 7.5 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-02T04:57:02.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:02 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:02 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:02 smithi070 ceph-mon[113635]: pgmap v191: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 7.5 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-02T04:57:03.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:03.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:03 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:03.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:03.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:03 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:04.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:04 smithi070 ceph-mon[113635]: pgmap v192: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-02T04:57:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:04 smithi179 ceph-mon[84288]: pgmap v192: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: pgmap v193: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 159 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:06.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:06.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:06.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:06 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: pgmap v193: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 159 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-02T04:57:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:57:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:06.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:06.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:06.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:06.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:06.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:06 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:08.555 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:08 smithi179 ceph-mon[84288]: pgmap v194: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-02T04:57:08.556 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:08 smithi179 ceph-mon[84288]: Upgrade: Updating mds.cephfs.smithi070.zvraae 2024-04-02T04:57:08.556 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:08 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:08.556 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:08 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.zvraae", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:57:08.556 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:08 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:08.556 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:08 smithi179 ceph-mon[84288]: Deploying daemon mds.cephfs.smithi070.zvraae on smithi070 2024-04-02T04:57:08.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:08 smithi070 ceph-mon[113635]: pgmap v194: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-02T04:57:08.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:08 smithi070 ceph-mon[113635]: Upgrade: Updating mds.cephfs.smithi070.zvraae 2024-04-02T04:57:08.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:08 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:08.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:08 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.zvraae", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:57:08.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:08 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:08.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:08 smithi070 ceph-mon[113635]: Deploying daemon mds.cephfs.smithi070.zvraae on smithi070 2024-04-02T04:57:09.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:09 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[113631]: 2024-04-02T04:57:09.381+0000 7f6fcc9da700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:57:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:10 smithi179 ceph-mon[84288]: pgmap v195: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 4.5 KiB/s wr, 3 op/s 2024-04-02T04:57:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:10 smithi179 ceph-mon[84288]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:57:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:10 smithi179 ceph-mon[84288]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:57:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:10 smithi179 ceph-mon[84288]: osdmap e93: 6 total, 6 up, 6 in 2024-04-02T04:57:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:10 smithi179 ceph-mon[84288]: Standby daemon mds.cephfs.smithi179.mzfimf assigned to filesystem cephfs as rank 0 2024-04-02T04:57:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:10 smithi179 ceph-mon[84288]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:57:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:10 smithi179 ceph-mon[84288]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:57:10.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:10 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi179.mzfimf=up:replay} 2 up:standby 2024-04-02T04:57:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:10 smithi070 ceph-mon[113635]: pgmap v195: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 4.5 KiB/s wr, 3 op/s 2024-04-02T04:57:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:10 smithi070 ceph-mon[113635]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:57:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:10 smithi070 ceph-mon[113635]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:57:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:10 smithi070 ceph-mon[113635]: osdmap e93: 6 total, 6 up, 6 in 2024-04-02T04:57:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:10 smithi070 ceph-mon[113635]: Standby daemon mds.cephfs.smithi179.mzfimf assigned to filesystem cephfs as rank 0 2024-04-02T04:57:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:10 smithi070 ceph-mon[113635]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:57:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:10 smithi070 ceph-mon[113635]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:57:10.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:10 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi179.mzfimf=up:replay} 2 up:standby 2024-04-02T04:57:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:12 smithi070 ceph-mon[113635]: pgmap v197: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 1 op/s 2024-04-02T04:57:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:12 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:57:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:12 smithi179 ceph-mon[84288]: pgmap v197: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 1 op/s 2024-04-02T04:57:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:12 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:57:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:14 smithi070 ceph-mon[113635]: pgmap v198: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 8.6 MiB/s rd, 3 op/s 2024-04-02T04:57:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:14 smithi179 ceph-mon[84288]: pgmap v198: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 8.6 MiB/s rd, 3 op/s 2024-04-02T04:57:16.466 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:16 smithi179 ceph-mon[84288]: pgmap v199: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-02T04:57:16.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:16 smithi070 ceph-mon[113635]: pgmap v199: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-02T04:57:18.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:18 smithi179 ceph-mon[84288]: pgmap v200: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-02T04:57:18.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:18 smithi070 ceph-mon[113635]: pgmap v200: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-02T04:57:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:19 smithi070 ceph-mon[113635]: reconnect by client.24309 192.168.0.1:0/2957635722 after 0 2024-04-02T04:57:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:19 smithi070 ceph-mon[113635]: reconnect by client.14532 192.168.0.1:0/3290270869 after 0.000999978 2024-04-02T04:57:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:19 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:reconnect 2024-04-02T04:57:19.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:19 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi179.mzfimf=up:reconnect} 2 up:standby 2024-04-02T04:57:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:19 smithi179 ceph-mon[84288]: reconnect by client.24309 192.168.0.1:0/2957635722 after 0 2024-04-02T04:57:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:19 smithi179 ceph-mon[84288]: reconnect by client.14532 192.168.0.1:0/3290270869 after 0.000999978 2024-04-02T04:57:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:19 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:reconnect 2024-04-02T04:57:19.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:19 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi179.mzfimf=up:reconnect} 2 up:standby 2024-04-02T04:57:20.439 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:20 smithi070 ceph-mon[113635]: pgmap v201: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-02T04:57:20.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:20 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:rejoin 2024-04-02T04:57:20.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:20 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi179.mzfimf=up:rejoin} 2 up:standby 2024-04-02T04:57:20.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:20 smithi070 ceph-mon[113635]: daemon mds.cephfs.smithi179.mzfimf is now active in filesystem cephfs as rank 0 2024-04-02T04:57:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:20 smithi179 ceph-mon[84288]: pgmap v201: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-02T04:57:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:20 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:rejoin 2024-04-02T04:57:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:20 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi179.mzfimf=up:rejoin} 2 up:standby 2024-04-02T04:57:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:20 smithi179 ceph-mon[84288]: daemon mds.cephfs.smithi179.mzfimf is now active in filesystem cephfs as rank 0 2024-04-02T04:57:21.519 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:21 smithi179 ceph-mon[84288]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:57:21.520 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:21 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:57:21.520 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:21 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:active 2024-04-02T04:57:21.520 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:21 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi179.mzfimf=up:active} 2 up:standby 2024-04-02T04:57:21.520 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:21.520 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:21.520 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:21.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:21 smithi070 ceph-mon[113635]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:57:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:21 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:57:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:21 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] up:active 2024-04-02T04:57:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:21 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi179.mzfimf=up:active} 2 up:standby 2024-04-02T04:57:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:22.405 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:22 smithi070 ceph-mon[113635]: pgmap v202: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 87 B/s wr, 7 op/s 2024-04-02T04:57:22.405 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:22 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.70:6828/3294303398,v1:172.21.15.70:6829/3294303398] up:boot 2024-04-02T04:57:22.405 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:22 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi179.mzfimf=up:active} 3 up:standby 2024-04-02T04:57:22.405 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:57:22.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.557+0000 7fad1dfec700 1 -- 172.21.15.70:0/3613156634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180fff50 msgr2=0x7fad18105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:22.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.557+0000 7fad1dfec700 1 --2- 172.21.15.70:0/3613156634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180fff50 0x7fad18105840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009910 tx=0x7fad0c009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:22.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.557+0000 7fad1dfec700 1 -- 172.21.15.70:0/3613156634 shutdown_connections 2024-04-02T04:57:22.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.557+0000 7fad1dfec700 1 --2- 172.21.15.70:0/3613156634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180fff50 0x7fad18105840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.557+0000 7fad1dfec700 1 --2- 172.21.15.70:0/3613156634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad180ff5a0 0x7fad180ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.557+0000 7fad1dfec700 1 -- 172.21.15.70:0/3613156634 >> 172.21.15.70:0/3613156634 conn(0x7fad18074e30 msgr2=0x7fad18075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:22.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.557+0000 7fad1dfec700 1 -- 172.21.15.70:0/3613156634 shutdown_connections 2024-04-02T04:57:22.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.557+0000 7fad1dfec700 1 -- 172.21.15.70:0/3613156634 wait complete. 2024-04-02T04:57:22.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.558+0000 7fad1dfec700 1 Processor -- start 2024-04-02T04:57:22.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.558+0000 7fad1dfec700 1 -- start start 2024-04-02T04:57:22.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad1dfec700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180ff5a0 0x7fad1819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:22.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad1dfec700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad180fff50 0x7fad1819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad1dfec700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1819d260 con 0x7fad180ff5a0 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad1dfec700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad18196710 con 0x7fad180fff50 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad177fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180ff5a0 0x7fad1819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad177fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180ff5a0 0x7fad1819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42278/0 (socket says 172.21.15.70:42278) 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad177fe700 1 -- 172.21.15.70:0/564859511 learned_addr learned my addr 172.21.15.70:0/564859511 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad16ffd700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad180fff50 0x7fad1819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad177fe700 1 -- 172.21.15.70:0/564859511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad180fff50 msgr2=0x7fad1819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad177fe700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad180fff50 0x7fad1819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad177fe700 1 -- 172.21.15.70:0/564859511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0095f0 con 0x7fad180ff5a0 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad177fe700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180ff5a0 0x7fad1819c690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e920 tx=0x7fad0800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.559+0000 7fad14ff9700 1 -- 172.21.15.70:0/564859511 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad0800c9d0 con 0x7fad180ff5a0 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.560+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad181969f0 con 0x7fad180ff5a0 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.560+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad18196f10 con 0x7fad180ff5a0 2024-04-02T04:57:22.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.560+0000 7fad14ff9700 1 -- 172.21.15.70:0/564859511 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad08010430 con 0x7fad180ff5a0 2024-04-02T04:57:22.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.560+0000 7fad14ff9700 1 -- 172.21.15.70:0/564859511 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad08018630 con 0x7fad180ff5a0 2024-04-02T04:57:22.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.561+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad04005320 con 0x7fad180ff5a0 2024-04-02T04:57:22.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.561+0000 7fad14ff9700 1 -- 172.21.15.70:0/564859511 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fad080105a0 con 0x7fad180ff5a0 2024-04-02T04:57:22.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.563+0000 7fad14ff9700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fad0006dd60 0x7fad00070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:22.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.563+0000 7fad14ff9700 1 -- 172.21.15.70:0/564859511 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6769+0+0 (secure 0 0 0) 0x7fad08014070 con 0x7fad180ff5a0 2024-04-02T04:57:22.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.563+0000 7fad16ffd700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fad0006dd60 0x7fad00070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:22.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.564+0000 7fad16ffd700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fad0006dd60 0x7fad00070220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0059d0 tx=0x7fad0c005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:22 smithi179 ceph-mon[84288]: pgmap v202: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 87 B/s wr, 7 op/s 2024-04-02T04:57:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:22 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.70:6828/3294303398,v1:172.21.15.70:6829/3294303398] up:boot 2024-04-02T04:57:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:22 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi179.mzfimf=up:active} 3 up:standby 2024-04-02T04:57:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:57:22.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.570+0000 7fad14ff9700 1 -- 172.21.15.70:0/564859511 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fad08053280 con 0x7fad180ff5a0 2024-04-02T04:57:22.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.803+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad04000bf0 con 0x7fad0006dd60 2024-04-02T04:57:22.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.804+0000 7fad14ff9700 1 -- 172.21.15.70:0/564859511 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fad04000bf0 con 0x7fad0006dd60 2024-04-02T04:57:22.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fad0006dd60 msgr2=0x7fad00070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:22.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fad0006dd60 0x7fad00070220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0059d0 tx=0x7fad0c005940 comp rx=0 tx=0).stop 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180ff5a0 msgr2=0x7fad1819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180ff5a0 0x7fad1819c690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e920 tx=0x7fad0800ec30 comp rx=0 tx=0).stop 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 shutdown_connections 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fad0006dd60 0x7fad00070220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad180ff5a0 0x7fad1819c690 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 --2- 172.21.15.70:0/564859511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad180fff50 0x7fad1819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 >> 172.21.15.70:0/564859511 conn(0x7fad18074e30 msgr2=0x7fad180fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 shutdown_connections 2024-04-02T04:57:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.807+0000 7fad1dfec700 1 -- 172.21.15.70:0/564859511 wait complete. 2024-04-02T04:57:22.826 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:57:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.924+0000 7f0dac563700 1 -- 172.21.15.70:0/967367762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 msgr2=0x7f0da41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.924+0000 7f0dac563700 1 --2- 172.21.15.70:0/967367762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 0x7f0da41021c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009910 tx=0x7f0da0009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.924+0000 7f0dac563700 1 -- 172.21.15.70:0/967367762 shutdown_connections 2024-04-02T04:57:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.924+0000 7f0dac563700 1 --2- 172.21.15.70:0/967367762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da4102700 0x7f0da410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.924+0000 7f0dac563700 1 --2- 172.21.15.70:0/967367762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 0x7f0da41021c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 -- 172.21.15.70:0/967367762 >> 172.21.15.70:0/967367762 conn(0x7f0da40fb610 msgr2=0x7f0da40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 -- 172.21.15.70:0/967367762 shutdown_connections 2024-04-02T04:57:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 -- 172.21.15.70:0/967367762 wait complete. 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 Processor -- start 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 -- start start 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 0x7f0da419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da4102700 0x7f0da419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419d240 con 0x7f0da4101de0 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.925+0000 7f0dac563700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da41966f0 con 0x7f0da4102700 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0daa2ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 0x7f0da419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0daa2ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 0x7f0da419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42294/0 (socket says 172.21.15.70:42294) 2024-04-02T04:57:22.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0daa2ff700 1 -- 172.21.15.70:0/2047553274 learned_addr learned my addr 172.21.15.70:0/2047553274 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:22.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0daa2ff700 1 -- 172.21.15.70:0/2047553274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da4102700 msgr2=0x7f0da419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:22.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0da9afe700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da4102700 0x7f0da419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:22.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0daa2ff700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da4102700 0x7f0da419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:22.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0daa2ff700 1 -- 172.21.15.70:0/2047553274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da00095f0 con 0x7f0da4101de0 2024-04-02T04:57:22.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0daa2ff700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 0x7f0da419c670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006010 tx=0x7f0da000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:22.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0d9b7fe700 1 -- 172.21.15.70:0/2047553274 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0da001d070 con 0x7f0da4101de0 2024-04-02T04:57:22.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0d9b7fe700 1 -- 172.21.15.70:0/2047553274 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0da0004fb0 con 0x7f0da4101de0 2024-04-02T04:57:22.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da4196970 con 0x7f0da4101de0 2024-04-02T04:57:22.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0d9b7fe700 1 -- 172.21.15.70:0/2047553274 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0da0022ea0 con 0x7f0da4101de0 2024-04-02T04:57:22.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.926+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da4196e60 con 0x7f0da4101de0 2024-04-02T04:57:22.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.928+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da41082f0 con 0x7f0da4101de0 2024-04-02T04:57:22.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.929+0000 7f0d9b7fe700 1 -- 172.21.15.70:0/2047553274 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f0da0022470 con 0x7f0da4101de0 2024-04-02T04:57:22.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.929+0000 7f0d9b7fe700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0d9006dd60 0x7f0d90070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:22.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.929+0000 7f0d9b7fe700 1 -- 172.21.15.70:0/2047553274 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6769+0+0 (secure 0 0 0) 0x7f0da008f600 con 0x7f0da4101de0 2024-04-02T04:57:22.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.929+0000 7f0da9afe700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0d9006dd60 0x7f0d90070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:22.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.930+0000 7f0da9afe700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0d9006dd60 0x7f0d90070220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0da4197b60 tx=0x7f0d94005d90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:22.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:22.934+0000 7f0d9b7fe700 1 -- 172.21.15.70:0/2047553274 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0da005c780 con 0x7f0da4101de0 2024-04-02T04:57:23.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.160+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0da4197130 con 0x7f0d9006dd60 2024-04-02T04:57:23.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.162+0000 7f0d9b7fe700 1 -- 172.21.15.70:0/2047553274 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0da4197130 con 0x7f0d9006dd60 2024-04-02T04:57:23.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0d9006dd60 msgr2=0x7f0d90070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0d9006dd60 0x7f0d90070220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0da4197b60 tx=0x7f0d94005d90 comp rx=0 tx=0).stop 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 msgr2=0x7f0da419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 0x7f0da419c670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006010 tx=0x7f0da000be20 comp rx=0 tx=0).stop 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 shutdown_connections 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0d9006dd60 0x7f0d90070220 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da4101de0 0x7f0da419c670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 --2- 172.21.15.70:0/2047553274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0da4102700 0x7f0da419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 >> 172.21.15.70:0/2047553274 conn(0x7f0da40fb610 msgr2=0x7f0da4105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 shutdown_connections 2024-04-02T04:57:23.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.164+0000 7f0dac563700 1 -- 172.21.15.70:0/2047553274 wait complete. 2024-04-02T04:57:23.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.272+0000 7fd7d6194700 1 -- 172.21.15.70:0/2316701429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 msgr2=0x7fd7d0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.272+0000 7fd7d6194700 1 --2- 172.21.15.70:0/2316701429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 0x7fd7d0100080 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009910 tx=0x7fd7c4009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:23.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.272+0000 7fd7d6194700 1 -- 172.21.15.70:0/2316701429 shutdown_connections 2024-04-02T04:57:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.272+0000 7fd7d6194700 1 --2- 172.21.15.70:0/2316701429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 0x7fd7d0100080 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.272+0000 7fd7d6194700 1 --2- 172.21.15.70:0/2316701429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7d0105bd0 0x7fd7d0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.272+0000 7fd7d6194700 1 -- 172.21.15.70:0/2316701429 >> 172.21.15.70:0/2316701429 conn(0x7fd7d0075670 msgr2=0x7fd7d0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.272+0000 7fd7d6194700 1 -- 172.21.15.70:0/2316701429 shutdown_connections 2024-04-02T04:57:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.272+0000 7fd7d6194700 1 -- 172.21.15.70:0/2316701429 wait complete. 2024-04-02T04:57:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.273+0000 7fd7d6194700 1 Processor -- start 2024-04-02T04:57:23.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.273+0000 7fd7d6194700 1 -- start start 2024-04-02T04:57:23.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7d6194700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 0x7fd7d0197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7d6194700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7d0105bd0 0x7fd7d0197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7d6194700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0198380 con 0x7fd7d00ffc20 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7d6194700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01984f0 con 0x7fd7d0105bd0 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7cf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 0x7fd7d0197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7cf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 0x7fd7d0197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42312/0 (socket says 172.21.15.70:42312) 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7cf7fe700 1 -- 172.21.15.70:0/3006480636 learned_addr learned my addr 172.21.15.70:0/3006480636 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7ceffd700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7d0105bd0 0x7fd7d0197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7cf7fe700 1 -- 172.21.15.70:0/3006480636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7d0105bd0 msgr2=0x7fd7d0197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7cf7fe700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7d0105bd0 0x7fd7d0197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7cf7fe700 1 -- 172.21.15.70:0/3006480636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40095f0 con 0x7fd7d00ffc20 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7cf7fe700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 0x7fd7d0197760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c8a0 tx=0x7fd7c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.274+0000 7fd7ccff9700 1 -- 172.21.15.70:0/3006480636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c000cea0 con 0x7fd7d00ffc20 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.275+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d01a07d0 con 0x7fd7d00ffc20 2024-04-02T04:57:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.275+0000 7fd7ccff9700 1 -- 172.21.15.70:0/3006480636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7c0004830 con 0x7fd7d00ffc20 2024-04-02T04:57:23.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.275+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d01a0d20 con 0x7fd7d00ffc20 2024-04-02T04:57:23.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.275+0000 7fd7ccff9700 1 -- 172.21.15.70:0/3006480636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c00055b0 con 0x7fd7d00ffc20 2024-04-02T04:57:23.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.276+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d01080d0 con 0x7fd7d00ffc20 2024-04-02T04:57:23.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.276+0000 7fd7ccff9700 1 -- 172.21.15.70:0/3006480636 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd7c000f450 con 0x7fd7d00ffc20 2024-04-02T04:57:23.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.277+0000 7fd7ccff9700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd7b806dd60 0x7fd7b8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:23.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.277+0000 7fd7ceffd700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd7b806dd60 0x7fd7b8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:23.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.277+0000 7fd7ccff9700 1 -- 172.21.15.70:0/3006480636 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6769+0+0 (secure 0 0 0) 0x7fd7c008d910 con 0x7fd7d00ffc20 2024-04-02T04:57:23.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.277+0000 7fd7ceffd700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd7b806dd60 0x7fd7b8070220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0068e30 tx=0x7fd7c401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:23.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.281+0000 7fd7ccff9700 1 -- 172.21.15.70:0/3006480636 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd7c0056d90 con 0x7fd7d00ffc20 2024-04-02T04:57:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.518+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7d0068c80 con 0x7fd7b806dd60 2024-04-02T04:57:23.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.529+0000 7fd7ccff9700 1 -- 172.21.15.70:0/3006480636 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd7d0068c80 con 0x7fd7b806dd60 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (6m) 22s ago 17m 25.8M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (18m) 22s ago 18m 9034k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (17m) 48s ago 17m 8493k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (4m) 22s ago 18m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (4m) 48s ago 17m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (6m) 22s ago 17m 79.7M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (33s) 22s ago 12m 12.3M - 19.0.0-1979-g6cec793d 78f830190839 2a4aa2b03d0d 2024-04-02T04:57:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 starting - - - - 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (12m) 48s ago 12m 19.1M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (12m) 48s ago 12m 20.4M - 18.2.1 5be31c24972a 22036b979e94 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (9m) 22s ago 19m 508M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (8m) 48s ago 17m 451M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (5m) 22s ago 19m 54.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (4m) 48s ago 16m 44.4M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 22s ago 18m 15.8M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (7m) 48s ago 17m 17.3M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (3m) 22s ago 16m 145M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (3m) 22s ago 15m 143M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (2m) 22s ago 15m 94.1M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (117s) 48s ago 14m 97.7M 2240M 19.0.0-1979-g6cec793d 78f830190839 f717fd6c8306 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (85s) 48s ago 14m 109M 2240M 19.0.0-1979-g6cec793d 78f830190839 2e5bbf3235ff 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (51s) 48s ago 13m 15.1M 2240M 19.0.0-1979-g6cec793d 78f830190839 71351459ac70 2024-04-02T04:57:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (7m) 22s ago 17m 65.1M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:57:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd7b806dd60 msgr2=0x7fd7b8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd7b806dd60 0x7fd7b8070220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0068e30 tx=0x7fd7c401a040 comp rx=0 tx=0).stop 2024-04-02T04:57:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 msgr2=0x7fd7d0197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 0x7fd7d0197760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c8a0 tx=0x7fd7c000cc60 comp rx=0 tx=0).stop 2024-04-02T04:57:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 shutdown_connections 2024-04-02T04:57:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd7b806dd60 0x7fd7b8070220 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7d00ffc20 0x7fd7d0197760 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 --2- 172.21.15.70:0/3006480636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7d0105bd0 0x7fd7d0197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.531+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 >> 172.21.15.70:0/3006480636 conn(0x7fd7d0075670 msgr2=0x7fd7d00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:23.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.532+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 shutdown_connections 2024-04-02T04:57:23.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.532+0000 7fd7d6194700 1 -- 172.21.15.70:0/3006480636 wait complete. 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 -- 172.21.15.70:0/3927207921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f001027a0 msgr2=0x7f0f0010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 --2- 172.21.15.70:0/3927207921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f001027a0 0x7f0f0010ac90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009910 tx=0x7f0eec009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 -- 172.21.15.70:0/3927207921 shutdown_connections 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 --2- 172.21.15.70:0/3927207921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f001027a0 0x7f0f0010ac90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 --2- 172.21.15.70:0/3927207921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f00101e80 0x7f0f00102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 -- 172.21.15.70:0/3927207921 >> 172.21.15.70:0/3927207921 conn(0x7f0f000fb610 msgr2=0x7f0f000fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 -- 172.21.15.70:0/3927207921 shutdown_connections 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 -- 172.21.15.70:0/3927207921 wait complete. 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.634+0000 7f0f070f8700 1 Processor -- start 2024-04-02T04:57:23.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f070f8700 1 -- start start 2024-04-02T04:57:23.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f070f8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f00101e80 0x7f0f000ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:23.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f070f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f001027a0 0x7f0f00100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:23.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f070f8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f00100560 con 0x7f0f00101e80 2024-04-02T04:57:23.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f070f8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f001006a0 con 0x7f0f001027a0 2024-04-02T04:57:23.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f04e94700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f00101e80 0x7f0f000ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f04e94700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f00101e80 0x7f0f000ffae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42324/0 (socket says 172.21.15.70:42324) 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f04e94700 1 -- 172.21.15.70:0/3914540460 learned_addr learned my addr 172.21.15.70:0/3914540460 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0efffff700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f001027a0 0x7f0f00100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f04e94700 1 -- 172.21.15.70:0/3914540460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f001027a0 msgr2=0x7f0f00100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f04e94700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f001027a0 0x7f0f00100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f04e94700 1 -- 172.21.15.70:0/3914540460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec0095f0 con 0x7f0f00101e80 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.635+0000 7f0f04e94700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f00101e80 0x7f0f000ffae0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400c930 tx=0x7f0ef400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.636+0000 7f0efdffb700 1 -- 172.21.15.70:0/3914540460 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ef4007ad0 con 0x7f0f00101e80 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.636+0000 7f0efdffb700 1 -- 172.21.15.70:0/3914540460 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ef400ce80 con 0x7f0f00101e80 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.636+0000 7f0efdffb700 1 -- 172.21.15.70:0/3914540460 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ef4018630 con 0x7f0f00101e80 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.636+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f001a2a70 con 0x7f0f00101e80 2024-04-02T04:57:23.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.636+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f001a2dd0 con 0x7f0f00101e80 2024-04-02T04:57:23.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.637+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f00108390 con 0x7f0f00101e80 2024-04-02T04:57:23.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.637+0000 7f0efdffb700 1 -- 172.21.15.70:0/3914540460 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f0ef4007c30 con 0x7f0f00101e80 2024-04-02T04:57:23.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.637+0000 7f0efdffb700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0ef006dd60 0x7f0ef0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:23.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.638+0000 7f0efdffb700 1 -- 172.21.15.70:0/3914540460 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6769+0+0 (secure 0 0 0) 0x7f0ef408dba0 con 0x7f0f00101e80 2024-04-02T04:57:23.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.638+0000 7f0efffff700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0ef006dd60 0x7f0ef0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:23.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.638+0000 7f0efffff700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0ef006dd60 0x7f0ef0070220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f001012d0 tx=0x7f0eec01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:23.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.642+0000 7f0efdffb700 1 -- 172.21.15.70:0/3914540460 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0ef40570a0 con 0x7f0f00101e80 2024-04-02T04:57:23.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.930+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f00100860 con 0x7f0f00101e80 2024-04-02T04:57:23.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.930+0000 7f0efdffb700 1 -- 172.21.15.70:0/3914540460 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f0ef405a760 con 0x7f0f00101e80 2024-04-02T04:57:23.935 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:57:23.935 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:57:23.935 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:57:23.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:57:23.935 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:57:23.935 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 12 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:57:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:57:23.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0ef006dd60 msgr2=0x7f0ef0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0ef006dd60 0x7f0ef0070220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f001012d0 tx=0x7f0eec01a040 comp rx=0 tx=0).stop 2024-04-02T04:57:23.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f00101e80 msgr2=0x7f0f000ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:23.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f00101e80 0x7f0f000ffae0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400c930 tx=0x7f0ef400cc40 comp rx=0 tx=0).stop 2024-04-02T04:57:23.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 shutdown_connections 2024-04-02T04:57:23.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0ef006dd60 0x7f0ef0070220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f00101e80 0x7f0f000ffae0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 --2- 172.21.15.70:0/3914540460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f001027a0 0x7f0f00100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:23.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 >> 172.21.15.70:0/3914540460 conn(0x7f0f000fb610 msgr2=0x7f0f001054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:23.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 shutdown_connections 2024-04-02T04:57:23.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:23.933+0000 7f0f070f8700 1 -- 172.21.15.70:0/3914540460 wait complete. 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 -- 172.21.15.70:0/138227713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 msgr2=0x7f93b8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 --2- 172.21.15.70:0/138227713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 0x7f93b8100180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009910 tx=0x7f93ac009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 -- 172.21.15.70:0/138227713 shutdown_connections 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 --2- 172.21.15.70:0/138227713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8100750 0x7f93b81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 --2- 172.21.15.70:0/138227713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 0x7f93b8100180 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 -- 172.21.15.70:0/138227713 >> 172.21.15.70:0/138227713 conn(0x7f93b80fb650 msgr2=0x7f93b80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 -- 172.21.15.70:0/138227713 shutdown_connections 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 -- 172.21.15.70:0/138227713 wait complete. 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 Processor -- start 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.038+0000 7f93befb0700 1 -- start start 2024-04-02T04:57:24.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93befb0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 0x7f93b8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93befb0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8100750 0x7f93b8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93befb0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81022c0 con 0x7f93b80ffda0 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93befb0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8102400 con 0x7f93b8100750 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93bcd4c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 0x7f93b8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93bcd4c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 0x7f93b8101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42350/0 (socket says 172.21.15.70:42350) 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93bcd4c700 1 -- 172.21.15.70:0/84123534 learned_addr learned my addr 172.21.15.70:0/84123534 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93b7fff700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8100750 0x7f93b8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93bcd4c700 1 -- 172.21.15.70:0/84123534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8100750 msgr2=0x7f93b8101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93bcd4c700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8100750 0x7f93b8101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93bcd4c700 1 -- 172.21.15.70:0/84123534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0095f0 con 0x7f93b80ffda0 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93b7fff700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8100750 0x7f93b8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93bcd4c700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 0x7f93b8101840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f93ac006010 tx=0x7f93ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:24.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93b5ffb700 1 -- 172.21.15.70:0/84123534 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93ac01d070 con 0x7f93b80ffda0 2024-04-02T04:57:24.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93b5ffb700 1 -- 172.21.15.70:0/84123534 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93ac004fb0 con 0x7f93b80ffda0 2024-04-02T04:57:24.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93b5ffb700 1 -- 172.21.15.70:0/84123534 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93ac00f690 con 0x7f93b80ffda0 2024-04-02T04:57:24.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.039+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b8102680 con 0x7f93b80ffda0 2024-04-02T04:57:24.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.040+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b81a2e20 con 0x7f93b80ffda0 2024-04-02T04:57:24.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.040+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b804f2e0 con 0x7f93b80ffda0 2024-04-02T04:57:24.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.041+0000 7f93b5ffb700 1 -- 172.21.15.70:0/84123534 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f93ac0228a0 con 0x7f93b80ffda0 2024-04-02T04:57:24.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.041+0000 7f93b5ffb700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f93a806dd60 0x7f93a8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.042+0000 7f93b5ffb700 1 -- 172.21.15.70:0/84123534 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6769+0+0 (secure 0 0 0) 0x7f93ac08f480 con 0x7f93b80ffda0 2024-04-02T04:57:24.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.042+0000 7f93b7fff700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f93a806dd60 0x7f93a8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:24.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.042+0000 7f93b7fff700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f93a806dd60 0x7f93a8070220 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93b8103030 tx=0x7f93a4005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:24.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.046+0000 7f93b5ffb700 1 -- 172.21.15.70:0/84123534 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f93ac058900 con 0x7f93b80ffda0 2024-04-02T04:57:24.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:24 smithi070 ceph-mon[113635]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:24.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:24 smithi070 ceph-mon[113635]: pgmap v203: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:57:24.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:24 smithi070 ceph-mon[113635]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:24.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:24 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3914540460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:24.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.309+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f93b81a3040 con 0x7f93b80ffda0 2024-04-02T04:57:24.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.309+0000 7f93b5ffb700 1 -- 172.21.15.70:0/84123534 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1889 (secure 0 0 0) 0x7f93ac05bfc0 con 0x7f93b80ffda0 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:e26 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 25 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:57:24.314 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:57:20.167349+0000 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 93 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:57:24.315 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=24253} 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 24253 members: 24253 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{0:24253} state up:active seq 188 join_fscid=1 addr [v2:172.21.15.179:6824/511108019,v1:172.21.15.179:6825/511108019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:57:24.316 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:57:24.317 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:57:24.317 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:57:24.317 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:57:24.317 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:57:24.317 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:57:24.317 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6828/3294303398,v1:172.21.15.70:6829/3294303398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f93a806dd60 msgr2=0x7f93a8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f93a806dd60 0x7f93a8070220 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93b8103030 tx=0x7f93a4005f50 comp rx=0 tx=0).stop 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 msgr2=0x7f93b8101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 0x7f93b8101840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f93ac006010 tx=0x7f93ac00be20 comp rx=0 tx=0).stop 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 shutdown_connections 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f93a806dd60 0x7f93a8070220 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b80ffda0 0x7f93b8101840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 --2- 172.21.15.70:0/84123534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8100750 0x7f93b8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 >> 172.21.15.70:0/84123534 conn(0x7f93b80fb650 msgr2=0x7f93b8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.311+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 shutdown_connections 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.312+0000 7f93befb0700 1 -- 172.21.15.70:0/84123534 wait complete. 2024-04-02T04:57:24.318 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 26 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 -- 172.21.15.70:0/1603658508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 msgr2=0x7ff73c111710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 --2- 172.21.15.70:0/1603658508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 0x7ff73c111710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff728009910 tx=0x7ff728009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 -- 172.21.15.70:0/1603658508 shutdown_connections 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 --2- 172.21.15.70:0/1603658508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 0x7ff73c111710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 --2- 172.21.15.70:0/1603658508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73c073130 0x7ff73c073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 -- 172.21.15.70:0/1603658508 >> 172.21.15.70:0/1603658508 conn(0x7ff73c0fc750 msgr2=0x7ff73c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 -- 172.21.15.70:0/1603658508 shutdown_connections 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 -- 172.21.15.70:0/1603658508 wait complete. 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 Processor -- start 2024-04-02T04:57:24.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 -- start start 2024-04-02T04:57:24.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73c073130 0x7ff73c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 0x7ff73c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c19db40 con 0x7ff73c073a50 2024-04-02T04:57:24.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.417+0000 7ff742bda700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c1a18d0 con 0x7ff73c073130 2024-04-02T04:57:24.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff73bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 0x7ff73c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:24.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff73bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 0x7ff73c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42368/0 (socket says 172.21.15.70:42368) 2024-04-02T04:57:24.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff73bfff700 1 -- 172.21.15.70:0/775950921 learned_addr learned my addr 172.21.15.70:0/775950921 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:24.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff73bfff700 1 -- 172.21.15.70:0/775950921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73c073130 msgr2=0x7ff73c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff73bfff700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73c073130 0x7ff73c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff73bfff700 1 -- 172.21.15.70:0/775950921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7280095f0 con 0x7ff73c073a50 2024-04-02T04:57:24.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff73bfff700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 0x7ff73c19d460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff728006010 tx=0x7ff72800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:24.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff739ffb700 1 -- 172.21.15.70:0/775950921 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72801d070 con 0x7ff73c073a50 2024-04-02T04:57:24.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c1a1bb0 con 0x7ff73c073a50 2024-04-02T04:57:24.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff739ffb700 1 -- 172.21.15.70:0/775950921 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff728004fb0 con 0x7ff73c073a50 2024-04-02T04:57:24.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff739ffb700 1 -- 172.21.15.70:0/775950921 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff728022ea0 con 0x7ff73c073a50 2024-04-02T04:57:24.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.418+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c1a2100 con 0x7ff73c073a50 2024-04-02T04:57:24.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.419+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c10ee90 con 0x7ff73c073a50 2024-04-02T04:57:24.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.420+0000 7ff739ffb700 1 -- 172.21.15.70:0/775950921 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff728022470 con 0x7ff73c073a50 2024-04-02T04:57:24.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.420+0000 7ff739ffb700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff72c06dd60 0x7ff72c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.420+0000 7ff740976700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff72c06dd60 0x7ff72c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:24.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.420+0000 7ff739ffb700 1 -- 172.21.15.70:0/775950921 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6769+0+0 (secure 0 0 0) 0x7ff72808f290 con 0x7ff73c073a50 2024-04-02T04:57:24.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.421+0000 7ff740976700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff72c06dd60 0x7ff72c070220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7300097b0 tx=0x7ff730006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:24.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.425+0000 7ff739ffb700 1 -- 172.21.15.70:0/775950921 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff7280588e0 con 0x7ff73c073a50 2024-04-02T04:57:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:24 smithi179 ceph-mon[84288]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:24 smithi179 ceph-mon[84288]: pgmap v203: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:57:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:24 smithi179 ceph-mon[84288]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:24 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3914540460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:24.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.644+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff73c1a23e0 con 0x7ff72c06dd60 2024-04-02T04:57:24.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.646+0000 7ff739ffb700 1 -- 172.21.15.70:0/775950921 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff73c1a23e0 con 0x7ff72c06dd60 2024-04-02T04:57:24.650 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:57:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:57:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:57:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:57:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:57:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:57:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "osd", 2024-04-02T04:57:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:57:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:57:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:57:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "13/23 daemons upgraded", 2024-04-02T04:57:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mds daemons", 2024-04-02T04:57:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:57:24.651 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:57:24.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff72c06dd60 msgr2=0x7ff72c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff72c06dd60 0x7ff72c070220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7300097b0 tx=0x7ff730006d20 comp rx=0 tx=0).stop 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 msgr2=0x7ff73c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 0x7ff73c19d460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff728006010 tx=0x7ff72800be20 comp rx=0 tx=0).stop 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 shutdown_connections 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff72c06dd60 0x7ff72c070220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73c073130 0x7ff73c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 --2- 172.21.15.70:0/775950921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c073a50 0x7ff73c19d460 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 >> 172.21.15.70:0/775950921 conn(0x7ff73c0fc750 msgr2=0x7ff73c103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 shutdown_connections 2024-04-02T04:57:24.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.648+0000 7ff742bda700 1 -- 172.21.15.70:0/775950921 wait complete. 2024-04-02T04:57:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.754+0000 7f5f78a99700 1 -- 172.21.15.70:0/2435459495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 msgr2=0x7f5f7410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.754+0000 7f5f78a99700 1 --2- 172.21.15.70:0/2435459495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 0x7f5f7410ac10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009910 tx=0x7f5f68009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.755+0000 7f5f78a99700 1 -- 172.21.15.70:0/2435459495 shutdown_connections 2024-04-02T04:57:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.755+0000 7f5f78a99700 1 --2- 172.21.15.70:0/2435459495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 0x7f5f7410ac10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.755+0000 7f5f78a99700 1 --2- 172.21.15.70:0/2435459495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f74101e00 0x7f5f741021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.755+0000 7f5f78a99700 1 -- 172.21.15.70:0/2435459495 >> 172.21.15.70:0/2435459495 conn(0x7f5f740fb650 msgr2=0x7f5f740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.755+0000 7f5f78a99700 1 -- 172.21.15.70:0/2435459495 shutdown_connections 2024-04-02T04:57:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.755+0000 7f5f78a99700 1 -- 172.21.15.70:0/2435459495 wait complete. 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.755+0000 7f5f78a99700 1 Processor -- start 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f78a99700 1 -- start start 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f78a99700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f74101e00 0x7f5f7419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f78a99700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 0x7f5f7419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f78a99700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7419d260 con 0x7f5f74102720 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f78a99700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f74196710 con 0x7f5f74101e00 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f71d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 0x7f5f7419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f71d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 0x7f5f7419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42380/0 (socket says 172.21.15.70:42380) 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f71d9b700 1 -- 172.21.15.70:0/312699283 learned_addr learned my addr 172.21.15.70:0/312699283 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f7259c700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f74101e00 0x7f5f7419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f71d9b700 1 -- 172.21.15.70:0/312699283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f74101e00 msgr2=0x7f5f7419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f71d9b700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f74101e00 0x7f5f7419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f71d9b700 1 -- 172.21.15.70:0/312699283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f680095f0 con 0x7f5f74102720 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f71d9b700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 0x7f5f7419cbd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800bd70 tx=0x7f5f6800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.756+0000 7f5f637fe700 1 -- 172.21.15.70:0/312699283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f6801d070 con 0x7f5f74102720 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.757+0000 7f5f637fe700 1 -- 172.21.15.70:0/312699283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f68005dc0 con 0x7f5f74102720 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.757+0000 7f5f637fe700 1 -- 172.21.15.70:0/312699283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f68017400 con 0x7f5f74102720 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.757+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f74196990 con 0x7f5f74102720 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.757+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f74196e80 con 0x7f5f74102720 2024-04-02T04:57:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.758+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f74108310 con 0x7f5f74102720 2024-04-02T04:57:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.758+0000 7f5f637fe700 1 -- 172.21.15.70:0/312699283 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5f6800f460 con 0x7f5f74102720 2024-04-02T04:57:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.758+0000 7f5f637fe700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f5c072170 0x7f5f5c074630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.759+0000 7f5f637fe700 1 -- 172.21.15.70:0/312699283 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6769+0+0 (secure 0 0 0) 0x7f5f6808efc0 con 0x7f5f74102720 2024-04-02T04:57:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.759+0000 7f5f7259c700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f5c072170 0x7f5f5c074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.759+0000 7f5f7259c700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f5c072170 0x7f5f5c074630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5f64005950 tx=0x7f5f6400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:24.763+0000 7f5f637fe700 1 -- 172.21.15.70:0/312699283 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f6805bf70 con 0x7f5f74102720 2024-04-02T04:57:25.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.059+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f74197cc0 con 0x7f5f74102720 2024-04-02T04:57:25.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.059+0000 7f5f637fe700 1 -- 172.21.15.70:0/312699283 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5f6805bb00 con 0x7f5f74102720 2024-04-02T04:57:25.064 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:57:25.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.061+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f5c072170 msgr2=0x7f5f5c074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:25.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f5c072170 0x7f5f5c074630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5f64005950 tx=0x7f5f6400b410 comp rx=0 tx=0).stop 2024-04-02T04:57:25.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 msgr2=0x7f5f7419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:25.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 0x7f5f7419cbd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800bd70 tx=0x7f5f6800bda0 comp rx=0 tx=0).stop 2024-04-02T04:57:25.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 shutdown_connections 2024-04-02T04:57:25.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f5f5c072170 0x7f5f5c074630 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:25.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f74101e00 0x7f5f7419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:25.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 --2- 172.21.15.70:0/312699283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f74102720 0x7f5f7419cbd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:25.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 >> 172.21.15.70:0/312699283 conn(0x7f5f740fb650 msgr2=0x7f5f74105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:25.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 shutdown_connections 2024-04-02T04:57:25.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:25.062+0000 7f5f78a99700 1 -- 172.21.15.70:0/312699283 wait complete. 2024-04-02T04:57:25.437 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:25 smithi070 ceph-mon[113635]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:25.438 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:25 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/84123534' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:57:25.438 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:25 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/312699283' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:57:25.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:25 smithi179 ceph-mon[84288]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:25.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:25 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/84123534' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:57:25.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:25 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/312699283' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:57:26.271 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:26 smithi070 ceph-mon[113635]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:26.271 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:26 smithi070 ceph-mon[113635]: pgmap v204: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 9.5 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T04:57:26.271 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:26.272 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:26 smithi179 ceph-mon[84288]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:26 smithi179 ceph-mon[84288]: pgmap v204: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 9.5 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T04:57:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:28.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:27 smithi070 ceph-mon[113635]: pgmap v205: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-02T04:57:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:28.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:27 smithi179 ceph-mon[84288]: pgmap v205: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-02T04:57:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:28 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:29.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:28 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:28 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:28 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:29 smithi070 ceph-mon[113635]: pgmap v206: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-02T04:57:30.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:29 smithi179 ceph-mon[84288]: pgmap v206: 65 pgs: 65 active+clean; 205 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-02T04:57:32.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: pgmap v207: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 845 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:57:32.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:32.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:32.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:32.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:57:32.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:32.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:32.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:32.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:32.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:32.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:32 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: pgmap v207: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 845 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:32.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:32.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:32.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:32 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:34.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:34 smithi179 ceph-mon[84288]: Upgrade: Updating mds.cephfs.smithi179.mzfimf 2024-04-02T04:57:34.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:34.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.mzfimf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:57:34.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:34 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:34.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:34 smithi179 ceph-mon[84288]: Deploying daemon mds.cephfs.smithi179.mzfimf on smithi179 2024-04-02T04:57:34.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:34 smithi179 ceph-mon[84288]: pgmap v208: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-04-02T04:57:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:34 smithi070 ceph-mon[113635]: Upgrade: Updating mds.cephfs.smithi179.mzfimf 2024-04-02T04:57:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.mzfimf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:57:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:34 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:34 smithi070 ceph-mon[113635]: Deploying daemon mds.cephfs.smithi179.mzfimf on smithi179 2024-04-02T04:57:34.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:34 smithi070 ceph-mon[113635]: pgmap v208: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-04-02T04:57:35.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:35 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[113631]: 2024-04-02T04:57:35.241+0000 7f6fcc9da700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:57:36.474 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:36 smithi179 ceph-mon[84288]: pgmap v209: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-02T04:57:36.474 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:36 smithi179 ceph-mon[84288]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:57:36.474 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:36 smithi179 ceph-mon[84288]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:57:36.474 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:36 smithi179 ceph-mon[84288]: osdmap e94: 6 total, 6 up, 6 in 2024-04-02T04:57:36.474 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:36 smithi179 ceph-mon[84288]: Standby daemon mds.cephfs.smithi179.atrmwz assigned to filesystem cephfs as rank 0 2024-04-02T04:57:36.474 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:36 smithi179 ceph-mon[84288]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:57:36.474 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:36 smithi179 ceph-mon[84288]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:57:36.474 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:36 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi179.atrmwz=up:replay} 2 up:standby 2024-04-02T04:57:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:36 smithi070 ceph-mon[113635]: pgmap v209: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-02T04:57:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:36 smithi070 ceph-mon[113635]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:57:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:36 smithi070 ceph-mon[113635]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:57:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:36 smithi070 ceph-mon[113635]: osdmap e94: 6 total, 6 up, 6 in 2024-04-02T04:57:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:36 smithi070 ceph-mon[113635]: Standby daemon mds.cephfs.smithi179.atrmwz assigned to filesystem cephfs as rank 0 2024-04-02T04:57:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:36 smithi070 ceph-mon[113635]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:57:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:36 smithi070 ceph-mon[113635]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:57:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:36 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi179.atrmwz=up:replay} 2 up:standby 2024-04-02T04:57:37.488 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:57:37.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:57:38.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:38 smithi070 ceph-mon[113635]: pgmap v211: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 409 KiB/s rd, 4.8 KiB/s wr, 2 op/s 2024-04-02T04:57:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:38 smithi179 ceph-mon[84288]: pgmap v211: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 409 KiB/s rd, 4.8 KiB/s wr, 2 op/s 2024-04-02T04:57:40.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:40 smithi070 ceph-mon[113635]: pgmap v212: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.8 KiB/s wr, 4 op/s 2024-04-02T04:57:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:40 smithi179 ceph-mon[84288]: pgmap v212: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.8 KiB/s wr, 4 op/s 2024-04-02T04:57:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:42 smithi070 ceph-mon[113635]: pgmap v213: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-02T04:57:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:42 smithi179 ceph-mon[84288]: pgmap v213: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-02T04:57:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:44 smithi070 ceph-mon[113635]: pgmap v214: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-02T04:57:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:44 smithi179 ceph-mon[84288]: pgmap v214: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-02T04:57:45.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:45 smithi070 ceph-mon[113635]: reconnect by client.14532 192.168.0.1:0/3290270869 after 0.00199996 2024-04-02T04:57:45.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:45 smithi070 ceph-mon[113635]: reconnect by client.24309 192.168.0.1:0/2957635722 after 0.00199996 2024-04-02T04:57:45.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:45 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:reconnect 2024-04-02T04:57:45.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:45 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi179.atrmwz=up:reconnect} 2 up:standby 2024-04-02T04:57:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:45 smithi179 ceph-mon[84288]: reconnect by client.14532 192.168.0.1:0/3290270869 after 0.00199996 2024-04-02T04:57:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:45 smithi179 ceph-mon[84288]: reconnect by client.24309 192.168.0.1:0/2957635722 after 0.00199996 2024-04-02T04:57:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:45 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:reconnect 2024-04-02T04:57:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:45 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi179.atrmwz=up:reconnect} 2 up:standby 2024-04-02T04:57:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:46 smithi070 ceph-mon[113635]: pgmap v215: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-02T04:57:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:46 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:rejoin 2024-04-02T04:57:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:46 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi179.atrmwz=up:rejoin} 2 up:standby 2024-04-02T04:57:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:46 smithi070 ceph-mon[113635]: daemon mds.cephfs.smithi179.atrmwz is now active in filesystem cephfs as rank 0 2024-04-02T04:57:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:46 smithi179 ceph-mon[84288]: pgmap v215: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-02T04:57:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:46 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:rejoin 2024-04-02T04:57:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:46 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi179.atrmwz=up:rejoin} 2 up:standby 2024-04-02T04:57:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:46 smithi179 ceph-mon[84288]: daemon mds.cephfs.smithi179.atrmwz is now active in filesystem cephfs as rank 0 2024-04-02T04:57:47.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:47 smithi070 ceph-mon[113635]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:57:47.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:47 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:57:47.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:47 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:active 2024-04-02T04:57:47.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:47 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi179.atrmwz=up:active} 2 up:standby 2024-04-02T04:57:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:47 smithi179 ceph-mon[84288]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:57:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:47 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:57:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:47 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] up:active 2024-04-02T04:57:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:47 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi179.atrmwz=up:active} 2 up:standby 2024-04-02T04:57:48.497 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:48 smithi179 ceph-mon[84288]: pgmap v216: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-02T04:57:48.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:48 smithi070 ceph-mon[113635]: pgmap v216: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-02T04:57:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:50 smithi070 ceph-mon[113635]: pgmap v217: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 6 op/s 2024-04-02T04:57:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:50 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:50 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:50.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:50 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:50 smithi179 ceph-mon[84288]: pgmap v217: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 6 op/s 2024-04-02T04:57:50.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:50 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:50.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:50 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:50.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:50 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:51.502 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:51 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.179:6824/1866330674,v1:172.21.15.179:6825/1866330674] up:boot 2024-04-02T04:57:51.502 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:51 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi179.atrmwz=up:active} 3 up:standby 2024-04-02T04:57:51.502 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:51 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:57:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:51 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.179:6824/1866330674,v1:172.21.15.179:6825/1866330674] up:boot 2024-04-02T04:57:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:51 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi179.atrmwz=up:active} 3 up:standby 2024-04-02T04:57:51.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:51 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:57:52.449 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:52 smithi179 ceph-mon[84288]: pgmap v218: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:57:52.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:52 smithi070 ceph-mon[113635]: pgmap v218: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:57:54.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:54 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:54.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:54 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:54.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:54 smithi179 ceph-mon[84288]: pgmap v219: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-02T04:57:54.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:54 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:54 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:54 smithi070 ceph-mon[113635]: pgmap v219: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-02T04:57:55.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 -- 172.21.15.70:0/1217560713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140fff50 msgr2=0x7f4c14105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 --2- 172.21.15.70:0/1217560713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140fff50 0x7f4c14105840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009910 tx=0x7f4c08009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 -- 172.21.15.70:0/1217560713 shutdown_connections 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 --2- 172.21.15.70:0/1217560713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140fff50 0x7f4c14105840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 --2- 172.21.15.70:0/1217560713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c140ff5a0 0x7f4c140ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 -- 172.21.15.70:0/1217560713 >> 172.21.15.70:0/1217560713 conn(0x7f4c14074e30 msgr2=0x7f4c14075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 -- 172.21.15.70:0/1217560713 shutdown_connections 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 -- 172.21.15.70:0/1217560713 wait complete. 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.176+0000 7f4c19985700 1 Processor -- start 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c19985700 1 -- start start 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c19985700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140ff5a0 0x7f4c1419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c19985700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c140fff50 0x7f4c1419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c19985700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1419d210 con 0x7f4c140ff5a0 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c19985700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c141966c0 con 0x7f4c140fff50 2024-04-02T04:57:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c12ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140ff5a0 0x7f4c1419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c12ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140ff5a0 0x7f4c1419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54374/0 (socket says 172.21.15.70:54374) 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c12ffd700 1 -- 172.21.15.70:0/558141465 learned_addr learned my addr 172.21.15.70:0/558141465 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c127fc700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c140fff50 0x7f4c1419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c12ffd700 1 -- 172.21.15.70:0/558141465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c140fff50 msgr2=0x7f4c1419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c12ffd700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c140fff50 0x7f4c1419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c12ffd700 1 -- 172.21.15.70:0/558141465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080095f0 con 0x7f4c140ff5a0 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.177+0000 7f4c12ffd700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140ff5a0 0x7f4c1419c640 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400d8d0 tx=0x7f4c0400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.178+0000 7f4c18983700 1 -- 172.21.15.70:0/558141465 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c040098a0 con 0x7f4c140ff5a0 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.178+0000 7f4c18983700 1 -- 172.21.15.70:0/558141465 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c04010460 con 0x7f4c140ff5a0 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.178+0000 7f4c18983700 1 -- 172.21.15.70:0/558141465 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0400f5f0 con 0x7f4c140ff5a0 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.178+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c141969a0 con 0x7f4c140ff5a0 2024-04-02T04:57:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.178+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c14196ec0 con 0x7f4c140ff5a0 2024-04-02T04:57:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.179+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c14109200 con 0x7f4c140ff5a0 2024-04-02T04:57:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.179+0000 7f4c18983700 1 -- 172.21.15.70:0/558141465 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f4c04009a00 con 0x7f4c140ff5a0 2024-04-02T04:57:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.180+0000 7f4c18983700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4bfc06dd60 0x7f4bfc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.180+0000 7f4c18983700 1 -- 172.21.15.70:0/558141465 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f4c0408db30 con 0x7f4c140ff5a0 2024-04-02T04:57:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.180+0000 7f4c127fc700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4bfc06dd60 0x7f4bfc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.180+0000 7f4c127fc700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4bfc06dd60 0x7f4bfc070220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006010 tx=0x7f4c0801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:55.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.184+0000 7f4c18983700 1 -- 172.21.15.70:0/558141465 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4c0405aa90 con 0x7f4c140ff5a0 2024-04-02T04:57:55.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:55 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:55.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:55 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.397+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c141970e0 con 0x7f4bfc06dd60 2024-04-02T04:57:55.402 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:55 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:55.403 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:55 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:55.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.398+0000 7f4c18983700 1 -- 172.21.15.70:0/558141465 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4c141970e0 con 0x7f4bfc06dd60 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4bfc06dd60 msgr2=0x7f4bfc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4bfc06dd60 0x7f4bfc070220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006010 tx=0x7f4c0801a040 comp rx=0 tx=0).stop 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140ff5a0 msgr2=0x7f4c1419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140ff5a0 0x7f4c1419c640 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400d8d0 tx=0x7f4c0400dbe0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 shutdown_connections 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4bfc06dd60 0x7f4bfc070220 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c140ff5a0 0x7f4c1419c640 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 --2- 172.21.15.70:0/558141465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c140fff50 0x7f4c1419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 >> 172.21.15.70:0/558141465 conn(0x7f4c14074e30 msgr2=0x7f4c140fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 shutdown_connections 2024-04-02T04:57:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.400+0000 7f4c19985700 1 -- 172.21.15.70:0/558141465 wait complete. 2024-04-02T04:57:55.415 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:57:55.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 -- 172.21.15.70:0/4105182896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 msgr2=0x7faa9c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 --2- 172.21.15.70:0/4105182896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 0x7faa9c0ff980 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faa98009910 tx=0x7faa98009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:55.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 -- 172.21.15.70:0/4105182896 shutdown_connections 2024-04-02T04:57:55.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 --2- 172.21.15.70:0/4105182896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa9c0fff50 0x7faa9c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 --2- 172.21.15.70:0/4105182896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 0x7faa9c0ff980 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 -- 172.21.15.70:0/4105182896 >> 172.21.15.70:0/4105182896 conn(0x7faa9c074e30 msgr2=0x7faa9c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 -- 172.21.15.70:0/4105182896 shutdown_connections 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 -- 172.21.15.70:0/4105182896 wait complete. 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 Processor -- start 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.505+0000 7faaa33d1700 1 -- start start 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa33d1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 0x7faa9c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa33d1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa9c0fff50 0x7faa9c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa33d1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c19d2d0 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa33d1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c196780 con 0x7faa9c0fff50 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa116d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 0x7faa9c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa116d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 0x7faa9c19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54394/0 (socket says 172.21.15.70:54394) 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa116d700 1 -- 172.21.15.70:0/3820799242 learned_addr learned my addr 172.21.15.70:0/3820799242 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:55.512 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa096c700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa9c0fff50 0x7faa9c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa116d700 1 -- 172.21.15.70:0/3820799242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa9c0fff50 msgr2=0x7faa9c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa116d700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa9c0fff50 0x7faa9c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa116d700 1 -- 172.21.15.70:0/3820799242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980095f0 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa116d700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 0x7faa9c19c700 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa98006010 tx=0x7faa9800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faa927fc700 1 -- 172.21.15.70:0/3820799242 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9801d070 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faa927fc700 1 -- 172.21.15.70:0/3820799242 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa98004fb0 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9c196a00 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faa927fc700 1 -- 172.21.15.70:0/3820799242 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa98022ea0 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.506+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c196ef0 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.507+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9c109270 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.508+0000 7faa927fc700 1 -- 172.21.15.70:0/3820799242 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7faa98022470 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.508+0000 7faa927fc700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faa8806dd60 0x7faa88070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.508+0000 7faa927fc700 1 -- 172.21.15.70:0/3820799242 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6855+0+0 (secure 0 0 0) 0x7faa9808f220 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.508+0000 7faaa096c700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faa8806dd60 0x7faa88070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.509+0000 7faaa096c700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faa8806dd60 0x7faa88070220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005fd0 tx=0x7faa8c00a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:55.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.513+0000 7faa927fc700 1 -- 172.21.15.70:0/3820799242 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faa9805c230 con 0x7faa9c0ff5a0 2024-04-02T04:57:55.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.725+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa9c197ce0 con 0x7faa8806dd60 2024-04-02T04:57:55.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.726+0000 7faa927fc700 1 -- 172.21.15.70:0/3820799242 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faa9c197ce0 con 0x7faa8806dd60 2024-04-02T04:57:55.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faa8806dd60 msgr2=0x7faa88070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faa8806dd60 0x7faa88070220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005fd0 tx=0x7faa8c00a560 comp rx=0 tx=0).stop 2024-04-02T04:57:55.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 msgr2=0x7faa9c19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 0x7faa9c19c700 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa98006010 tx=0x7faa9800be20 comp rx=0 tx=0).stop 2024-04-02T04:57:55.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 shutdown_connections 2024-04-02T04:57:55.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faa8806dd60 0x7faa88070220 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa9c0ff5a0 0x7faa9c19c700 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 --2- 172.21.15.70:0/3820799242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa9c0fff50 0x7faa9c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 >> 172.21.15.70:0/3820799242 conn(0x7faa9c074e30 msgr2=0x7faa9c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:55.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 shutdown_connections 2024-04-02T04:57:55.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.729+0000 7faaa33d1700 1 -- 172.21.15.70:0/3820799242 wait complete. 2024-04-02T04:57:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.831+0000 7fb8484e8700 1 -- 172.21.15.70:0/1003385231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 msgr2=0x7fb840100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.831+0000 7fb8484e8700 1 --2- 172.21.15.70:0/1003385231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 0x7fb840100b60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009910 tx=0x7fb83c009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 -- 172.21.15.70:0/1003385231 shutdown_connections 2024-04-02T04:57:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 --2- 172.21.15.70:0/1003385231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840101130 0x7fb8401047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 --2- 172.21.15.70:0/1003385231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 0x7fb840100b60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 -- 172.21.15.70:0/1003385231 >> 172.21.15.70:0/1003385231 conn(0x7fb840074e30 msgr2=0x7fb840075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 -- 172.21.15.70:0/1003385231 shutdown_connections 2024-04-02T04:57:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 -- 172.21.15.70:0/1003385231 wait complete. 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 Processor -- start 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 -- start start 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 0x7fb840197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840101130 0x7fb840197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8401982b0 con 0x7fb840100780 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.832+0000 7fb8484e8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8401983f0 con 0x7fb840101130 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb846284700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 0x7fb840197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb846284700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 0x7fb840197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54412/0 (socket says 172.21.15.70:54412) 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb846284700 1 -- 172.21.15.70:0/1049425601 learned_addr learned my addr 172.21.15.70:0/1049425601 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb845a83700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840101130 0x7fb840197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb846284700 1 -- 172.21.15.70:0/1049425601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840101130 msgr2=0x7fb840197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb846284700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840101130 0x7fb840197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb846284700 1 -- 172.21.15.70:0/1049425601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c0095f0 con 0x7fb840100780 2024-04-02T04:57:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb846284700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 0x7fb840197690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb83c006010 tx=0x7fb83c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb8377fe700 1 -- 172.21.15.70:0/1049425601 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb83c01d070 con 0x7fb840100780 2024-04-02T04:57:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb8377fe700 1 -- 172.21.15.70:0/1049425601 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb83c005dc0 con 0x7fb840100780 2024-04-02T04:57:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8401a0700 con 0x7fb840100780 2024-04-02T04:57:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb8377fe700 1 -- 172.21.15.70:0/1049425601 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb83c017400 con 0x7fb840100780 2024-04-02T04:57:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.833+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8401a0b70 con 0x7fb840100780 2024-04-02T04:57:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.834+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb840108120 con 0x7fb840100780 2024-04-02T04:57:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.835+0000 7fb8377fe700 1 -- 172.21.15.70:0/1049425601 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb83c003a60 con 0x7fb840100780 2024-04-02T04:57:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.835+0000 7fb8377fe700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82c06dd60 0x7fb82c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.835+0000 7fb8377fe700 1 -- 172.21.15.70:0/1049425601 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fb83c08f040 con 0x7fb840100780 2024-04-02T04:57:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.835+0000 7fb845a83700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82c06dd60 0x7fb82c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.835+0000 7fb845a83700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82c06dd60 0x7fb82c070220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb8401004f0 tx=0x7fb83000b3c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:55.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:55.840+0000 7fb8377fe700 1 -- 172.21.15.70:0/1049425601 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb83c0584f0 con 0x7fb840100780 2024-04-02T04:57:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.041+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8400ffba0 con 0x7fb82c06dd60 2024-04-02T04:57:56.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.052+0000 7fb8377fe700 1 -- 172.21.15.70:0/1049425601 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb830015070 con 0x7fb82c06dd60 2024-04-02T04:57:56.057 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:57:56.057 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (7m) 29s ago 18m 25.8M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:57:56.057 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (18m) 29s ago 18m 9084k - 18.2.1 5be31c24972a b0d22ce36524 2024-04-02T04:57:56.057 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (17m) 3s ago 17m 9034k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:57:56.057 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (4m) 29s ago 18m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:57:56.057 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (4m) 3s ago 17m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (7m) 29s ago 18m 79.7M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (65s) 29s ago 12m 13.4M - 19.0.0-1979-g6cec793d 78f830190839 2a4aa2b03d0d 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (35s) 29s ago 12m 11.5M - 19.0.0-1979-g6cec793d 78f830190839 3b0eb47520be 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (12m) 3s ago 12m 88.9M - 18.2.1 5be31c24972a 8b26d91d131f 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (6s) 3s ago 12m 11.6M - 19.0.0-1979-g6cec793d 78f830190839 49c2ed39fbf7 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (9m) 29s ago 19m 509M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (8m) 3s ago 17m 451M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (5m) 29s ago 20m 54.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (5m) 3s ago 17m 47.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:57:56.058 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 29s ago 18m 15.9M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:57:56.059 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (8m) 3s ago 17m 16.9M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:57:56.059 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (4m) 29s ago 16m 147M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:57:56.059 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (3m) 29s ago 16m 143M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:57:56.059 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (3m) 29s ago 15m 95.0M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T04:57:56.059 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (2m) 3s ago 15m 109M 2240M 19.0.0-1979-g6cec793d 78f830190839 f717fd6c8306 2024-04-02T04:57:56.059 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (117s) 3s ago 14m 111M 2240M 19.0.0-1979-g6cec793d 78f830190839 2e5bbf3235ff 2024-04-02T04:57:56.059 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (84s) 3s ago 14m 101M 2240M 19.0.0-1979-g6cec793d 78f830190839 71351459ac70 2024-04-02T04:57:56.059 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (7m) 29s ago 18m 65.1M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:57:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.054+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82c06dd60 msgr2=0x7fb82c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.054+0000 7fb8484e8700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82c06dd60 0x7fb82c070220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb8401004f0 tx=0x7fb83000b3c0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.054+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 msgr2=0x7fb840197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.054+0000 7fb8484e8700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 0x7fb840197690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb83c006010 tx=0x7fb83c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:57:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.055+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 shutdown_connections 2024-04-02T04:57:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.055+0000 7fb8484e8700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82c06dd60 0x7fb82c070220 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.055+0000 7fb8484e8700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840100780 0x7fb840197690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.055+0000 7fb8484e8700 1 --2- 172.21.15.70:0/1049425601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb840101130 0x7fb840197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.055+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 >> 172.21.15.70:0/1049425601 conn(0x7fb840074e30 msgr2=0x7fb8400fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.055+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 shutdown_connections 2024-04-02T04:57:56.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.055+0000 7fb8484e8700 1 -- 172.21.15.70:0/1049425601 wait complete. 2024-04-02T04:57:56.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 -- 172.21.15.70:0/3903609889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 msgr2=0x7f54b4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 --2- 172.21.15.70:0/3903609889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 0x7f54b4101280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009910 tx=0x7f54a4009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:56.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 -- 172.21.15.70:0/3903609889 shutdown_connections 2024-04-02T04:57:56.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 --2- 172.21.15.70:0/3903609889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b4101850 0x7f54b41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 --2- 172.21.15.70:0/3903609889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 0x7f54b4101280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 -- 172.21.15.70:0/3903609889 >> 172.21.15.70:0/3903609889 conn(0x7f54b40fc750 msgr2=0x7f54b40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:56.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 -- 172.21.15.70:0/3903609889 shutdown_connections 2024-04-02T04:57:56.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 -- 172.21.15.70:0/3903609889 wait complete. 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.158+0000 7f54b9ddd700 1 Processor -- start 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b9ddd700 1 -- start start 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b9ddd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 0x7f54b419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b9ddd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b4101850 0x7f54b419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b9ddd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b419d2e0 con 0x7f54b4100ea0 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b9ddd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4196790 con 0x7f54b4101850 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 0x7f54b419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 0x7f54b419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54426/0 (socket says 172.21.15.70:54426) 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b37fe700 1 -- 172.21.15.70:0/1887942224 learned_addr learned my addr 172.21.15.70:0/1887942224 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b2ffd700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b4101850 0x7f54b419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b37fe700 1 -- 172.21.15.70:0/1887942224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b4101850 msgr2=0x7f54b419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b37fe700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b4101850 0x7f54b419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b37fe700 1 -- 172.21.15.70:0/1887942224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40095f0 con 0x7f54b4100ea0 2024-04-02T04:57:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.159+0000 7f54b37fe700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 0x7f54b419c710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f54a4006010 tx=0x7f54a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.160+0000 7f54b0ff9700 1 -- 172.21.15.70:0/1887942224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a401d070 con 0x7f54b4100ea0 2024-04-02T04:57:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.160+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b4196a10 con 0x7f54b4100ea0 2024-04-02T04:57:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.160+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b4196f00 con 0x7f54b4100ea0 2024-04-02T04:57:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.160+0000 7f54b0ff9700 1 -- 172.21.15.70:0/1887942224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54a4005dc0 con 0x7f54b4100ea0 2024-04-02T04:57:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.160+0000 7f54b0ff9700 1 -- 172.21.15.70:0/1887942224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a4017400 con 0x7f54b4100ea0 2024-04-02T04:57:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.161+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b4109240 con 0x7f54b4100ea0 2024-04-02T04:57:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.161+0000 7f54b0ff9700 1 -- 172.21.15.70:0/1887942224 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f54a4003a60 con 0x7f54b4100ea0 2024-04-02T04:57:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.162+0000 7f54b0ff9700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f549c06dd60 0x7f549c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.162+0000 7f54b0ff9700 1 -- 172.21.15.70:0/1887942224 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f54a408f020 con 0x7f54b4100ea0 2024-04-02T04:57:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.162+0000 7f54b2ffd700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f549c06dd60 0x7f549c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.162+0000 7f54b2ffd700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f549c06dd60 0x7f549c070220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f54b4197c00 tx=0x7f54a800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:56.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.170+0000 7f54b0ff9700 1 -- 172.21.15.70:0/1887942224 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f54a405c000 con 0x7f54b4100ea0 2024-04-02T04:57:56.455 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:56 smithi179 ceph-mon[84288]: pgmap v220: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:57:56.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.458+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f54b404f2e0 con 0x7f54b4100ea0 2024-04-02T04:57:56.463 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:56 smithi070 ceph-mon[113635]: pgmap v220: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.459+0000 7f54b0ff9700 1 -- 172.21.15.70:0/1887942224 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f54a405bb90 con 0x7f54b4100ea0 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:57:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 3 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 13 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:57:56.465 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f549c06dd60 msgr2=0x7f549c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f549c06dd60 0x7f549c070220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f54b4197c00 tx=0x7f54a800b410 comp rx=0 tx=0).stop 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 msgr2=0x7f54b419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 0x7f54b419c710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f54a4006010 tx=0x7f54a400bd70 comp rx=0 tx=0).stop 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 shutdown_connections 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f549c06dd60 0x7f549c070220 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54b4100ea0 0x7f54b419c710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 --2- 172.21.15.70:0/1887942224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b4101850 0x7f54b419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 >> 172.21.15.70:0/1887942224 conn(0x7f54b40fc750 msgr2=0x7f54b40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:56.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 shutdown_connections 2024-04-02T04:57:56.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.461+0000 7f54b9ddd700 1 -- 172.21.15.70:0/1887942224 wait complete. 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 -- 172.21.15.70:0/1826606014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 msgr2=0x7f3d14105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 --2- 172.21.15.70:0/1826606014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 0x7f3d14105fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009910 tx=0x7f3d04009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 -- 172.21.15.70:0/1826606014 shutdown_connections 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 --2- 172.21.15.70:0/1826606014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d140ffb40 0x7f3d140fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 --2- 172.21.15.70:0/1826606014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 0x7f3d14105fd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 -- 172.21.15.70:0/1826606014 >> 172.21.15.70:0/1826606014 conn(0x7f3d140fb650 msgr2=0x7f3d140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 -- 172.21.15.70:0/1826606014 shutdown_connections 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 -- 172.21.15.70:0/1826606014 wait complete. 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.565+0000 7f3d19bac700 1 Processor -- start 2024-04-02T04:57:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d19bac700 1 -- start start 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d19bac700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d140ffb40 0x7f3d1419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d19bac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 0x7f3d1419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d19bac700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1419d170 con 0x7f3d14105bf0 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d19bac700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14196620 con 0x7f3d140ffb40 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d12ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 0x7f3d1419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d12ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 0x7f3d1419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54448/0 (socket says 172.21.15.70:54448) 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d12ffd700 1 -- 172.21.15.70:0/950515172 learned_addr learned my addr 172.21.15.70:0/950515172 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d137fe700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d140ffb40 0x7f3d1419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d12ffd700 1 -- 172.21.15.70:0/950515172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d140ffb40 msgr2=0x7f3d1419c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d12ffd700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d140ffb40 0x7f3d1419c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d12ffd700 1 -- 172.21.15.70:0/950515172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040095f0 con 0x7f3d14105bf0 2024-04-02T04:57:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d12ffd700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 0x7f3d1419cae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800e920 tx=0x7f3d0800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.566+0000 7f3d10ff9700 1 -- 172.21.15.70:0/950515172 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0800c9d0 con 0x7f3d14105bf0 2024-04-02T04:57:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.567+0000 7f3d10ff9700 1 -- 172.21.15.70:0/950515172 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d08010430 con 0x7f3d14105bf0 2024-04-02T04:57:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.567+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d14196900 con 0x7f3d14105bf0 2024-04-02T04:57:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.567+0000 7f3d10ff9700 1 -- 172.21.15.70:0/950515172 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d08018630 con 0x7f3d14105bf0 2024-04-02T04:57:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.567+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d14196e50 con 0x7f3d14105bf0 2024-04-02T04:57:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.567+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1404f2e0 con 0x7f3d14105bf0 2024-04-02T04:57:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.568+0000 7f3d10ff9700 1 -- 172.21.15.70:0/950515172 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f3d080105a0 con 0x7f3d14105bf0 2024-04-02T04:57:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.569+0000 7f3d10ff9700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3cfc06dd60 0x7f3cfc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.569+0000 7f3d10ff9700 1 -- 172.21.15.70:0/950515172 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f3d08014070 con 0x7f3d14105bf0 2024-04-02T04:57:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.569+0000 7f3d137fe700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3cfc06dd60 0x7f3cfc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.569+0000 7f3d137fe700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3cfc06dd60 0x7f3cfc070220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3d040059d0 tx=0x7f3d04005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:56.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.573+0000 7f3d10ff9700 1 -- 172.21.15.70:0/950515172 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3d08053280 con 0x7f3d14105bf0 2024-04-02T04:57:56.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.816+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3d14197b30 con 0x7f3d14105bf0 2024-04-02T04:57:56.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.816+0000 7f3d10ff9700 1 -- 172.21.15.70:0/950515172 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1892 (secure 0 0 0) 0x7f3d0805aeb0 con 0x7f3d14105bf0 2024-04-02T04:57:56.821 INFO:teuthology.orchestra.run.smithi070.stdout:e32 2024-04-02T04:57:56.821 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:57:56.821 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:57:56.821 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:57:56.821 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 31 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:57:46.167773+0000 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:57:56.822 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 94 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=24267} 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:57:56.823 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 24267 members: 24267 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{0:24267} state up:active seq 193 join_fscid=1 addr [v2:172.21.15.179:6826/3541837059,v1:172.21.15.179:6827/3541837059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6828/3294303398,v1:172.21.15.70:6829/3294303398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:57:56.824 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:44287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.179:6824/1866330674,v1:172.21.15.179:6825/1866330674] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:57:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3cfc06dd60 msgr2=0x7f3cfc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3cfc06dd60 0x7f3cfc070220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3d040059d0 tx=0x7f3d04005940 comp rx=0 tx=0).stop 2024-04-02T04:57:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 msgr2=0x7f3d1419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 0x7f3d1419cae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800e920 tx=0x7f3d0800ec30 comp rx=0 tx=0).stop 2024-04-02T04:57:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 shutdown_connections 2024-04-02T04:57:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3cfc06dd60 0x7f3cfc070220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d140ffb40 0x7f3d1419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 --2- 172.21.15.70:0/950515172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d14105bf0 0x7f3d1419cae0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.819+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 >> 172.21.15.70:0/950515172 conn(0x7f3d140fb650 msgr2=0x7f3d14068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.820+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 shutdown_connections 2024-04-02T04:57:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.820+0000 7f3d19bac700 1 -- 172.21.15.70:0/950515172 wait complete. 2024-04-02T04:57:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.924+0000 7f95b14ee700 1 -- 172.21.15.70:0/2884572848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 msgr2=0x7f95ac105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.924+0000 7f95b14ee700 1 --2- 172.21.15.70:0/2884572848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 0x7f95ac105730 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f959c009910 tx=0x7f959c009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.924+0000 7f95b14ee700 1 -- 172.21.15.70:0/2884572848 shutdown_connections 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.924+0000 7f95b14ee700 1 --2- 172.21.15.70:0/2884572848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac0ff340 0x7f95ac0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.924+0000 7f95b14ee700 1 --2- 172.21.15.70:0/2884572848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 0x7f95ac105730 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.924+0000 7f95b14ee700 1 -- 172.21.15.70:0/2884572848 >> 172.21.15.70:0/2884572848 conn(0x7f95ac074e30 msgr2=0x7f95ac075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.924+0000 7f95b14ee700 1 -- 172.21.15.70:0/2884572848 shutdown_connections 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.924+0000 7f95b14ee700 1 -- 172.21.15.70:0/2884572848 wait complete. 2024-04-02T04:57:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95b14ee700 1 Processor -- start 2024-04-02T04:57:56.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95b14ee700 1 -- start start 2024-04-02T04:57:56.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95b14ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac0ff340 0x7f95ac19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95b14ee700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 0x7f95ac19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95b14ee700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac19d240 con 0x7f95ac105350 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95b14ee700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1966c0 con 0x7f95ac0ff340 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95aa7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 0x7f95ac19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95aa7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 0x7f95ac19cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35530/0 (socket says 172.21.15.70:35530) 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95aa7fc700 1 -- 172.21.15.70:0/4120693321 learned_addr learned my addr 172.21.15.70:0/4120693321 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95aaffd700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac0ff340 0x7f95ac19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95aa7fc700 1 -- 172.21.15.70:0/4120693321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac0ff340 msgr2=0x7f95ac19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95aa7fc700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac0ff340 0x7f95ac19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.925+0000 7f95aa7fc700 1 -- 172.21.15.70:0/4120693321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0095f0 con 0x7f95ac105350 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.926+0000 7f95aa7fc700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 0x7f95ac19cb60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95a000db60 tx=0x7f95a000de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.926+0000 7f9593fff700 1 -- 172.21.15.70:0/4120693321 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95a0007c30 con 0x7f95ac105350 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.926+0000 7f9593fff700 1 -- 172.21.15.70:0/4120693321 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95a00044e0 con 0x7f95ac105350 2024-04-02T04:57:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.926+0000 7f9593fff700 1 -- 172.21.15.70:0/4120693321 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95a0005250 con 0x7f95ac105350 2024-04-02T04:57:56.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.926+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac1969a0 con 0x7f95ac105350 2024-04-02T04:57:56.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.926+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac196ef0 con 0x7f95ac105350 2024-04-02T04:57:56.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.927+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac107850 con 0x7f95ac105350 2024-04-02T04:57:56.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.927+0000 7f9593fff700 1 -- 172.21.15.70:0/4120693321 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f95a0005470 con 0x7f95ac105350 2024-04-02T04:57:56.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.928+0000 7f9593fff700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f959406dd60 0x7f9594070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:56.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.928+0000 7f9593fff700 1 -- 172.21.15.70:0/4120693321 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f95a008dac0 con 0x7f95ac105350 2024-04-02T04:57:56.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.928+0000 7f95aaffd700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f959406dd60 0x7f9594070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:56.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.928+0000 7f95aaffd700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f959406dd60 0x7f9594070220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f959c006010 tx=0x7f959c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:56.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:56.932+0000 7f9593fff700 1 -- 172.21.15.70:0/4120693321 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f95a005aa20 con 0x7f95ac105350 2024-04-02T04:57:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.143+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95ac197170 con 0x7f959406dd60 2024-04-02T04:57:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.144+0000 7f9593fff700 1 -- 172.21.15.70:0/4120693321 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f95ac197170 con 0x7f959406dd60 2024-04-02T04:57:57.148 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:57:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:57:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:57:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:57:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: "osd", 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "15/23 daemons upgraded", 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mds daemons", 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:57:57.149 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:57:57.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f959406dd60 msgr2=0x7f9594070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f959406dd60 0x7f9594070220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f959c006010 tx=0x7f959c01a040 comp rx=0 tx=0).stop 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 msgr2=0x7f95ac19cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 0x7f95ac19cb60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95a000db60 tx=0x7f95a000de70 comp rx=0 tx=0).stop 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 shutdown_connections 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f959406dd60 0x7f9594070220 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac0ff340 0x7f95ac19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 --2- 172.21.15.70:0/4120693321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ac105350 0x7f95ac19cb60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 >> 172.21.15.70:0/4120693321 conn(0x7f95ac074e30 msgr2=0x7f95ac0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 shutdown_connections 2024-04-02T04:57:57.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.146+0000 7f95b14ee700 1 -- 172.21.15.70:0/4120693321 wait complete. 2024-04-02T04:57:57.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.250+0000 7f81f2837700 1 -- 172.21.15.70:0/1075806578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec101850 msgr2=0x7f81ec1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:57.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.250+0000 7f81f2837700 1 --2- 172.21.15.70:0/1075806578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec101850 0x7f81ec1058a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009910 tx=0x7f81e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:57:57.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.250+0000 7f81f2837700 1 -- 172.21.15.70:0/1075806578 shutdown_connections 2024-04-02T04:57:57.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.250+0000 7f81f2837700 1 --2- 172.21.15.70:0/1075806578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec101850 0x7f81ec1058a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.250+0000 7f81f2837700 1 --2- 172.21.15.70:0/1075806578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec100ea0 0x7f81ec101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.250+0000 7f81f2837700 1 -- 172.21.15.70:0/1075806578 >> 172.21.15.70:0/1075806578 conn(0x7f81ec0fc750 msgr2=0x7f81ec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:57.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.250+0000 7f81f2837700 1 -- 172.21.15.70:0/1075806578 shutdown_connections 2024-04-02T04:57:57.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.250+0000 7f81f2837700 1 -- 172.21.15.70:0/1075806578 wait complete. 2024-04-02T04:57:57.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81f2837700 1 Processor -- start 2024-04-02T04:57:57.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81f2837700 1 -- start start 2024-04-02T04:57:57.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81f2837700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec100ea0 0x7f81ec19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:57.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81f2837700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec101850 0x7f81ec19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:57.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81f2837700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec19d360 con 0x7f81ec100ea0 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81f2837700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec196810 con 0x7f81ec101850 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81ebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec100ea0 0x7f81ec19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81ebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec100ea0 0x7f81ec19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35550/0 (socket says 172.21.15.70:35550) 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81ebfff700 1 -- 172.21.15.70:0/3602549005 learned_addr learned my addr 172.21.15.70:0/3602549005 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.251+0000 7f81eb7fe700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec101850 0x7f81ec19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81ebfff700 1 -- 172.21.15.70:0/3602549005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec101850 msgr2=0x7f81ec19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81ebfff700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec101850 0x7f81ec19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81ebfff700 1 -- 172.21.15.70:0/3602549005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e00095f0 con 0x7f81ec100ea0 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81ebfff700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec100ea0 0x7f81ec19c790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00b4b0 tx=0x7f81dc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81e97fa700 1 -- 172.21.15.70:0/3602549005 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81dc011800 con 0x7f81ec100ea0 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81e97fa700 1 -- 172.21.15.70:0/3602549005 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81dc011e20 con 0x7f81ec100ea0 2024-04-02T04:57:57.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81e97fa700 1 -- 172.21.15.70:0/3602549005 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81dc00f2e0 con 0x7f81ec100ea0 2024-04-02T04:57:57.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec196af0 con 0x7f81ec100ea0 2024-04-02T04:57:57.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.252+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec197040 con 0x7f81ec100ea0 2024-04-02T04:57:57.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.253+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec1091d0 con 0x7f81ec100ea0 2024-04-02T04:57:57.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.254+0000 7f81e97fa700 1 -- 172.21.15.70:0/3602549005 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f81dc010380 con 0x7f81ec100ea0 2024-04-02T04:57:57.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.254+0000 7f81e97fa700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f81d406dd60 0x7f81d4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:57:57.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.254+0000 7f81e97fa700 1 -- 172.21.15.70:0/3602549005 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f81dc08d800 con 0x7f81ec100ea0 2024-04-02T04:57:57.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.254+0000 7f81eb7fe700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f81d406dd60 0x7f81d4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:57:57.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.254+0000 7f81eb7fe700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f81d406dd60 0x7f81d4070220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0fd160 tx=0x7f81e001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:57:57.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.258+0000 7f81e97fa700 1 -- 172.21.15.70:0/3602549005 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f81dc05a760 con 0x7f81ec100ea0 2024-04-02T04:57:57.283 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:57 smithi179 ceph-mon[84288]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:57.283 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:57 smithi179 ceph-mon[84288]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:57.283 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:57 smithi179 ceph-mon[84288]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:57.283 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:57.283 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:57 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:57.284 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:57 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1887942224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:57.284 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:57 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/950515172' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:57:57.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:57 smithi070 ceph-mon[113635]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:57.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:57 smithi070 ceph-mon[113635]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:57.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:57 smithi070 ceph-mon[113635]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:57.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:57.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:57 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:57.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:57 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1887942224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:57.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:57 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/950515172' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:57:57.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.532+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f81ec04f2e0 con 0x7f81ec100ea0 2024-04-02T04:57:57.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.532+0000 7f81e97fa700 1 -- 172.21.15.70:0/3602549005 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f81dc05a2f0 con 0x7f81ec100ea0 2024-04-02T04:57:57.537 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:57:57.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.534+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f81d406dd60 msgr2=0x7f81d4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:57.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.534+0000 7f81f2837700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f81d406dd60 0x7f81d4070220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0fd160 tx=0x7f81e001a040 comp rx=0 tx=0).stop 2024-04-02T04:57:57.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec100ea0 msgr2=0x7f81ec19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:57:57.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec100ea0 0x7f81ec19c790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00b4b0 tx=0x7f81dc00b7c0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 shutdown_connections 2024-04-02T04:57:57.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f81d406dd60 0x7f81d4070220 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ec100ea0 0x7f81ec19c790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 --2- 172.21.15.70:0/3602549005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec101850 0x7f81ec19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:57:57.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 >> 172.21.15.70:0/3602549005 conn(0x7f81ec0fc750 msgr2=0x7f81ec0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:57:57.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 shutdown_connections 2024-04-02T04:57:57.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:57:57.535+0000 7f81f2837700 1 -- 172.21.15.70:0/3602549005 wait complete. 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: pgmap v221: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 686 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3602549005' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:58.437 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:58.438 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:58.438 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:58.438 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:57:58 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: pgmap v221: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 686 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3602549005' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:58.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:58.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:58.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:57:58.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:57:58 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:00.315 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:00 smithi179 ceph-mon[84288]: pgmap v222: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:58:00.315 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:00 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:00.315 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:00 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.atrmwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:58:00.315 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:00 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:00 smithi070 ceph-mon[113635]: pgmap v222: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:58:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:00 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:00 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi179.atrmwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:58:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:00 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:01.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:01 smithi179 ceph-mon[84288]: Upgrade: Updating mds.cephfs.smithi179.atrmwz 2024-04-02T04:58:01.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:01 smithi179 ceph-mon[84288]: Deploying daemon mds.cephfs.smithi179.atrmwz on smithi179 2024-04-02T04:58:01.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:01 smithi070 ceph-mon[113635]: Upgrade: Updating mds.cephfs.smithi179.atrmwz 2024-04-02T04:58:01.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:01 smithi070 ceph-mon[113635]: Deploying daemon mds.cephfs.smithi179.atrmwz on smithi179 2024-04-02T04:58:02.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:01 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[113631]: 2024-04-02T04:58:01.604+0000 7f6fcc9da700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:58:02.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: pgmap v223: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-02T04:58:02.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:58:02.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:58:02.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: osdmap e95: 6 total, 6 up, 6 in 2024-04-02T04:58:02.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: Standby daemon mds.cephfs.smithi070.mnzrwt assigned to filesystem cephfs as rank 0 2024-04-02T04:58:02.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:58:02.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:58:02.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi070.mnzrwt=up:replay} 2 up:standby 2024-04-02T04:58:02.323 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:02 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:58:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: pgmap v223: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-04-02T04:58:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:58:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:58:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: osdmap e95: 6 total, 6 up, 6 in 2024-04-02T04:58:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: Standby daemon mds.cephfs.smithi070.mnzrwt assigned to filesystem cephfs as rank 0 2024-04-02T04:58:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:58:02.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:58:02.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi070.mnzrwt=up:replay} 2 up:standby 2024-04-02T04:58:02.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:02 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:58:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:04 smithi070 ceph-mon[113635]: pgmap v225: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-04-02T04:58:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:04 smithi179 ceph-mon[84288]: pgmap v225: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-04-02T04:58:06.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:06 smithi070 ceph-mon[113635]: pgmap v226: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s rd, 2 op/s 2024-04-02T04:58:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:06 smithi179 ceph-mon[84288]: pgmap v226: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s rd, 2 op/s 2024-04-02T04:58:08.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:08 smithi070 ceph-mon[113635]: pgmap v227: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 9.8 MiB/s rd, 3 op/s 2024-04-02T04:58:08.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:08 smithi179 ceph-mon[84288]: pgmap v227: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 9.8 MiB/s rd, 3 op/s 2024-04-02T04:58:10.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:10 smithi070 ceph-mon[113635]: pgmap v228: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-02T04:58:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:10 smithi179 ceph-mon[84288]: pgmap v228: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-02T04:58:11.466 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:11 smithi179 ceph-mon[84288]: reconnect by client.14532 192.168.0.1:0/3290270869 after 0.000999972 2024-04-02T04:58:11.466 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:11 smithi179 ceph-mon[84288]: reconnect by client.24309 192.168.0.1:0/2957635722 after 0.000999972 2024-04-02T04:58:11.466 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:11 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] up:reconnect 2024-04-02T04:58:11.466 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:11 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi070.mnzrwt=up:reconnect} 2 up:standby 2024-04-02T04:58:11.467 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:11.467 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:11.467 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:11 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] up:rejoin 2024-04-02T04:58:11.467 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:11 smithi179 ceph-mon[84288]: fsmap cephfs:1/1 {0=cephfs.smithi070.mnzrwt=up:rejoin} 2 up:standby 2024-04-02T04:58:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:11 smithi070 ceph-mon[113635]: reconnect by client.14532 192.168.0.1:0/3290270869 after 0.000999972 2024-04-02T04:58:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:11 smithi070 ceph-mon[113635]: reconnect by client.24309 192.168.0.1:0/2957635722 after 0.000999972 2024-04-02T04:58:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:11 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] up:reconnect 2024-04-02T04:58:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:11 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi070.mnzrwt=up:reconnect} 2 up:standby 2024-04-02T04:58:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:11 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] up:rejoin 2024-04-02T04:58:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:11 smithi070 ceph-mon[113635]: fsmap cephfs:1/1 {0=cephfs.smithi070.mnzrwt=up:rejoin} 2 up:standby 2024-04-02T04:58:12.508 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:12 smithi179 ceph-mon[84288]: pgmap v229: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-02T04:58:12.508 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:12 smithi179 ceph-mon[84288]: daemon mds.cephfs.smithi070.mnzrwt is now active in filesystem cephfs as rank 0 2024-04-02T04:58:12.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:12 smithi070 ceph-mon[113635]: pgmap v229: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-02T04:58:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:12 smithi070 ceph-mon[113635]: daemon mds.cephfs.smithi070.mnzrwt is now active in filesystem cephfs as rank 0 2024-04-02T04:58:13.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:13 smithi070 ceph-mon[113635]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:58:13.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:13 smithi070 ceph-mon[113635]: Cluster is now healthy 2024-04-02T04:58:13.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:13 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] up:active 2024-04-02T04:58:13.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:13 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 2 up:standby 2024-04-02T04:58:13.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:13 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:13.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:13 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:13.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:13 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:13.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:13 smithi179 ceph-mon[84288]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:58:13.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:13 smithi179 ceph-mon[84288]: Cluster is now healthy 2024-04-02T04:58:13.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:13 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] up:active 2024-04-02T04:58:13.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:13 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 2 up:standby 2024-04-02T04:58:13.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:13 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:13.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:13 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:13.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:13 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:14.458 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:14 smithi179 ceph-mon[84288]: pgmap v230: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 88 B/s wr, 5 op/s 2024-04-02T04:58:14.458 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:14 smithi179 ceph-mon[84288]: mds.? [v2:172.21.15.179:6826/1653234055,v1:172.21.15.179:6827/1653234055] up:boot 2024-04-02T04:58:14.458 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:14 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:58:14.458 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:14 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:58:14.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:14 smithi070 ceph-mon[113635]: pgmap v230: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 88 B/s wr, 5 op/s 2024-04-02T04:58:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:14 smithi070 ceph-mon[113635]: mds.? [v2:172.21.15.179:6826/1653234055,v1:172.21.15.179:6827/1653234055] up:boot 2024-04-02T04:58:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:14 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:58:14.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:14 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:58:16.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:16 smithi070 ceph-mon[113635]: pgmap v231: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T04:58:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:16 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:16.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:16 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:16.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:16 smithi179 ceph-mon[84288]: pgmap v231: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T04:58:16.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:16 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:16.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:16 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:18.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:17 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:18.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:17 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:18.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:17 smithi179 ceph-mon[84288]: pgmap v232: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T04:58:18.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:18.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:17 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:18.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:17 smithi070 ceph-mon[113635]: pgmap v232: 65 pgs: 65 active+clean; 205 MiB data, 863 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T04:58:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:20 smithi070 ceph-mon[113635]: pgmap v233: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T04:58:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:20 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:20 smithi179 ceph-mon[84288]: pgmap v233: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 8.5 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T04:58:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:20.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:20 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all mds 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.mnzrwt"}]': finished 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.zvraae"}]': finished 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi179.atrmwz"}]': finished 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi179.mzfimf"}]': finished 2024-04-02T04:58:21.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:21 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-02T04:58:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all mds 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.mnzrwt"}]: dispatch 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.mnzrwt"}]': finished 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.zvraae"}]: dispatch 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.zvraae"}]': finished 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi179.atrmwz"}]: dispatch 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi179.atrmwz"}]': finished 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi179.mzfimf"}]: dispatch 2024-04-02T04:58:21.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi179.mzfimf"}]': finished 2024-04-02T04:58:21.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:21 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-02T04:58:22.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: Upgrade: Setting filesystem cephfs Joinable 2024-04-02T04:58:22.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: pgmap v234: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-02T04:58:22.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-02T04:58:22.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:58:22.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:22.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:22.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:22.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:22.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:22 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: Upgrade: Setting filesystem cephfs Joinable 2024-04-02T04:58:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: pgmap v234: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-02T04:58:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-02T04:58:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:58:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:22.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:22 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:23.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:23 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all rgw 2024-04-02T04:58:23.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:23 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all rbd-mirror 2024-04-02T04:58:23.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:23 smithi070 ceph-mon[113635]: Upgrade: Updating ceph-exporter.smithi070 (1/2) 2024-04-02T04:58:23.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:23 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:23.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:23 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:58:23.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:23 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:23.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:23 smithi070 ceph-mon[113635]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-04-02T04:58:23.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:23 smithi070 ceph-mon[113635]: pgmap v235: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 345 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:58:23.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:23 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all rgw 2024-04-02T04:58:23.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:23 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all rbd-mirror 2024-04-02T04:58:23.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:23 smithi179 ceph-mon[84288]: Upgrade: Updating ceph-exporter.smithi070 (1/2) 2024-04-02T04:58:23.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:23 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:23.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:23 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:58:23.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:23 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:23.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:23 smithi179 ceph-mon[84288]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-04-02T04:58:23.823 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:23 smithi179 ceph-mon[84288]: pgmap v235: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 345 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-02T04:58:26.361 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:26 smithi070 ceph-mon[113635]: pgmap v236: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-02T04:58:26.361 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:26.361 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:26 smithi179 ceph-mon[84288]: pgmap v236: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-02T04:58:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:27.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.644+0000 7fbde790e700 1 -- 172.21.15.70:0/3840657016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde00ffc20 msgr2=0x7fbde0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:27.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.644+0000 7fbde790e700 1 --2- 172.21.15.70:0/3840657016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde00ffc20 0x7fbde0100080 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009910 tx=0x7fbdd0009c20 comp rx=0 tx=0).stop 2024-04-02T04:58:27.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.644+0000 7fbde790e700 1 -- 172.21.15.70:0/3840657016 shutdown_connections 2024-04-02T04:58:27.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.644+0000 7fbde790e700 1 --2- 172.21.15.70:0/3840657016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde00ffc20 0x7fbde0100080 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:27.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.644+0000 7fbde790e700 1 --2- 172.21.15.70:0/3840657016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde0105bd0 0x7fbde0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:27.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.644+0000 7fbde790e700 1 -- 172.21.15.70:0/3840657016 >> 172.21.15.70:0/3840657016 conn(0x7fbde0075670 msgr2=0x7fbde0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:27.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.644+0000 7fbde790e700 1 -- 172.21.15.70:0/3840657016 shutdown_connections 2024-04-02T04:58:27.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.644+0000 7fbde790e700 1 -- 172.21.15.70:0/3840657016 wait complete. 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde790e700 1 Processor -- start 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde790e700 1 -- start start 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde790e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde00ffc20 0x7fbde019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde790e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde0105bd0 0x7fbde019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde790e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde019d200 con 0x7fbde0105bd0 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde790e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde01966b0 con 0x7fbde00ffc20 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde4ea9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde0105bd0 0x7fbde019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde4ea9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde0105bd0 0x7fbde019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55716/0 (socket says 172.21.15.70:55716) 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde4ea9700 1 -- 172.21.15.70:0/4135215408 learned_addr learned my addr 172.21.15.70:0/4135215408 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.645+0000 7fbde56aa700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde00ffc20 0x7fbde019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbde4ea9700 1 -- 172.21.15.70:0/4135215408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde00ffc20 msgr2=0x7fbde019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbde4ea9700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde00ffc20 0x7fbde019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbde4ea9700 1 -- 172.21.15.70:0/4135215408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00095f0 con 0x7fbde0105bd0 2024-04-02T04:58:27.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbde4ea9700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde0105bd0 0x7fbde019cb70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b5c0 tx=0x7fbdd000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:27.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbdd67fc700 1 -- 172.21.15.70:0/4135215408 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd001d070 con 0x7fbde0105bd0 2024-04-02T04:58:27.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde0196930 con 0x7fbde0105bd0 2024-04-02T04:58:27.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbdd67fc700 1 -- 172.21.15.70:0/4135215408 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdd0005e00 con 0x7fbde0105bd0 2024-04-02T04:58:27.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbdd67fc700 1 -- 172.21.15.70:0/4135215408 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd0017400 con 0x7fbde0105bd0 2024-04-02T04:58:27.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.646+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde0196e20 con 0x7fbde0105bd0 2024-04-02T04:58:27.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.647+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde004f2e0 con 0x7fbde0105bd0 2024-04-02T04:58:27.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.648+0000 7fbdd67fc700 1 -- 172.21.15.70:0/4135215408 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbdd0003a60 con 0x7fbde0105bd0 2024-04-02T04:58:27.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.648+0000 7fbdd67fc700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbdcc06dd60 0x7fbdcc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:27.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.648+0000 7fbdd67fc700 1 -- 172.21.15.70:0/4135215408 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fbdd008f190 con 0x7fbde0105bd0 2024-04-02T04:58:27.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.648+0000 7fbde56aa700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbdcc06dd60 0x7fbdcc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:27.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.648+0000 7fbde56aa700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbdcc06dd60 0x7fbdcc070220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbddc007be0 tx=0x7fbddc0073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:27.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.653+0000 7fbdd67fc700 1 -- 172.21.15.70:0/4135215408 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbdd005c0a0 con 0x7fbde0105bd0 2024-04-02T04:58:27.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.887+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde0197150 con 0x7fbdcc06dd60 2024-04-02T04:58:27.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.888+0000 7fbdd67fc700 1 -- 172.21.15.70:0/4135215408 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fbde0197150 con 0x7fbdcc06dd60 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbdcc06dd60 msgr2=0x7fbdcc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbdcc06dd60 0x7fbdcc070220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbddc007be0 tx=0x7fbddc0073d0 comp rx=0 tx=0).stop 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde0105bd0 msgr2=0x7fbde019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde0105bd0 0x7fbde019cb70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b5c0 tx=0x7fbdd000bd70 comp rx=0 tx=0).stop 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 shutdown_connections 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbdcc06dd60 0x7fbdcc070220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde00ffc20 0x7fbde019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 --2- 172.21.15.70:0/4135215408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbde0105bd0 0x7fbde019cb70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:27.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.890+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 >> 172.21.15.70:0/4135215408 conn(0x7fbde0075670 msgr2=0x7fbde0068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:27.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.891+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 shutdown_connections 2024-04-02T04:58:27.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.891+0000 7fbde790e700 1 -- 172.21.15.70:0/4135215408 wait complete. 2024-04-02T04:58:27.905 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.996+0000 7f0cc7106700 1 -- 172.21.15.70:0/3988693325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00fffb0 msgr2=0x7f0cc01058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.996+0000 7f0cc7106700 1 --2- 172.21.15.70:0/3988693325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00fffb0 0x7f0cc01058e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009910 tx=0x7f0cac009c20 comp rx=0 tx=0).stop 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.996+0000 7f0cc7106700 1 -- 172.21.15.70:0/3988693325 shutdown_connections 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.996+0000 7f0cc7106700 1 --2- 172.21.15.70:0/3988693325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00fffb0 0x7f0cc01058e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.996+0000 7f0cc7106700 1 --2- 172.21.15.70:0/3988693325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cc00ff600 0x7f0cc00ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.996+0000 7f0cc7106700 1 -- 172.21.15.70:0/3988693325 >> 172.21.15.70:0/3988693325 conn(0x7f0cc0074df0 msgr2=0x7f0cc0075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.996+0000 7f0cc7106700 1 -- 172.21.15.70:0/3988693325 shutdown_connections 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.996+0000 7f0cc7106700 1 -- 172.21.15.70:0/3988693325 wait complete. 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc7106700 1 Processor -- start 2024-04-02T04:58:28.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc7106700 1 -- start start 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc7106700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00ff600 0x7f0cc0100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc7106700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cc00fffb0 0x7f0cc0100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc7106700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc0101000 con 0x7f0cc00ff600 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc7106700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc0101140 con 0x7f0cc00fffb0 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc4ea2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00ff600 0x7f0cc0100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc4ea2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00ff600 0x7f0cc0100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55744/0 (socket says 172.21.15.70:55744) 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.997+0000 7f0cc4ea2700 1 -- 172.21.15.70:0/4094262438 learned_addr learned my addr 172.21.15.70:0/4094262438 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cbffff700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cc00fffb0 0x7f0cc0100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cc4ea2700 1 -- 172.21.15.70:0/4094262438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cc00fffb0 msgr2=0x7f0cc0100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cc4ea2700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cc00fffb0 0x7f0cc0100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cc4ea2700 1 -- 172.21.15.70:0/4094262438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac0095f0 con 0x7f0cc00ff600 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cc4ea2700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00ff600 0x7f0cc0100580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400b4b0 tx=0x7f0cb400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cbdffb700 1 -- 172.21.15.70:0/4094262438 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb4011800 con 0x7f0cc00ff600 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cbdffb700 1 -- 172.21.15.70:0/4094262438 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cb4011e20 con 0x7f0cc00ff600 2024-04-02T04:58:28.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc01a2b80 con 0x7f0cc00ff600 2024-04-02T04:58:28.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cbdffb700 1 -- 172.21.15.70:0/4094262438 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb400f2e0 con 0x7f0cc00ff600 2024-04-02T04:58:28.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.998+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc01a2f90 con 0x7f0cc00ff600 2024-04-02T04:58:28.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:27.999+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc0109270 con 0x7f0cc00ff600 2024-04-02T04:58:28.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.000+0000 7f0cbdffb700 1 -- 172.21.15.70:0/4094262438 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f0cb4010380 con 0x7f0cc00ff600 2024-04-02T04:58:28.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.000+0000 7f0cbdffb700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0cb006dd60 0x7f0cb0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.001+0000 7f0cbdffb700 1 -- 172.21.15.70:0/4094262438 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f0cb408d810 con 0x7f0cc00ff600 2024-04-02T04:58:28.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.001+0000 7f0cbffff700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0cb006dd60 0x7f0cb0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.001+0000 7f0cbffff700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0cb006dd60 0x7f0cb0070220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0101d70 tx=0x7f0cac01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:28.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.005+0000 7f0cbdffb700 1 -- 172.21.15.70:0/4094262438 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0cb4056d20 con 0x7f0cc00ff600 2024-04-02T04:58:28.202 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:28.202 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:27 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:28.202 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:27 smithi179 ceph-mon[84288]: pgmap v237: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-04-02T04:58:28.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.224+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cc00611d0 con 0x7f0cb006dd60 2024-04-02T04:58:28.229 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:28.229 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:27 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:28.229 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:27 smithi070 ceph-mon[113635]: pgmap v237: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-04-02T04:58:28.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.225+0000 7f0cbdffb700 1 -- 172.21.15.70:0/4094262438 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f0cc00611d0 con 0x7f0cb006dd60 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.227+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0cb006dd60 msgr2=0x7f0cb0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.227+0000 7f0cc7106700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0cb006dd60 0x7f0cb0070220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0101d70 tx=0x7f0cac01a040 comp rx=0 tx=0).stop 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.227+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00ff600 msgr2=0x7f0cc0100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.227+0000 7f0cc7106700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00ff600 0x7f0cc0100580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400b4b0 tx=0x7f0cb400b870 comp rx=0 tx=0).stop 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.227+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 shutdown_connections 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.228+0000 7f0cc7106700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0cb006dd60 0x7f0cb0070220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.228+0000 7f0cc7106700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cc00ff600 0x7f0cc0100580 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.228+0000 7f0cc7106700 1 --2- 172.21.15.70:0/4094262438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cc00fffb0 0x7f0cc0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.228+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 >> 172.21.15.70:0/4094262438 conn(0x7f0cc0074df0 msgr2=0x7f0cc00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.228+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 shutdown_connections 2024-04-02T04:58:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.228+0000 7f0cc7106700 1 -- 172.21.15.70:0/4094262438 wait complete. 2024-04-02T04:58:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.333+0000 7fa5344fa700 1 -- 172.21.15.70:0/2581836703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0fff50 msgr2=0x7fa52c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.333+0000 7fa5344fa700 1 --2- 172.21.15.70:0/2581836703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0fff50 0x7fa52c105840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009910 tx=0x7fa51c009c20 comp rx=0 tx=0).stop 2024-04-02T04:58:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.333+0000 7fa5344fa700 1 -- 172.21.15.70:0/2581836703 shutdown_connections 2024-04-02T04:58:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.333+0000 7fa5344fa700 1 --2- 172.21.15.70:0/2581836703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0fff50 0x7fa52c105840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.333+0000 7fa5344fa700 1 --2- 172.21.15.70:0/2581836703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa52c0ff5a0 0x7fa52c0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.333+0000 7fa5344fa700 1 -- 172.21.15.70:0/2581836703 >> 172.21.15.70:0/2581836703 conn(0x7fa52c074e30 msgr2=0x7fa52c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.333+0000 7fa5344fa700 1 -- 172.21.15.70:0/2581836703 shutdown_connections 2024-04-02T04:58:28.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.333+0000 7fa5344fa700 1 -- 172.21.15.70:0/2581836703 wait complete. 2024-04-02T04:58:28.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa5344fa700 1 Processor -- start 2024-04-02T04:58:28.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa5344fa700 1 -- start start 2024-04-02T04:58:28.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa5344fa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0ff5a0 0x7fa52c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa5344fa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa52c0fff50 0x7fa52c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa5344fa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c19d1d0 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa5344fa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c196710 con 0x7fa52c0fff50 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa532296700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0ff5a0 0x7fa52c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa532296700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0ff5a0 0x7fa52c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55772/0 (socket says 172.21.15.70:55772) 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.334+0000 7fa532296700 1 -- 172.21.15.70:0/2380166779 learned_addr learned my addr 172.21.15.70:0/2380166779 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa531a95700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa52c0fff50 0x7fa52c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa532296700 1 -- 172.21.15.70:0/2380166779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa52c0fff50 msgr2=0x7fa52c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa532296700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa52c0fff50 0x7fa52c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa532296700 1 -- 172.21.15.70:0/2380166779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0095f0 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa531a95700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa52c0fff50 0x7fa52c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:58:28.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa532296700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0ff5a0 0x7fa52c19c690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c930 tx=0x7fa52800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:28.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa5237fe700 1 -- 172.21.15.70:0/2380166779 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa528007ad0 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa5237fe700 1 -- 172.21.15.70:0/2380166779 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa528007c30 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa5237fe700 1 -- 172.21.15.70:0/2380166779 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa528018610 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c1969f0 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.335+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c196ef0 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.336+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c109200 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.337+0000 7fa5237fe700 1 -- 172.21.15.70:0/2380166779 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa52800f450 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.337+0000 7fa5237fe700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa51806dd60 0x7fa518070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.337+0000 7fa5237fe700 1 -- 172.21.15.70:0/2380166779 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fa52808ded0 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.337+0000 7fa531a95700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa51806dd60 0x7fa518070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.338+0000 7fa531a95700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa51806dd60 0x7fa518070220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa52c197b80 tx=0x7fa51c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:28.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.342+0000 7fa5237fe700 1 -- 172.21.15.70:0/2380166779 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa52805ade0 con 0x7fa52c0ff5a0 2024-04-02T04:58:28.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.547+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa52c197d30 con 0x7fa51806dd60 2024-04-02T04:58:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.559+0000 7fa5237fe700 1 -- 172.21.15.70:0/2380166779 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fa52c197d30 con 0x7fa51806dd60 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (8m) 61s ago 19m 25.8M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 starting - - - - 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (18m) 12s ago 18m 9068k - 18.2.1 5be31c24972a fbc73637d767 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (5m) 61s ago 19m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (5m) 12s ago 18m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (7m) 61s ago 18m 79.7M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (98s) 61s ago 13m 13.4M - 19.0.0-1979-g6cec793d 78f830190839 2a4aa2b03d0d 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (67s) 61s ago 13m 11.5M - 19.0.0-1979-g6cec793d 78f830190839 3b0eb47520be 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (16s) 12s ago 13m 12.6M - 19.0.0-1979-g6cec793d 78f830190839 ecf548e68baf 2024-04-02T04:58:28.564 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (39s) 12s ago 13m 12.7M - 19.0.0-1979-g6cec793d 78f830190839 49c2ed39fbf7 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (10m) 61s ago 20m 509M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (9m) 12s ago 18m 451M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (6m) 61s ago 20m 54.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (6m) 12s ago 18m 48.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 61s ago 19m 15.9M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (8m) 12s ago 18m 16.9M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (4m) 61s ago 17m 147M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (4m) 61s ago 16m 143M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (3m) 61s ago 16m 95.0M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (3m) 12s ago 15m 109M 2240M 19.0.0-1979-g6cec793d 78f830190839 f717fd6c8306 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (2m) 12s ago 15m 112M 2240M 19.0.0-1979-g6cec793d 78f830190839 2e5bbf3235ff 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (116s) 12s ago 15m 101M 2240M 19.0.0-1979-g6cec793d 78f830190839 71351459ac70 2024-04-02T04:58:28.565 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (8m) 61s ago 18m 65.1M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:58:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa51806dd60 msgr2=0x7fa518070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa51806dd60 0x7fa518070220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa52c197b80 tx=0x7fa51c01a040 comp rx=0 tx=0).stop 2024-04-02T04:58:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0ff5a0 msgr2=0x7fa52c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0ff5a0 0x7fa52c19c690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c930 tx=0x7fa52800ccf0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 shutdown_connections 2024-04-02T04:58:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa51806dd60 0x7fa518070220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0ff5a0 0x7fa52c19c690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 --2- 172.21.15.70:0/2380166779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa52c0fff50 0x7fa52c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 >> 172.21.15.70:0/2380166779 conn(0x7fa52c074e30 msgr2=0x7fa52c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 shutdown_connections 2024-04-02T04:58:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.561+0000 7fa5344fa700 1 -- 172.21.15.70:0/2380166779 wait complete. 2024-04-02T04:58:28.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.663+0000 7f694f344700 1 -- 172.21.15.70:0/1697203610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6948105bd0 msgr2=0x7f6948105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.663+0000 7f694f344700 1 --2- 172.21.15.70:0/1697203610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6948105bd0 0x7f6948105fb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6944009910 tx=0x7f6944009c20 comp rx=0 tx=0).stop 2024-04-02T04:58:28.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.663+0000 7f694f344700 1 -- 172.21.15.70:0/1697203610 shutdown_connections 2024-04-02T04:58:28.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.663+0000 7f694f344700 1 --2- 172.21.15.70:0/1697203610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69480ffc20 0x7f6948100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.663+0000 7f694f344700 1 --2- 172.21.15.70:0/1697203610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6948105bd0 0x7f6948105fb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.663+0000 7f694f344700 1 -- 172.21.15.70:0/1697203610 >> 172.21.15.70:0/1697203610 conn(0x7f6948075670 msgr2=0x7f6948075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:28.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.663+0000 7f694f344700 1 -- 172.21.15.70:0/1697203610 shutdown_connections 2024-04-02T04:58:28.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.663+0000 7f694f344700 1 -- 172.21.15.70:0/1697203610 wait complete. 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694f344700 1 Processor -- start 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694f344700 1 -- start start 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694f344700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480ffc20 0x7f694819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694f344700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6948105bd0 0x7f694819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694f344700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694819d190 con 0x7f69480ffc20 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694f344700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69481966a0 con 0x7f6948105bd0 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694d0e0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480ffc20 0x7f694819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694d0e0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480ffc20 0x7f694819c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55788/0 (socket says 172.21.15.70:55788) 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694d0e0700 1 -- 172.21.15.70:0/3836687656 learned_addr learned my addr 172.21.15.70:0/3836687656 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:58:28.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694c8df700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6948105bd0 0x7f694819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694d0e0700 1 -- 172.21.15.70:0/3836687656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6948105bd0 msgr2=0x7f694819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694d0e0700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6948105bd0 0x7f694819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694d0e0700 1 -- 172.21.15.70:0/3836687656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440095f0 con 0x7f69480ffc20 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.664+0000 7f694d0e0700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480ffc20 0x7f694819c600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6944006010 tx=0x7f694400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.665+0000 7f693e7fc700 1 -- 172.21.15.70:0/3836687656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694401d070 con 0x7f69480ffc20 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.665+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6948196920 con 0x7f69480ffc20 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.665+0000 7f693e7fc700 1 -- 172.21.15.70:0/3836687656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f694400f460 con 0x7f69480ffc20 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.665+0000 7f693e7fc700 1 -- 172.21.15.70:0/3836687656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6944017630 con 0x7f69480ffc20 2024-04-02T04:58:28.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.665+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6948196e10 con 0x7f69480ffc20 2024-04-02T04:58:28.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.666+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694804f2e0 con 0x7f69480ffc20 2024-04-02T04:58:28.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.667+0000 7f693e7fc700 1 -- 172.21.15.70:0/3836687656 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f694400f5d0 con 0x7f69480ffc20 2024-04-02T04:58:28.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.667+0000 7f693e7fc700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f693406dd60 0x7f6934070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:28.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.667+0000 7f693e7fc700 1 -- 172.21.15.70:0/3836687656 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f694408f280 con 0x7f69480ffc20 2024-04-02T04:58:28.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.667+0000 7f694c8df700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f693406dd60 0x7f6934070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:28.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.668+0000 7f694c8df700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f693406dd60 0x7f6934070220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6948197e90 tx=0x7f693800a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:28.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:28.672+0000 7f693e7fc700 1 -- 172.21.15.70:0/3836687656 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f694405c190 con 0x7f69480ffc20 2024-04-02T04:58:29.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:28 smithi179 ceph-mon[84288]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:58:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:28 smithi179 ceph-mon[84288]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:58:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:28 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:28 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:58:29.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:28 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:29.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.075+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6948197000 con 0x7f69480ffc20 2024-04-02T04:58:29.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.076+0000 7f693e7fc700 1 -- 172.21.15.70:0/3836687656 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f694405bd20 con 0x7f69480ffc20 2024-04-02T04:58:29.081 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:58:29.081 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:58:29.081 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:58:29.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:58:29.081 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:58:29.081 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 14 2024-04-02T04:58:29.082 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:58:29.083 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:58:29.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f693406dd60 msgr2=0x7f6934070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f693406dd60 0x7f6934070220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6948197e90 tx=0x7f693800a380 comp rx=0 tx=0).stop 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480ffc20 msgr2=0x7f694819c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480ffc20 0x7f694819c600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6944006010 tx=0x7f694400bd70 comp rx=0 tx=0).stop 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 shutdown_connections 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f693406dd60 0x7f6934070220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480ffc20 0x7f694819c600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 --2- 172.21.15.70:0/3836687656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6948105bd0 0x7f694819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 >> 172.21.15.70:0/3836687656 conn(0x7f6948075670 msgr2=0x7f6948103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 shutdown_connections 2024-04-02T04:58:29.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.078+0000 7f694f344700 1 -- 172.21.15.70:0/3836687656 wait complete. 2024-04-02T04:58:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.181+0000 7f142bfbb700 1 -- 172.21.15.70:0/1708232325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1424100750 msgr2=0x7f14241047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.181+0000 7f142bfbb700 1 --2- 172.21.15.70:0/1708232325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1424100750 0x7f14241047a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1414009910 tx=0x7f1414009c20 comp rx=0 tx=0).stop 2024-04-02T04:58:29.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.182+0000 7f142bfbb700 1 -- 172.21.15.70:0/1708232325 shutdown_connections 2024-04-02T04:58:29.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.182+0000 7f142bfbb700 1 --2- 172.21.15.70:0/1708232325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1424100750 0x7f14241047a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.182+0000 7f142bfbb700 1 --2- 172.21.15.70:0/1708232325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14240ffda0 0x7f1424100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.182+0000 7f142bfbb700 1 -- 172.21.15.70:0/1708232325 >> 172.21.15.70:0/1708232325 conn(0x7f14240fb650 msgr2=0x7f14240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.182+0000 7f142bfbb700 1 -- 172.21.15.70:0/1708232325 shutdown_connections 2024-04-02T04:58:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.182+0000 7f142bfbb700 1 -- 172.21.15.70:0/1708232325 wait complete. 2024-04-02T04:58:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f142bfbb700 1 Processor -- start 2024-04-02T04:58:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f142bfbb700 1 -- start start 2024-04-02T04:58:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f142bfbb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14240ffda0 0x7f14241977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f142bfbb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1424100750 0x7f1424197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f142bfbb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1424198320 con 0x7f14240ffda0 2024-04-02T04:58:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f142bfbb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1424198460 con 0x7f1424100750 2024-04-02T04:58:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f1429d57700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14240ffda0 0x7f14241977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f1429d57700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14240ffda0 0x7f14241977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55806/0 (socket says 172.21.15.70:55806) 2024-04-02T04:58:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f1429d57700 1 -- 172.21.15.70:0/502852894 learned_addr learned my addr 172.21.15.70:0/502852894 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:58:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f1429556700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1424100750 0x7f1424197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f1429d57700 1 -- 172.21.15.70:0/502852894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1424100750 msgr2=0x7f1424197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f1429d57700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1424100750 0x7f1424197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f1429d57700 1 -- 172.21.15.70:0/502852894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14140095f0 con 0x7f14240ffda0 2024-04-02T04:58:29.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f1429d57700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14240ffda0 0x7f14241977c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f142000e920 tx=0x7f142000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:29.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.183+0000 7f141affd700 1 -- 172.21.15.70:0/502852894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f142000c9d0 con 0x7f14240ffda0 2024-04-02T04:58:29.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.184+0000 7f141affd700 1 -- 172.21.15.70:0/502852894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1420010430 con 0x7f14240ffda0 2024-04-02T04:58:29.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.184+0000 7f141affd700 1 -- 172.21.15.70:0/502852894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1420018630 con 0x7f14240ffda0 2024-04-02T04:58:29.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.184+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14241a0830 con 0x7f14240ffda0 2024-04-02T04:58:29.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.184+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14241a0cf0 con 0x7f14240ffda0 2024-04-02T04:58:29.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.185+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14241081a0 con 0x7f14240ffda0 2024-04-02T04:58:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.186+0000 7f141affd700 1 -- 172.21.15.70:0/502852894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f14200105a0 con 0x7f14240ffda0 2024-04-02T04:58:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.186+0000 7f141affd700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f141006dd60 0x7f1410070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.186+0000 7f141affd700 1 -- 172.21.15.70:0/502852894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f1420014070 con 0x7f14240ffda0 2024-04-02T04:58:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.186+0000 7f1429556700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f141006dd60 0x7f1410070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.186+0000 7f1429556700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f141006dd60 0x7f1410070220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1414005b40 tx=0x7f1414005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:29.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.191+0000 7f141affd700 1 -- 172.21.15.70:0/502852894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1420053280 con 0x7f14240ffda0 2024-04-02T04:58:29.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:28 smithi070 ceph-mon[113635]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:58:29.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:28 smithi070 ceph-mon[113635]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:58:29.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:28 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:29.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:28 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi179", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:58:29.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:28 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:29.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.435+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1424069500 con 0x7f14240ffda0 2024-04-02T04:58:29.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.435+0000 7f141affd700 1 -- 172.21.15.70:0/502852894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1936 (secure 0 0 0) 0x7f1420057350 con 0x7f14240ffda0 2024-04-02T04:58:29.440 INFO:teuthology.orchestra.run.smithi070.stdout:e39 2024-04-02T04:58:29.440 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:58:29.440 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:58:29.440 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 39 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:58:20.330302+0000 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T04:58:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 95 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=34388} 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T04:58:29.442 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 34388 members: 34388 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:34388} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T04:58:29.443 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T04:58:29.444 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6828/3294303398,v1:172.21.15.70:6829/3294303398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:58:29.444 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:44287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.179:6824/1866330674,v1:172.21.15.179:6825/1866330674] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:58:29.444 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.179:6826/1653234055,v1:172.21.15.179:6827/1653234055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.437+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f141006dd60 msgr2=0x7f1410070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.437+0000 7f142bfbb700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f141006dd60 0x7f1410070220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1414005b40 tx=0x7f1414005a90 comp rx=0 tx=0).stop 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.437+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14240ffda0 msgr2=0x7f14241977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.437+0000 7f142bfbb700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14240ffda0 0x7f14241977c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f142000e920 tx=0x7f142000ec30 comp rx=0 tx=0).stop 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.437+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 shutdown_connections 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.437+0000 7f142bfbb700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f141006dd60 0x7f1410070220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.437+0000 7f142bfbb700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14240ffda0 0x7f14241977c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.438+0000 7f142bfbb700 1 --2- 172.21.15.70:0/502852894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1424100750 0x7f1424197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.438+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 >> 172.21.15.70:0/502852894 conn(0x7f14240fb650 msgr2=0x7f14240fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.438+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 shutdown_connections 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.438+0000 7f142bfbb700 1 -- 172.21.15.70:0/502852894 wait complete. 2024-04-02T04:58:29.445 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 39 2024-04-02T04:58:29.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.542+0000 7f2be110d700 1 -- 172.21.15.70:0/4075709366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc106cd0 msgr2=0x7f2bdc1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.542+0000 7f2be110d700 1 --2- 172.21.15.70:0/4075709366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc106cd0 0x7f2bdc1070b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009910 tx=0x7f2bcc009c20 comp rx=0 tx=0).stop 2024-04-02T04:58:29.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.542+0000 7f2be110d700 1 -- 172.21.15.70:0/4075709366 shutdown_connections 2024-04-02T04:58:29.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.542+0000 7f2be110d700 1 --2- 172.21.15.70:0/4075709366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc068580 0x7f2bdc0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.542+0000 7f2be110d700 1 --2- 172.21.15.70:0/4075709366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc106cd0 0x7f2bdc1070b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.542+0000 7f2be110d700 1 -- 172.21.15.70:0/4075709366 >> 172.21.15.70:0/4075709366 conn(0x7f2bdc075670 msgr2=0x7f2bdc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:29.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.542+0000 7f2be110d700 1 -- 172.21.15.70:0/4075709366 shutdown_connections 2024-04-02T04:58:29.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.542+0000 7f2be110d700 1 -- 172.21.15.70:0/4075709366 wait complete. 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.543+0000 7f2be110d700 1 Processor -- start 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2be110d700 1 -- start start 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2be110d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc068580 0x7f2bdc19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2be110d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc106cd0 0x7f2bdc19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2be110d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc19d230 con 0x7f2bdc068580 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2be110d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1966e0 con 0x7f2bdc106cd0 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2bdad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc068580 0x7f2bdc19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2bdad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc068580 0x7f2bdc19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55820/0 (socket says 172.21.15.70:55820) 2024-04-02T04:58:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2bdad9d700 1 -- 172.21.15.70:0/1245668192 learned_addr learned my addr 172.21.15.70:0/1245668192 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2bda59c700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc106cd0 0x7f2bdc19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2bdad9d700 1 -- 172.21.15.70:0/1245668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc106cd0 msgr2=0x7f2bdc19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2bdad9d700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc106cd0 0x7f2bdc19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2bdad9d700 1 -- 172.21.15.70:0/1245668192 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc0095f0 con 0x7f2bdc068580 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.544+0000 7f2bdad9d700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc068580 0x7f2bdc19c660 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc006010 tx=0x7f2bcc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.545+0000 7f2bc3fff700 1 -- 172.21.15.70:0/1245668192 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bcc01d070 con 0x7f2bdc068580 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.545+0000 7f2bc3fff700 1 -- 172.21.15.70:0/1245668192 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bcc005dc0 con 0x7f2bdc068580 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.545+0000 7f2bc3fff700 1 -- 172.21.15.70:0/1245668192 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bcc017400 con 0x7f2bdc068580 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.545+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc196960 con 0x7f2bdc068580 2024-04-02T04:58:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.545+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc196e50 con 0x7f2bdc068580 2024-04-02T04:58:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.546+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bdc1091f0 con 0x7f2bdc068580 2024-04-02T04:58:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.546+0000 7f2bc3fff700 1 -- 172.21.15.70:0/1245668192 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2bcc017560 con 0x7f2bdc068580 2024-04-02T04:58:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.546+0000 7f2bc3fff700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2bc406dd60 0x7f2bc4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.547+0000 7f2bda59c700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2bc406dd60 0x7f2bc4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.547+0000 7f2bc3fff700 1 -- 172.21.15.70:0/1245668192 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f2bcc08f0a0 con 0x7f2bdc068580 2024-04-02T04:58:29.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.547+0000 7f2bda59c700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2bc406dd60 0x7f2bc4070220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc197b50 tx=0x7f2bd000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:29.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.551+0000 7f2bc3fff700 1 -- 172.21.15.70:0/1245668192 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2bcc05c030 con 0x7f2bdc068580 2024-04-02T04:58:29.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.770+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bdc1970c0 con 0x7f2bc406dd60 2024-04-02T04:58:29.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.772+0000 7f2bc3fff700 1 -- 172.21.15.70:0/1245668192 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f2bdc1970c0 con 0x7f2bc406dd60 2024-04-02T04:58:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:58:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:58:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-04-02T04:58:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:58:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-04-02T04:58:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "crash", 2024-04-02T04:58:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-04-02T04:58:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "mds", 2024-04-02T04:58:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-04-02T04:58:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd" 2024-04-02T04:58:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-04-02T04:58:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "16/23 daemons upgraded", 2024-04-02T04:58:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-04-02T04:58:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:58:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:58:29.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2bc406dd60 msgr2=0x7f2bc4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2bc406dd60 0x7f2bc4070220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc197b50 tx=0x7f2bd000b410 comp rx=0 tx=0).stop 2024-04-02T04:58:29.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc068580 msgr2=0x7f2bdc19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc068580 0x7f2bdc19c660 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc006010 tx=0x7f2bcc00bd70 comp rx=0 tx=0).stop 2024-04-02T04:58:29.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 shutdown_connections 2024-04-02T04:58:29.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f2bc406dd60 0x7f2bc4070220 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bdc068580 0x7f2bdc19c660 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 --2- 172.21.15.70:0/1245668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc106cd0 0x7f2bdc19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 >> 172.21.15.70:0/1245668192 conn(0x7f2bdc075670 msgr2=0x7f2bdc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:29.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 shutdown_connections 2024-04-02T04:58:29.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.774+0000 7f2be110d700 1 -- 172.21.15.70:0/1245668192 wait complete. 2024-04-02T04:58:29.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.879+0000 7f0023c5d700 1 -- 172.21.15.70:0/1617183038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c101850 msgr2=0x7f001c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.879+0000 7f0023c5d700 1 --2- 172.21.15.70:0/1617183038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c101850 0x7f001c1058a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0010009910 tx=0x7f0010009c20 comp rx=0 tx=0).stop 2024-04-02T04:58:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.879+0000 7f0023c5d700 1 -- 172.21.15.70:0/1617183038 shutdown_connections 2024-04-02T04:58:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.879+0000 7f0023c5d700 1 --2- 172.21.15.70:0/1617183038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c101850 0x7f001c1058a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.879+0000 7f0023c5d700 1 --2- 172.21.15.70:0/1617183038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f001c100ea0 0x7f001c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.879+0000 7f0023c5d700 1 -- 172.21.15.70:0/1617183038 >> 172.21.15.70:0/1617183038 conn(0x7f001c0fc750 msgr2=0x7f001c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.879+0000 7f0023c5d700 1 -- 172.21.15.70:0/1617183038 shutdown_connections 2024-04-02T04:58:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.879+0000 7f0023c5d700 1 -- 172.21.15.70:0/1617183038 wait complete. 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.880+0000 7f0023c5d700 1 Processor -- start 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.880+0000 7f0023c5d700 1 -- start start 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.880+0000 7f0023c5d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c100ea0 0x7f001c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.880+0000 7f0023c5d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f001c101850 0x7f001c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.880+0000 7f0023c5d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c19d2d0 con 0x7f001c100ea0 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.880+0000 7f0023c5d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c196780 con 0x7f001c101850 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f00219f9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c100ea0 0x7f001c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f00219f9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c100ea0 0x7f001c19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55834/0 (socket says 172.21.15.70:55834) 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f00219f9700 1 -- 172.21.15.70:0/353634991 learned_addr learned my addr 172.21.15.70:0/353634991 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f00211f8700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f001c101850 0x7f001c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f00219f9700 1 -- 172.21.15.70:0/353634991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f001c101850 msgr2=0x7f001c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:29.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f00219f9700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f001c101850 0x7f001c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:29.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f00219f9700 1 -- 172.21.15.70:0/353634991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00100095f0 con 0x7f001c100ea0 2024-04-02T04:58:29.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f00219f9700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c100ea0 0x7f001c19c700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f00140060b0 tx=0x7f001400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:29.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f001affd700 1 -- 172.21.15.70:0/353634991 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001400dda0 con 0x7f001c100ea0 2024-04-02T04:58:29.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f001affd700 1 -- 172.21.15.70:0/353634991 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f001400f040 con 0x7f001c100ea0 2024-04-02T04:58:29.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f001affd700 1 -- 172.21.15.70:0/353634991 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0014014790 con 0x7f001c100ea0 2024-04-02T04:58:29.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c196a60 con 0x7f001c100ea0 2024-04-02T04:58:29.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.881+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c196fb0 con 0x7f001c100ea0 2024-04-02T04:58:29.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.882+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c1092a0 con 0x7f001c100ea0 2024-04-02T04:58:29.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.883+0000 7f001affd700 1 -- 172.21.15.70:0/353634991 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f0014014980 con 0x7f001c100ea0 2024-04-02T04:58:29.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.883+0000 7f001affd700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f000406dd60 0x7f0004070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:58:29.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.884+0000 7f00211f8700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f000406dd60 0x7f0004070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:58:29.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.884+0000 7f001affd700 1 -- 172.21.15.70:0/353634991 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f001408d800 con 0x7f001c100ea0 2024-04-02T04:58:29.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.884+0000 7f00211f8700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f000406dd60 0x7f0004070220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f001c197bf0 tx=0x7f001001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:58:29.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:29.888+0000 7f001affd700 1 -- 172.21.15.70:0/353634991 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f001405a720 con 0x7f001c100ea0 2024-04-02T04:58:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:29 smithi070 ceph-mon[113635]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:58:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:29 smithi070 ceph-mon[113635]: Upgrade: Updating ceph-exporter.smithi179 (2/2) 2024-04-02T04:58:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:29 smithi070 ceph-mon[113635]: Deploying daemon ceph-exporter.smithi179 on smithi179 2024-04-02T04:58:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:29 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3836687656' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:29 smithi070 ceph-mon[113635]: pgmap v238: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-02T04:58:30.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:29 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/502852894' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:58:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.215+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f001c04f2e0 con 0x7f001c100ea0 2024-04-02T04:58:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.215+0000 7f001affd700 1 -- 172.21.15.70:0/353634991 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f001405a2b0 con 0x7f001c100ea0 2024-04-02T04:58:30.220 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f000406dd60 msgr2=0x7f0004070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f000406dd60 0x7f0004070220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f001c197bf0 tx=0x7f001001a040 comp rx=0 tx=0).stop 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c100ea0 msgr2=0x7f001c19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c100ea0 0x7f001c19c700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f00140060b0 tx=0x7f001400d3b0 comp rx=0 tx=0).stop 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 shutdown_connections 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f000406dd60 0x7f0004070220 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f001c100ea0 0x7f001c19c700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 --2- 172.21.15.70:0/353634991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f001c101850 0x7f001c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 >> 172.21.15.70:0/353634991 conn(0x7f001c0fc750 msgr2=0x7f001c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:58:30.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 shutdown_connections 2024-04-02T04:58:30.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:58:30.217+0000 7f0023c5d700 1 -- 172.21.15.70:0/353634991 wait complete. 2024-04-02T04:58:30.320 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:29 smithi179 ceph-mon[84288]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:58:30.320 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:29 smithi179 ceph-mon[84288]: Upgrade: Updating ceph-exporter.smithi179 (2/2) 2024-04-02T04:58:30.320 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:29 smithi179 ceph-mon[84288]: Deploying daemon ceph-exporter.smithi179 on smithi179 2024-04-02T04:58:30.320 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:29 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3836687656' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:30.320 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:29 smithi179 ceph-mon[84288]: pgmap v238: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-02T04:58:30.320 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:29 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/502852894' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:58:31.143 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:30 smithi179 ceph-mon[84288]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:58:31.143 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:30 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/353634991' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:58:31.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:30 smithi070 ceph-mon[113635]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:58:31.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:30 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/353634991' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:58:32.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:31 smithi179 ceph-mon[84288]: pgmap v239: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 4.0 KiB/s wr, 1 op/s 2024-04-02T04:58:32.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:31 smithi070 ceph-mon[113635]: pgmap v239: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail; 4.0 KiB/s wr, 1 op/s 2024-04-02T04:58:33.513 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:33 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:33.513 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:33 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:33.513 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:33 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:33.513 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:33 smithi070 ceph-mon[113635]: pgmap v240: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:33 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:33 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:33 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:33 smithi179 ceph-mon[84288]: pgmap v240: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:36.029 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:35 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:35 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:36.030 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:35 smithi070 ceph-mon[113635]: pgmap v241: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:36.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:35 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:36.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:35 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:36.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:35 smithi179 ceph-mon[84288]: pgmap v241: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:37.072 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:36 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:37.073 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:36 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:37.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:36 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:37.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:36 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:38.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:38.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:37 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:38.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:37 smithi070 ceph-mon[113635]: pgmap v242: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:38.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:38.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:37 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:38.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:37 smithi179 ceph-mon[84288]: pgmap v242: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:39.128 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:38 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:39.128 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:38 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:39.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:38 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:39.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:38 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:40.231 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:40 smithi070 ceph-mon[113635]: pgmap v243: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:40.231 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:40.231 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:40 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:40 smithi179 ceph-mon[84288]: pgmap v243: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:40 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:41.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: pgmap v244: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:42 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-02T04:58:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: pgmap v244: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:42.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:42 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-02T04:58:43.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: Upgrade: Setting filesystem cephfs Joinable 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi070"}]: dispatch 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi070"}]': finished 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi179"}]: dispatch 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi179"}]': finished 2024-04-02T04:58:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-02T04:58:43.532 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-02T04:58:43.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-02T04:58:43.534 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-02T04:58:43.534 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-02T04:58:43.534 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-02T04:58:43.534 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-02T04:58:43.534 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:43.535 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:43.536 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: Upgrade: Setting filesystem cephfs Joinable 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: fsmap cephfs:1 {0=cephfs.smithi070.mnzrwt=up:active} 3 up:standby 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi070"}]: dispatch 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi070"}]': finished 2024-04-02T04:58:43.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi179"}]: dispatch 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi179"}]': finished 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-02T04:58:43.575 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-02T04:58:43.576 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:43.577 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:43.578 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:43.578 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:43.578 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:43.578 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:43.578 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:43.578 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:44 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all ceph-exporter 2024-04-02T04:58:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:44 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all iscsi 2024-04-02T04:58:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:44 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all nfs 2024-04-02T04:58:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:44 smithi070 ceph-mon[113635]: Upgrade: Setting container_image for all nvmeof 2024-04-02T04:58:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:44 smithi070 ceph-mon[113635]: Upgrade: Finalizing container_image settings 2024-04-02T04:58:44.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:44 smithi070 ceph-mon[113635]: Upgrade: Complete! 2024-04-02T04:58:44.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:44 smithi070 ceph-mon[113635]: pgmap v245: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:44 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all ceph-exporter 2024-04-02T04:58:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:44 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all iscsi 2024-04-02T04:58:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:44 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all nfs 2024-04-02T04:58:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:44 smithi179 ceph-mon[84288]: Upgrade: Setting container_image for all nvmeof 2024-04-02T04:58:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:44 smithi179 ceph-mon[84288]: Upgrade: Finalizing container_image settings 2024-04-02T04:58:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:44 smithi179 ceph-mon[84288]: Upgrade: Complete! 2024-04-02T04:58:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:44 smithi179 ceph-mon[84288]: pgmap v245: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:46.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:46 smithi070 ceph-mon[113635]: pgmap v246: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:46 smithi179 ceph-mon[84288]: pgmap v246: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:47.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:58:48.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:48 smithi070 ceph-mon[113635]: pgmap v247: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:48 smithi179 ceph-mon[84288]: pgmap v247: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:50.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:50 smithi070 ceph-mon[113635]: pgmap v248: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:50 smithi179 ceph-mon[84288]: pgmap v248: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:52.333 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:52 smithi070 ceph-mon[113635]: pgmap v249: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:52.478 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:52 smithi179 ceph-mon[84288]: pgmap v249: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:54.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:54 smithi070 ceph-mon[113635]: pgmap v250: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:54.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:54 smithi179 ceph-mon[84288]: pgmap v250: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:56 smithi070 ceph-mon[113635]: pgmap v251: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:56 smithi179 ceph-mon[84288]: pgmap v251: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:58:58 smithi070 ceph-mon[113635]: pgmap v252: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:58:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:58:58 smithi179 ceph-mon[84288]: pgmap v252: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:00.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.324+0000 7f87c39be700 1 -- 172.21.15.70:0/1386940670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 msgr2=0x7f87bc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:00.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.324+0000 7f87c39be700 1 --2- 172.21.15.70:0/1386940670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 0x7f87bc101260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009910 tx=0x7f87b8009c20 comp rx=0 tx=0).stop 2024-04-02T04:59:00.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.324+0000 7f87c39be700 1 -- 172.21.15.70:0/1386940670 shutdown_connections 2024-04-02T04:59:00.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.324+0000 7f87c39be700 1 --2- 172.21.15.70:0/1386940670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87bc101830 0x7f87bc105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:00.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.324+0000 7f87c39be700 1 --2- 172.21.15.70:0/1386940670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 0x7f87bc101260 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:00.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.324+0000 7f87c39be700 1 -- 172.21.15.70:0/1386940670 >> 172.21.15.70:0/1386940670 conn(0x7f87bc0fc710 msgr2=0x7f87bc0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:00.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c39be700 1 -- 172.21.15.70:0/1386940670 shutdown_connections 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c39be700 1 -- 172.21.15.70:0/1386940670 wait complete. 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c39be700 1 Processor -- start 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c39be700 1 -- start start 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c39be700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 0x7f87bc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c39be700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87bc101830 0x7f87bc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c39be700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc073b10 con 0x7f87bc100e80 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c39be700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc073c50 con 0x7f87bc101830 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c175a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 0x7f87bc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c175a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 0x7f87bc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35326/0 (socket says 172.21.15.70:35326) 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.325+0000 7f87c175a700 1 -- 172.21.15.70:0/2667756110 learned_addr learned my addr 172.21.15.70:0/2667756110 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87c0f59700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87bc101830 0x7f87bc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87c175a700 1 -- 172.21.15.70:0/2667756110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87bc101830 msgr2=0x7f87bc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:00.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87c175a700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87bc101830 0x7f87bc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:00.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87c175a700 1 -- 172.21.15.70:0/2667756110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80095f0 con 0x7f87bc100e80 2024-04-02T04:59:00.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87c175a700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 0x7f87bc073090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006010 tx=0x7f87b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:00.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87b27fc700 1 -- 172.21.15.70:0/2667756110 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b801d070 con 0x7f87bc100e80 2024-04-02T04:59:00.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87b27fc700 1 -- 172.21.15.70:0/2667756110 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87b8004fb0 con 0x7f87bc100e80 2024-04-02T04:59:00.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87b27fc700 1 -- 172.21.15.70:0/2667756110 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b8022ea0 con 0x7f87bc100e80 2024-04-02T04:59:00.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc073ed0 con 0x7f87bc100e80 2024-04-02T04:59:00.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.326+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc1a2e40 con 0x7f87bc100e80 2024-04-02T04:59:00.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.327+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc109220 con 0x7f87bc100e80 2024-04-02T04:59:00.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.328+0000 7f87b27fc700 1 -- 172.21.15.70:0/2667756110 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f87b8022470 con 0x7f87bc100e80 2024-04-02T04:59:00.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.328+0000 7f87b27fc700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f87a806dd60 0x7f87a8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:00.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.329+0000 7f87b27fc700 1 -- 172.21.15.70:0/2667756110 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f87b808f260 con 0x7f87bc100e80 2024-04-02T04:59:00.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.329+0000 7f87c0f59700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f87a806dd60 0x7f87a8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:00.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.329+0000 7f87c0f59700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f87a806dd60 0x7f87a8070220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f87ac006fd0 tx=0x7f87ac008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:00.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.334+0000 7f87b27fc700 1 -- 172.21.15.70:0/2667756110 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f87b8058810 con 0x7f87bc100e80 2024-04-02T04:59:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:00 smithi070 ceph-mon[113635]: pgmap v253: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:00.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.554+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87bc0611d0 con 0x7f87a806dd60 2024-04-02T04:59:00.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.555+0000 7f87b27fc700 1 -- 172.21.15.70:0/2667756110 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f87bc0611d0 con 0x7f87a806dd60 2024-04-02T04:59:00.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f87a806dd60 msgr2=0x7f87a8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:00.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f87a806dd60 0x7f87a8070220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f87ac006fd0 tx=0x7f87ac008040 comp rx=0 tx=0).stop 2024-04-02T04:59:00.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 msgr2=0x7f87bc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:00.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 0x7f87bc073090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006010 tx=0x7f87b800be20 comp rx=0 tx=0).stop 2024-04-02T04:59:00.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 shutdown_connections 2024-04-02T04:59:00.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f87a806dd60 0x7f87a8070220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:00.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87bc100e80 0x7f87bc073090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:00.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 --2- 172.21.15.70:0/2667756110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87bc101830 0x7f87bc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:00.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 >> 172.21.15.70:0/2667756110 conn(0x7f87bc0fc710 msgr2=0x7f87bc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:00.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 shutdown_connections 2024-04-02T04:59:00.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:00.558+0000 7f87c39be700 1 -- 172.21.15.70:0/2667756110 wait complete. 2024-04-02T04:59:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:00 smithi179 ceph-mon[84288]: pgmap v253: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:01.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch ps' 2024-04-02T04:59:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:59:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:02 smithi070 ceph-mon[113635]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:59:02.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:02 smithi070 ceph-mon[113635]: pgmap v254: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:02.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:02 smithi179 ceph-mon[84288]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:59:02.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:02 smithi179 ceph-mon[84288]: pgmap v254: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 -- 172.21.15.70:0/63871882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba40ffc20 msgr2=0x7f1ba4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 --2- 172.21.15.70:0/63871882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4100080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009910 tx=0x7f1b90009c20 comp rx=0 tx=0).stop 2024-04-02T04:59:03.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 -- 172.21.15.70:0/63871882 shutdown_connections 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 --2- 172.21.15.70:0/63871882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba40ffc20 0x7f1ba4100080 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 --2- 172.21.15.70:0/63871882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba4105bd0 0x7f1ba4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 -- 172.21.15.70:0/63871882 >> 172.21.15.70:0/63871882 conn(0x7f1ba4075670 msgr2=0x7f1ba4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 -- 172.21.15.70:0/63871882 shutdown_connections 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 -- 172.21.15.70:0/63871882 wait complete. 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 Processor -- start 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.235+0000 7f1baac7e700 1 -- start start 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1baac7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba40ffc20 0x7f1ba419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1baac7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba4105bd0 0x7f1ba419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1baac7e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4195620 con 0x7f1ba4105bd0 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1baac7e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4195790 con 0x7f1ba40ffc20 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1ba3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba4105bd0 0x7f1ba419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:03.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1ba3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba4105bd0 0x7f1ba419bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35346/0 (socket says 172.21.15.70:35346) 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1ba3fff700 1 -- 172.21.15.70:0/2480223259 learned_addr learned my addr 172.21.15.70:0/2480223259 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1ba8a1a700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba40ffc20 0x7f1ba419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1ba3fff700 1 -- 172.21.15.70:0/2480223259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba40ffc20 msgr2=0x7f1ba419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1ba3fff700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba40ffc20 0x7f1ba419b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1ba3fff700 1 -- 172.21.15.70:0/2480223259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900095f0 con 0x7f1ba4105bd0 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.236+0000 7f1ba3fff700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba4105bd0 0x7f1ba419bb70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1b90006010 tx=0x7f1b9000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.237+0000 7f1ba1ffb700 1 -- 172.21.15.70:0/2480223259 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b9001d070 con 0x7f1ba4105bd0 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.237+0000 7f1ba1ffb700 1 -- 172.21.15.70:0/2480223259 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b90005dc0 con 0x7f1ba4105bd0 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.237+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba4195a10 con 0x7f1ba4105bd0 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.237+0000 7f1ba1ffb700 1 -- 172.21.15.70:0/2480223259 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b90017400 con 0x7f1ba4105bd0 2024-04-02T04:59:03.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.237+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba4195f00 con 0x7f1ba4105bd0 2024-04-02T04:59:03.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.238+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba404f2e0 con 0x7f1ba4105bd0 2024-04-02T04:59:03.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.239+0000 7f1ba1ffb700 1 -- 172.21.15.70:0/2480223259 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f1b90003a60 con 0x7f1ba4105bd0 2024-04-02T04:59:03.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.239+0000 7f1ba1ffb700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1b9406ff70 0x7f1b94072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:03.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.240+0000 7f1ba1ffb700 1 -- 172.21.15.70:0/2480223259 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f1b9008f170 con 0x7f1ba4105bd0 2024-04-02T04:59:03.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.240+0000 7f1ba8a1a700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1b9406ff70 0x7f1b94072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:03.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.240+0000 7f1ba8a1a700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1b9406ff70 0x7f1b94072430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005950 tx=0x7f1b9800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:03.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.244+0000 7f1ba1ffb700 1 -- 172.21.15.70:0/2480223259 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1b90058550 con 0x7f1ba4105bd0 2024-04-02T04:59:03.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.532+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ba4196120 con 0x7f1b9406ff70 2024-04-02T04:59:03.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.543+0000 7f1ba1ffb700 1 -- 172.21.15.70:0/2480223259 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1ba4196120 con 0x7f1b9406ff70 2024-04-02T04:59:03.547 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (8m) 26s ago 19m 25.1M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (37s) 26s ago 19m 7696k - 19.0.0-1979-g6cec793d 78f830190839 cfa5eeab524f 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (31s) 28s ago 18m 6941k - 19.0.0-1979-g6cec793d 78f830190839 4045497c5672 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (5m) 26s ago 19m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (5m) 28s ago 18m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (8m) 26s ago 19m 78.4M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (2m) 26s ago 14m 86.3M - 19.0.0-1979-g6cec793d 78f830190839 2a4aa2b03d0d 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (102s) 26s ago 14m 12.9M - 19.0.0-1979-g6cec793d 78f830190839 3b0eb47520be 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (51s) 28s ago 13m 13.6M - 19.0.0-1979-g6cec793d 78f830190839 ecf548e68baf 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (74s) 28s ago 14m 12.9M - 19.0.0-1979-g6cec793d 78f830190839 49c2ed39fbf7 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (10m) 26s ago 21m 510M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (9m) 28s ago 18m 451M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (7m) 26s ago 21m 58.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T04:59:03.548 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (6m) 28s ago 18m 49.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (9m) 26s ago 19m 15.6M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (9m) 28s ago 18m 16.9M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (5m) 26s ago 17m 147M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (4m) 26s ago 17m 144M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (4m) 26s ago 16m 96.6M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (3m) 28s ago 16m 110M 2240M 19.0.0-1979-g6cec793d 78f830190839 f717fd6c8306 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (3m) 28s ago 16m 113M 2240M 19.0.0-1979-g6cec793d 78f830190839 2e5bbf3235ff 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (2m) 28s ago 15m 102M 2240M 19.0.0-1979-g6cec793d 78f830190839 71351459ac70 2024-04-02T04:59:03.549 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (8m) 26s ago 19m 67.3M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T04:59:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1b9406ff70 msgr2=0x7f1b94072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1b9406ff70 0x7f1b94072430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005950 tx=0x7f1b9800b410 comp rx=0 tx=0).stop 2024-04-02T04:59:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba4105bd0 msgr2=0x7f1ba419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba4105bd0 0x7f1ba419bb70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1b90006010 tx=0x7f1b9000bd70 comp rx=0 tx=0).stop 2024-04-02T04:59:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 shutdown_connections 2024-04-02T04:59:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1b9406ff70 0x7f1b94072430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba40ffc20 0x7f1ba419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 --2- 172.21.15.70:0/2480223259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba4105bd0 0x7f1ba419bb70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 >> 172.21.15.70:0/2480223259 conn(0x7f1ba4075670 msgr2=0x7f1ba40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 shutdown_connections 2024-04-02T04:59:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:03.545+0000 7f1baac7e700 1 -- 172.21.15.70:0/2480223259 wait complete. 2024-04-02T04:59:04.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch upgrade status' 2024-04-02T04:59:04.504 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:59:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:04 smithi070 ceph-mon[113635]: pgmap v255: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:04 smithi179 ceph-mon[84288]: pgmap v255: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:05.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:05 smithi070 ceph-mon[113635]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:59:05.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:05 smithi179 ceph-mon[84288]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:59:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.207+0000 7fed3232b700 1 -- 172.21.15.70:0/1506693017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c100750 msgr2=0x7fed2c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.207+0000 7fed3232b700 1 --2- 172.21.15.70:0/1506693017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c100750 0x7fed2c1047a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fed20009910 tx=0x7fed20009c20 comp rx=0 tx=0).stop 2024-04-02T04:59:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.208+0000 7fed3232b700 1 -- 172.21.15.70:0/1506693017 shutdown_connections 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.208+0000 7fed3232b700 1 --2- 172.21.15.70:0/1506693017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c100750 0x7fed2c1047a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.208+0000 7fed3232b700 1 --2- 172.21.15.70:0/1506693017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2c0ffda0 0x7fed2c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.208+0000 7fed3232b700 1 -- 172.21.15.70:0/1506693017 >> 172.21.15.70:0/1506693017 conn(0x7fed2c0fb650 msgr2=0x7fed2c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.208+0000 7fed3232b700 1 -- 172.21.15.70:0/1506693017 shutdown_connections 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.208+0000 7fed3232b700 1 -- 172.21.15.70:0/1506693017 wait complete. 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.208+0000 7fed3232b700 1 Processor -- start 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.208+0000 7fed3232b700 1 -- start start 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed3232b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c0ffda0 0x7fed2c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed3232b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2c100750 0x7fed2c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed3232b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c19d240 con 0x7fed2c0ffda0 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed3232b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1966f0 con 0x7fed2c100750 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed2bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c0ffda0 0x7fed2c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed2bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c0ffda0 0x7fed2c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35360/0 (socket says 172.21.15.70:35360) 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed2bfff700 1 -- 172.21.15.70:0/6954528 learned_addr learned my addr 172.21.15.70:0/6954528 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed2b7fe700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2c100750 0x7fed2c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed2bfff700 1 -- 172.21.15.70:0/6954528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2c100750 msgr2=0x7fed2c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed2bfff700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2c100750 0x7fed2c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed2bfff700 1 -- 172.21.15.70:0/6954528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200095f0 con 0x7fed2c0ffda0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed2bfff700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c0ffda0 0x7fed2c19c670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00d8d0 tx=0x7fed1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed297fa700 1 -- 172.21.15.70:0/6954528 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed1c009940 con 0x7fed2c0ffda0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c1969d0 con 0x7fed2c0ffda0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed297fa700 1 -- 172.21.15.70:0/6954528 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed1c010460 con 0x7fed2c0ffda0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.209+0000 7fed297fa700 1 -- 172.21.15.70:0/6954528 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed1c00f5f0 con 0x7fed2c0ffda0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.210+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c196f20 con 0x7fed2c0ffda0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.211+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2c04f2e0 con 0x7fed2c0ffda0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.211+0000 7fed297fa700 1 -- 172.21.15.70:0/6954528 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fed1c0105d0 con 0x7fed2c0ffda0 2024-04-02T04:59:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.211+0000 7fed297fa700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fed1406dd60 0x7fed14070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:06.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.212+0000 7fed297fa700 1 -- 172.21.15.70:0/6954528 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fed1c08dbc0 con 0x7fed2c0ffda0 2024-04-02T04:59:06.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.212+0000 7fed2b7fe700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fed1406dd60 0x7fed14070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:06.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.212+0000 7fed2b7fe700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fed1406dd60 0x7fed14070220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fed2c197b60 tx=0x7fed2001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:06.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.216+0000 7fed297fa700 1 -- 172.21.15.70:0/6954528 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fed1c05ab50 con 0x7fed2c0ffda0 2024-04-02T04:59:06.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.441+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed2c197230 con 0x7fed1406dd60 2024-04-02T04:59:06.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.443+0000 7fed297fa700 1 -- 172.21.15.70:0/6954528 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fed2c197230 con 0x7fed1406dd60 2024-04-02T04:59:06.447 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:59:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": null, 2024-04-02T04:59:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": false, 2024-04-02T04:59:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "", 2024-04-02T04:59:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-04-02T04:59:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": null, 2024-04-02T04:59:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-04-02T04:59:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-04-02T04:59:06.448 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:59:06.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fed1406dd60 msgr2=0x7fed14070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:06.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fed1406dd60 0x7fed14070220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fed2c197b60 tx=0x7fed2001a040 comp rx=0 tx=0).stop 2024-04-02T04:59:06.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c0ffda0 msgr2=0x7fed2c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:06.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c0ffda0 0x7fed2c19c670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00d8d0 tx=0x7fed1c00dc90 comp rx=0 tx=0).stop 2024-04-02T04:59:06.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 shutdown_connections 2024-04-02T04:59:06.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fed1406dd60 0x7fed14070220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:06.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed2c0ffda0 0x7fed2c19c670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:06.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 --2- 172.21.15.70:0/6954528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2c100750 0x7fed2c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:06.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 >> 172.21.15.70:0/6954528 conn(0x7fed2c0fb650 msgr2=0x7fed2c104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:06.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 shutdown_connections 2024-04-02T04:59:06.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:06.445+0000 7fed3232b700 1 -- 172.21.15.70:0/6954528 wait complete. 2024-04-02T04:59:06.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:06 smithi070 ceph-mon[113635]: pgmap v256: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:06 smithi179 ceph-mon[84288]: pgmap v256: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:07.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph health detail' 2024-04-02T04:59:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:59:08.504 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:08 smithi070 ceph-mon[113635]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:59:08.504 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:08 smithi070 ceph-mon[113635]: pgmap v257: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:08.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:08 smithi179 ceph-mon[84288]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:59:08.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:08 smithi179 ceph-mon[84288]: pgmap v257: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.050+0000 7fbd40db8700 1 -- 172.21.15.70:0/3251806814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 msgr2=0x7fbd3c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.050+0000 7fbd40db8700 1 --2- 172.21.15.70:0/3251806814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 0x7fbd3c1047a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009910 tx=0x7fbd30009c20 comp rx=0 tx=0).stop 2024-04-02T04:59:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.050+0000 7fbd40db8700 1 -- 172.21.15.70:0/3251806814 shutdown_connections 2024-04-02T04:59:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.050+0000 7fbd40db8700 1 --2- 172.21.15.70:0/3251806814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 0x7fbd3c1047a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.050+0000 7fbd40db8700 1 --2- 172.21.15.70:0/3251806814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3c0ffda0 0x7fbd3c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.050+0000 7fbd40db8700 1 -- 172.21.15.70:0/3251806814 >> 172.21.15.70:0/3251806814 conn(0x7fbd3c0fb650 msgr2=0x7fbd3c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.050+0000 7fbd40db8700 1 -- 172.21.15.70:0/3251806814 shutdown_connections 2024-04-02T04:59:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.050+0000 7fbd40db8700 1 -- 172.21.15.70:0/3251806814 wait complete. 2024-04-02T04:59:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd40db8700 1 Processor -- start 2024-04-02T04:59:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd40db8700 1 -- start start 2024-04-02T04:59:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd40db8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3c0ffda0 0x7fbd3c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd40db8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 0x7fbd3c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd40db8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19f2e0 con 0x7fbd3c100750 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd40db8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c198900 con 0x7fbd3c0ffda0 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd39d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 0x7fbd3c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd39d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 0x7fbd3c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51542/0 (socket says 172.21.15.70:51542) 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.051+0000 7fbd39d9b700 1 -- 172.21.15.70:0/159761300 learned_addr learned my addr 172.21.15.70:0/159761300 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd3a59c700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3c0ffda0 0x7fbd3c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd39d9b700 1 -- 172.21.15.70:0/159761300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3c0ffda0 msgr2=0x7fbd3c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd39d9b700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3c0ffda0 0x7fbd3c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd39d9b700 1 -- 172.21.15.70:0/159761300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300095f0 con 0x7fbd3c100750 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd39d9b700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 0x7fbd3c19ed50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000bda0 tx=0x7fbd3000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd2b7fe700 1 -- 172.21.15.70:0/159761300 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd3001d070 con 0x7fbd3c100750 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c198b80 con 0x7fbd3c100750 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd2b7fe700 1 -- 172.21.15.70:0/159761300 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd30004fb0 con 0x7fbd3c100750 2024-04-02T04:59:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd2b7fe700 1 -- 172.21.15.70:0/159761300 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd3000f460 con 0x7fbd3c100750 2024-04-02T04:59:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.052+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c199070 con 0x7fbd3c100750 2024-04-02T04:59:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.053+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c04f2e0 con 0x7fbd3c100750 2024-04-02T04:59:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.054+0000 7fbd2b7fe700 1 -- 172.21.15.70:0/159761300 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbd30022800 con 0x7fbd3c100750 2024-04-02T04:59:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.054+0000 7fbd2b7fe700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd2406dd60 0x7fbd24070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.054+0000 7fbd2b7fe700 1 -- 172.21.15.70:0/159761300 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fbd3008f380 con 0x7fbd3c100750 2024-04-02T04:59:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.054+0000 7fbd3a59c700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd2406dd60 0x7fbd24070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.055+0000 7fbd3a59c700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd2406dd60 0x7fbd24070220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c005ea0 tx=0x7fbd2c005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:09.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.059+0000 7fbd2b7fe700 1 -- 172.21.15.70:0/159761300 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbd3005c4e0 con 0x7fbd3c100750 2024-04-02T04:59:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.383+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbd3c1992e0 con 0x7fbd3c100750 2024-04-02T04:59:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.383+0000 7fbd2b7fe700 1 -- 172.21.15.70:0/159761300 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbd30027020 con 0x7fbd3c100750 2024-04-02T04:59:09.388 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-04-02T04:59:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.385+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd2406dd60 msgr2=0x7fbd24070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.385+0000 7fbd40db8700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd2406dd60 0x7fbd24070220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c005ea0 tx=0x7fbd2c005e30 comp rx=0 tx=0).stop 2024-04-02T04:59:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.385+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 msgr2=0x7fbd3c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.385+0000 7fbd40db8700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 0x7fbd3c19ed50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000bda0 tx=0x7fbd3000be80 comp rx=0 tx=0).stop 2024-04-02T04:59:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.386+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 shutdown_connections 2024-04-02T04:59:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.386+0000 7fbd40db8700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd2406dd60 0x7fbd24070220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:09.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.386+0000 7fbd40db8700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3c0ffda0 0x7fbd3c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.386+0000 7fbd40db8700 1 --2- 172.21.15.70:0/159761300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3c100750 0x7fbd3c19ed50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.386+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 >> 172.21.15.70:0/159761300 conn(0x7fbd3c0fb650 msgr2=0x7fbd3c103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.386+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 shutdown_connections 2024-04-02T04:59:09.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:09.386+0000 7fbd40db8700 1 -- 172.21.15.70:0/159761300 wait complete. 2024-04-02T04:59:10.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions' 2024-04-02T04:59:10.291 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:59:10.412 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:10 smithi070 ceph-mon[113635]: pgmap v258: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:10.412 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:10 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/159761300' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:59:10.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:10 smithi179 ceph-mon[84288]: pgmap v258: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:10.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:10 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/159761300' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:59:11.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:11.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.967+0000 7f0c26b66700 1 -- 172.21.15.70:0/943509407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 msgr2=0x7f0c200fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:11.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.967+0000 7f0c26b66700 1 --2- 172.21.15.70:0/943509407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 0x7f0c200fffd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009910 tx=0x7f0c14009c20 comp rx=0 tx=0).stop 2024-04-02T04:59:11.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.967+0000 7f0c26b66700 1 -- 172.21.15.70:0/943509407 shutdown_connections 2024-04-02T04:59:11.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.967+0000 7f0c26b66700 1 --2- 172.21.15.70:0/943509407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c20100510 0x7f0c201049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:11.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.967+0000 7f0c26b66700 1 --2- 172.21.15.70:0/943509407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 0x7f0c200fffd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:11.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.967+0000 7f0c26b66700 1 -- 172.21.15.70:0/943509407 >> 172.21.15.70:0/943509407 conn(0x7f0c200fb650 msgr2=0x7f0c200fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:11.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.968+0000 7f0c26b66700 1 -- 172.21.15.70:0/943509407 shutdown_connections 2024-04-02T04:59:11.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.968+0000 7f0c26b66700 1 -- 172.21.15.70:0/943509407 wait complete. 2024-04-02T04:59:11.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.968+0000 7f0c26b66700 1 Processor -- start 2024-04-02T04:59:11.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.968+0000 7f0c26b66700 1 -- start start 2024-04-02T04:59:11.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c26b66700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 0x7f0c2019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c26b66700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c20100510 0x7f0c2019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c26b66700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2019db50 con 0x7f0c200ffbf0 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c26b66700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a18e0 con 0x7f0c20100510 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c24902700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 0x7f0c2019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c24902700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 0x7f0c2019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51556/0 (socket says 172.21.15.70:51556) 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c24902700 1 -- 172.21.15.70:0/4207068846 learned_addr learned my addr 172.21.15.70:0/4207068846 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c1ffff700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c20100510 0x7f0c2019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c24902700 1 -- 172.21.15.70:0/4207068846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c20100510 msgr2=0x7f0c2019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c24902700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c20100510 0x7f0c2019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c24902700 1 -- 172.21.15.70:0/4207068846 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140095f0 con 0x7f0c200ffbf0 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c1ffff700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c20100510 0x7f0c2019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c24902700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 0x7f0c2019cf30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c140092e0 tx=0x7f0c1400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:11.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c1dffb700 1 -- 172.21.15.70:0/4207068846 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c1401d070 con 0x7f0c200ffbf0 2024-04-02T04:59:11.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c1dffb700 1 -- 172.21.15.70:0/4207068846 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c14005e00 con 0x7f0c200ffbf0 2024-04-02T04:59:11.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c201a1b60 con 0x7f0c200ffbf0 2024-04-02T04:59:11.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.969+0000 7f0c1dffb700 1 -- 172.21.15.70:0/4207068846 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c14017400 con 0x7f0c200ffbf0 2024-04-02T04:59:11.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.970+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c201a2050 con 0x7f0c200ffbf0 2024-04-02T04:59:11.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.971+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c20102140 con 0x7f0c200ffbf0 2024-04-02T04:59:11.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.971+0000 7f0c1dffb700 1 -- 172.21.15.70:0/4207068846 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f0c1400f460 con 0x7f0c200ffbf0 2024-04-02T04:59:11.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.972+0000 7f0c1dffb700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0c1006dd60 0x7f0c10070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:59:11.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.972+0000 7f0c1dffb700 1 -- 172.21.15.70:0/4207068846 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f0c1408f200 con 0x7f0c200ffbf0 2024-04-02T04:59:11.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.972+0000 7f0c1ffff700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0c1006dd60 0x7f0c10070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:59:11.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.972+0000 7f0c1ffff700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0c1006dd60 0x7f0c10070220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c005950 tx=0x7f0c0c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:59:11.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:11.976+0000 7f0c1dffb700 1 -- 172.21.15.70:0/4207068846 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0c140585e0 con 0x7f0c200ffbf0 2024-04-02T04:59:12.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.286+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c2004f2e0 con 0x7f0c200ffbf0 2024-04-02T04:59:12.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.287+0000 7f0c1dffb700 1 -- 172.21.15.70:0/4207068846 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0c1405bca0 con 0x7f0c200ffbf0 2024-04-02T04:59:12.291 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T04:59:12.291 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T04:59:12.291 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:59:12.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 14 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T04:59:12.292 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T04:59:12.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0c1006dd60 msgr2=0x7f0c10070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:12.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0c1006dd60 0x7f0c10070220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c005950 tx=0x7f0c0c0058e0 comp rx=0 tx=0).stop 2024-04-02T04:59:12.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 msgr2=0x7f0c2019cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:59:12.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 0x7f0c2019cf30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c140092e0 tx=0x7f0c1400be20 comp rx=0 tx=0).stop 2024-04-02T04:59:12.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 shutdown_connections 2024-04-02T04:59:12.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f0c1006dd60 0x7f0c10070220 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:12.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c200ffbf0 0x7f0c2019cf30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:12.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 --2- 172.21.15.70:0/4207068846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c20100510 0x7f0c2019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:59:12.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 >> 172.21.15.70:0/4207068846 conn(0x7f0c200fb650 msgr2=0x7f0c20111060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:59:12.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 shutdown_connections 2024-04-02T04:59:12.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T04:59:12.289+0000 7f0c26b66700 1 -- 172.21.15.70:0/4207068846 wait complete. 2024-04-02T04:59:12.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:12 smithi070 ceph-mon[113635]: pgmap v259: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:12 smithi179 ceph-mon[84288]: pgmap v259: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:12.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-02T04:59:13.212 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:59:13.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:13 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/4207068846' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:59:13.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:13 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/4207068846' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:59:14.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:14 smithi070 ceph-mon[113635]: pgmap v260: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:14 smithi179 ceph-mon[84288]: pgmap v260: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:14.903 INFO:teuthology.orchestra.run.smithi070.stdout:wait for servicemap items w/ changing names to refresh 2024-04-02T04:59:15.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'sleep 60' 2024-04-02T04:59:15.806 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T04:59:16.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:16 smithi070 ceph-mon[113635]: pgmap v261: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:16.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:16 smithi179 ceph-mon[84288]: pgmap v261: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:18.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:18 smithi070 ceph-mon[113635]: pgmap v262: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:18.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:18 smithi179 ceph-mon[84288]: pgmap v262: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:20.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:20 smithi070 ceph-mon[113635]: pgmap v263: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:20 smithi179 ceph-mon[84288]: pgmap v263: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:22.517 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:22 smithi070 ceph-mon[113635]: pgmap v264: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:22 smithi179 ceph-mon[84288]: pgmap v264: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:24 smithi070 ceph-mon[113635]: pgmap v265: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:24 smithi179 ceph-mon[84288]: pgmap v265: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:26.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:26 smithi070 ceph-mon[113635]: pgmap v266: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:26 smithi179 ceph-mon[84288]: pgmap v266: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:28.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:28 smithi070 ceph-mon[113635]: pgmap v267: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:28.574 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:28 smithi179 ceph-mon[84288]: pgmap v267: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:30.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:30 smithi070 ceph-mon[113635]: pgmap v268: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:30.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:30 smithi179 ceph-mon[84288]: pgmap v268: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:32 smithi070 ceph-mon[113635]: pgmap v269: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:32 smithi179 ceph-mon[84288]: pgmap v269: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:34.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:34 smithi070 ceph-mon[113635]: pgmap v270: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:34 smithi179 ceph-mon[84288]: pgmap v270: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:36 smithi179 ceph-mon[84288]: pgmap v271: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:36.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:36 smithi070 ceph-mon[113635]: pgmap v271: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:38 smithi179 ceph-mon[84288]: pgmap v272: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:38.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:38 smithi070 ceph-mon[113635]: pgmap v272: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:40 smithi179 ceph-mon[84288]: pgmap v273: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:40 smithi070 ceph-mon[113635]: pgmap v273: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:41.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:42 smithi179 ceph-mon[84288]: pgmap v274: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:42.725 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:42 smithi070 ceph-mon[113635]: pgmap v274: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:43 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:59:43.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:43 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:59:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:44 smithi179 ceph-mon[84288]: pgmap v275: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:59:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:59:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:59:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:59:44.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:44 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:59:44.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:44 smithi070 ceph-mon[113635]: pgmap v275: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:44.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:59:44.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:59:44.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:59:44.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:59:44.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:44 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T04:59:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:46 smithi179 ceph-mon[84288]: pgmap v276: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:46 smithi070 ceph-mon[113635]: pgmap v276: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:48 smithi179 ceph-mon[84288]: pgmap v277: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:48 smithi070 ceph-mon[113635]: pgmap v277: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:50 smithi179 ceph-mon[84288]: pgmap v278: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:50.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:50 smithi070 ceph-mon[113635]: pgmap v278: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:52 smithi179 ceph-mon[84288]: pgmap v279: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:52.581 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:52 smithi070 ceph-mon[113635]: pgmap v279: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:54.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:53 smithi070 ceph-mon[113635]: pgmap v280: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:54.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:53 smithi179 ceph-mon[84288]: pgmap v280: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:56 smithi070 ceph-mon[113635]: pgmap v281: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:56.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:56 smithi179 ceph-mon[84288]: pgmap v281: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 04:59:58 smithi070 ceph-mon[113635]: pgmap v282: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:59:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 04:59:58 smithi179 ceph-mon[84288]: pgmap v282: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:00.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:00 smithi070 ceph-mon[113635]: pgmap v283: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:00.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:00 smithi070 ceph-mon[113635]: overall HEALTH_OK 2024-04-02T05:00:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:00 smithi179 ceph-mon[84288]: pgmap v283: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:00.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:00 smithi179 ceph-mon[84288]: overall HEALTH_OK 2024-04-02T05:00:02.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:02 smithi070 ceph-mon[113635]: pgmap v284: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:02.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:02 smithi179 ceph-mon[84288]: pgmap v284: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:04 smithi070 ceph-mon[113635]: pgmap v285: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:04.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:04 smithi179 ceph-mon[84288]: pgmap v285: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:06.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:06 smithi070 ceph-mon[113635]: pgmap v286: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:06 smithi179 ceph-mon[84288]: pgmap v286: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:08.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:08 smithi070 ceph-mon[113635]: pgmap v287: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:08.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:08 smithi179 ceph-mon[84288]: pgmap v287: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:10.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:10 smithi070 ceph-mon[113635]: pgmap v288: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:10 smithi179 ceph-mon[84288]: pgmap v288: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:11.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:00:11.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:00:12.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:12 smithi070 ceph-mon[113635]: pgmap v289: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:12 smithi179 ceph-mon[84288]: pgmap v289: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:14.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:14 smithi070 ceph-mon[113635]: pgmap v290: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:14.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:14 smithi179 ceph-mon[84288]: pgmap v290: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:16.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:16 smithi070 ceph-mon[113635]: pgmap v291: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:16.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:16 smithi179 ceph-mon[84288]: pgmap v291: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:18.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch ps' 2024-04-02T05:00:18.347 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:18 smithi070 ceph-mon[113635]: pgmap v292: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:18.544 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T05:00:18.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:18 smithi179 ceph-mon[84288]: pgmap v292: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:20.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.140+0000 7fde3a2b6700 1 -- 172.21.15.70:0/1193082386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 msgr2=0x7fde340689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.140+0000 7fde3a2b6700 1 --2- 172.21.15.70:0/1193082386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 0x7fde340689e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009910 tx=0x7fde1c009c20 comp rx=0 tx=0).stop 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.140+0000 7fde3a2b6700 1 -- 172.21.15.70:0/1193082386 shutdown_connections 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.140+0000 7fde3a2b6700 1 --2- 172.21.15.70:0/1193082386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 0x7fde340689e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.140+0000 7fde3a2b6700 1 --2- 172.21.15.70:0/1193082386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde34106cd0 0x7fde341070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.140+0000 7fde3a2b6700 1 -- 172.21.15.70:0/1193082386 >> 172.21.15.70:0/1193082386 conn(0x7fde34075670 msgr2=0x7fde34075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.140+0000 7fde3a2b6700 1 -- 172.21.15.70:0/1193082386 shutdown_connections 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.140+0000 7fde3a2b6700 1 -- 172.21.15.70:0/1193082386 wait complete. 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde3a2b6700 1 Processor -- start 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde3a2b6700 1 -- start start 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde3a2b6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 0x7fde340ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde3a2b6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde34106cd0 0x7fde34100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde3a2b6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34100770 con 0x7fde34068580 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde3a2b6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde341008b0 con 0x7fde34106cd0 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde33fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 0x7fde340ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde33fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 0x7fde340ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36136/0 (socket says 172.21.15.70:36136) 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.141+0000 7fde33fff700 1 -- 172.21.15.70:0/2935804410 learned_addr learned my addr 172.21.15.70:0/2935804410 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde337fe700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde34106cd0 0x7fde34100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde33fff700 1 -- 172.21.15.70:0/2935804410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde34106cd0 msgr2=0x7fde34100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde33fff700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde34106cd0 0x7fde34100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde33fff700 1 -- 172.21.15.70:0/2935804410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0095f0 con 0x7fde34068580 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde337fe700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde34106cd0 0x7fde34100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde33fff700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 0x7fde340ffcf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde2400d8d0 tx=0x7fde2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:20.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde317fa700 1 -- 172.21.15.70:0/2935804410 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde24009940 con 0x7fde34068580 2024-04-02T05:00:20.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde341a2a30 con 0x7fde34068580 2024-04-02T05:00:20.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde317fa700 1 -- 172.21.15.70:0/2935804410 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde24010460 con 0x7fde34068580 2024-04-02T05:00:20.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde317fa700 1 -- 172.21.15.70:0/2935804410 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde2400f5f0 con 0x7fde34068580 2024-04-02T05:00:20.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.142+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde341a2e40 con 0x7fde34068580 2024-04-02T05:00:20.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.143+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde341091d0 con 0x7fde34068580 2024-04-02T05:00:20.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.143+0000 7fde317fa700 1 -- 172.21.15.70:0/2935804410 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fde240105d0 con 0x7fde34068580 2024-04-02T05:00:20.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.144+0000 7fde317fa700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fde2006dd60 0x7fde20070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:20.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.144+0000 7fde337fe700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fde2006dd60 0x7fde20070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:20.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.144+0000 7fde317fa700 1 -- 172.21.15.70:0/2935804410 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fde2408dd10 con 0x7fde34068580 2024-04-02T05:00:20.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.145+0000 7fde337fe700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fde2006dd60 0x7fde20070220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00b5c0 tx=0x7fde1c0053a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:20.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.149+0000 7fde317fa700 1 -- 172.21.15.70:0/2935804410 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fde240570f0 con 0x7fde34068580 2024-04-02T05:00:20.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.373+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 --> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde340611d0 con 0x7fde2006dd60 2024-04-02T05:00:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.383+0000 7fde317fa700 1 -- 172.21.15.70:0/2935804410 <== mgr.34102 v2:172.21.15.70:6800/2021202469 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fde340611d0 con 0x7fde2006dd60 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (9m) 103s ago 20m 25.1M - 0.25.0 c8568f914cd2 5a972a9c0c4b 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi070 smithi070 running (113s) 103s ago 21m 7696k - 19.0.0-1979-g6cec793d 78f830190839 cfa5eeab524f 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-exporter.smithi179 smithi179 running (108s) 105s ago 20m 6941k - 19.0.0-1979-g6cec793d 78f830190839 4045497c5672 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (7m) 103s ago 21m 6665k - 19.0.0-1979-g6cec793d 78f830190839 c4fefa07ff3f 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi179 smithi179 running (7m) 105s ago 20m 6656k - 19.0.0-1979-g6cec793d 78f830190839 2e48da8dc819 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (9m) 103s ago 20m 78.4M - 9.4.12 2bacad6d85d8 f11c67a6c489 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.mnzrwt smithi070 running (3m) 103s ago 15m 86.3M - 19.0.0-1979-g6cec793d 78f830190839 2a4aa2b03d0d 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.zvraae smithi070 running (2m) 103s ago 15m 12.9M - 19.0.0-1979-g6cec793d 78f830190839 3b0eb47520be 2024-04-02T05:00:20.388 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.atrmwz smithi179 running (2m) 105s ago 15m 13.6M - 19.0.0-1979-g6cec793d 78f830190839 ecf548e68baf 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi179.mzfimf smithi179 running (2m) 105s ago 15m 12.9M - 19.0.0-1979-g6cec793d 78f830190839 49c2ed39fbf7 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.igidvr smithi070 *:8443,9283,8765 running (11m) 103s ago 22m 510M - 19.0.0-1979-g6cec793d 78f830190839 4cff16389ec4 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi179.labmvs smithi179 *:8443,9283,8765 running (11m) 105s ago 19m 451M - 19.0.0-1979-g6cec793d 78f830190839 82bf35bdf507 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (8m) 103s ago 22m 58.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 e6a7191c4a7f 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi179 smithi179 running (7m) 105s ago 19m 49.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 1cbe8f43e3c6 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (10m) 103s ago 21m 15.6M - 1.5.0 0da6a335fe13 711e9036a895 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi179 smithi179 *:9100 running (10m) 105s ago 20m 16.9M - 1.5.0 0da6a335fe13 f473a2e2fdba 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (6m) 103s ago 19m 147M 1216M 19.0.0-1979-g6cec793d 78f830190839 0dd09ba7bb69 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (6m) 103s ago 18m 144M 1216M 19.0.0-1979-g6cec793d 78f830190839 10ef3364d940 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (5m) 103s ago 18m 96.6M 1216M 19.0.0-1979-g6cec793d 78f830190839 87cd16d907e6 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi179 running (4m) 105s ago 17m 110M 2240M 19.0.0-1979-g6cec793d 78f830190839 f717fd6c8306 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi179 running (4m) 105s ago 17m 113M 2240M 19.0.0-1979-g6cec793d 78f830190839 2e5bbf3235ff 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi179 running (3m) 105s ago 16m 102M 2240M 19.0.0-1979-g6cec793d 78f830190839 71351459ac70 2024-04-02T05:00:20.389 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (10m) 103s ago 20m 67.3M - 2.43.0 a07b618ecd1d 4b33390120dc 2024-04-02T05:00:20.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.385+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fde2006dd60 msgr2=0x7fde20070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.385+0000 7fde3a2b6700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fde2006dd60 0x7fde20070220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00b5c0 tx=0x7fde1c0053a0 comp rx=0 tx=0).stop 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.385+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 msgr2=0x7fde340ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.385+0000 7fde3a2b6700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 0x7fde340ffcf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde2400d8d0 tx=0x7fde2400dc90 comp rx=0 tx=0).stop 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.386+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 shutdown_connections 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.386+0000 7fde3a2b6700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fde2006dd60 0x7fde20070220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.386+0000 7fde3a2b6700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde34068580 0x7fde340ffcf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.386+0000 7fde3a2b6700 1 --2- 172.21.15.70:0/2935804410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde34106cd0 0x7fde34100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.386+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 >> 172.21.15.70:0/2935804410 conn(0x7fde34075670 msgr2=0x7fde340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.386+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 shutdown_connections 2024-04-02T05:00:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:20.386+0000 7fde3a2b6700 1 -- 172.21.15.70:0/2935804410 wait complete. 2024-04-02T05:00:20.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:20 smithi070 ceph-mon[113635]: pgmap v293: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:20 smithi179 ceph-mon[84288]: pgmap v293: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:21.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions' 2024-04-02T05:00:21.332 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T05:00:22.451 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:22 smithi070 ceph-mon[113635]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T05:00:22.451 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:22 smithi070 ceph-mon[113635]: pgmap v294: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:22 smithi179 ceph-mon[84288]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T05:00:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:22 smithi179 ceph-mon[84288]: pgmap v294: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:22.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.977+0000 7f06e1da4700 1 -- 172.21.15.70:0/1394947426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc101850 msgr2=0x7f06dc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:22.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.977+0000 7f06e1da4700 1 --2- 172.21.15.70:0/1394947426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc101850 0x7f06dc1058a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009910 tx=0x7f06d0009c20 comp rx=0 tx=0).stop 2024-04-02T05:00:22.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.977+0000 7f06e1da4700 1 -- 172.21.15.70:0/1394947426 shutdown_connections 2024-04-02T05:00:22.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.977+0000 7f06e1da4700 1 --2- 172.21.15.70:0/1394947426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc101850 0x7f06dc1058a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:22.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.977+0000 7f06e1da4700 1 --2- 172.21.15.70:0/1394947426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06dc100ea0 0x7f06dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:22.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.977+0000 7f06e1da4700 1 -- 172.21.15.70:0/1394947426 >> 172.21.15.70:0/1394947426 conn(0x7f06dc0fc750 msgr2=0x7f06dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:22.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.978+0000 7f06e1da4700 1 -- 172.21.15.70:0/1394947426 shutdown_connections 2024-04-02T05:00:22.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.978+0000 7f06e1da4700 1 -- 172.21.15.70:0/1394947426 wait complete. 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.978+0000 7f06e1da4700 1 Processor -- start 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.978+0000 7f06e1da4700 1 -- start start 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06e1da4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc100ea0 0x7f06dc19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06e1da4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06dc101850 0x7f06dc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06e1da4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc19d410 con 0x7f06dc100ea0 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06e1da4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc1968c0 con 0x7f06dc101850 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06db7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc100ea0 0x7f06dc19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06db7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc100ea0 0x7f06dc19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36164/0 (socket says 172.21.15.70:36164) 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06db7fe700 1 -- 172.21.15.70:0/1528598294 learned_addr learned my addr 172.21.15.70:0/1528598294 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:22.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06db7fe700 1 -- 172.21.15.70:0/1528598294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06dc101850 msgr2=0x7f06dc19cd80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T05:00:22.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06db7fe700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06dc101850 0x7f06dc19cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:22.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06db7fe700 1 -- 172.21.15.70:0/1528598294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d00095f0 con 0x7f06dc100ea0 2024-04-02T05:00:22.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06db7fe700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc100ea0 0x7f06dc19c840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00b4d0 tx=0x7f06cc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:22.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06d8ff9700 1 -- 172.21.15.70:0/1528598294 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06cc011800 con 0x7f06dc100ea0 2024-04-02T05:00:22.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc196ba0 con 0x7f06dc100ea0 2024-04-02T05:00:22.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06d8ff9700 1 -- 172.21.15.70:0/1528598294 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06cc011e20 con 0x7f06dc100ea0 2024-04-02T05:00:22.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06d8ff9700 1 -- 172.21.15.70:0/1528598294 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06cc00f2e0 con 0x7f06dc100ea0 2024-04-02T05:00:22.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.979+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc1970f0 con 0x7f06dc100ea0 2024-04-02T05:00:22.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.980+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc1092a0 con 0x7f06dc100ea0 2024-04-02T05:00:22.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.981+0000 7f06d8ff9700 1 -- 172.21.15.70:0/1528598294 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f06cc010380 con 0x7f06dc100ea0 2024-04-02T05:00:22.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.981+0000 7f06d8ff9700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f06c406dd60 0x7f06c4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:22.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.981+0000 7f06d8ff9700 1 -- 172.21.15.70:0/1528598294 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f06cc08d8f0 con 0x7f06dc100ea0 2024-04-02T05:00:22.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.981+0000 7f06daffd700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f06c406dd60 0x7f06c4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:22.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.982+0000 7f06daffd700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f06c406dd60 0x7f06c4070220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06d0000c00 tx=0x7f06d0005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:22.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:22.986+0000 7f06d8ff9700 1 -- 172.21.15.70:0/1528598294 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f06cc05a800 con 0x7f06dc100ea0 2024-04-02T05:00:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.322+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f06dc04f2e0 con 0x7f06dc100ea0 2024-04-02T05:00:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.323+0000 7f06d8ff9700 1 -- 172.21.15.70:0/1528598294 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f06cc05a390 con 0x7f06dc100ea0 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T05:00:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T05:00:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-04-02T05:00:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T05:00:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-04-02T05:00:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-04-02T05:00:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 14 2024-04-02T05:00:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-04-02T05:00:23.329 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-04-02T05:00:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f06c406dd60 msgr2=0x7f06c4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f06c406dd60 0x7f06c4070220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06d0000c00 tx=0x7f06d0005fb0 comp rx=0 tx=0).stop 2024-04-02T05:00:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc100ea0 msgr2=0x7f06dc19c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc100ea0 0x7f06dc19c840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00b4d0 tx=0x7f06cc00b890 comp rx=0 tx=0).stop 2024-04-02T05:00:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 shutdown_connections 2024-04-02T05:00:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f06c406dd60 0x7f06c4070220 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06dc100ea0 0x7f06dc19c840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 --2- 172.21.15.70:0/1528598294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06dc101850 0x7f06dc19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 >> 172.21.15.70:0/1528598294 conn(0x7f06dc0fc750 msgr2=0x7f06dc0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 shutdown_connections 2024-04-02T05:00:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:23.326+0000 7f06e1da4700 1 -- 172.21.15.70:0/1528598294 wait complete. 2024-04-02T05:00:23.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-02T05:00:24.133 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T05:00:24.528 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:24 smithi070 ceph-mon[113635]: pgmap v295: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:24.528 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:24 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1528598294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T05:00:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:24 smithi179 ceph-mon[84288]: pgmap v295: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:24 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1528598294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T05:00:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 -- 172.21.15.70:0/972842823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 msgr2=0x7fb83c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 --2- 172.21.15.70:0/972842823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 0x7fb83c105890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb830009910 tx=0x7fb830009c20 comp rx=0 tx=0).stop 2024-04-02T05:00:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 -- 172.21.15.70:0/972842823 shutdown_connections 2024-04-02T05:00:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 --2- 172.21.15.70:0/972842823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 0x7fb83c105890 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 --2- 172.21.15.70:0/972842823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb83c100e90 0x7fb83c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 -- 172.21.15.70:0/972842823 >> 172.21.15.70:0/972842823 conn(0x7fb83c0fc740 msgr2=0x7fb83c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 -- 172.21.15.70:0/972842823 shutdown_connections 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 -- 172.21.15.70:0/972842823 wait complete. 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 Processor -- start 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.807+0000 7fb84203c700 1 -- start start 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb84203c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb83c100e90 0x7fb83c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb84203c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 0x7fb83c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb84203c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c19d350 con 0x7fb83c101840 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb84203c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c196800 con 0x7fb83c100e90 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb83affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 0x7fb83c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb83affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 0x7fb83c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36172/0 (socket says 172.21.15.70:36172) 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb83affd700 1 -- 172.21.15.70:0/546443021 learned_addr learned my addr 172.21.15.70:0/546443021 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb83b7fe700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb83c100e90 0x7fb83c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb83affd700 1 -- 172.21.15.70:0/546443021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb83c100e90 msgr2=0x7fb83c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb83affd700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb83c100e90 0x7fb83c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb83affd700 1 -- 172.21.15.70:0/546443021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300095f0 con 0x7fb83c101840 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb83affd700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 0x7fb83c19ccc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb8300092e0 tx=0x7fb83000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb838ff9700 1 -- 172.21.15.70:0/546443021 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb83001d070 con 0x7fb83c101840 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83c196a80 con 0x7fb83c101840 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb838ff9700 1 -- 172.21.15.70:0/546443021 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb83000f460 con 0x7fb83c101840 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.808+0000 7fb838ff9700 1 -- 172.21.15.70:0/546443021 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8300175d0 con 0x7fb83c101840 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.809+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c196f70 con 0x7fb83c101840 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.809+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83c1091e0 con 0x7fb83c101840 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.810+0000 7fb838ff9700 1 -- 172.21.15.70:0/546443021 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb83000f5d0 con 0x7fb83c101840 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.811+0000 7fb838ff9700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82406dd60 0x7fb824070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.811+0000 7fb838ff9700 1 -- 172.21.15.70:0/546443021 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fb83008f160 con 0x7fb83c101840 2024-04-02T05:00:25.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.811+0000 7fb83b7fe700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82406dd60 0x7fb824070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:25.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.811+0000 7fb83b7fe700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82406dd60 0x7fb824070220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009a10 tx=0x7fb82c009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:25.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:25.816+0000 7fb838ff9700 1 -- 172.21.15.70:0/546443021 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb83005c070 con 0x7fb83c101840 2024-04-02T05:00:26.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.252+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb83c04f2e0 con 0x7fb83c101840 2024-04-02T05:00:26.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.253+0000 7fb838ff9700 1 -- 172.21.15.70:0/546443021 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb83005bc00 con 0x7fb83c101840 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82406dd60 msgr2=0x7fb824070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82406dd60 0x7fb824070220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009a10 tx=0x7fb82c009190 comp rx=0 tx=0).stop 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 msgr2=0x7fb83c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 0x7fb83c19ccc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb8300092e0 tx=0x7fb83000bd70 comp rx=0 tx=0).stop 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 shutdown_connections 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb82406dd60 0x7fb824070220 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb83c100e90 0x7fb83c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 --2- 172.21.15.70:0/546443021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb83c101840 0x7fb83c19ccc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.256+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 >> 172.21.15.70:0/546443021 conn(0x7fb83c0fc740 msgr2=0x7fb83c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.257+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 shutdown_connections 2024-04-02T05:00:26.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:26.257+0000 7fb84203c700 1 -- 172.21.15.70:0/546443021 wait complete. 2024-04-02T05:00:26.276 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-04-02T05:00:26.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:26 smithi070 ceph-mon[113635]: pgmap v296: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:00:26.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:26 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/546443021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T05:00:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:26 smithi179 ceph-mon[84288]: pgmap v296: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:00:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:26 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/546443021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T05:00:26.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-02T05:00:27.081 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T05:00:28.527 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:28 smithi070 ceph-mon[113635]: pgmap v297: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:28 smithi179 ceph-mon[84288]: pgmap v297: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.729+0000 7fc041bbf700 1 -- 172.21.15.70:0/2521505010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c100ea0 msgr2=0x7fc03c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.729+0000 7fc041bbf700 1 --2- 172.21.15.70:0/2521505010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c100ea0 0x7fc03c101280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009910 tx=0x7fc02c009c20 comp rx=0 tx=0).stop 2024-04-02T05:00:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.729+0000 7fc041bbf700 1 -- 172.21.15.70:0/2521505010 shutdown_connections 2024-04-02T05:00:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.729+0000 7fc041bbf700 1 --2- 172.21.15.70:0/2521505010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc03c101850 0x7fc03c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.729+0000 7fc041bbf700 1 --2- 172.21.15.70:0/2521505010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c100ea0 0x7fc03c101280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.729+0000 7fc041bbf700 1 -- 172.21.15.70:0/2521505010 >> 172.21.15.70:0/2521505010 conn(0x7fc03c0fc750 msgr2=0x7fc03c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.729+0000 7fc041bbf700 1 -- 172.21.15.70:0/2521505010 shutdown_connections 2024-04-02T05:00:28.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.729+0000 7fc041bbf700 1 -- 172.21.15.70:0/2521505010 wait complete. 2024-04-02T05:00:28.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc041bbf700 1 Processor -- start 2024-04-02T05:00:28.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc041bbf700 1 -- start start 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc041bbf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc03c100ea0 0x7fc03c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc041bbf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c101850 0x7fc03c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc041bbf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19f4b0 con 0x7fc03c101850 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc041bbf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c198960 con 0x7fc03c100ea0 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc03affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c101850 0x7fc03c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc03affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c101850 0x7fc03c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55390/0 (socket says 172.21.15.70:55390) 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc03affd700 1 -- 172.21.15.70:0/447113028 learned_addr learned my addr 172.21.15.70:0/447113028 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.730+0000 7fc03b7fe700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc03c100ea0 0x7fc03c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:28.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc03affd700 1 -- 172.21.15.70:0/447113028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc03c100ea0 msgr2=0x7fc03c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc03affd700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc03c100ea0 0x7fc03c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc03affd700 1 -- 172.21.15.70:0/447113028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0095f0 con 0x7fc03c101850 2024-04-02T05:00:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc03affd700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c101850 0x7fc03c19ee20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc03000d900 tx=0x7fc03000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc038ff9700 1 -- 172.21.15.70:0/447113028 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc030009900 con 0x7fc03c101850 2024-04-02T05:00:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc038ff9700 1 -- 172.21.15.70:0/447113028 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc030010460 con 0x7fc03c101850 2024-04-02T05:00:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c198c40 con 0x7fc03c101850 2024-04-02T05:00:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc038ff9700 1 -- 172.21.15.70:0/447113028 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03000f5f0 con 0x7fc03c101850 2024-04-02T05:00:28.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.731+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c199190 con 0x7fc03c101850 2024-04-02T05:00:28.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.733+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c105020 con 0x7fc03c101850 2024-04-02T05:00:28.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.733+0000 7fc038ff9700 1 -- 172.21.15.70:0/447113028 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc03000f820 con 0x7fc03c101850 2024-04-02T05:00:28.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.733+0000 7fc038ff9700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc02406dd60 0x7fc024070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:28.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.733+0000 7fc038ff9700 1 -- 172.21.15.70:0/447113028 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fc03008d7e0 con 0x7fc03c101850 2024-04-02T05:00:28.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.733+0000 7fc03b7fe700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc02406dd60 0x7fc024070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:28.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.734+0000 7fc03b7fe700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc02406dd60 0x7fc024070220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00b5c0 tx=0x7fc02c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:28.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:28.738+0000 7fc038ff9700 1 -- 172.21.15.70:0/447113028 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc03005b790 con 0x7fc03c101850 2024-04-02T05:00:29.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.037+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc03c199e70 con 0x7fc03c101850 2024-04-02T05:00:29.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.038+0000 7fc038ff9700 1 -- 172.21.15.70:0/447113028 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc03005b320 con 0x7fc03c101850 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.039+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc02406dd60 msgr2=0x7fc024070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc02406dd60 0x7fc024070220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00b5c0 tx=0x7fc02c01a040 comp rx=0 tx=0).stop 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c101850 msgr2=0x7fc03c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c101850 0x7fc03c19ee20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc03000d900 tx=0x7fc03000dc10 comp rx=0 tx=0).stop 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 shutdown_connections 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fc02406dd60 0x7fc024070220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc03c100ea0 0x7fc03c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 --2- 172.21.15.70:0/447113028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc03c101850 0x7fc03c19ee20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 >> 172.21.15.70:0/447113028 conn(0x7fc03c0fc750 msgr2=0x7fc03c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:29.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 shutdown_connections 2024-04-02T05:00:29.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:29.040+0000 7fc041bbf700 1 -- 172.21.15.70:0/447113028 wait complete. 2024-04-02T05:00:29.054 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)" 2024-04-02T05:00:29.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:29 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/447113028' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T05:00:29.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:29 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/447113028' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T05:00:29.699 DEBUG:teuthology.parallel:result is None 2024-04-02T05:00:29.699 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T05:00:29.712 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T05:00:29.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T05:00:29.932 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/mon.smithi070/config 2024-04-02T05:00:30.309 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:30 smithi070 ceph-mon[113635]: pgmap v298: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:30.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:30 smithi179 ceph-mon[84288]: pgmap v298: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:31.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.450+0000 7f7c799a0700 1 -- 172.21.15.70:0/2956448270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 msgr2=0x7f7c74105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:31.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 --2- 172.21.15.70:0/2956448270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 0x7f7c74105890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009910 tx=0x7f7c68009c20 comp rx=0 tx=0).stop 2024-04-02T05:00:31.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 -- 172.21.15.70:0/2956448270 shutdown_connections 2024-04-02T05:00:31.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 --2- 172.21.15.70:0/2956448270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 0x7f7c74105890 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:31.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 --2- 172.21.15.70:0/2956448270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c74100e90 0x7f7c74101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 -- 172.21.15.70:0/2956448270 >> 172.21.15.70:0/2956448270 conn(0x7f7c740fc740 msgr2=0x7f7c740feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 -- 172.21.15.70:0/2956448270 shutdown_connections 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 -- 172.21.15.70:0/2956448270 wait complete. 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 Processor -- start 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.451+0000 7f7c799a0700 1 -- start start 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c799a0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c74100e90 0x7f7c7419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c799a0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 0x7f7c7419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c799a0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7419d240 con 0x7f7c74101840 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c799a0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741966f0 con 0x7f7c74100e90 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c727fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 0x7f7c7419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c727fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 0x7f7c7419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55400/0 (socket says 172.21.15.70:55400) 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c727fc700 1 -- 172.21.15.70:0/69180318 learned_addr learned my addr 172.21.15.70:0/69180318 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c72ffd700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c74100e90 0x7f7c7419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c727fc700 1 -- 172.21.15.70:0/69180318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c74100e90 msgr2=0x7f7c7419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c727fc700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c74100e90 0x7f7c7419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c727fc700 1 -- 172.21.15.70:0/69180318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680095f0 con 0x7f7c74101840 2024-04-02T05:00:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c727fc700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 0x7f7c7419cbb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800bd70 tx=0x7f7c6800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c7899e700 1 -- 172.21.15.70:0/69180318 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c6801d070 con 0x7f7c74101840 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c7899e700 1 -- 172.21.15.70:0/69180318 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c68005dc0 con 0x7f7c74101840 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c74196970 con 0x7f7c74101840 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.452+0000 7f7c7899e700 1 -- 172.21.15.70:0/69180318 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c68017400 con 0x7f7c74101840 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.453+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c74196e60 con 0x7f7c74101840 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.454+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c74109290 con 0x7f7c74101840 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.454+0000 7f7c7899e700 1 -- 172.21.15.70:0/69180318 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f7c6800f460 con 0x7f7c74101840 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.455+0000 7f7c7899e700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7c5c06dd60 0x7f7c5c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.455+0000 7f7c72ffd700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7c5c06dd60 0x7f7c5c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.455+0000 7f7c7899e700 1 -- 172.21.15.70:0/69180318 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f7c6808f150 con 0x7f7c74101840 2024-04-02T05:00:31.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.455+0000 7f7c72ffd700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7c5c06dd60 0x7f7c5c070220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005950 tx=0x7f7c6400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.459+0000 7f7c7899e700 1 -- 172.21.15.70:0/69180318 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7c6805c060 con 0x7f7c74101840 2024-04-02T05:00:31.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.703+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7c7404f2e0 con 0x7f7c74101840 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.703+0000 7f7c7899e700 1 -- 172.21.15.70:0/69180318 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1936 (secure 0 0 0) 0x7f7c6805bbf0 con 0x7f7c74101840 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:e40 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 40 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-04-02T04:44:52.788932+0000 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-04-02T04:58:41.729436+0000 2024-04-02T05:00:31.708 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 95 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=34388} 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-04-02T05:00:31.709 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:qdb_cluster leader: 34388 members: 34388 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.mnzrwt{0:34388} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.70:6826/17804049,v1:172.21.15.70:6827/17804049] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:31.710 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.zvraae{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6828/3294303398,v1:172.21.15.70:6829/3294303398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T05:00:31.711 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.mzfimf{-1:44287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.179:6824/1866330674,v1:172.21.15.179:6825/1866330674] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T05:00:31.711 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi179.atrmwz{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.179:6826/1653234055,v1:172.21.15.179:6827/1653234055] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T05:00:31.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7c5c06dd60 msgr2=0x7f7c5c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:31.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7c5c06dd60 0x7f7c5c070220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005950 tx=0x7f7c6400b410 comp rx=0 tx=0).stop 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 msgr2=0x7f7c7419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 0x7f7c7419cbb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800bd70 tx=0x7f7c6800bda0 comp rx=0 tx=0).stop 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 shutdown_connections 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7c5c06dd60 0x7f7c5c070220 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c74100e90 0x7f7c7419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 --2- 172.21.15.70:0/69180318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c74101840 0x7f7c7419cbb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 >> 172.21.15.70:0/69180318 conn(0x7f7c740fc740 msgr2=0x7f7c740fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.706+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 shutdown_connections 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:31.707+0000 7f7c799a0700 1 -- 172.21.15.70:0/69180318 wait complete. 2024-04-02T05:00:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 40 2024-04-02T05:00:32.382 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-02T05:00:32.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T05:00:32.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.530+0000 7f218710a640 1 -- 172.21.15.70:0/2909892566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 msgr2=0x7f21800ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.530+0000 7f218710a640 1 --2- 172.21.15.70:0/2909892566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 0x7f21800ff440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2174009880 tx=0x7f217402bd10 comp rx=0 tx=0).stop 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.530+0000 7f218710a640 1 -- 172.21.15.70:0/2909892566 shutdown_connections 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.530+0000 7f218710a640 1 --2- 172.21.15.70:0/2909892566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 0x7f21800ff440 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.530+0000 7f218710a640 1 --2- 172.21.15.70:0/2909892566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2180103f20 0x7f2180104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.530+0000 7f218710a640 1 -- 172.21.15.70:0/2909892566 >> 172.21.15.70:0/2909892566 conn(0x7f2180075b40 msgr2=0x7f2180075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.530+0000 7f218710a640 1 -- 172.21.15.70:0/2909892566 shutdown_connections 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.530+0000 7f218710a640 1 -- 172.21.15.70:0/2909892566 wait complete. 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f218710a640 1 Processor -- start 2024-04-02T05:00:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f218710a640 1 -- start start 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f218710a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 0x7f2180196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f218710a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2180103f20 0x7f2180197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f218710a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21801976f0 con 0x7f21800fefe0 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f218710a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180197860 con 0x7f2180103f20 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f2184ea6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 0x7f2180196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f2184ea6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 0x7f2180196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55412/0 (socket says 172.21.15.70:55412) 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f2184ea6640 1 -- 172.21.15.70:0/1183869431 learned_addr learned my addr 172.21.15.70:0/1183869431 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f2184ea6640 1 -- 172.21.15.70:0/1183869431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2180103f20 msgr2=0x7f2180197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f2184ea6640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2180103f20 0x7f2180197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.531+0000 7f2184ea6640 1 -- 172.21.15.70:0/1183869431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174009560 con 0x7f21800fefe0 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.532+0000 7f2184ea6640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 0x7f2180196be0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f217800ba50 tx=0x7f217800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.532+0000 7f217dffb640 1 -- 172.21.15.70:0/1183869431 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f217800cb60 con 0x7f21800fefe0 2024-04-02T05:00:32.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.532+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218019fc50 con 0x7f21800fefe0 2024-04-02T05:00:32.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.532+0000 7f217dffb640 1 -- 172.21.15.70:0/1183869431 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f217800ccc0 con 0x7f21800fefe0 2024-04-02T05:00:32.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.532+0000 7f217dffb640 1 -- 172.21.15.70:0/1183869431 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21780077e0 con 0x7f21800fefe0 2024-04-02T05:00:32.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.532+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21801a00d0 con 0x7f21800fefe0 2024-04-02T05:00:32.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.533+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218004e040 con 0x7f21800fefe0 2024-04-02T05:00:32.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.533+0000 7f217dffb640 1 -- 172.21.15.70:0/1183869431 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2178007960 con 0x7f21800fefe0 2024-04-02T05:00:32.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.534+0000 7f217dffb640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f216806dd90 0x7f2168070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:32.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.534+0000 7f217dffb640 1 -- 172.21.15.70:0/1183869431 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f217808d750 con 0x7f21800fefe0 2024-04-02T05:00:32.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.534+0000 7f217ffff640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f216806dd90 0x7f2168070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:32.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.534+0000 7f217ffff640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f216806dd90 0x7f2168070250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2174036040 tx=0x7f2174000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.542+0000 7f217dffb640 1 -- 172.21.15.70:0/1183869431 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2178056be0 con 0x7f21800fefe0 2024-04-02T05:00:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:32 smithi179 ceph-mon[84288]: pgmap v299: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:32.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:32 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/69180318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T05:00:32.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.720+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2180067940 con 0x7f21800fefe0 2024-04-02T05:00:32.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.721+0000 7f217dffb640 1 -- 172.21.15.70:0/1183869431 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17334 (secure 0 0 0) 0x7f217805a2a0 con 0x7f21800fefe0 2024-04-02T05:00:32.726 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:32.726 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":95,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:37:49.993595+0000","modified":"2024-04-02T04:58:01.605043+0000","last_up_change":"2024-04-02T04:56:39.019903+0000","last_in_change":"2024-04-02T04:43:12.453452+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:42:18.484513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-02T04:44:51.427296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-02T04:44:52.024713+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b5e5510-c4db-4073-b470-c777b557d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6803","nonce":4011619577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6805","nonce":4011619577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6809","nonce":4011619577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6807","nonce":4011619577}]},"public_addr":"172.21.15.70:6803/4011619577","cluster_addr":"172.21.15.70:6805/4011619577","heartbeat_back_addr":"172.21.15.70:6809/4011619577","heartbeat_front_addr":"172.21.15.70:6807/4011619577","state":["exists","up"]},{"osd":1,"uuid":"b751ceaf-b734-4280-ba22-82683b9776ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":64,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6811","nonce":2318524655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6813","nonce":2318524655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6817","nonce":2318524655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6815","nonce":2318524655}]},"public_addr":"172.21.15.70:6811/2318524655","cluster_addr":"172.21.15.70:6813/2318524655","heartbeat_back_addr":"172.21.15.70:6817/2318524655","heartbeat_front_addr":"172.21.15.70:6815/2318524655","state":["exists","up"]},{"osd":2,"uuid":"32080ab6-7809-4523-aab7-ca7a6d263ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6819","nonce":347493648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6821","nonce":347493648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6825","nonce":347493648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6823","nonce":347493648}]},"public_addr":"172.21.15.70:6819/347493648","cluster_addr":"172.21.15.70:6821/347493648","heartbeat_back_addr":"172.21.15.70:6825/347493648","heartbeat_front_addr":"172.21.15.70:6823/347493648","state":["exists","up"]},{"osd":3,"uuid":"0d71a0bb-5e7a-4339-afe7-06174dea4986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6801","nonce":657192449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6803","nonce":657192449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6807","nonce":657192449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6805","nonce":657192449}]},"public_addr":"172.21.15.179:6801/657192449","cluster_addr":"172.21.15.179:6803/657192449","heartbeat_back_addr":"172.21.15.179:6807/657192449","heartbeat_front_addr":"172.21.15.179:6805/657192449","state":["exists","up"]},{"osd":4,"uuid":"754e6f09-2a8e-42e2-a998-5f5a1addcb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6809","nonce":229347770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6811","nonce":229347770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6815","nonce":229347770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6813","nonce":229347770}]},"public_addr":"172.21.15.179:6809/229347770","cluster_addr":"172.21.15.179:6811/229347770","heartbeat_back_addr":"172.21.15.179:6815/229347770","heartbeat_front_addr":"172.21.15.179:6813/229347770","state":["exists","up"]},{"osd":5,"uuid":"46a73805-a770-4ed1-96f4-e64cf88810eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6817","nonce":2533703533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6819","nonce":2533703533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6823","nonce":2533703533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6821","nonce":2533703533}]},"public_addr":"172.21.15.179:6817/2533703533","cluster_addr":"172.21.15.179:6819/2533703533","heartbeat_back_addr":"172.21.15.179:6823/2533703533","heartbeat_front_addr":"172.21.15.179:6821/2533703533","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:17.145441+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:44.622049+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:14.717560+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:38.493463+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:04.319117+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:30.171864+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.15","mappings":[{"from":0,"to":5}]},{"pgid":"2.16","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1f","mappings":[{"from":0,"to":1},{"from":3,"to":2}]},{"pgid":"3.3","mappings":[{"from":4,"to":1}]},{"pgid":"3.11","mappings":[{"from":3,"to":1}]},{"pgid":"3.13","mappings":[{"from":3,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6827/3541837059":"2024-04-03T04:58:01.605007+0000","172.21.15.179:6824/511108019":"2024-04-03T04:57:35.242205+0000","172.21.15.179:6826/3541837059":"2024-04-03T04:58:01.605007+0000","172.21.15.70:6829/2586361462":"2024-04-03T04:57:09.380867+0000","172.21.15.70:6800/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1721127786":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1700662291":"2024-04-03T04:47:27.805482+0000","172.21.15.179:0/1433338238":"2024-04-03T04:48:41.679801+0000","172.21.15.179:0/1900906037":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6801/146104126":"2024-04-03T04:52:11.062737+0000","172.21.15.179:6825/511108019":"2024-04-03T04:57:35.242205+0000","172.21.15.70:6800/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/2463094374":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/940716355":"2024-04-03T04:38:18.397033+0000","172.21.15.179:0/1600665743":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6828/2586361462":"2024-04-03T04:57:09.380867+0000","172.21.15.70:0/3512109322":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.179:0/4148120600":"2024-04-03T04:48:41.679801+0000","172.21.15.179:6828/1862820759":"2024-04-03T04:48:41.679801+0000","172.21.15.70:0/2446527261":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/1353385001":"2024-04-03T04:47:27.805482+0000","172.21.15.179:0/1702319761":"2024-04-03T04:48:41.679801+0000","172.21.15.179:6829/1862820759":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6827/1351191990":"2024-04-03T04:56:43.851170+0000","172.21.15.70:6801/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/4149690148":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/39315361":"2024-04-03T04:47:27.805482+0000","172.21.15.70:6801/2814127860":"2024-04-03T04:47:27.805482+0000","172.21.15.70:0/110321404":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/4005781191":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/3239652190":"2024-04-03T04:52:11.062737+0000","172.21.15.70:0/2716917447":"2024-04-03T04:47:27.805482+0000","172.21.15.70:0/2937797857":"2024-04-03T04:39:58.163164+0000","172.21.15.179:0/546971533":"2024-04-03T04:48:41.679801+0000","172.21.15.70:0/2778101694":"2024-04-03T04:52:11.062737+0000","172.21.15.70:6826/1351191990":"2024-04-03T04:56:43.851170+0000","172.21.15.70:6800/146104126":"2024-04-03T04:52:11.062737+0000","172.21.15.70:6801/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2814127860":"2024-04-03T04:47:27.805482+0000","172.21.15.70:6801/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/2760078492":"2024-04-03T04:52:11.062737+0000","172.21.15.70:0/3184091580":"2024-04-03T04:52:11.062737+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T05:00:32.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f216806dd90 msgr2=0x7f2168070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:32.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f216806dd90 0x7f2168070250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2174036040 tx=0x7f2174000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 msgr2=0x7f2180196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:32.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 0x7f2180196be0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f217800ba50 tx=0x7f217800bf10 comp rx=0 tx=0).stop 2024-04-02T05:00:32.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 shutdown_connections 2024-04-02T05:00:32.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2180103f20 0x7f2180197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f216806dd90 0x7f2168070250 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 --2- 172.21.15.70:0/1183869431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21800fefe0 0x7f2180196be0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 >> 172.21.15.70:0/1183869431 conn(0x7f2180075b40 msgr2=0x7f21800fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:32.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 shutdown_connections 2024-04-02T05:00:32.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.723+0000 7f218710a640 1 -- 172.21.15.70:0/1183869431 wait complete. 2024-04-02T05:00:32.737 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:32 smithi070 ceph-mon[113635]: pgmap v299: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:32.737 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:32 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/69180318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T05:00:32.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:42:18.484513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-02T04:44:51.427296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-02T04:44:52.024713+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T05:00:32.739 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T05:00:32.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.841+0000 7f51c3fb6640 1 -- 172.21.15.70:0/563464263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc100c60 msgr2=0x7f51bc104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.841+0000 7f51c3fb6640 1 --2- 172.21.15.70:0/563464263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc100c60 0x7f51bc104ca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009880 tx=0x7f51ac02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.841+0000 7f51c3fb6640 1 -- 172.21.15.70:0/563464263 shutdown_connections 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.841+0000 7f51c3fb6640 1 --2- 172.21.15.70:0/563464263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc100c60 0x7f51bc104ca0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.841+0000 7f51c3fb6640 1 --2- 172.21.15.70:0/563464263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51bc1002b0 0x7f51bc100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.841+0000 7f51c3fb6640 1 -- 172.21.15.70:0/563464263 >> 172.21.15.70:0/563464263 conn(0x7f51bc0fbb30 msgr2=0x7f51bc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.841+0000 7f51c3fb6640 1 -- 172.21.15.70:0/563464263 shutdown_connections 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.841+0000 7f51c3fb6640 1 -- 172.21.15.70:0/563464263 wait complete. 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c3fb6640 1 Processor -- start 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c3fb6640 1 -- start start 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c3fb6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc1002b0 0x7f51bc195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c3fb6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51bc196330 0x7f51bc19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c3fb6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc196930 con 0x7f51bc1002b0 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c3fb6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc196aa0 con 0x7f51bc196330 2024-04-02T05:00:32.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c1d52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc1002b0 0x7f51bc195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c1d52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc1002b0 0x7f51bc195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55426/0 (socket says 172.21.15.70:55426) 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c1d52640 1 -- 172.21.15.70:0/3126457041 learned_addr learned my addr 172.21.15.70:0/3126457041 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.842+0000 7f51c1551640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51bc196330 0x7f51bc19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51c1d52640 1 -- 172.21.15.70:0/3126457041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51bc196330 msgr2=0x7f51bc19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51c1d52640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51bc196330 0x7f51bc19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51c1d52640 1 -- 172.21.15.70:0/3126457041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac009560 con 0x7f51bc1002b0 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51c1d52640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc1002b0 0x7f51bc195df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51b800d720 tx=0x7f51b800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51b2ffd640 1 -- 172.21.15.70:0/3126457041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b80051f0 con 0x7f51bc1002b0 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51b2ffd640 1 -- 172.21.15.70:0/3126457041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51b8009ce0 con 0x7f51bc1002b0 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51b2ffd640 1 -- 172.21.15.70:0/3126457041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b8010460 con 0x7f51bc1002b0 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc19ad90 con 0x7f51bc1002b0 2024-04-02T05:00:32.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.843+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc19b2f0 con 0x7f51bc1002b0 2024-04-02T05:00:32.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.844+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc19b5c0 con 0x7f51bc1002b0 2024-04-02T05:00:32.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.844+0000 7f51b2ffd640 1 -- 172.21.15.70:0/3126457041 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f51b8005350 con 0x7f51bc1002b0 2024-04-02T05:00:32.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.845+0000 7f51b2ffd640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f51a806dcd0 0x7f51a8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:32.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.845+0000 7f51b2ffd640 1 -- 172.21.15.70:0/3126457041 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f51b808d730 con 0x7f51bc1002b0 2024-04-02T05:00:32.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.845+0000 7f51c1551640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f51a806dcd0 0x7f51a8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:32.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.845+0000 7f51c1551640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f51a806dcd0 0x7f51a8070190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51ac000c00 tx=0x7f51ac005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:32.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:32.854+0000 7f51b2ffd640 1 -- 172.21.15.70:0/3126457041 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f51b8056c40 con 0x7f51bc1002b0 2024-04-02T05:00:33.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.035+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51bc197140 con 0x7f51bc1002b0 2024-04-02T05:00:33.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.035+0000 7f51b2ffd640 1 -- 172.21.15.70:0/3126457041 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51b805a300 con 0x7f51bc1002b0 2024-04-02T05:00:33.040 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-04-02T05:00:33.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f51a806dcd0 msgr2=0x7f51a8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f51a806dcd0 0x7f51a8070190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51ac000c00 tx=0x7f51ac005fd0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc1002b0 msgr2=0x7f51bc195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc1002b0 0x7f51bc195df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51b800d720 tx=0x7f51b800dbe0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 shutdown_connections 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51bc196330 0x7f51bc19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f51a806dcd0 0x7f51a8070190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 --2- 172.21.15.70:0/3126457041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51bc1002b0 0x7f51bc195df0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 >> 172.21.15.70:0/3126457041 conn(0x7f51bc0fbb30 msgr2=0x7f51bc0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 shutdown_connections 2024-04-02T05:00:33.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.037+0000 7f51c3fb6640 1 -- 172.21.15.70:0/3126457041 wait complete. 2024-04-02T05:00:33.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-02T05:00:33.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.153+0000 7f5785377640 1 -- 172.21.15.70:0/3788605918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5780067f70 msgr2=0x7f5780104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.153+0000 7f5785377640 1 --2- 172.21.15.70:0/3788605918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5780067f70 0x7f5780104c70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5768009880 tx=0x7f576802bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.153+0000 7f5785377640 1 -- 172.21.15.70:0/3788605918 shutdown_connections 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.153+0000 7f5785377640 1 --2- 172.21.15.70:0/3788605918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5780067f70 0x7f5780104c70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.153+0000 7f5785377640 1 --2- 172.21.15.70:0/3788605918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57800675c0 0x7f57800679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.153+0000 7f5785377640 1 -- 172.21.15.70:0/3788605918 >> 172.21.15.70:0/3788605918 conn(0x7f5780074a50 msgr2=0x7f5780074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.153+0000 7f5785377640 1 -- 172.21.15.70:0/3788605918 shutdown_connections 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.153+0000 7f5785377640 1 -- 172.21.15.70:0/3788605918 wait complete. 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f5785377640 1 Processor -- start 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f5785377640 1 -- start start 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f5785377640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57800675c0 0x7f57800ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f5785377640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57800ff600 0x7f57800ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f5785377640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57801a1f40 con 0x7f57800675c0 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f5785377640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57801a2080 con 0x7f57800ff600 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f577effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57800675c0 0x7f57800ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f577effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57800675c0 0x7f57800ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55448/0 (socket says 172.21.15.70:55448) 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.154+0000 7f577effd640 1 -- 172.21.15.70:0/61721031 learned_addr learned my addr 172.21.15.70:0/61721031 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f577effd640 1 -- 172.21.15.70:0/61721031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57800ff600 msgr2=0x7f57800ffa60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f577effd640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57800ff600 0x7f57800ffa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f577effd640 1 -- 172.21.15.70:0/61721031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768009560 con 0x7f57800675c0 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f577effd640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57800675c0 0x7f57800ff0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f577000c920 tx=0x7f577000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f577cff9640 1 -- 172.21.15.70:0/61721031 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5770007b80 con 0x7f57800675c0 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f577cff9640 1 -- 172.21.15.70:0/61721031 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57700054e0 con 0x7f57800675c0 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57801a21c0 con 0x7f57800675c0 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f577cff9640 1 -- 172.21.15.70:0/61721031 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f577000f450 con 0x7f57800675c0 2024-04-02T05:00:33.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.155+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57801a2720 con 0x7f57800675c0 2024-04-02T05:00:33.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.156+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578004e040 con 0x7f57800675c0 2024-04-02T05:00:33.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.156+0000 7f577cff9640 1 -- 172.21.15.70:0/61721031 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5770004f60 con 0x7f57800675c0 2024-04-02T05:00:33.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.157+0000 7f577cff9640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f576c06dcd0 0x7f576c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.157+0000 7f57767fc640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f576c06dcd0 0x7f576c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:33.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.157+0000 7f577cff9640 1 -- 172.21.15.70:0/61721031 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f577008db30 con 0x7f57800675c0 2024-04-02T05:00:33.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.157+0000 7f57767fc640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f576c06dcd0 0x7f576c070190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5768036040 tx=0x7f5768005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.162+0000 7f577cff9640 1 -- 172.21.15.70:0/61721031 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5770056f10 con 0x7f57800675c0 2024-04-02T05:00:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.350+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f5780108580 con 0x7f57800675c0 2024-04-02T05:00:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.350+0000 7f577cff9640 1 -- 172.21.15.70:0/61721031 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7f577005a5d0 con 0x7f57800675c0 2024-04-02T05:00:33.355 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-04-02T05:00:33.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f576c06dcd0 msgr2=0x7f576c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f576c06dcd0 0x7f576c070190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5768036040 tx=0x7f5768005fd0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57800675c0 msgr2=0x7f57800ff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57800675c0 0x7f57800ff0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f577000c920 tx=0x7f577000cde0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 shutdown_connections 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57800ff600 0x7f57800ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f576c06dcd0 0x7f576c070190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 --2- 172.21.15.70:0/61721031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57800675c0 0x7f57800ff0c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 >> 172.21.15.70:0/61721031 conn(0x7f5780074a50 msgr2=0x7f57800fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 shutdown_connections 2024-04-02T05:00:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.352+0000 7f5785377640 1 -- 172.21.15.70:0/61721031 wait complete. 2024-04-02T05:00:33.368 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:33 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1183869431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T05:00:33.368 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:33 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3126457041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T05:00:33.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 -- 172.21.15.70:0/3989161821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ffb50 msgr2=0x7fbd84103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 --2- 172.21.15.70:0/3989161821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ffb50 0x7fbd84103b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009880 tx=0x7fbd7402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 -- 172.21.15.70:0/3989161821 shutdown_connections 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 --2- 172.21.15.70:0/3989161821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ffb50 0x7fbd84103b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 --2- 172.21.15.70:0/3989161821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd840ff1a0 0x7fbd840ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 -- 172.21.15.70:0/3989161821 >> 172.21.15.70:0/3989161821 conn(0x7fbd840faa20 msgr2=0x7fbd840fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 -- 172.21.15.70:0/3989161821 shutdown_connections 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 -- 172.21.15.70:0/3989161821 wait complete. 2024-04-02T05:00:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 Processor -- start 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.475+0000 7fbd89784640 1 -- start start 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd89784640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ff1a0 0x7fbd84195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd89784640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd84196310 0x7fbd8419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd89784640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd84196910 con 0x7fbd840ff1a0 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd89784640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd84196a80 con 0x7fbd84196310 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd82ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ff1a0 0x7fbd84195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd82ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ff1a0 0x7fbd84195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55472/0 (socket says 172.21.15.70:55472) 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd82ffd640 1 -- 172.21.15.70:0/3770107726 learned_addr learned my addr 172.21.15.70:0/3770107726 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd827fc640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd84196310 0x7fbd8419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd82ffd640 1 -- 172.21.15.70:0/3770107726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd84196310 msgr2=0x7fbd8419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd82ffd640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd84196310 0x7fbd8419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd82ffd640 1 -- 172.21.15.70:0/3770107726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd74009560 con 0x7fbd840ff1a0 2024-04-02T05:00:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd82ffd640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ff1a0 0x7fbd84195dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800d720 tx=0x7fbd7800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:33.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd6bfff640 1 -- 172.21.15.70:0/3770107726 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd780051f0 con 0x7fbd840ff1a0 2024-04-02T05:00:33.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd6bfff640 1 -- 172.21.15.70:0/3770107726 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd78009ce0 con 0x7fbd840ff1a0 2024-04-02T05:00:33.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8419ad70 con 0x7fbd840ff1a0 2024-04-02T05:00:33.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd6bfff640 1 -- 172.21.15.70:0/3770107726 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd78010460 con 0x7fbd840ff1a0 2024-04-02T05:00:33.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.476+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8419b2d0 con 0x7fbd840ff1a0 2024-04-02T05:00:33.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.477+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8404e040 con 0x7fbd840ff1a0 2024-04-02T05:00:33.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.478+0000 7fbd6bfff640 1 -- 172.21.15.70:0/3770107726 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbd78005350 con 0x7fbd840ff1a0 2024-04-02T05:00:33.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.478+0000 7fbd6bfff640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd6c06dcd0 0x7fbd6c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.478+0000 7fbd827fc640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd6c06dcd0 0x7fbd6c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:33.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.478+0000 7fbd6bfff640 1 -- 172.21.15.70:0/3770107726 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fbd7808d860 con 0x7fbd840ff1a0 2024-04-02T05:00:33.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.479+0000 7fbd827fc640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd6c06dcd0 0x7fbd6c070190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd74006010 tx=0x7fbd74000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:33.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.483+0000 7fbd6bfff640 1 -- 172.21.15.70:0/3770107726 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbd78056c40 con 0x7fbd840ff1a0 2024-04-02T05:00:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:33 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1183869431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T05:00:33.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:33 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3126457041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T05:00:33.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.661+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fbd8419b550 con 0x7fbd840ff1a0 2024-04-02T05:00:33.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.662+0000 7fbd6bfff640 1 -- 172.21.15.70:0/3770107726 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7fbd7805a300 con 0x7fbd840ff1a0 2024-04-02T05:00:33.666 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-04-02T05:00:33.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd6c06dcd0 msgr2=0x7fbd6c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd6c06dcd0 0x7fbd6c070190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd74006010 tx=0x7fbd74000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ff1a0 msgr2=0x7fbd84195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ff1a0 0x7fbd84195dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800d720 tx=0x7fbd7800dbe0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 shutdown_connections 2024-04-02T05:00:33.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd84196310 0x7fbd8419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbd6c06dcd0 0x7fbd6c070190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 --2- 172.21.15.70:0/3770107726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd840ff1a0 0x7fbd84195dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 >> 172.21.15.70:0/3770107726 conn(0x7fbd840faa20 msgr2=0x7fbd841033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:33.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 shutdown_connections 2024-04-02T05:00:33.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.664+0000 7fbd89784640 1 -- 172.21.15.70:0/3770107726 wait complete. 2024-04-02T05:00:33.679 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.776+0000 7f65da3ae640 1 -- 172.21.15.70:0/1133988943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 msgr2=0x7f65d40ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.776+0000 7f65da3ae640 1 --2- 172.21.15.70:0/1133988943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 0x7f65d40ff580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009880 tx=0x7f65bc02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.776+0000 7f65da3ae640 1 -- 172.21.15.70:0/1133988943 shutdown_connections 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.776+0000 7f65da3ae640 1 --2- 172.21.15.70:0/1133988943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d40ffb50 0x7f65d4103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.776+0000 7f65da3ae640 1 --2- 172.21.15.70:0/1133988943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 0x7f65d40ff580 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.776+0000 7f65da3ae640 1 -- 172.21.15.70:0/1133988943 >> 172.21.15.70:0/1133988943 conn(0x7f65d40faa40 msgr2=0x7f65d40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.776+0000 7f65da3ae640 1 -- 172.21.15.70:0/1133988943 shutdown_connections 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.776+0000 7f65da3ae640 1 -- 172.21.15.70:0/1133988943 wait complete. 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65da3ae640 1 Processor -- start 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65da3ae640 1 -- start start 2024-04-02T05:00:33.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65da3ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 0x7f65d419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65da3ae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d419c010 0x7f65d4195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65da3ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d419c5e0 con 0x7f65d40ff1a0 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65da3ae640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4196080 con 0x7f65d419c010 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 0x7f65d419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 0x7f65d419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55490/0 (socket says 172.21.15.70:55490) 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.777+0000 7f65d3fff640 1 -- 172.21.15.70:0/475565432 learned_addr learned my addr 172.21.15.70:0/475565432 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d37fe640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d419c010 0x7f65d4195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d3fff640 1 -- 172.21.15.70:0/475565432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d419c010 msgr2=0x7f65d4195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d3fff640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d419c010 0x7f65d4195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d3fff640 1 -- 172.21.15.70:0/475565432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc009560 con 0x7f65d40ff1a0 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d37fe640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d419c010 0x7f65d4195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d3fff640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 0x7f65d419bad0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009230 tx=0x7f65bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d17fa640 1 -- 172.21.15.70:0/475565432 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65bc0035f0 con 0x7f65d40ff1a0 2024-04-02T05:00:33.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4196300 con 0x7f65d40ff1a0 2024-04-02T05:00:33.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d17fa640 1 -- 172.21.15.70:0/475565432 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65bc003c10 con 0x7f65d40ff1a0 2024-04-02T05:00:33.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65d17fa640 1 -- 172.21.15.70:0/475565432 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65bc034440 con 0x7f65d40ff1a0 2024-04-02T05:00:33.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.778+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d4196800 con 0x7f65d40ff1a0 2024-04-02T05:00:33.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.779+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d404e040 con 0x7f65d40ff1a0 2024-04-02T05:00:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.780+0000 7f65d17fa640 1 -- 172.21.15.70:0/475565432 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f65bc004fd0 con 0x7f65d40ff1a0 2024-04-02T05:00:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.780+0000 7f65d17fa640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f65c006dd90 0x7f65c0070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.781+0000 7f65d37fe640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f65c006dd90 0x7f65c0070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.781+0000 7f65d17fa640 1 -- 172.21.15.70:0/475565432 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f65bc0b01a0 con 0x7f65d40ff1a0 2024-04-02T05:00:33.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.781+0000 7f65d37fe640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f65c006dd90 0x7f65c0070250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65c40058c0 tx=0x7f65c4005d30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:33.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:33.785+0000 7f65d17fa640 1 -- 172.21.15.70:0/475565432 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f65bc079580 con 0x7f65d40ff1a0 2024-04-02T05:00:34.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.031+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65d4196fe0 con 0x7f65d40ff1a0 2024-04-02T05:00:34.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.031+0000 7f65d17fa640 1 -- 172.21.15.70:0/475565432 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5207 (secure 0 0 0) 0x7f65bc07cc40 con 0x7f65d40ff1a0 2024-04-02T05:00:34.036 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:34.036 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44315,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6827/1653234055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1653234055},{"type":"v1","addr":"172.21.15.179:6827","nonce":1653234055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:58:41.729436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":34,"state":"up:active","state_seq":23,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34388,"qdb_cluster":[34388]},"id":1}]} 2024-04-02T05:00:34.036 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 40 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f65c006dd90 msgr2=0x7f65c0070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f65c006dd90 0x7f65c0070250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65c40058c0 tx=0x7f65c4005d30 comp rx=0 tx=0).stop 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 msgr2=0x7f65d419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 0x7f65d419bad0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009230 tx=0x7f65bc004040 comp rx=0 tx=0).stop 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 shutdown_connections 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d419c010 0x7f65d4195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f65c006dd90 0x7f65c0070250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 --2- 172.21.15.70:0/475565432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d40ff1a0 0x7f65d419bad0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 >> 172.21.15.70:0/475565432 conn(0x7f65d40faa40 msgr2=0x7f65d41032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 shutdown_connections 2024-04-02T05:00:34.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.034+0000 7f65da3ae640 1 -- 172.21.15.70:0/475565432 wait complete. 2024-04-02T05:00:34.051 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 13, 'max_mds': 1, 'flags': 18} 2024-04-02T05:00:34.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-04-02T05:00:34.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.172+0000 7f8d25284640 1 -- 172.21.15.70:0/3058924064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 msgr2=0x7f8d20100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.172+0000 7f8d25284640 1 --2- 172.21.15.70:0/3058924064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 0x7f8d20100680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009880 tx=0x7f8d1002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.172+0000 7f8d25284640 1 -- 172.21.15.70:0/3058924064 shutdown_connections 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.172+0000 7f8d25284640 1 --2- 172.21.15.70:0/3058924064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d20100c50 0x7f8d20104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.172+0000 7f8d25284640 1 --2- 172.21.15.70:0/3058924064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 0x7f8d20100680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.172+0000 7f8d25284640 1 -- 172.21.15.70:0/3058924064 >> 172.21.15.70:0/3058924064 conn(0x7f8d200fbb20 msgr2=0x7f8d200fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.172+0000 7f8d25284640 1 -- 172.21.15.70:0/3058924064 shutdown_connections 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.172+0000 7f8d25284640 1 -- 172.21.15.70:0/3058924064 wait complete. 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d25284640 1 Processor -- start 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d25284640 1 -- start start 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d25284640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 0x7f8d20197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d25284640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d201984e0 0x7f8d2019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d25284640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20198ae0 con 0x7f8d201002a0 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d25284640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20198c50 con 0x7f8d201984e0 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d1effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 0x7f8d20197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d1effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 0x7f8d20197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55516/0 (socket says 172.21.15.70:55516) 2024-04-02T05:00:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d1effd640 1 -- 172.21.15.70:0/3612163008 learned_addr learned my addr 172.21.15.70:0/3612163008 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d17fff640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d201984e0 0x7f8d2019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d1effd640 1 -- 172.21.15.70:0/3612163008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d201984e0 msgr2=0x7f8d2019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d1effd640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d201984e0 0x7f8d2019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d1effd640 1 -- 172.21.15.70:0/3612163008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d10009560 con 0x7f8d201002a0 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.173+0000 7f8d1effd640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 0x7f8d20197fa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009430 tx=0x7f8d10004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.174+0000 7f8d1cff9640 1 -- 172.21.15.70:0/3612163008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d100035f0 con 0x7f8d201002a0 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.174+0000 7f8d1cff9640 1 -- 172.21.15.70:0/3612163008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d10003c10 con 0x7f8d201002a0 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.174+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2019cee0 con 0x7f8d201002a0 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.174+0000 7f8d1cff9640 1 -- 172.21.15.70:0/3612163008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d10005670 con 0x7f8d201002a0 2024-04-02T05:00:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.174+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2019d440 con 0x7f8d201002a0 2024-04-02T05:00:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.175+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2004e040 con 0x7f8d201002a0 2024-04-02T05:00:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.175+0000 7f8d1cff9640 1 -- 172.21.15.70:0/3612163008 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f8d10004dd0 con 0x7f8d201002a0 2024-04-02T05:00:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.175+0000 7f8d1cff9640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8d0c06dc80 0x7f8d0c070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.176+0000 7f8d1cff9640 1 -- 172.21.15.70:0/3612163008 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f8d100b0040 con 0x7f8d201002a0 2024-04-02T05:00:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.176+0000 7f8d17fff640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8d0c06dc80 0x7f8d0c070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.176+0000 7f8d17fff640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8d0c06dc80 0x7f8d0c070140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d200696e0 tx=0x7f8d0800b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:34.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.180+0000 7f8d1cff9640 1 -- 172.21.15.70:0/3612163008 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d100794d0 con 0x7f8d201002a0 2024-04-02T05:00:34.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.394+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7f8d201992f0 con 0x7f8d201002a0 2024-04-02T05:00:34.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.395+0000 7f8d1cff9640 1 -- 172.21.15.70:0/3612163008 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v40) v1 ==== 107+0+4858 (secure 0 0 0) 0x7f8d1007cb90 con 0x7f8d201002a0 2024-04-02T05:00:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:34.399 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14480,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6829/2586361462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2586361462},{"type":"v1","addr":"172.21.15.70:6829","nonce":2586361462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:45:08.643957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6827/1351191990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1351191990},{"type":"v1","addr":"172.21.15.70:6827","nonce":1351191990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:34.399 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 14 2024-04-02T05:00:34.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.396+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8d0c06dc80 msgr2=0x7f8d0c070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.396+0000 7f8d25284640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8d0c06dc80 0x7f8d0c070140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d200696e0 tx=0x7f8d0800b390 comp rx=0 tx=0).stop 2024-04-02T05:00:34.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.396+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 msgr2=0x7f8d20197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.396+0000 7f8d25284640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 0x7f8d20197fa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009430 tx=0x7f8d10004040 comp rx=0 tx=0).stop 2024-04-02T05:00:34.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.397+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 shutdown_connections 2024-04-02T05:00:34.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.397+0000 7f8d25284640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d201984e0 0x7f8d2019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.397+0000 7f8d25284640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8d0c06dc80 0x7f8d0c070140 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.397+0000 7f8d25284640 1 --2- 172.21.15.70:0/3612163008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d201002a0 0x7f8d20197fa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.397+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 >> 172.21.15.70:0/3612163008 conn(0x7f8d200fbb20 msgr2=0x7f8d200fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:34.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.397+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 shutdown_connections 2024-04-02T05:00:34.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.397+0000 7f8d25284640 1 -- 172.21.15.70:0/3612163008 wait complete. 2024-04-02T05:00:34.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:34 smithi070 ceph-mon[113635]: pgmap v300: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:34.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:34 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/61721031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T05:00:34.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:34 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3770107726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T05:00:34.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:34 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/475565432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T05:00:34.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-04-02T05:00:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.517+0000 7f225110a640 1 -- 172.21.15.70:0/1710795541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 msgr2=0x7f224c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.517+0000 7f225110a640 1 --2- 172.21.15.70:0/1710795541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 0x7f224c0ff640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2240009880 tx=0x7f224002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.517+0000 7f225110a640 1 -- 172.21.15.70:0/1710795541 shutdown_connections 2024-04-02T05:00:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.517+0000 7f225110a640 1 --2- 172.21.15.70:0/1710795541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f224c0ffc10 0x7f224c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.517+0000 7f225110a640 1 --2- 172.21.15.70:0/1710795541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 0x7f224c0ff640 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.517+0000 7f225110a640 1 -- 172.21.15.70:0/1710795541 >> 172.21.15.70:0/1710795541 conn(0x7f224c074a50 msgr2=0x7f224c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.517+0000 7f225110a640 1 -- 172.21.15.70:0/1710795541 shutdown_connections 2024-04-02T05:00:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.517+0000 7f225110a640 1 -- 172.21.15.70:0/1710795541 wait complete. 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.518+0000 7f225110a640 1 Processor -- start 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.518+0000 7f225110a640 1 -- start start 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.518+0000 7f225110a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 0x7f224c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.518+0000 7f225110a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f224c19afa0 0x7f224c194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.518+0000 7f225110a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c19b570 con 0x7f224c0ff260 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.518+0000 7f225110a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c195010 con 0x7f224c19afa0 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 0x7f224c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f224ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 0x7f224c19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55534/0 (socket says 172.21.15.70:55534) 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f224ad9d640 1 -- 172.21.15.70:0/930932551 learned_addr learned my addr 172.21.15.70:0/930932551 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f224ad9d640 1 -- 172.21.15.70:0/930932551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f224c19afa0 msgr2=0x7f224c194ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f224ad9d640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f224c19afa0 0x7f224c194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f224ad9d640 1 -- 172.21.15.70:0/930932551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2240009560 con 0x7f224c0ff260 2024-04-02T05:00:34.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f224ad9d640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 0x7f224c19aa60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2240009230 tx=0x7f2240004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:34.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f2233fff640 1 -- 172.21.15.70:0/930932551 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22400035f0 con 0x7f224c0ff260 2024-04-02T05:00:34.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f2233fff640 1 -- 172.21.15.70:0/930932551 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2240003c10 con 0x7f224c0ff260 2024-04-02T05:00:34.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c195290 con 0x7f224c0ff260 2024-04-02T05:00:34.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f2233fff640 1 -- 172.21.15.70:0/930932551 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2240005670 con 0x7f224c0ff260 2024-04-02T05:00:34.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.519+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c195790 con 0x7f224c0ff260 2024-04-02T05:00:34.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.520+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c04e040 con 0x7f224c0ff260 2024-04-02T05:00:34.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.521+0000 7f2233fff640 1 -- 172.21.15.70:0/930932551 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2240004dd0 con 0x7f224c0ff260 2024-04-02T05:00:34.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.522+0000 7f2233fff640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f223406dc80 0x7f2234070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.522+0000 7f2233fff640 1 -- 172.21.15.70:0/930932551 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f22400b0120 con 0x7f224c0ff260 2024-04-02T05:00:34.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.522+0000 7f224a59c640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f223406dc80 0x7f2234070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:34.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.522+0000 7f224a59c640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f223406dc80 0x7f2234070140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f223c005fd0 tx=0x7f223c005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:34.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.526+0000 7f2233fff640 1 -- 172.21.15.70:0/930932551 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2240079500 con 0x7f224c0ff260 2024-04-02T05:00:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:34 smithi179 ceph-mon[84288]: pgmap v300: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:34 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/61721031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T05:00:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:34 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3770107726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T05:00:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:34 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/475565432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T05:00:34.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.743+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7f224c107480 con 0x7f224c0ff260 2024-04-02T05:00:34.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.744+0000 7f2233fff640 1 -- 172.21.15.70:0/930932551 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v40) v1 ==== 107+0+4070 (secure 0 0 0) 0x7f224007cbc0 con 0x7f224c0ff260 2024-04-02T05:00:34.748 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:34.748 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14480,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.70:6829/2586361462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2586361462},{"type":"v1","addr":"172.21.15.70:6829","nonce":2586361462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:56:43.851188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:34.748 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 15 2024-04-02T05:00:34.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f223406dc80 msgr2=0x7f2234070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f223406dc80 0x7f2234070140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f223c005fd0 tx=0x7f223c005850 comp rx=0 tx=0).stop 2024-04-02T05:00:34.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 msgr2=0x7f224c19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 0x7f224c19aa60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2240009230 tx=0x7f2240004040 comp rx=0 tx=0).stop 2024-04-02T05:00:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 shutdown_connections 2024-04-02T05:00:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f224c19afa0 0x7f224c194ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f223406dc80 0x7f2234070140 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 --2- 172.21.15.70:0/930932551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c0ff260 0x7f224c19aa60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 >> 172.21.15.70:0/930932551 conn(0x7f224c074a50 msgr2=0x7f224c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 shutdown_connections 2024-04-02T05:00:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.746+0000 7f225110a640 1 -- 172.21.15.70:0/930932551 wait complete. 2024-04-02T05:00:34.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-02T05:00:34.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.859+0000 7f3c4038e640 1 -- 172.21.15.70:0/1974385258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38067f70 msgr2=0x7f3c38104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.859+0000 7f3c4038e640 1 --2- 172.21.15.70:0/1974385258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38067f70 0x7f3c38104c70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009880 tx=0x7f3c2802bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.859+0000 7f3c4038e640 1 -- 172.21.15.70:0/1974385258 shutdown_connections 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.859+0000 7f3c4038e640 1 --2- 172.21.15.70:0/1974385258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c38067f70 0x7f3c38104c70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.859+0000 7f3c4038e640 1 --2- 172.21.15.70:0/1974385258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c380675c0 0x7f3c380679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.859+0000 7f3c4038e640 1 -- 172.21.15.70:0/1974385258 >> 172.21.15.70:0/1974385258 conn(0x7f3c38074a50 msgr2=0x7f3c38074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.859+0000 7f3c4038e640 1 -- 172.21.15.70:0/1974385258 shutdown_connections 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.859+0000 7f3c4038e640 1 -- 172.21.15.70:0/1974385258 wait complete. 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c4038e640 1 Processor -- start 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c4038e640 1 -- start start 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c4038e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c380675c0 0x7f3c38195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c4038e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c381962c0 0x7f3c3819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c4038e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381968c0 con 0x7f3c381962c0 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c4038e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38196a30 con 0x7f3c380675c0 2024-04-02T05:00:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c3d929640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c381962c0 0x7f3c3819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c3d929640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c381962c0 0x7f3c3819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55554/0 (socket says 172.21.15.70:55554) 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.860+0000 7f3c3d929640 1 -- 172.21.15.70:0/1019156822 learned_addr learned my addr 172.21.15.70:0/1019156822 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c3d929640 1 -- 172.21.15.70:0/1019156822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c380675c0 msgr2=0x7f3c38195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c3e12a640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c380675c0 0x7f3c38195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c3d929640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c380675c0 0x7f3c38195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c3d929640 1 -- 172.21.15.70:0/1019156822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28009560 con 0x7f3c381962c0 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c3d929640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c381962c0 0x7f3c3819a720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3c28036040 tx=0x7f3c2802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/1019156822 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c280035f0 con 0x7f3c381962c0 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/1019156822 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c28003c10 con 0x7f3c381962c0 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/1019156822 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c28034440 con 0x7f3c381962c0 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3819acc0 con 0x7f3c381962c0 2024-04-02T05:00:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.861+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3819b220 con 0x7f3c381962c0 2024-04-02T05:00:34.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.862+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3804e040 con 0x7f3c381962c0 2024-04-02T05:00:34.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.862+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/1019156822 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f3c28003750 con 0x7f3c381962c0 2024-04-02T05:00:34.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.863+0000 7f3c2f7fe640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3c2406dcd0 0x7f3c24070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:34.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.863+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/1019156822 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f3c280b00c0 con 0x7f3c381962c0 2024-04-02T05:00:34.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.863+0000 7f3c3e12a640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3c2406dcd0 0x7f3c24070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:34.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.863+0000 7f3c3e12a640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3c2406dcd0 0x7f3c24070190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c34005fd0 tx=0x7f3c34005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:34.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:34.867+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/1019156822 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3c280794a0 con 0x7f3c381962c0 2024-04-02T05:00:35.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.083+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f3c38108580 con 0x7f3c381962c0 2024-04-02T05:00:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.084+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/1019156822 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v40) v1 ==== 107+0+4081 (secure 0 0 0) 0x7f3c2807cb60 con 0x7f3c381962c0 2024-04-02T05:00:35.088 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:35.088 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:56:43.855740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi070.zvraae","rank":0,"incarnation":16,"state":"up:replay","state_seq":2,"addr":"172.21.15.70:6829/2586361462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2586361462},{"type":"v1","addr":"172.21.15.70:6829","nonce":2586361462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:35.088 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2024-04-02T05:00:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3c2406dcd0 msgr2=0x7f3c24070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3c2406dcd0 0x7f3c24070190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c34005fd0 tx=0x7f3c34005850 comp rx=0 tx=0).stop 2024-04-02T05:00:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c381962c0 msgr2=0x7f3c3819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c381962c0 0x7f3c3819a720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3c28036040 tx=0x7f3c2802c850 comp rx=0 tx=0).stop 2024-04-02T05:00:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 shutdown_connections 2024-04-02T05:00:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f3c2406dcd0 0x7f3c24070190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c381962c0 0x7f3c3819a720 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 --2- 172.21.15.70:0/1019156822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c380675c0 0x7f3c38195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 >> 172.21.15.70:0/1019156822 conn(0x7f3c38074a50 msgr2=0x7f3c380fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 shutdown_connections 2024-04-02T05:00:35.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.086+0000 7f3c4038e640 1 -- 172.21.15.70:0/1019156822 wait complete. 2024-04-02T05:00:35.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-02T05:00:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.204+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3122416834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0675c0 msgr2=0x7ff9cc0679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.204+0000 7ff9d0c41640 1 --2- 172.21.15.70:0/3122416834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0675c0 0x7ff9cc0679a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009880 tx=0x7ff9bc02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.204+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3122416834 shutdown_connections 2024-04-02T05:00:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.204+0000 7ff9d0c41640 1 --2- 172.21.15.70:0/3122416834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc067f70 0x7ff9cc104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.204+0000 7ff9d0c41640 1 --2- 172.21.15.70:0/3122416834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0675c0 0x7ff9cc0679a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.204+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3122416834 >> 172.21.15.70:0/3122416834 conn(0x7ff9cc074a50 msgr2=0x7ff9cc074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.204+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3122416834 shutdown_connections 2024-04-02T05:00:35.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.204+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3122416834 wait complete. 2024-04-02T05:00:35.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9d0c41640 1 Processor -- start 2024-04-02T05:00:35.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9d0c41640 1 -- start start 2024-04-02T05:00:35.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9d0c41640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0675c0 0x7ff9cc0ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9d0c41640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0ff600 0x7ff9cc0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9d0c41640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1a1f70 con 0x7ff9cc0ff600 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9d0c41640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1a20b0 con 0x7ff9cc0675c0 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9c9d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0ff600 0x7ff9cc0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9c9d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0ff600 0x7ff9cc0ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55564/0 (socket says 172.21.15.70:55564) 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9c9d9b640 1 -- 172.21.15.70:0/3992911475 learned_addr learned my addr 172.21.15.70:0/3992911475 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9ca59c640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0675c0 0x7ff9cc0ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.205+0000 7ff9c9d9b640 1 -- 172.21.15.70:0/3992911475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0675c0 msgr2=0x7ff9cc0ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9c9d9b640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0675c0 0x7ff9cc0ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9c9d9b640 1 -- 172.21.15.70:0/3992911475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc009560 con 0x7ff9cc0ff600 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9ca59c640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0675c0 0x7ff9cc0ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9c9d9b640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0ff600 0x7ff9cc0ffa60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000e9a0 tx=0x7ff9c000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9bb7fe640 1 -- 172.21.15.70:0/3992911475 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c000cc90 con 0x7ff9cc0ff600 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc1a21f0 con 0x7ff9cc0ff600 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9bb7fe640 1 -- 172.21.15.70:0/3992911475 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9c000cdf0 con 0x7ff9cc0ff600 2024-04-02T05:00:35.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9bb7fe640 1 -- 172.21.15.70:0/3992911475 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c0010430 con 0x7ff9cc0ff600 2024-04-02T05:00:35.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.206+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc1a2750 con 0x7ff9cc0ff600 2024-04-02T05:00:35.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.207+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc04e040 con 0x7ff9cc0ff600 2024-04-02T05:00:35.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.207+0000 7ff9bb7fe640 1 -- 172.21.15.70:0/3992911475 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff9c0005020 con 0x7ff9cc0ff600 2024-04-02T05:00:35.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.208+0000 7ff9bb7fe640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff9b406dcd0 0x7ff9b4070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.208+0000 7ff9bb7fe640 1 -- 172.21.15.70:0/3992911475 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7ff9c0014070 con 0x7ff9cc0ff600 2024-04-02T05:00:35.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.208+0000 7ff9ca59c640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff9b406dcd0 0x7ff9b4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:35.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.208+0000 7ff9ca59c640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff9b406dcd0 0x7ff9b4070190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc000c00 tx=0x7ff9bc02cb40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:35.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.212+0000 7ff9bb7fe640 1 -- 172.21.15.70:0/3992911475 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff9c0057620 con 0x7ff9cc0ff600 2024-04-02T05:00:35.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.441+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7ff9cc068010 con 0x7ff9cc0ff600 2024-04-02T05:00:35.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.441+0000 7ff9bb7fe640 1 -- 172.21.15.70:0/3992911475 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v40) v1 ==== 107+0+4922 (secure 0 0 0) 0x7ff9c005ace0 con 0x7ff9cc0ff600 2024-04-02T05:00:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:35.446 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:56:43.855740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi070.zvraae","rank":0,"incarnation":16,"state":"up:replay","state_seq":2,"addr":"172.21.15.70:6829/2586361462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2586361462},{"type":"v1","addr":"172.21.15.70:6829","nonce":2586361462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:35.446 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-04-02T05:00:35.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff9b406dcd0 msgr2=0x7ff9b4070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff9b406dcd0 0x7ff9b4070190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc000c00 tx=0x7ff9bc02cb40 comp rx=0 tx=0).stop 2024-04-02T05:00:35.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0ff600 msgr2=0x7ff9cc0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0ff600 0x7ff9cc0ffa60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000e9a0 tx=0x7ff9c000ee60 comp rx=0 tx=0).stop 2024-04-02T05:00:35.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 shutdown_connections 2024-04-02T05:00:35.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff9b406dcd0 0x7ff9b4070190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9cc0ff600 0x7ff9cc0ffa60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 --2- 172.21.15.70:0/3992911475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc0675c0 0x7ff9cc0ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 >> 172.21.15.70:0/3992911475 conn(0x7ff9cc074a50 msgr2=0x7ff9cc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:35.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 shutdown_connections 2024-04-02T05:00:35.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.444+0000 7ff9d0c41640 1 -- 172.21.15.70:0/3992911475 wait complete. 2024-04-02T05:00:35.458 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:35 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3612163008' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-02T05:00:35.458 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:35 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/930932551' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-02T05:00:35.458 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:35 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1019156822' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-02T05:00:35.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-02T05:00:35.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.558+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1759764341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 msgr2=0x7fb0e81006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.558+0000 7fb0ee65b640 1 --2- 172.21.15.70:0/1759764341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 0x7fb0e81006b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009880 tx=0x7fb0d802bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.558+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1759764341 shutdown_connections 2024-04-02T05:00:35.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.558+0000 7fb0ee65b640 1 --2- 172.21.15.70:0/1759764341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e8100c80 0x7fb0e8104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.558+0000 7fb0ee65b640 1 --2- 172.21.15.70:0/1759764341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 0x7fb0e81006b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.558+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1759764341 >> 172.21.15.70:0/1759764341 conn(0x7fb0e80fbb50 msgr2=0x7fb0e80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:35.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.558+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1759764341 shutdown_connections 2024-04-02T05:00:35.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.558+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1759764341 wait complete. 2024-04-02T05:00:35.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0ee65b640 1 Processor -- start 2024-04-02T05:00:35.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0ee65b640 1 -- start start 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0ee65b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 0x7fb0e8195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0ee65b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e8196310 0x7fb0e819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0ee65b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8196910 con 0x7fb0e81002d0 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0ee65b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8196a80 con 0x7fb0e8196310 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 0x7fb0e8195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 0x7fb0e8195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55576/0 (socket says 172.21.15.70:55576) 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.559+0000 7fb0e7fff640 1 -- 172.21.15.70:0/1244247876 learned_addr learned my addr 172.21.15.70:0/1244247876 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0e7fff640 1 -- 172.21.15.70:0/1244247876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e8196310 msgr2=0x7fb0e819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0e7fff640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e8196310 0x7fb0e819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0e7fff640 1 -- 172.21.15.70:0/1244247876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d8009560 con 0x7fb0e81002d0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0e7fff640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 0x7fb0e8195dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009430 tx=0x7fb0d8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0e57fa640 1 -- 172.21.15.70:0/1244247876 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d80035f0 con 0x7fb0e81002d0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e819ad10 con 0x7fb0e81002d0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0e57fa640 1 -- 172.21.15.70:0/1244247876 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0d8003c10 con 0x7fb0e81002d0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0e57fa640 1 -- 172.21.15.70:0/1244247876 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d8034440 con 0x7fb0e81002d0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.560+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e819b270 con 0x7fb0e81002d0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.561+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e804e040 con 0x7fb0e81002d0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.562+0000 7fb0e57fa640 1 -- 172.21.15.70:0/1244247876 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb0d8004fd0 con 0x7fb0e81002d0 2024-04-02T05:00:35.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.562+0000 7fb0e57fa640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb0d006dcd0 0x7fb0d0070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.562+0000 7fb0e57fa640 1 -- 172.21.15.70:0/1244247876 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fb0d80b0110 con 0x7fb0e81002d0 2024-04-02T05:00:35.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.562+0000 7fb0e77fe640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb0d006dcd0 0x7fb0d0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:35.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.562+0000 7fb0e77fe640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb0d006dcd0 0x7fb0d0070190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8067c50 tx=0x7fb0dc005bf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:35.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.567+0000 7fb0e57fa640 1 -- 172.21.15.70:0/1244247876 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb0d80794f0 con 0x7fb0e81002d0 2024-04-02T05:00:35.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:35 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3612163008' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-02T05:00:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:35 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/930932551' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-02T05:00:35.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:35 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1019156822' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-02T05:00:35.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.783+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7fb0e8197120 con 0x7fb0e81002d0 2024-04-02T05:00:35.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.783+0000 7fb0e57fa640 1 -- 172.21.15.70:0/1244247876 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v40) v1 ==== 107+0+4927 (secure 0 0 0) 0x7fb0d807cbb0 con 0x7fb0e81002d0 2024-04-02T05:00:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:35.788 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:56:52.757800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi070.zvraae","rank":0,"incarnation":16,"state":"up:reconnect","state_seq":179,"addr":"172.21.15.70:6829/2586361462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2586361462},{"type":"v1","addr":"172.21.15.70:6829","nonce":2586361462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:35.788 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2024-04-02T05:00:35.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.785+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb0d006dcd0 msgr2=0x7fb0d0070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.785+0000 7fb0ee65b640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb0d006dcd0 0x7fb0d0070190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8067c50 tx=0x7fb0dc005bf0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.785+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 msgr2=0x7fb0e8195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.785+0000 7fb0ee65b640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 0x7fb0e8195dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009430 tx=0x7fb0d8004040 comp rx=0 tx=0).stop 2024-04-02T05:00:35.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.785+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 shutdown_connections 2024-04-02T05:00:35.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.785+0000 7fb0ee65b640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e8196310 0x7fb0e819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.785+0000 7fb0ee65b640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb0d006dcd0 0x7fb0d0070190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.785+0000 7fb0ee65b640 1 --2- 172.21.15.70:0/1244247876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e81002d0 0x7fb0e8195dd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.786+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 >> 172.21.15.70:0/1244247876 conn(0x7fb0e80fbb50 msgr2=0x7fb0e80fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:35.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.786+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 shutdown_connections 2024-04-02T05:00:35.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.786+0000 7fb0ee65b640 1 -- 172.21.15.70:0/1244247876 wait complete. 2024-04-02T05:00:35.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-02T05:00:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 -- 172.21.15.70:0/17731826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a800ffc10 msgr2=0x7f7a80103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 --2- 172.21.15.70:0/17731826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a800ffc10 0x7f7a80103b70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009880 tx=0x7f7a7002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 -- 172.21.15.70:0/17731826 shutdown_connections 2024-04-02T05:00:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 --2- 172.21.15.70:0/17731826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a800ffc10 0x7f7a80103b70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 --2- 172.21.15.70:0/17731826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a800ff260 0x7f7a800ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 -- 172.21.15.70:0/17731826 >> 172.21.15.70:0/17731826 conn(0x7f7a80074a50 msgr2=0x7f7a80074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 -- 172.21.15.70:0/17731826 shutdown_connections 2024-04-02T05:00:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 -- 172.21.15.70:0/17731826 wait complete. 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 Processor -- start 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.900+0000 7f7a87441640 1 -- start start 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a87441640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a800ff260 0x7f7a80196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a87441640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a801970b0 0x7f7a80197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a87441640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8019fbe0 con 0x7f7a801970b0 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a87441640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8019fd20 con 0x7f7a800ff260 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a849dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a801970b0 0x7f7a80197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a849dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a801970b0 0x7f7a80197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55596/0 (socket says 172.21.15.70:55596) 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a849dc640 1 -- 172.21.15.70:0/686786021 learned_addr learned my addr 172.21.15.70:0/686786021 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a851dd640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a800ff260 0x7f7a80196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a849dc640 1 -- 172.21.15.70:0/686786021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a800ff260 msgr2=0x7f7a80196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a849dc640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a800ff260 0x7f7a80196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a849dc640 1 -- 172.21.15.70:0/686786021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a70009560 con 0x7f7a801970b0 2024-04-02T05:00:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a849dc640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a801970b0 0x7f7a80197510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7a70036040 tx=0x7f7a7002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a767fc640 1 -- 172.21.15.70:0/686786021 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a700035f0 con 0x7f7a801970b0 2024-04-02T05:00:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a767fc640 1 -- 172.21.15.70:0/686786021 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a70003c10 con 0x7f7a801970b0 2024-04-02T05:00:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8019ff20 con 0x7f7a801970b0 2024-04-02T05:00:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a767fc640 1 -- 172.21.15.70:0/686786021 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a70034440 con 0x7f7a801970b0 2024-04-02T05:00:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.901+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a801a03a0 con 0x7f7a801970b0 2024-04-02T05:00:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.902+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8004e040 con 0x7f7a801970b0 2024-04-02T05:00:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.903+0000 7f7a767fc640 1 -- 172.21.15.70:0/686786021 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f7a70003750 con 0x7f7a801970b0 2024-04-02T05:00:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.903+0000 7f7a767fc640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7a6c06dcd0 0x7f7a6c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.903+0000 7f7a767fc640 1 -- 172.21.15.70:0/686786021 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f7a700affd0 con 0x7f7a801970b0 2024-04-02T05:00:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.903+0000 7f7a851dd640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7a6c06dcd0 0x7f7a6c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.904+0000 7f7a851dd640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7a6c06dcd0 0x7f7a6c070190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00bbe0 tx=0x7f7a7c00b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:35.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:35.908+0000 7f7a767fc640 1 -- 172.21.15.70:0/686786021 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7a70079460 con 0x7f7a801970b0 2024-04-02T05:00:36.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.245+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f7a80107480 con 0x7f7a801970b0 2024-04-02T05:00:36.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.246+0000 7f7a767fc640 1 -- 172.21.15.70:0/686786021 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v40) v1 ==== 107+0+4924 (secure 0 0 0) 0x7f7a7007cb20 con 0x7f7a801970b0 2024-04-02T05:00:36.250 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:36.250 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:56:53.176045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi070.zvraae","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":180,"addr":"172.21.15.70:6829/2586361462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2586361462},{"type":"v1","addr":"172.21.15.70:6829","nonce":2586361462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:36.250 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 19 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7a6c06dcd0 msgr2=0x7f7a6c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7a6c06dcd0 0x7f7a6c070190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00bbe0 tx=0x7f7a7c00b390 comp rx=0 tx=0).stop 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a801970b0 msgr2=0x7f7a80197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a801970b0 0x7f7a80197510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7a70036040 tx=0x7f7a7002c850 comp rx=0 tx=0).stop 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 shutdown_connections 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f7a6c06dcd0 0x7f7a6c070190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a801970b0 0x7f7a80197510 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 --2- 172.21.15.70:0/686786021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a800ff260 0x7f7a80196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 >> 172.21.15.70:0/686786021 conn(0x7f7a80074a50 msgr2=0x7f7a800fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:36.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 shutdown_connections 2024-04-02T05:00:36.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.248+0000 7f7a87441640 1 -- 172.21.15.70:0/686786021 wait complete. 2024-04-02T05:00:36.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.361+0000 7f1c05af7640 1 -- 172.21.15.70:0/2557669662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 msgr2=0x7f1c000679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.361+0000 7f1c05af7640 1 --2- 172.21.15.70:0/2557669662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 0x7f1c000679a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009880 tx=0x7f1bf002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.361+0000 7f1c05af7640 1 -- 172.21.15.70:0/2557669662 shutdown_connections 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.361+0000 7f1c05af7640 1 --2- 172.21.15.70:0/2557669662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c00067f70 0x7f1c00104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.361+0000 7f1c05af7640 1 --2- 172.21.15.70:0/2557669662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 0x7f1c000679a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.361+0000 7f1c05af7640 1 -- 172.21.15.70:0/2557669662 >> 172.21.15.70:0/2557669662 conn(0x7f1c00074a50 msgr2=0x7f1c00074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.361+0000 7f1c05af7640 1 -- 172.21.15.70:0/2557669662 shutdown_connections 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.361+0000 7f1c05af7640 1 -- 172.21.15.70:0/2557669662 wait complete. 2024-04-02T05:00:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1c05af7640 1 Processor -- start 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1c05af7640 1 -- start start 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1c05af7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 0x7f1c00195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1c05af7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c001962d0 0x7f1c0019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1c05af7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c00196840 con 0x7f1c000675c0 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1c05af7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c001969b0 con 0x7f1c001962d0 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1bff7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 0x7f1c00195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1bff7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 0x7f1c00195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55606/0 (socket says 172.21.15.70:55606) 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.362+0000 7f1bff7fe640 1 -- 172.21.15.70:0/3088843626 learned_addr learned my addr 172.21.15.70:0/3088843626 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1bff7fe640 1 -- 172.21.15.70:0/3088843626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c001962d0 msgr2=0x7f1c0019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1bff7fe640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c001962d0 0x7f1c0019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1bff7fe640 1 -- 172.21.15.70:0/3088843626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf0009560 con 0x7f1c000675c0 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1bff7fe640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 0x7f1c00195d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0000c00 tx=0x7f1bf0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1bfcff9640 1 -- 172.21.15.70:0/3088843626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bf00037c0 con 0x7f1c000675c0 2024-04-02T05:00:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1bfcff9640 1 -- 172.21.15.70:0/3088843626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bf0003de0 con 0x7f1c000675c0 2024-04-02T05:00:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0019acd0 con 0x7f1c000675c0 2024-04-02T05:00:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1bfcff9640 1 -- 172.21.15.70:0/3088843626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bf0034460 con 0x7f1c000675c0 2024-04-02T05:00:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.363+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0019b1e0 con 0x7f1c000675c0 2024-04-02T05:00:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.364+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0004e040 con 0x7f1c000675c0 2024-04-02T05:00:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.365+0000 7f1bfcff9640 1 -- 172.21.15.70:0/3088843626 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f1bf0003920 con 0x7f1c000675c0 2024-04-02T05:00:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.365+0000 7f1bfcff9640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1be806dcd0 0x7f1be8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.365+0000 7f1bfcff9640 1 -- 172.21.15.70:0/3088843626 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f1bf00b01b0 con 0x7f1c000675c0 2024-04-02T05:00:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.365+0000 7f1bfeffd640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1be806dcd0 0x7f1be8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:36.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.365+0000 7f1bfeffd640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1be806dcd0 0x7f1be8070190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c00072e40 tx=0x7f1bf4005c70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:36.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.373+0000 7f1bfcff9640 1 -- 172.21.15.70:0/3088843626 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1bf0079650 con 0x7f1c000675c0 2024-04-02T05:00:36.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:36 smithi070 ceph-mon[113635]: pgmap v301: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:36.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:36 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3992911475' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-02T05:00:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:36 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1244247876' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-02T05:00:36.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:36 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/686786021' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-02T05:00:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:36 smithi179 ceph-mon[84288]: pgmap v301: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:36 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3992911475' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-02T05:00:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:36 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1244247876' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-02T05:00:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:36 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/686786021' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-02T05:00:36.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.585+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f1c00108580 con 0x7f1c000675c0 2024-04-02T05:00:36.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.586+0000 7f1bfcff9640 1 -- 172.21.15.70:0/3088843626 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v40) v1 ==== 107+0+4933 (secure 0 0 0) 0x7f1bf007cd10 con 0x7f1c000675c0 2024-04-02T05:00:36.590 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:36.591 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:56:55.179081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi070.zvraae","rank":0,"incarnation":16,"state":"up:active","state_seq":181,"addr":"172.21.15.70:6829/2586361462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2586361462},{"type":"v1","addr":"172.21.15.70:6829","nonce":2586361462}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14480,"qdb_cluster":[14480]},"id":1}]} 2024-04-02T05:00:36.591 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1be806dcd0 msgr2=0x7f1be8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1be806dcd0 0x7f1be8070190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c00072e40 tx=0x7f1bf4005c70 comp rx=0 tx=0).stop 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 msgr2=0x7f1c00195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 0x7f1c00195d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0000c00 tx=0x7f1bf0004270 comp rx=0 tx=0).stop 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 shutdown_connections 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c001962d0 0x7f1c0019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f1be806dcd0 0x7f1be8070190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 --2- 172.21.15.70:0/3088843626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c000675c0 0x7f1c00195d90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 >> 172.21.15.70:0/3088843626 conn(0x7f1c00074a50 msgr2=0x7f1c000fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 shutdown_connections 2024-04-02T05:00:36.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.589+0000 7f1c05af7640 1 -- 172.21.15.70:0/3088843626 wait complete. 2024-04-02T05:00:36.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-02T05:00:36.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.709+0000 7faf72520640 1 -- 172.21.15.70:0/3879364295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 msgr2=0x7faf6c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.709+0000 7faf72520640 1 --2- 172.21.15.70:0/3879364295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 0x7faf6c0ff640 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf60009880 tx=0x7faf6002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.709+0000 7faf72520640 1 -- 172.21.15.70:0/3879364295 shutdown_connections 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.709+0000 7faf72520640 1 --2- 172.21.15.70:0/3879364295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c0ffc10 0x7faf6c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.709+0000 7faf72520640 1 --2- 172.21.15.70:0/3879364295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 0x7faf6c0ff640 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.709+0000 7faf72520640 1 -- 172.21.15.70:0/3879364295 >> 172.21.15.70:0/3879364295 conn(0x7faf6c074a50 msgr2=0x7faf6c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.709+0000 7faf72520640 1 -- 172.21.15.70:0/3879364295 shutdown_connections 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.709+0000 7faf72520640 1 -- 172.21.15.70:0/3879364295 wait complete. 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf72520640 1 Processor -- start 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf72520640 1 -- start start 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf72520640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 0x7faf6c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf72520640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c197150 0x7faf6c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:36.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf72520640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c19fc80 con 0x7faf6c0ff260 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf72520640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c19fdc0 con 0x7faf6c197150 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf6bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 0x7faf6c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf6bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 0x7faf6c196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56444/0 (socket says 172.21.15.70:56444) 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf6bfff640 1 -- 172.21.15.70:0/2030870505 learned_addr learned my addr 172.21.15.70:0/2030870505 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf6bfff640 1 -- 172.21.15.70:0/2030870505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c197150 msgr2=0x7faf6c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf6bfff640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c197150 0x7faf6c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.710+0000 7faf6bfff640 1 -- 172.21.15.70:0/2030870505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009560 con 0x7faf6c0ff260 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.711+0000 7faf6bfff640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 0x7faf6c196c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faf60009430 tx=0x7faf60005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.711+0000 7faf697fa640 1 -- 172.21.15.70:0/2030870505 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf600044a0 con 0x7faf6c0ff260 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.711+0000 7faf697fa640 1 -- 172.21.15.70:0/2030870505 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf60003ab0 con 0x7faf6c0ff260 2024-04-02T05:00:36.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.711+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c19ffc0 con 0x7faf6c0ff260 2024-04-02T05:00:36.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.711+0000 7faf697fa640 1 -- 172.21.15.70:0/2030870505 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf600055f0 con 0x7faf6c0ff260 2024-04-02T05:00:36.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.711+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c1a0440 con 0x7faf6c0ff260 2024-04-02T05:00:36.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.712+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c04e040 con 0x7faf6c0ff260 2024-04-02T05:00:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.713+0000 7faf697fa640 1 -- 172.21.15.70:0/2030870505 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7faf60004e60 con 0x7faf6c0ff260 2024-04-02T05:00:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.713+0000 7faf697fa640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf5406dcd0 0x7faf54070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.713+0000 7faf697fa640 1 -- 172.21.15.70:0/2030870505 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7faf600b00b0 con 0x7faf6c0ff260 2024-04-02T05:00:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.713+0000 7faf6b7fe640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf5406dcd0 0x7faf54070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.714+0000 7faf6b7fe640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf5406dcd0 0x7faf54070190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0058c0 tx=0x7faf5c005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:36.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.718+0000 7faf697fa640 1 -- 172.21.15.70:0/2030870505 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faf60079490 con 0x7faf6c0ff260 2024-04-02T05:00:36.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.933+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7faf6c107480 con 0x7faf6c0ff260 2024-04-02T05:00:36.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.934+0000 7faf697fa640 1 -- 172.21.15.70:0/2030870505 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v40) v1 ==== 107+0+4132 (secure 0 0 0) 0x7faf6007cb50 con 0x7faf6c0ff260 2024-04-02T05:00:36.937 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:36.937 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:09.381135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:36.938 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 21 2024-04-02T05:00:36.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf5406dcd0 msgr2=0x7faf54070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf5406dcd0 0x7faf54070190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0058c0 tx=0x7faf5c005850 comp rx=0 tx=0).stop 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 msgr2=0x7faf6c196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 0x7faf6c196c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faf60009430 tx=0x7faf60005e50 comp rx=0 tx=0).stop 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 shutdown_connections 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c197150 0x7faf6c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7faf5406dcd0 0x7faf54070190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 --2- 172.21.15.70:0/2030870505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf6c0ff260 0x7faf6c196c10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 >> 172.21.15.70:0/2030870505 conn(0x7faf6c074a50 msgr2=0x7faf6c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 shutdown_connections 2024-04-02T05:00:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:36.936+0000 7faf72520640 1 -- 172.21.15.70:0/2030870505 wait complete. 2024-04-02T05:00:36.952 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-02T05:00:37.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.050+0000 7f72c6a0b640 1 -- 172.21.15.70:0/242846466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c00ffc10 msgr2=0x7f72c0103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.050+0000 7f72c6a0b640 1 --2- 172.21.15.70:0/242846466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c00ffc10 0x7f72c0103b70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009880 tx=0x7f72b402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.050+0000 7f72c6a0b640 1 -- 172.21.15.70:0/242846466 shutdown_connections 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.050+0000 7f72c6a0b640 1 --2- 172.21.15.70:0/242846466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c00ffc10 0x7f72c0103b70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.050+0000 7f72c6a0b640 1 --2- 172.21.15.70:0/242846466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c00ff260 0x7f72c00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.050+0000 7f72c6a0b640 1 -- 172.21.15.70:0/242846466 >> 172.21.15.70:0/242846466 conn(0x7f72c0074a50 msgr2=0x7f72c0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.051+0000 7f72c6a0b640 1 -- 172.21.15.70:0/242846466 shutdown_connections 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.051+0000 7f72c6a0b640 1 -- 172.21.15.70:0/242846466 wait complete. 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.051+0000 7f72c6a0b640 1 Processor -- start 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.051+0000 7f72c6a0b640 1 -- start start 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.051+0000 7f72c6a0b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c00ff260 0x7f72c0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.051+0000 7f72c6a0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c01970f0 0x7f72c0197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.051+0000 7f72c6a0b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c019fc20 con 0x7f72c01970f0 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.051+0000 7f72c6a0b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c019fd60 con 0x7f72c00ff260 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c01970f0 0x7f72c0197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c01970f0 0x7f72c0197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56456/0 (socket says 172.21.15.70:56456) 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bf7fe640 1 -- 172.21.15.70:0/3451823709 learned_addr learned my addr 172.21.15.70:0/3451823709 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bffff640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c00ff260 0x7f72c0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bf7fe640 1 -- 172.21.15.70:0/3451823709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c00ff260 msgr2=0x7f72c0196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bf7fe640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c00ff260 0x7f72c0196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bf7fe640 1 -- 172.21.15.70:0/3451823709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4009560 con 0x7f72c01970f0 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bf7fe640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c01970f0 0x7f72c0197550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f72b4036040 tx=0x7f72b402c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bd7fa640 1 -- 172.21.15.70:0/3451823709 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b40035f0 con 0x7f72c01970f0 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c019ff60 con 0x7f72c01970f0 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bd7fa640 1 -- 172.21.15.70:0/3451823709 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72b4003c10 con 0x7f72c01970f0 2024-04-02T05:00:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72bd7fa640 1 -- 172.21.15.70:0/3451823709 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b4005630 con 0x7f72c01970f0 2024-04-02T05:00:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.052+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c01a03e0 con 0x7f72c01970f0 2024-04-02T05:00:37.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.053+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c004e040 con 0x7f72c01970f0 2024-04-02T05:00:37.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.054+0000 7f72bd7fa640 1 -- 172.21.15.70:0/3451823709 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f72b4003750 con 0x7f72c01970f0 2024-04-02T05:00:37.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.054+0000 7f72bd7fa640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f72a806dcd0 0x7f72a8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.054+0000 7f72bd7fa640 1 -- 172.21.15.70:0/3451823709 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f72b40b0080 con 0x7f72c01970f0 2024-04-02T05:00:37.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.054+0000 7f72bffff640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f72a806dcd0 0x7f72a8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.054+0000 7f72bffff640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f72a806dcd0 0x7f72a8070190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009b60 tx=0x7f72b0005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:37.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.059+0000 7f72bd7fa640 1 -- 172.21.15.70:0/3451823709 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f72b4079460 con 0x7f72c01970f0 2024-04-02T05:00:37.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.325+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f72c0107480 con 0x7f72c01970f0 2024-04-02T05:00:37.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.326+0000 7f72bd7fa640 1 -- 172.21.15.70:0/3451823709 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v40) v1 ==== 107+0+4143 (secure 0 0 0) 0x7f72b407cb20 con 0x7f72c01970f0 2024-04-02T05:00:37.329 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:37.329 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:09.385586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":0,"incarnation":22,"state":"up:replay","state_seq":3,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:37.329 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 22 2024-04-02T05:00:37.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f72a806dcd0 msgr2=0x7f72a8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f72a806dcd0 0x7f72a8070190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009b60 tx=0x7f72b0005f90 comp rx=0 tx=0).stop 2024-04-02T05:00:37.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c01970f0 msgr2=0x7f72c0197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c01970f0 0x7f72c0197550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f72b4036040 tx=0x7f72b402c750 comp rx=0 tx=0).stop 2024-04-02T05:00:37.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 shutdown_connections 2024-04-02T05:00:37.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f72a806dcd0 0x7f72a8070190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72c01970f0 0x7f72c0197550 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 --2- 172.21.15.70:0/3451823709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72c00ff260 0x7f72c0196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 >> 172.21.15.70:0/3451823709 conn(0x7f72c0074a50 msgr2=0x7f72c00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:37.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 shutdown_connections 2024-04-02T05:00:37.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.328+0000 7f72c6a0b640 1 -- 172.21.15.70:0/3451823709 wait complete. 2024-04-02T05:00:37.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-02T05:00:37.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.446+0000 7f274cff0640 1 -- 172.21.15.70:0/2640355105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ffc10 msgr2=0x7f2748103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.446+0000 7f274cff0640 1 --2- 172.21.15.70:0/2640355105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ffc10 0x7f2748103b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f273c009880 tx=0x7f273c02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.446+0000 7f274cff0640 1 -- 172.21.15.70:0/2640355105 shutdown_connections 2024-04-02T05:00:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.446+0000 7f274cff0640 1 --2- 172.21.15.70:0/2640355105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ffc10 0x7f2748103b70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.446+0000 7f274cff0640 1 --2- 172.21.15.70:0/2640355105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27480ff260 0x7f27480ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.446+0000 7f274cff0640 1 -- 172.21.15.70:0/2640355105 >> 172.21.15.70:0/2640355105 conn(0x7f2748074a50 msgr2=0x7f2748074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.446+0000 7f274cff0640 1 -- 172.21.15.70:0/2640355105 shutdown_connections 2024-04-02T05:00:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.446+0000 7f274cff0640 1 -- 172.21.15.70:0/2640355105 wait complete. 2024-04-02T05:00:37.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274cff0640 1 Processor -- start 2024-04-02T05:00:37.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274cff0640 1 -- start start 2024-04-02T05:00:37.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274cff0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ff260 0x7f274819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274cff0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f274819af60 0x7f2748194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274cff0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274819b530 con 0x7f27480ff260 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274cff0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2748194fd0 con 0x7f274819af60 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ff260 0x7f274819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ff260 0x7f274819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56470/0 (socket says 172.21.15.70:56470) 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274659c640 1 -- 172.21.15.70:0/572889727 learned_addr learned my addr 172.21.15.70:0/572889727 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f2745d9b640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f274819af60 0x7f2748194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274659c640 1 -- 172.21.15.70:0/572889727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f274819af60 msgr2=0x7f2748194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.447+0000 7f274659c640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f274819af60 0x7f2748194a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.448+0000 7f274659c640 1 -- 172.21.15.70:0/572889727 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c009560 con 0x7f27480ff260 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.448+0000 7f274659c640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ff260 0x7f274819aa20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f273800e970 tx=0x7f273800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.448+0000 7f27377fe640 1 -- 172.21.15.70:0/572889727 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273800cc90 con 0x7f27480ff260 2024-04-02T05:00:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.448+0000 7f27377fe640 1 -- 172.21.15.70:0/572889727 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f273800cdf0 con 0x7f27480ff260 2024-04-02T05:00:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.448+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27481952b0 con 0x7f27480ff260 2024-04-02T05:00:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.448+0000 7f27377fe640 1 -- 172.21.15.70:0/572889727 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2738010430 con 0x7f27480ff260 2024-04-02T05:00:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.448+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2748195810 con 0x7f27480ff260 2024-04-02T05:00:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.449+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274804e040 con 0x7f27480ff260 2024-04-02T05:00:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.450+0000 7f27377fe640 1 -- 172.21.15.70:0/572889727 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f27380046c0 con 0x7f27480ff260 2024-04-02T05:00:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.450+0000 7f27377fe640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f273006dcd0 0x7f2730070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.450+0000 7f2745d9b640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f273006dcd0 0x7f2730070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.450+0000 7f27377fe640 1 -- 172.21.15.70:0/572889727 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f2738014070 con 0x7f27480ff260 2024-04-02T05:00:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.451+0000 7f2745d9b640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f273006dcd0 0x7f2730070190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f273c006010 tx=0x7f273c000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:37.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.455+0000 7f27377fe640 1 -- 172.21.15.70:0/572889727 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f27380576c0 con 0x7f27480ff260 2024-04-02T05:00:37.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:37 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3088843626' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-02T05:00:37.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:37 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2030870505' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-02T05:00:37.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.667+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f2748195fa0 con 0x7f27480ff260 2024-04-02T05:00:37.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.667+0000 7f27377fe640 1 -- 172.21.15.70:0/572889727 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v40) v1 ==== 107+0+4148 (secure 0 0 0) 0x7f273805ad80 con 0x7f27480ff260 2024-04-02T05:00:37.671 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:37.671 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:18.046930+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":0,"incarnation":22,"state":"up:reconnect","state_seq":186,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:37.671 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 23 2024-04-02T05:00:37.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f273006dcd0 msgr2=0x7f2730070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f273006dcd0 0x7f2730070190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f273c006010 tx=0x7f273c000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ff260 msgr2=0x7f274819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ff260 0x7f274819aa20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f273800e970 tx=0x7f273800ed70 comp rx=0 tx=0).stop 2024-04-02T05:00:37.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 shutdown_connections 2024-04-02T05:00:37.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f274819af60 0x7f2748194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f273006dcd0 0x7f2730070190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 --2- 172.21.15.70:0/572889727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480ff260 0x7f274819aa20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 >> 172.21.15.70:0/572889727 conn(0x7f2748074a50 msgr2=0x7f27480fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:37.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 shutdown_connections 2024-04-02T05:00:37.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.669+0000 7f274cff0640 1 -- 172.21.15.70:0/572889727 wait complete. 2024-04-02T05:00:37.683 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:37 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3088843626' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-02T05:00:37.683 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:37 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2030870505' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-02T05:00:37.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-02T05:00:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/4135523181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 msgr2=0x7f4b040ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 --2- 172.21.15.70:0/4135523181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 0x7f4b040ff640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009880 tx=0x7f4af402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/4135523181 shutdown_connections 2024-04-02T05:00:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 --2- 172.21.15.70:0/4135523181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b040ffc10 0x7f4b04103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 --2- 172.21.15.70:0/4135523181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 0x7f4b040ff640 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/4135523181 >> 172.21.15.70:0/4135523181 conn(0x7f4b04074a50 msgr2=0x7f4b04074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/4135523181 shutdown_connections 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/4135523181 wait complete. 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 Processor -- start 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.793+0000 7f4b0a1cb640 1 -- start start 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b0a1cb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 0x7f4b04196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b0a1cb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b04197150 0x7f4b041975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b0a1cb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0419fc80 con 0x7f4b040ff260 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b0a1cb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0419fdc0 con 0x7f4b04197150 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 0x7f4b04196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 0x7f4b04196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56482/0 (socket says 172.21.15.70:56482) 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b037fe640 1 -- 172.21.15.70:0/3440510238 learned_addr learned my addr 172.21.15.70:0/3440510238 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b02ffd640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b04197150 0x7f4b041975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b037fe640 1 -- 172.21.15.70:0/3440510238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b04197150 msgr2=0x7f4b041975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b037fe640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b04197150 0x7f4b041975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b037fe640 1 -- 172.21.15.70:0/3440510238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af4009560 con 0x7f4b040ff260 2024-04-02T05:00:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b02ffd640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b04197150 0x7f4b041975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T05:00:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b037fe640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 0x7f4b04196c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009430 tx=0x7f4af4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b00ff9640 1 -- 172.21.15.70:0/3440510238 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af40035f0 con 0x7f4b040ff260 2024-04-02T05:00:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b00ff9640 1 -- 172.21.15.70:0/3440510238 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4af4003c10 con 0x7f4b040ff260 2024-04-02T05:00:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0419ffc0 con 0x7f4b040ff260 2024-04-02T05:00:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b00ff9640 1 -- 172.21.15.70:0/3440510238 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af4005670 con 0x7f4b040ff260 2024-04-02T05:00:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.794+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b041a03f0 con 0x7f4b040ff260 2024-04-02T05:00:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.795+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0404e040 con 0x7f4b040ff260 2024-04-02T05:00:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.796+0000 7f4b00ff9640 1 -- 172.21.15.70:0/3440510238 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f4af4004dd0 con 0x7f4b040ff260 2024-04-02T05:00:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.796+0000 7f4b00ff9640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4aec06dc80 0x7f4aec070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.796+0000 7f4b00ff9640 1 -- 172.21.15.70:0/3440510238 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f4af40b00f0 con 0x7f4b040ff260 2024-04-02T05:00:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.796+0000 7f4b02ffd640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4aec06dc80 0x7f4aec070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.797+0000 7f4b02ffd640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4aec06dc80 0x7f4aec070140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4af80058c0 tx=0x7f4af8005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:37.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:37.801+0000 7f4b00ff9640 1 -- 172.21.15.70:0/3440510238 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4af40794d0 con 0x7f4b040ff260 2024-04-02T05:00:38.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.014+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f4b040ffd00 con 0x7f4b040ff260 2024-04-02T05:00:38.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.015+0000 7f4b00ff9640 1 -- 172.21.15.70:0/3440510238 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v40) v1 ==== 107+0+4145 (secure 0 0 0) 0x7f4af407cb90 con 0x7f4b040ff260 2024-04-02T05:00:38.018 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:38.018 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:18.159575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":187,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:38.018 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2024-04-02T05:00:38.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4aec06dc80 msgr2=0x7f4aec070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4aec06dc80 0x7f4aec070140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4af80058c0 tx=0x7f4af8005850 comp rx=0 tx=0).stop 2024-04-02T05:00:38.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 msgr2=0x7f4b04196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 0x7f4b04196c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009430 tx=0x7f4af4004040 comp rx=0 tx=0).stop 2024-04-02T05:00:38.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 shutdown_connections 2024-04-02T05:00:38.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b04197150 0x7f4b041975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f4aec06dc80 0x7f4aec070140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 --2- 172.21.15.70:0/3440510238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b040ff260 0x7f4b04196c10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 >> 172.21.15.70:0/3440510238 conn(0x7f4b04074a50 msgr2=0x7f4b040fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:38.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 shutdown_connections 2024-04-02T05:00:38.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.017+0000 7f4b0a1cb640 1 -- 172.21.15.70:0/3440510238 wait complete. 2024-04-02T05:00:38.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-02T05:00:38.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.131+0000 7f9adbb31640 1 -- 172.21.15.70:0/2541369807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ffb30 msgr2=0x7f9ad4103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.131+0000 7f9adbb31640 1 --2- 172.21.15.70:0/2541369807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ffb30 0x7f9ad4103b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009880 tx=0x7f9ac402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.131+0000 7f9adbb31640 1 -- 172.21.15.70:0/2541369807 shutdown_connections 2024-04-02T05:00:38.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.131+0000 7f9adbb31640 1 --2- 172.21.15.70:0/2541369807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ffb30 0x7f9ad4103b70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.131+0000 7f9adbb31640 1 --2- 172.21.15.70:0/2541369807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad40ff180 0x7f9ad40ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.131+0000 7f9adbb31640 1 -- 172.21.15.70:0/2541369807 >> 172.21.15.70:0/2541369807 conn(0x7f9ad40faa00 msgr2=0x7f9ad40fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:38.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.131+0000 7f9adbb31640 1 -- 172.21.15.70:0/2541369807 shutdown_connections 2024-04-02T05:00:38.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.131+0000 7f9adbb31640 1 -- 172.21.15.70:0/2541369807 wait complete. 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9adbb31640 1 Processor -- start 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9adbb31640 1 -- start start 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9adbb31640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ff180 0x7f9ad4196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9adbb31640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad4197160 0x7f9ad41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9adbb31640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad419fc90 con 0x7f9ad40ff180 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9adbb31640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad419fdd0 con 0x7f9ad4197160 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9ad98cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ff180 0x7f9ad4196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9ad98cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ff180 0x7f9ad4196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56502/0 (socket says 172.21.15.70:56502) 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9ad98cd640 1 -- 172.21.15.70:0/3229808856 learned_addr learned my addr 172.21.15.70:0/3229808856 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9ad90cc640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad4197160 0x7f9ad41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9ad98cd640 1 -- 172.21.15.70:0/3229808856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad4197160 msgr2=0x7f9ad41975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9ad98cd640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad4197160 0x7f9ad41975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9ad98cd640 1 -- 172.21.15.70:0/3229808856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4009560 con 0x7f9ad40ff180 2024-04-02T05:00:38.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.132+0000 7f9ad98cd640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ff180 0x7f9ad4196c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000d720 tx=0x7f9ad000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:38.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.133+0000 7f9acaffd640 1 -- 172.21.15.70:0/3229808856 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ad00051f0 con 0x7f9ad40ff180 2024-04-02T05:00:38.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.133+0000 7f9acaffd640 1 -- 172.21.15.70:0/3229808856 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ad0009ce0 con 0x7f9ad40ff180 2024-04-02T05:00:38.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.133+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad419ffd0 con 0x7f9ad40ff180 2024-04-02T05:00:38.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.133+0000 7f9acaffd640 1 -- 172.21.15.70:0/3229808856 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ad0010460 con 0x7f9ad40ff180 2024-04-02T05:00:38.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.133+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad41a04b0 con 0x7f9ad40ff180 2024-04-02T05:00:38.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.134+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad404e040 con 0x7f9ad40ff180 2024-04-02T05:00:38.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.134+0000 7f9acaffd640 1 -- 172.21.15.70:0/3229808856 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f9ad0005350 con 0x7f9ad40ff180 2024-04-02T05:00:38.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.135+0000 7f9acaffd640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9ac006dcd0 0x7f9ac0070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.135+0000 7f9acaffd640 1 -- 172.21.15.70:0/3229808856 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f9ad008d7e0 con 0x7f9ad40ff180 2024-04-02T05:00:38.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.135+0000 7f9ad90cc640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9ac006dcd0 0x7f9ac0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.135+0000 7f9ad90cc640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9ac006dcd0 0x7f9ac0070190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4006010 tx=0x7f9ac4000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:38.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.140+0000 7f9acaffd640 1 -- 172.21.15.70:0/3229808856 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9ad0056c40 con 0x7f9ad40ff180 2024-04-02T05:00:38.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.429+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f9ad4107480 con 0x7f9ad40ff180 2024-04-02T05:00:38.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.430+0000 7f9acaffd640 1 -- 172.21.15.70:0/3229808856 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v40) v1 ==== 107+0+4154 (secure 0 0 0) 0x7f9ad005a300 con 0x7f9ad40ff180 2024-04-02T05:00:38.433 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:38.433 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:20.167349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":0,"incarnation":22,"state":"up:active","state_seq":188,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24253,"qdb_cluster":[24253]},"id":1}]} 2024-04-02T05:00:38.433 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 25 2024-04-02T05:00:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9ac006dcd0 msgr2=0x7f9ac0070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9ac006dcd0 0x7f9ac0070190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4006010 tx=0x7f9ac4000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ff180 msgr2=0x7f9ad4196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ff180 0x7f9ad4196c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000d720 tx=0x7f9ad000dbe0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 shutdown_connections 2024-04-02T05:00:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad4197160 0x7f9ad41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f9ac006dcd0 0x7f9ac0070190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 --2- 172.21.15.70:0/3229808856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad40ff180 0x7f9ad4196c20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 >> 172.21.15.70:0/3229808856 conn(0x7f9ad40faa00 msgr2=0x7f9ad40fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 shutdown_connections 2024-04-02T05:00:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.432+0000 7f9adbb31640 1 -- 172.21.15.70:0/3229808856 wait complete. 2024-04-02T05:00:38.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-02T05:00:38.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.545+0000 7f7133388640 1 -- 172.21.15.70:0/87102567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c0ff260 msgr2=0x7f712c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.545+0000 7f7133388640 1 --2- 172.21.15.70:0/87102567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c0ff260 0x7f712c0ff640 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7128009880 tx=0x7f712802bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.545+0000 7f7133388640 1 -- 172.21.15.70:0/87102567 shutdown_connections 2024-04-02T05:00:38.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.545+0000 7f7133388640 1 --2- 172.21.15.70:0/87102567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712c0ffc10 0x7f712c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.545+0000 7f7133388640 1 --2- 172.21.15.70:0/87102567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c0ff260 0x7f712c0ff640 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.545+0000 7f7133388640 1 -- 172.21.15.70:0/87102567 >> 172.21.15.70:0/87102567 conn(0x7f712c074a50 msgr2=0x7f712c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.545+0000 7f7133388640 1 -- 172.21.15.70:0/87102567 shutdown_connections 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.545+0000 7f7133388640 1 -- 172.21.15.70:0/87102567 wait complete. 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7133388640 1 Processor -- start 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7133388640 1 -- start start 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7133388640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712c0ff260 0x7f712c196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7133388640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c197130 0x7f712c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7133388640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19fc60 con 0x7f712c197130 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7133388640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19fda0 con 0x7f712c0ff260 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7130923640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c197130 0x7f712c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7130923640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c197130 0x7f712c197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56518/0 (socket says 172.21.15.70:56518) 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7130923640 1 -- 172.21.15.70:0/3009916464 learned_addr learned my addr 172.21.15.70:0/3009916464 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:38.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7131124640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712c0ff260 0x7f712c196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7130923640 1 -- 172.21.15.70:0/3009916464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712c0ff260 msgr2=0x7f712c196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7130923640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712c0ff260 0x7f712c196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.546+0000 7f7130923640 1 -- 172.21.15.70:0/3009916464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128009560 con 0x7f712c197130 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.547+0000 7f7130923640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c197130 0x7f712c197590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f711c00d8d0 tx=0x7f711c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.547+0000 7f71227fc640 1 -- 172.21.15.70:0/3009916464 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711c00bea0 con 0x7f712c197130 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.547+0000 7f71227fc640 1 -- 172.21.15.70:0/3009916464 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f711c009c00 con 0x7f712c197130 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.547+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c1a0000 con 0x7f712c197130 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.547+0000 7f71227fc640 1 -- 172.21.15.70:0/3009916464 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711c010460 con 0x7f712c197130 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.547+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c1a04e0 con 0x7f712c197130 2024-04-02T05:00:38.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.548+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c04e040 con 0x7f712c197130 2024-04-02T05:00:38.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.548+0000 7f71227fc640 1 -- 172.21.15.70:0/3009916464 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f711c003f10 con 0x7f712c197130 2024-04-02T05:00:38.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.549+0000 7f71227fc640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f711806dcd0 0x7f7118070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.549+0000 7f71227fc640 1 -- 172.21.15.70:0/3009916464 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f711c08d9d0 con 0x7f712c197130 2024-04-02T05:00:38.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.549+0000 7f7131124640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f711806dcd0 0x7f7118070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.549+0000 7f7131124640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f711806dcd0 0x7f7118070190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7128006010 tx=0x7f7128005dd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:38.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.553+0000 7f71227fc640 1 -- 172.21.15.70:0/3009916464 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f711c056db0 con 0x7f712c197130 2024-04-02T05:00:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:38 smithi179 ceph-mon[84288]: pgmap v302: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:38 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3451823709' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-02T05:00:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:38 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/572889727' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-02T05:00:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:38 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3440510238' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-02T05:00:38.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.773+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f712c0ffda0 con 0x7f712c197130 2024-04-02T05:00:38.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.774+0000 7f71227fc640 1 -- 172.21.15.70:0/3009916464 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v40) v1 ==== 107+0+5001 (secure 0 0 0) 0x7f711c05a470 con 0x7f712c197130 2024-04-02T05:00:38.778 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:38.778 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:20.167349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24253":{"gid":24253,"name":"cephfs.smithi179.mzfimf","rank":0,"incarnation":22,"state":"up:active","state_seq":188,"addr":"172.21.15.179:6825/511108019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":511108019},{"type":"v1","addr":"172.21.15.179:6825","nonce":511108019}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24253,"qdb_cluster":[24253]},"id":1}]} 2024-04-02T05:00:38.779 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 26 2024-04-02T05:00:38.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:38 smithi070 ceph-mon[113635]: pgmap v302: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:38.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:38 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3451823709' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-02T05:00:38.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:38 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/572889727' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-02T05:00:38.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:38 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3440510238' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-02T05:00:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f711806dcd0 msgr2=0x7f7118070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f711806dcd0 0x7f7118070190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7128006010 tx=0x7f7128005dd0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c197130 msgr2=0x7f712c197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c197130 0x7f712c197590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f711c00d8d0 tx=0x7f711c00dd90 comp rx=0 tx=0).stop 2024-04-02T05:00:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 shutdown_connections 2024-04-02T05:00:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f711806dcd0 0x7f7118070190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f712c197130 0x7f712c197590 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 --2- 172.21.15.70:0/3009916464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712c0ff260 0x7f712c196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 >> 172.21.15.70:0/3009916464 conn(0x7f712c074a50 msgr2=0x7f712c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 shutdown_connections 2024-04-02T05:00:38.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.776+0000 7f7133388640 1 -- 172.21.15.70:0/3009916464 wait complete. 2024-04-02T05:00:38.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-02T05:00:38.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 -- 172.21.15.70:0/2946946687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0ffc10 msgr2=0x7f838c103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 --2- 172.21.15.70:0/2946946687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0ffc10 0x7f838c103b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8380009880 tx=0x7f838002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 -- 172.21.15.70:0/2946946687 shutdown_connections 2024-04-02T05:00:38.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 --2- 172.21.15.70:0/2946946687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0ffc10 0x7f838c103b70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 --2- 172.21.15.70:0/2946946687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838c0ff260 0x7f838c0ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 -- 172.21.15.70:0/2946946687 >> 172.21.15.70:0/2946946687 conn(0x7f838c074a50 msgr2=0x7f838c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 -- 172.21.15.70:0/2946946687 shutdown_connections 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 -- 172.21.15.70:0/2946946687 wait complete. 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 Processor -- start 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.920+0000 7f8393d50640 1 -- start start 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f8393d50640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838c0ff260 0x7f838c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f8393d50640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c198660 0x7f838c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f8393d50640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c198c60 con 0x7f838c198660 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f8393d50640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c198dd0 con 0x7f838c0ff260 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f83912eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c198660 0x7f838c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f83912eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c198660 0x7f838c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56538/0 (socket says 172.21.15.70:56538) 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f83912eb640 1 -- 172.21.15.70:0/1311172331 learned_addr learned my addr 172.21.15.70:0/1311172331 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f8391aec640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838c0ff260 0x7f838c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f83912eb640 1 -- 172.21.15.70:0/1311172331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838c0ff260 msgr2=0x7f838c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f83912eb640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838c0ff260 0x7f838c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f83912eb640 1 -- 172.21.15.70:0/1311172331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8380009560 con 0x7f838c198660 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f8391aec640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838c0ff260 0x7f838c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f83912eb640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c198660 0x7f838c19cac0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8380036040 tx=0x7f8380004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f838affd640 1 -- 172.21.15.70:0/1311172331 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83800037c0 con 0x7f838c198660 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f838affd640 1 -- 172.21.15.70:0/1311172331 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8380003de0 con 0x7f838c198660 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c19d060 con 0x7f838c198660 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f838affd640 1 -- 172.21.15.70:0/1311172331 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8380034440 con 0x7f838c198660 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.921+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c19d5c0 con 0x7f838c198660 2024-04-02T05:00:38.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.922+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c04e040 con 0x7f838c198660 2024-04-02T05:00:38.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.923+0000 7f838affd640 1 -- 172.21.15.70:0/1311172331 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f8380004fb0 con 0x7f838c198660 2024-04-02T05:00:38.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.923+0000 7f838affd640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f837406dcd0 0x7f8374070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:38.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.923+0000 7f838affd640 1 -- 172.21.15.70:0/1311172331 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f83800b01e0 con 0x7f838c198660 2024-04-02T05:00:38.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.923+0000 7f8391aec640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f837406dcd0 0x7f8374070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.924+0000 7f8391aec640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f837406dcd0 0x7f8374070190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8384009fa0 tx=0x7f8384009570 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:38.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:38.928+0000 7f838affd640 1 -- 172.21.15.70:0/1311172331 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f83800795c0 con 0x7f838c198660 2024-04-02T05:00:39.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.144+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f838c19d900 con 0x7f838c198660 2024-04-02T05:00:39.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.145+0000 7f838affd640 1 -- 172.21.15.70:0/1311172331 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v40) v1 ==== 107+0+4200 (secure 0 0 0) 0x7f838007cc80 con 0x7f838c198660 2024-04-02T05:00:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:39.148 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:35.242621+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:39.148 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 27 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f837406dcd0 msgr2=0x7f8374070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f837406dcd0 0x7f8374070190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8384009fa0 tx=0x7f8384009570 comp rx=0 tx=0).stop 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c198660 msgr2=0x7f838c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c198660 0x7f838c19cac0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8380036040 tx=0x7f8380004270 comp rx=0 tx=0).stop 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 shutdown_connections 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f837406dcd0 0x7f8374070190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c198660 0x7f838c19cac0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 --2- 172.21.15.70:0/1311172331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838c0ff260 0x7f838c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.147+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 >> 172.21.15.70:0/1311172331 conn(0x7f838c074a50 msgr2=0x7f838c067c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.148+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 shutdown_connections 2024-04-02T05:00:39.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.148+0000 7f8393d50640 1 -- 172.21.15.70:0/1311172331 wait complete. 2024-04-02T05:00:39.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-02T05:00:39.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.264+0000 7fdf7b592640 1 -- 172.21.15.70:0/1778551694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf74067f70 msgr2=0x7fdf74104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.264+0000 7fdf7b592640 1 --2- 172.21.15.70:0/1778551694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf74067f70 0x7fdf74104c70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009880 tx=0x7fdf6402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.264+0000 7fdf7b592640 1 -- 172.21.15.70:0/1778551694 shutdown_connections 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.264+0000 7fdf7b592640 1 --2- 172.21.15.70:0/1778551694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf74067f70 0x7fdf74104c70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.264+0000 7fdf7b592640 1 --2- 172.21.15.70:0/1778551694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf740675c0 0x7fdf740679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.264+0000 7fdf7b592640 1 -- 172.21.15.70:0/1778551694 >> 172.21.15.70:0/1778551694 conn(0x7fdf74074a50 msgr2=0x7fdf74074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.264+0000 7fdf7b592640 1 -- 172.21.15.70:0/1778551694 shutdown_connections 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.264+0000 7fdf7b592640 1 -- 172.21.15.70:0/1778551694 wait complete. 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7b592640 1 Processor -- start 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7b592640 1 -- start start 2024-04-02T05:00:39.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7b592640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740675c0 0x7fdf740ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7b592640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf740ff600 0x7fdf740ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7b592640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741a20c0 con 0x7fdf740675c0 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7b592640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741a2200 con 0x7fdf740ff600 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7932e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740675c0 0x7fdf740ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7932e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740675c0 0x7fdf740ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56552/0 (socket says 172.21.15.70:56552) 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf7932e640 1 -- 172.21.15.70:0/2067002318 learned_addr learned my addr 172.21.15.70:0/2067002318 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.265+0000 7fdf78b2d640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf740ff600 0x7fdf740ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf7932e640 1 -- 172.21.15.70:0/2067002318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf740ff600 msgr2=0x7fdf740ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf7932e640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf740ff600 0x7fdf740ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf7932e640 1 -- 172.21.15.70:0/2067002318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64009560 con 0x7fdf740675c0 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf7932e640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740675c0 0x7fdf740ff0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b450 tx=0x7fdf7000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf6a7fc640 1 -- 172.21.15.70:0/2067002318 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf70011860 con 0x7fdf740675c0 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf6a7fc640 1 -- 172.21.15.70:0/2067002318 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf70011e80 con 0x7fdf740675c0 2024-04-02T05:00:39.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf741a2340 con 0x7fdf740675c0 2024-04-02T05:00:39.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf6a7fc640 1 -- 172.21.15.70:0/2067002318 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7000f570 con 0x7fdf740675c0 2024-04-02T05:00:39.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.266+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf741a28a0 con 0x7fdf740675c0 2024-04-02T05:00:39.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.267+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7404e040 con 0x7fdf740675c0 2024-04-02T05:00:39.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.267+0000 7fdf6a7fc640 1 -- 172.21.15.70:0/2067002318 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fdf700103e0 con 0x7fdf740675c0 2024-04-02T05:00:39.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.268+0000 7fdf6a7fc640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006dcd0 0x7fdf60070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.268+0000 7fdf78b2d640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006dcd0 0x7fdf60070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:39.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.268+0000 7fdf6a7fc640 1 -- 172.21.15.70:0/2067002318 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fdf7008dbf0 con 0x7fdf740675c0 2024-04-02T05:00:39.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.268+0000 7fdf78b2d640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006dcd0 0x7fdf60070190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf64000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:39.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.273+0000 7fdf6a7fc640 1 -- 172.21.15.70:0/2067002318 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdf70056fd0 con 0x7fdf740675c0 2024-04-02T05:00:39.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.508+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fdf74108580 con 0x7fdf740675c0 2024-04-02T05:00:39.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.508+0000 7fdf6a7fc640 1 -- 172.21.15.70:0/2067002318 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v40) v1 ==== 107+0+4211 (secure 0 0 0) 0x7fdf7005a690 con 0x7fdf740675c0 2024-04-02T05:00:39.512 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:39.512 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:35.247379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":0,"incarnation":28,"state":"up:replay","state_seq":2,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:39.512 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 28 2024-04-02T05:00:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.510+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006dcd0 msgr2=0x7fdf60070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.510+0000 7fdf7b592640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006dcd0 0x7fdf60070190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf64000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.510+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740675c0 msgr2=0x7fdf740ff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.510+0000 7fdf7b592640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740675c0 0x7fdf740ff0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b450 tx=0x7fdf7000b910 comp rx=0 tx=0).stop 2024-04-02T05:00:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.511+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 shutdown_connections 2024-04-02T05:00:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.511+0000 7fdf7b592640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf740ff600 0x7fdf740ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.511+0000 7fdf7b592640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fdf6006dcd0 0x7fdf60070190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.511+0000 7fdf7b592640 1 --2- 172.21.15.70:0/2067002318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf740675c0 0x7fdf740ff0c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.511+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 >> 172.21.15.70:0/2067002318 conn(0x7fdf74074a50 msgr2=0x7fdf740fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:39.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.511+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 shutdown_connections 2024-04-02T05:00:39.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.511+0000 7fdf7b592640 1 -- 172.21.15.70:0/2067002318 wait complete. 2024-04-02T05:00:39.524 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:39 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3229808856' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-02T05:00:39.524 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:39 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3009916464' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-02T05:00:39.524 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:39 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1311172331' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-02T05:00:39.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-02T05:00:39.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:39 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3229808856' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-02T05:00:39.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:39 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3009916464' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-02T05:00:39.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:39 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1311172331' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-02T05:00:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.625+0000 7f786eeae640 1 -- 172.21.15.70:0/1857636337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78680ffb30 msgr2=0x7f7868103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.625+0000 7f786eeae640 1 --2- 172.21.15.70:0/1857636337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78680ffb30 0x7f7868103b70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7854009880 tx=0x7f785402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.625+0000 7f786eeae640 1 -- 172.21.15.70:0/1857636337 shutdown_connections 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.625+0000 7f786eeae640 1 --2- 172.21.15.70:0/1857636337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78680ffb30 0x7f7868103b70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.625+0000 7f786eeae640 1 --2- 172.21.15.70:0/1857636337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680ff180 0x7f78680ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.625+0000 7f786eeae640 1 -- 172.21.15.70:0/1857636337 >> 172.21.15.70:0/1857636337 conn(0x7f78680faa00 msgr2=0x7f78680fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.625+0000 7f786eeae640 1 -- 172.21.15.70:0/1857636337 shutdown_connections 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.625+0000 7f786eeae640 1 -- 172.21.15.70:0/1857636337 wait complete. 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f786eeae640 1 Processor -- start 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f786eeae640 1 -- start start 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f786eeae640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680ff180 0x7f7868196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f786eeae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7868197160 0x7f78681975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f786eeae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786819fc90 con 0x7f7868197160 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f786eeae640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786819fdd0 con 0x7f78680ff180 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f7867fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7868197160 0x7f78681975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f7867fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7868197160 0x7f78681975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56568/0 (socket says 172.21.15.70:56568) 2024-04-02T05:00:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f7867fff640 1 -- 172.21.15.70:0/1349538386 learned_addr learned my addr 172.21.15.70:0/1349538386 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.626+0000 7f786cc4a640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680ff180 0x7f7868196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f7867fff640 1 -- 172.21.15.70:0/1349538386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680ff180 msgr2=0x7f7868196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f7867fff640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680ff180 0x7f7868196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f7867fff640 1 -- 172.21.15.70:0/1349538386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854009560 con 0x7f7868197160 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f7867fff640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7868197160 0x7f78681975c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7854036040 tx=0x7f785402c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f7865ffb640 1 -- 172.21.15.70:0/1349538386 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78540035f0 con 0x7f7868197160 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f7865ffb640 1 -- 172.21.15.70:0/1349538386 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7854003c10 con 0x7f7868197160 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786819ffd0 con 0x7f7868197160 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f7865ffb640 1 -- 172.21.15.70:0/1349538386 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7854005630 con 0x7f7868197160 2024-04-02T05:00:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.627+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78681a0450 con 0x7f7868197160 2024-04-02T05:00:39.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.628+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786804e040 con 0x7f7868197160 2024-04-02T05:00:39.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.629+0000 7f7865ffb640 1 -- 172.21.15.70:0/1349538386 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f7854003750 con 0x7f7868197160 2024-04-02T05:00:39.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.629+0000 7f7865ffb640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f785806dcd0 0x7f7858070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.629+0000 7f7865ffb640 1 -- 172.21.15.70:0/1349538386 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f78540affd0 con 0x7f7868197160 2024-04-02T05:00:39.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.629+0000 7f786cc4a640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f785806dcd0 0x7f7858070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:39.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.629+0000 7f786cc4a640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f785806dcd0 0x7f7858070190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f785c009bd0 tx=0x7f785c005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:39.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.634+0000 7f7865ffb640 1 -- 172.21.15.70:0/1349538386 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7854079460 con 0x7f7868197160 2024-04-02T05:00:39.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.852+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f7868107480 con 0x7f7868197160 2024-04-02T05:00:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.853+0000 7f7865ffb640 1 -- 172.21.15.70:0/1349538386 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v40) v1 ==== 107+0+4216 (secure 0 0 0) 0x7f785407cb20 con 0x7f7868197160 2024-04-02T05:00:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:39.856 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:43.751932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":191,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 29 2024-04-02T05:00:39.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f785806dcd0 msgr2=0x7f7858070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f785806dcd0 0x7f7858070190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f785c009bd0 tx=0x7f785c005f90 comp rx=0 tx=0).stop 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7868197160 msgr2=0x7f78681975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7868197160 0x7f78681975c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7854036040 tx=0x7f785402c750 comp rx=0 tx=0).stop 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 shutdown_connections 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f785806dcd0 0x7f7858070190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7868197160 0x7f78681975c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 --2- 172.21.15.70:0/1349538386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78680ff180 0x7f7868196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 >> 172.21.15.70:0/1349538386 conn(0x7f78680faa00 msgr2=0x7f78680fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 shutdown_connections 2024-04-02T05:00:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.855+0000 7f786eeae640 1 -- 172.21.15.70:0/1349538386 wait complete. 2024-04-02T05:00:39.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-02T05:00:39.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 -- 172.21.15.70:0/3009455735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ffc10 msgr2=0x7fbf04103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 --2- 172.21.15.70:0/3009455735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ffc10 0x7fbf04103b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009880 tx=0x7fbef802bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 -- 172.21.15.70:0/3009455735 shutdown_connections 2024-04-02T05:00:39.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 --2- 172.21.15.70:0/3009455735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ffc10 0x7fbf04103b70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 --2- 172.21.15.70:0/3009455735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf040ff260 0x7fbf040ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 -- 172.21.15.70:0/3009455735 >> 172.21.15.70:0/3009455735 conn(0x7fbf04074a50 msgr2=0x7fbf04074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:39.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 -- 172.21.15.70:0/3009455735 shutdown_connections 2024-04-02T05:00:39.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 -- 172.21.15.70:0/3009455735 wait complete. 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 Processor -- start 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.970+0000 7fbf09a51640 1 -- start start 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf09a51640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ff260 0x7fbf04100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf09a51640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04101150 0x7fbf041015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf09a51640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf041a1f10 con 0x7fbf040ff260 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf09a51640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf041a2050 con 0x7fbf04101150 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ff260 0x7fbf04100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ff260 0x7fbf04100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56578/0 (socket says 172.21.15.70:56578) 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf02ffd640 1 -- 172.21.15.70:0/2167991198 learned_addr learned my addr 172.21.15.70:0/2167991198 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf02ffd640 1 -- 172.21.15.70:0/2167991198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04101150 msgr2=0x7fbf041015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf02ffd640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04101150 0x7fbf041015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf02ffd640 1 -- 172.21.15.70:0/2167991198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef8009560 con 0x7fbf040ff260 2024-04-02T05:00:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf02ffd640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ff260 0x7fbf04100c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbef400d900 tx=0x7fbef400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf08a4f640 1 -- 172.21.15.70:0/2167991198 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbef4009c00 con 0x7fbf040ff260 2024-04-02T05:00:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf041a2190 con 0x7fbf040ff260 2024-04-02T05:00:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf08a4f640 1 -- 172.21.15.70:0/2167991198 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbef4009d60 con 0x7fbf040ff260 2024-04-02T05:00:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf08a4f640 1 -- 172.21.15.70:0/2167991198 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbef40049d0 con 0x7fbf040ff260 2024-04-02T05:00:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.971+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf041a26f0 con 0x7fbf040ff260 2024-04-02T05:00:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.972+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0404e040 con 0x7fbf040ff260 2024-04-02T05:00:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.973+0000 7fbf08a4f640 1 -- 172.21.15.70:0/2167991198 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbef4004020 con 0x7fbf040ff260 2024-04-02T05:00:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.973+0000 7fbf08a4f640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbeec06dcd0 0x7fbeec070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.973+0000 7fbf027fc640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbeec06dcd0 0x7fbeec070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.973+0000 7fbf08a4f640 1 -- 172.21.15.70:0/2167991198 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fbef408d9d0 con 0x7fbf040ff260 2024-04-02T05:00:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.974+0000 7fbf027fc640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbeec06dcd0 0x7fbeec070190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbef8036040 tx=0x7fbef8000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:39.978+0000 7fbf08a4f640 1 -- 172.21.15.70:0/2167991198 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbef4056db0 con 0x7fbf040ff260 2024-04-02T05:00:40.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.224+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fbf040ffe60 con 0x7fbf040ff260 2024-04-02T05:00:40.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.225+0000 7fbf08a4f640 1 -- 172.21.15.70:0/2167991198 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v40) v1 ==== 107+0+4213 (secure 0 0 0) 0x7fbf040ffe60 con 0x7fbf040ff260 2024-04-02T05:00:40.228 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:40.229 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:44.162868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":192,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:40.229 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 30 2024-04-02T05:00:40.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbeec06dcd0 msgr2=0x7fbeec070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:40.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbeec06dcd0 0x7fbeec070190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbef8036040 tx=0x7fbef8000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ff260 msgr2=0x7fbf04100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ff260 0x7fbf04100c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbef400d900 tx=0x7fbef400ddc0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 shutdown_connections 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04101150 0x7fbf041015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fbeec06dcd0 0x7fbeec070190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 --2- 172.21.15.70:0/2167991198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf040ff260 0x7fbf04100c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 >> 172.21.15.70:0/2167991198 conn(0x7fbf04074a50 msgr2=0x7fbf04103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 shutdown_connections 2024-04-02T05:00:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.227+0000 7fbf09a51640 1 -- 172.21.15.70:0/2167991198 wait complete. 2024-04-02T05:00:40.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.343+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2207675328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e80ff260 msgr2=0x7fb8e80ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.343+0000 7fb8eca8e640 1 --2- 172.21.15.70:0/2207675328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e80ff260 0x7fb8e80ff640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009880 tx=0x7fb8e002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.343+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2207675328 shutdown_connections 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.343+0000 7fb8eca8e640 1 --2- 172.21.15.70:0/2207675328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e80ffc10 0x7fb8e8103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.343+0000 7fb8eca8e640 1 --2- 172.21.15.70:0/2207675328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e80ff260 0x7fb8e80ff640 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.343+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2207675328 >> 172.21.15.70:0/2207675328 conn(0x7fb8e8074a50 msgr2=0x7fb8e8074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.343+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2207675328 shutdown_connections 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.343+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2207675328 wait complete. 2024-04-02T05:00:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8eca8e640 1 Processor -- start 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8eca8e640 1 -- start start 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8eca8e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e80ff260 0x7fb8e8194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8eca8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e81951f0 0x7fb8e81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8eca8e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e81957f0 con 0x7fb8e81951f0 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8eca8e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e8195930 con 0x7fb8e80ff260 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8e5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e81951f0 0x7fb8e81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8e5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e81951f0 0x7fb8e81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56592/0 (socket says 172.21.15.70:56592) 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8e5d9b640 1 -- 172.21.15.70:0/2537006758 learned_addr learned my addr 172.21.15.70:0/2537006758 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8e659c640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e80ff260 0x7fb8e8194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8e5d9b640 1 -- 172.21.15.70:0/2537006758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e80ff260 msgr2=0x7fb8e8194cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8e5d9b640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e80ff260 0x7fb8e8194cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8e5d9b640 1 -- 172.21.15.70:0/2537006758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0009560 con 0x7fb8e81951f0 2024-04-02T05:00:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.344+0000 7fb8e5d9b640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e81951f0 0x7fb8e81985a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00cc60 tx=0x7fb8dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.345+0000 7fb8d77fe640 1 -- 172.21.15.70:0/2537006758 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8dc004b20 con 0x7fb8e81951f0 2024-04-02T05:00:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.345+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e8198ba0 con 0x7fb8e81951f0 2024-04-02T05:00:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.345+0000 7fb8d77fe640 1 -- 172.21.15.70:0/2537006758 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8dc004c80 con 0x7fb8e81951f0 2024-04-02T05:00:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.345+0000 7fb8d77fe640 1 -- 172.21.15.70:0/2537006758 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8dc00f680 con 0x7fb8e81951f0 2024-04-02T05:00:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.345+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e8199100 con 0x7fb8e81951f0 2024-04-02T05:00:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.346+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e81993e0 con 0x7fb8e81951f0 2024-04-02T05:00:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.346+0000 7fb8d77fe640 1 -- 172.21.15.70:0/2537006758 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb8dc005020 con 0x7fb8e81951f0 2024-04-02T05:00:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.346+0000 7fb8d77fe640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb8d006dcd0 0x7fb8d0070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.347+0000 7fb8d77fe640 1 -- 172.21.15.70:0/2537006758 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fb8dc08ded0 con 0x7fb8e81951f0 2024-04-02T05:00:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.347+0000 7fb8e659c640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb8d006dcd0 0x7fb8d0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.347+0000 7fb8e659c640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb8d006dcd0 0x7fb8d0070190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0006010 tx=0x7fb8e0005b50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:40.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.351+0000 7fb8d77fe640 1 -- 172.21.15.70:0/2537006758 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb8dc057360 con 0x7fb8e81951f0 2024-04-02T05:00:40.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:40 smithi070 ceph-mon[113635]: pgmap v303: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:40.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:40 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2067002318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-02T05:00:40.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:40 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1349538386' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-02T05:00:40.530 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:40 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2167991198' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-02T05:00:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:40 smithi179 ceph-mon[84288]: pgmap v303: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:40 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2067002318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-02T05:00:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:40 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1349538386' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-02T05:00:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:40 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2167991198' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-02T05:00:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.569+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fb8e8196010 con 0x7fb8e81951f0 2024-04-02T05:00:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.570+0000 7fb8d77fe640 1 -- 172.21.15.70:0/2537006758 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v40) v1 ==== 107+0+4222 (secure 0 0 0) 0x7fb8dc01d090 con 0x7fb8e81951f0 2024-04-02T05:00:40.573 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:40.573 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:46.167773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":0,"incarnation":28,"state":"up:active","state_seq":193,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24267,"qdb_cluster":[24267]},"id":1}]} 2024-04-02T05:00:40.574 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 31 2024-04-02T05:00:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb8d006dcd0 msgr2=0x7fb8d0070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb8d006dcd0 0x7fb8d0070190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0006010 tx=0x7fb8e0005b50 comp rx=0 tx=0).stop 2024-04-02T05:00:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e81951f0 msgr2=0x7fb8e81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e81951f0 0x7fb8e81985a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00cc60 tx=0x7fb8dc00a620 comp rx=0 tx=0).stop 2024-04-02T05:00:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 shutdown_connections 2024-04-02T05:00:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb8d006dcd0 0x7fb8d0070190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8e81951f0 0x7fb8e81985a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 --2- 172.21.15.70:0/2537006758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e80ff260 0x7fb8e8194cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 >> 172.21.15.70:0/2537006758 conn(0x7fb8e8074a50 msgr2=0x7fb8e80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 shutdown_connections 2024-04-02T05:00:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.572+0000 7fb8eca8e640 1 -- 172.21.15.70:0/2537006758 wait complete. 2024-04-02T05:00:40.586 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 -- 172.21.15.70:0/1173551532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 msgr2=0x7f765010c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 --2- 172.21.15.70:0/1173551532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 0x7f765010c3a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7640009880 tx=0x7f764002bdc0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 -- 172.21.15.70:0/1173551532 shutdown_connections 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 --2- 172.21.15.70:0/1173551532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 0x7f765010c3a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 --2- 172.21.15.70:0/1173551532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650072670 0x7f7650072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 -- 172.21.15.70:0/1173551532 >> 172.21.15.70:0/1173551532 conn(0x7f76500fbb30 msgr2=0x7f76500fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 -- 172.21.15.70:0/1173551532 shutdown_connections 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 -- 172.21.15.70:0/1173551532 wait complete. 2024-04-02T05:00:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 Processor -- start 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.687+0000 7f76573c9640 1 -- start start 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76573c9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650072670 0x7f7650197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76573c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 0x7f7650198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76573c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650198ad0 con 0x7f7650072f90 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76573c9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765019c8a0 con 0x7f7650072670 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7654964640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 0x7f7650198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7654964640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 0x7f7650198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56604/0 (socket says 172.21.15.70:56604) 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7654964640 1 -- 172.21.15.70:0/4017641980 learned_addr learned my addr 172.21.15.70:0/4017641980 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7655165640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650072670 0x7f7650197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7654964640 1 -- 172.21.15.70:0/4017641980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650072670 msgr2=0x7f7650197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7654964640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650072670 0x7f7650197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7654964640 1 -- 172.21.15.70:0/4017641980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7640009560 con 0x7f7650072f90 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7655165640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650072670 0x7f7650197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f7654964640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 0x7f7650198440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76400099a0 tx=0x7f7640004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76467fc640 1 -- 172.21.15.70:0/4017641980 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76400035f0 con 0x7f7650072f90 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76467fc640 1 -- 172.21.15.70:0/4017641980 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7640003c10 con 0x7f7650072f90 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765019cb20 con 0x7f7650072f90 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76467fc640 1 -- 172.21.15.70:0/4017641980 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7640005690 con 0x7f7650072f90 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.688+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765019d080 con 0x7f7650072f90 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.689+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765004e040 con 0x7f7650072f90 2024-04-02T05:00:40.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.690+0000 7f76467fc640 1 -- 172.21.15.70:0/4017641980 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f7640004ed0 con 0x7f7650072f90 2024-04-02T05:00:40.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.690+0000 7f76467fc640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f763c06dcd0 0x7f763c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:40.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.690+0000 7f7655165640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f763c06dcd0 0x7f763c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:40.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.691+0000 7f76467fc640 1 -- 172.21.15.70:0/4017641980 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f76400b07a0 con 0x7f7650072f90 2024-04-02T05:00:40.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.691+0000 7f7655165640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f763c06dcd0 0x7f763c070190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f764c0058c0 tx=0x7f764c005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:40.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.695+0000 7f76467fc640 1 -- 172.21.15.70:0/4017641980 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7640079c00 con 0x7f7650072f90 2024-04-02T05:00:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.998+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f7650109a10 con 0x7f7650072f90 2024-04-02T05:00:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:40.998+0000 7f76467fc640 1 -- 172.21.15.70:0/4017641980 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v40) v1 ==== 107+0+5072 (secure 0 0 0) 0x7f764007d2c0 con 0x7f7650072f90 2024-04-02T05:00:41.001 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:41.002 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:57:46.167773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi179.atrmwz","rank":0,"incarnation":28,"state":"up:active","state_seq":193,"addr":"172.21.15.179:6827/3541837059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":3541837059},{"type":"v1","addr":"172.21.15.179:6827","nonce":3541837059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24267,"qdb_cluster":[24267]},"id":1}]} 2024-04-02T05:00:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2024-04-02T05:00:41.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f763c06dcd0 msgr2=0x7f763c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f763c06dcd0 0x7f763c070190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f764c0058c0 tx=0x7f764c005850 comp rx=0 tx=0).stop 2024-04-02T05:00:41.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 msgr2=0x7f7650198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 0x7f7650198440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76400099a0 tx=0x7f7640004270 comp rx=0 tx=0).stop 2024-04-02T05:00:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 shutdown_connections 2024-04-02T05:00:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f763c06dcd0 0x7f763c070190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7650072f90 0x7f7650198440 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 --2- 172.21.15.70:0/4017641980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650072670 0x7f7650197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 >> 172.21.15.70:0/4017641980 conn(0x7f76500fbb30 msgr2=0x7f7650106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 shutdown_connections 2024-04-02T05:00:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.001+0000 7f76573c9640 1 -- 172.21.15.70:0/4017641980 wait complete. 2024-04-02T05:00:41.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-02T05:00:41.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 -- 172.21.15.70:0/1593771764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40500ffc10 msgr2=0x7f4050103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 --2- 172.21.15.70:0/1593771764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40500ffc10 0x7f4050103b70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4038009880 tx=0x7f403802bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 -- 172.21.15.70:0/1593771764 shutdown_connections 2024-04-02T05:00:41.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 --2- 172.21.15.70:0/1593771764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40500ffc10 0x7f4050103b70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 --2- 172.21.15.70:0/1593771764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40500ff260 0x7f40500ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 -- 172.21.15.70:0/1593771764 >> 172.21.15.70:0/1593771764 conn(0x7f4050074a50 msgr2=0x7f4050074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 -- 172.21.15.70:0/1593771764 shutdown_connections 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 -- 172.21.15.70:0/1593771764 wait complete. 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 Processor -- start 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.115+0000 7f40554d2640 1 -- start start 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40554d2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40500ff260 0x7f4050196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40554d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197090 0x7f40501974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40554d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019fc50 con 0x7f4050197090 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40554d2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019fd90 con 0x7f40500ff260 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40467fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197090 0x7f40501974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40467fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197090 0x7f40501974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56618/0 (socket says 172.21.15.70:56618) 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40467fc640 1 -- 172.21.15.70:0/4104614610 learned_addr learned my addr 172.21.15.70:0/4104614610 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f404effd640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40500ff260 0x7f4050196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40467fc640 1 -- 172.21.15.70:0/4104614610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40500ff260 msgr2=0x7f4050196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40467fc640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40500ff260 0x7f4050196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40467fc640 1 -- 172.21.15.70:0/4104614610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038009560 con 0x7f4050197090 2024-04-02T05:00:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40467fc640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197090 0x7f40501974f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4038036040 tx=0x7f403802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f404cff9640 1 -- 172.21.15.70:0/4104614610 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40380035f0 con 0x7f4050197090 2024-04-02T05:00:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405019ff30 con 0x7f4050197090 2024-04-02T05:00:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f404cff9640 1 -- 172.21.15.70:0/4104614610 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4038003c10 con 0x7f4050197090 2024-04-02T05:00:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.116+0000 7f404cff9640 1 -- 172.21.15.70:0/4104614610 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4038034440 con 0x7f4050197090 2024-04-02T05:00:41.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.117+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40501a0400 con 0x7f4050197090 2024-04-02T05:00:41.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.117+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405004e040 con 0x7f4050197090 2024-04-02T05:00:41.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.118+0000 7f404cff9640 1 -- 172.21.15.70:0/4104614610 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f4038003750 con 0x7f4050197090 2024-04-02T05:00:41.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.118+0000 7f404cff9640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403c06dcd0 0x7f403c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.118+0000 7f404cff9640 1 -- 172.21.15.70:0/4104614610 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f40380affb0 con 0x7f4050197090 2024-04-02T05:00:41.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.119+0000 7f404effd640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403c06dcd0 0x7f403c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.119+0000 7f404effd640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403c06dcd0 0x7f403c070190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4040005fd0 tx=0x7f404000b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:41.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.123+0000 7f404cff9640 1 -- 172.21.15.70:0/4104614610 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4038079440 con 0x7f4050197090 2024-04-02T05:00:41.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.343+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f4050107480 con 0x7f4050197090 2024-04-02T05:00:41.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.344+0000 7f404cff9640 1 -- 172.21.15.70:0/4104614610 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v40) v1 ==== 107+0+4268 (secure 0 0 0) 0x7f403807cb00 con 0x7f4050197090 2024-04-02T05:00:41.347 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:41.347 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:58:01.605348+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:41.348 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 33 2024-04-02T05:00:41.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403c06dcd0 msgr2=0x7f403c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403c06dcd0 0x7f403c070190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4040005fd0 tx=0x7f404000b390 comp rx=0 tx=0).stop 2024-04-02T05:00:41.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197090 msgr2=0x7f40501974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197090 0x7f40501974f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4038036040 tx=0x7f403802c850 comp rx=0 tx=0).stop 2024-04-02T05:00:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 shutdown_connections 2024-04-02T05:00:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f403c06dcd0 0x7f403c070190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4050197090 0x7f40501974f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 --2- 172.21.15.70:0/4104614610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40500ff260 0x7f4050196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 >> 172.21.15.70:0/4104614610 conn(0x7f4050074a50 msgr2=0x7f40500fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 shutdown_connections 2024-04-02T05:00:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.346+0000 7f40554d2640 1 -- 172.21.15.70:0/4104614610 wait complete. 2024-04-02T05:00:41.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-02T05:00:41.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.474+0000 7f079327d640 1 -- 172.21.15.70:0/1050417426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ffc10 msgr2=0x7f078c103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.474+0000 7f079327d640 1 --2- 172.21.15.70:0/1050417426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ffc10 0x7f078c103b70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f077c009880 tx=0x7f077c02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.474+0000 7f079327d640 1 -- 172.21.15.70:0/1050417426 shutdown_connections 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.474+0000 7f079327d640 1 --2- 172.21.15.70:0/1050417426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ffc10 0x7f078c103b70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.474+0000 7f079327d640 1 --2- 172.21.15.70:0/1050417426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f078c0ff260 0x7f078c0ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.474+0000 7f079327d640 1 -- 172.21.15.70:0/1050417426 >> 172.21.15.70:0/1050417426 conn(0x7f078c074a50 msgr2=0x7f078c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.474+0000 7f079327d640 1 -- 172.21.15.70:0/1050417426 shutdown_connections 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.474+0000 7f079327d640 1 -- 172.21.15.70:0/1050417426 wait complete. 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f079327d640 1 Processor -- start 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f079327d640 1 -- start start 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f079327d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ff260 0x7f078c195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f079327d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f078c196370 0x7f078c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f079327d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1968e0 con 0x7f078c0ff260 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f079327d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c196a50 con 0x7f078c196370 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f0791019640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ff260 0x7f078c195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f0791019640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ff260 0x7f078c195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56642/0 (socket says 172.21.15.70:56642) 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f0791019640 1 -- 172.21.15.70:0/168310195 learned_addr learned my addr 172.21.15.70:0/168310195 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f0790818640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f078c196370 0x7f078c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f0791019640 1 -- 172.21.15.70:0/168310195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f078c196370 msgr2=0x7f078c19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f0791019640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f078c196370 0x7f078c19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.475+0000 7f0791019640 1 -- 172.21.15.70:0/168310195 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c009560 con 0x7f078c0ff260 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.476+0000 7f0790818640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f078c196370 0x7f078c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.476+0000 7f0791019640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ff260 0x7f078c195e30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f078800c8a0 tx=0x7f078800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.476+0000 7f07827fc640 1 -- 172.21.15.70:0/168310195 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0788007b80 con 0x7f078c0ff260 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.476+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c19ad70 con 0x7f078c0ff260 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.476+0000 7f07827fc640 1 -- 172.21.15.70:0/168310195 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0788007ce0 con 0x7f078c0ff260 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.476+0000 7f07827fc640 1 -- 172.21.15.70:0/168310195 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0788018810 con 0x7f078c0ff260 2024-04-02T05:00:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.476+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c19b280 con 0x7f078c0ff260 2024-04-02T05:00:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.477+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c04e040 con 0x7f078c0ff260 2024-04-02T05:00:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.477+0000 7f07827fc640 1 -- 172.21.15.70:0/168310195 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f0788018a80 con 0x7f078c0ff260 2024-04-02T05:00:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.478+0000 7f07827fc640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f077806dcd0 0x7f0778070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.478+0000 7f07827fc640 1 -- 172.21.15.70:0/168310195 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f078808df80 con 0x7f078c0ff260 2024-04-02T05:00:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.478+0000 7f0790818640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f077806dcd0 0x7f0778070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.478+0000 7f0790818640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f077806dcd0 0x7f0778070190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f077c005ec0 tx=0x7f077c005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:41.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.482+0000 7f07827fc640 1 -- 172.21.15.70:0/168310195 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0788057360 con 0x7f078c0ff260 2024-04-02T05:00:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:41 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2537006758' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-02T05:00:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:41 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/4017641980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-02T05:00:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:00:41.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.694+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f078c19b560 con 0x7f078c0ff260 2024-04-02T05:00:41.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.694+0000 7f07827fc640 1 -- 172.21.15.70:0/168310195 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v40) v1 ==== 107+0+4347 (secure 0 0 0) 0x7f078805aa20 con 0x7f078c0ff260 2024-04-02T05:00:41.697 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:41.698 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:58:01.610412+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:41.698 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 34 2024-04-02T05:00:41.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f077806dcd0 msgr2=0x7f0778070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f077806dcd0 0x7f0778070190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f077c005ec0 tx=0x7f077c005fd0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ff260 msgr2=0x7f078c195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ff260 0x7f078c195e30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f078800c8a0 tx=0x7f078800cd60 comp rx=0 tx=0).stop 2024-04-02T05:00:41.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 shutdown_connections 2024-04-02T05:00:41.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f078c196370 0x7f078c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f077806dcd0 0x7f0778070190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 --2- 172.21.15.70:0/168310195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f078c0ff260 0x7f078c195e30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 >> 172.21.15.70:0/168310195 conn(0x7f078c074a50 msgr2=0x7f078c103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:41.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 shutdown_connections 2024-04-02T05:00:41.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.697+0000 7f079327d640 1 -- 172.21.15.70:0/168310195 wait complete. 2024-04-02T05:00:41.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:41 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2537006758' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-02T05:00:41.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:41 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/4017641980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-02T05:00:41.711 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:00:41.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-02T05:00:41.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.813+0000 7f2995fe4640 1 -- 172.21.15.70:0/121577374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29900ffc10 msgr2=0x7f2990103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.813+0000 7f2995fe4640 1 --2- 172.21.15.70:0/121577374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29900ffc10 0x7f2990103b70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2984009880 tx=0x7f298402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.813+0000 7f2995fe4640 1 -- 172.21.15.70:0/121577374 shutdown_connections 2024-04-02T05:00:41.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.813+0000 7f2995fe4640 1 --2- 172.21.15.70:0/121577374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29900ffc10 0x7f2990103b70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.813+0000 7f2995fe4640 1 --2- 172.21.15.70:0/121577374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29900ff260 0x7f29900ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.813+0000 7f2995fe4640 1 -- 172.21.15.70:0/121577374 >> 172.21.15.70:0/121577374 conn(0x7f2990074a50 msgr2=0x7f2990074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:41.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.813+0000 7f2995fe4640 1 -- 172.21.15.70:0/121577374 shutdown_connections 2024-04-02T05:00:41.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.813+0000 7f2995fe4640 1 -- 172.21.15.70:0/121577374 wait complete. 2024-04-02T05:00:41.818 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.814+0000 7f2995fe4640 1 Processor -- start 2024-04-02T05:00:41.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.814+0000 7f2995fe4640 1 -- start start 2024-04-02T05:00:41.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.814+0000 7f2995fe4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29900ff260 0x7f299019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.814+0000 7f2995fe4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299019af60 0x7f2990194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.814+0000 7f2995fe4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299019b530 con 0x7f299019af60 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.814+0000 7f2995fe4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2990194fd0 con 0x7f29900ff260 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299019af60 0x7f2990194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299019af60 0x7f2990194a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56648/0 (socket says 172.21.15.70:56648) 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298effd640 1 -- 172.21.15.70:0/733763848 learned_addr learned my addr 172.21.15.70:0/733763848 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298f7fe640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29900ff260 0x7f299019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298effd640 1 -- 172.21.15.70:0/733763848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29900ff260 msgr2=0x7f299019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298effd640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29900ff260 0x7f299019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298effd640 1 -- 172.21.15.70:0/733763848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2984009560 con 0x7f299019af60 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298f7fe640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29900ff260 0x7f299019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298effd640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299019af60 0x7f2990194a90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2984009230 tx=0x7f298402c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298cff9640 1 -- 172.21.15.70:0/733763848 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29840035f0 con 0x7f299019af60 2024-04-02T05:00:41.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298cff9640 1 -- 172.21.15.70:0/733763848 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2984003c10 con 0x7f299019af60 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f298cff9640 1 -- 172.21.15.70:0/733763848 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2984005600 con 0x7f299019af60 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2990195250 con 0x7f299019af60 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.815+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2990195700 con 0x7f299019af60 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.816+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299004e040 con 0x7f299019af60 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.817+0000 7f298cff9640 1 -- 172.21.15.70:0/733763848 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2984004dd0 con 0x7f299019af60 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.817+0000 7f298cff9640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f297806dcd0 0x7f2978070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.817+0000 7f298f7fe640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f297806dcd0 0x7f2978070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.817+0000 7f298cff9640 1 -- 172.21.15.70:0/733763848 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f29840b00c0 con 0x7f299019af60 2024-04-02T05:00:41.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.817+0000 7f298f7fe640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f297806dcd0 0x7f2978070190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2980009b50 tx=0x7f2980005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:41.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:41.822+0000 7f298cff9640 1 -- 172.21.15.70:0/733763848 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f29840794a0 con 0x7f299019af60 2024-04-02T05:00:42.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.037+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f2990107480 con 0x7f299019af60 2024-04-02T05:00:42.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.038+0000 7f298cff9640 1 -- 172.21.15.70:0/733763848 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v40) v1 ==== 107+0+4351 (secure 0 0 0) 0x7f298407cb60 con 0x7f299019af60 2024-04-02T05:00:42.042 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:42.042 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:58:10.154432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:42.042 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 35 2024-04-02T05:00:42.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f297806dcd0 msgr2=0x7f2978070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f297806dcd0 0x7f2978070190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2980009b50 tx=0x7f2980005f90 comp rx=0 tx=0).stop 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299019af60 msgr2=0x7f2990194a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299019af60 0x7f2990194a90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2984009230 tx=0x7f298402c750 comp rx=0 tx=0).stop 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 shutdown_connections 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f297806dcd0 0x7f2978070190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f299019af60 0x7f2990194a90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 --2- 172.21.15.70:0/733763848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29900ff260 0x7f299019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 >> 172.21.15.70:0/733763848 conn(0x7f2990074a50 msgr2=0x7f29900fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 shutdown_connections 2024-04-02T05:00:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.040+0000 7f2995fe4640 1 -- 172.21.15.70:0/733763848 wait complete. 2024-04-02T05:00:42.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-02T05:00:42.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.154+0000 7fb35215c640 1 -- 172.21.15.70:0/1296078378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 msgr2=0x7fb34c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.154+0000 7fb35215c640 1 --2- 172.21.15.70:0/1296078378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 0x7fb34c0ff640 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009880 tx=0x7fb33c02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.154+0000 7fb35215c640 1 -- 172.21.15.70:0/1296078378 shutdown_connections 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.154+0000 7fb35215c640 1 --2- 172.21.15.70:0/1296078378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb34c0ffc10 0x7fb34c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.154+0000 7fb35215c640 1 --2- 172.21.15.70:0/1296078378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 0x7fb34c0ff640 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.154+0000 7fb35215c640 1 -- 172.21.15.70:0/1296078378 >> 172.21.15.70:0/1296078378 conn(0x7fb34c074a50 msgr2=0x7fb34c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.154+0000 7fb35215c640 1 -- 172.21.15.70:0/1296078378 shutdown_connections 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.154+0000 7fb35215c640 1 -- 172.21.15.70:0/1296078378 wait complete. 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb35215c640 1 Processor -- start 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb35215c640 1 -- start start 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb35215c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 0x7fb34c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb35215c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb34c197150 0x7fb34c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb35215c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c19fc80 con 0x7fb34c0ff260 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb35215c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c19fdc0 con 0x7fb34c197150 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 0x7fb34c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 0x7fb34c196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56662/0 (socket says 172.21.15.70:56662) 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb34b7fe640 1 -- 172.21.15.70:0/1381048334 learned_addr learned my addr 172.21.15.70:0/1381048334 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.155+0000 7fb34affd640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb34c197150 0x7fb34c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb34b7fe640 1 -- 172.21.15.70:0/1381048334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb34c197150 msgr2=0x7fb34c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb34b7fe640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb34c197150 0x7fb34c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb34b7fe640 1 -- 172.21.15.70:0/1381048334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c009560 con 0x7fb34c0ff260 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb34b7fe640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 0x7fb34c196c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009430 tx=0x7fb33c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb348ff9640 1 -- 172.21.15.70:0/1381048334 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33c0035f0 con 0x7fb34c0ff260 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb348ff9640 1 -- 172.21.15.70:0/1381048334 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb33c003c10 con 0x7fb34c0ff260 2024-04-02T05:00:42.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c19ffc0 con 0x7fb34c0ff260 2024-04-02T05:00:42.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb348ff9640 1 -- 172.21.15.70:0/1381048334 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33c005670 con 0x7fb34c0ff260 2024-04-02T05:00:42.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.156+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c1a03f0 con 0x7fb34c0ff260 2024-04-02T05:00:42.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.157+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c04e040 con 0x7fb34c0ff260 2024-04-02T05:00:42.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.158+0000 7fb348ff9640 1 -- 172.21.15.70:0/1381048334 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb33c004dd0 con 0x7fb34c0ff260 2024-04-02T05:00:42.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.158+0000 7fb348ff9640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb33406dc80 0x7fb334070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.159+0000 7fb348ff9640 1 -- 172.21.15.70:0/1381048334 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fb33c0b0000 con 0x7fb34c0ff260 2024-04-02T05:00:42.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.159+0000 7fb34affd640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb33406dc80 0x7fb334070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:42.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.159+0000 7fb34affd640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb33406dc80 0x7fb334070140 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb340005fd0 tx=0x7fb34000b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:42.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.163+0000 7fb348ff9640 1 -- 172.21.15.70:0/1381048334 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb33c079490 con 0x7fb34c0ff260 2024-04-02T05:00:42.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.397+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7fb34c0ffda0 con 0x7fb34c0ff260 2024-04-02T05:00:42.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.397+0000 7fb348ff9640 1 -- 172.21.15.70:0/1381048334 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v40) v1 ==== 107+0+4348 (secure 0 0 0) 0x7fb33c07cb50 con 0x7fb34c0ff260 2024-04-02T05:00:42.401 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:42.401 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:58:10.178352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T05:00:42.401 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 36 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb33406dc80 msgr2=0x7fb334070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb33406dc80 0x7fb334070140 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb340005fd0 tx=0x7fb34000b390 comp rx=0 tx=0).stop 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 msgr2=0x7fb34c196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 0x7fb34c196c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009430 tx=0x7fb33c004040 comp rx=0 tx=0).stop 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 shutdown_connections 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb34c197150 0x7fb34c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb33406dc80 0x7fb334070140 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 --2- 172.21.15.70:0/1381048334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb34c0ff260 0x7fb34c196c10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 >> 172.21.15.70:0/1381048334 conn(0x7fb34c074a50 msgr2=0x7fb34c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:42.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 shutdown_connections 2024-04-02T05:00:42.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.400+0000 7fb35215c640 1 -- 172.21.15.70:0/1381048334 wait complete. 2024-04-02T05:00:42.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-02T05:00:42.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.521+0000 7fa66f4fb640 1 -- 172.21.15.70:0/3234055443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6680675c0 msgr2=0x7fa6680679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.521+0000 7fa66f4fb640 1 --2- 172.21.15.70:0/3234055443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6680675c0 0x7fa6680679a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa660009c20 tx=0x7fa66002c1b0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.521+0000 7fa66f4fb640 1 -- 172.21.15.70:0/3234055443 shutdown_connections 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.521+0000 7fa66f4fb640 1 --2- 172.21.15.70:0/3234055443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa668067f70 0x7fa668104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.521+0000 7fa66f4fb640 1 --2- 172.21.15.70:0/3234055443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6680675c0 0x7fa6680679a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.521+0000 7fa66f4fb640 1 -- 172.21.15.70:0/3234055443 >> 172.21.15.70:0/3234055443 conn(0x7fa668074a50 msgr2=0x7fa668074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.521+0000 7fa66f4fb640 1 -- 172.21.15.70:0/3234055443 shutdown_connections 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.521+0000 7fa66f4fb640 1 -- 172.21.15.70:0/3234055443 wait complete. 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66f4fb640 1 Processor -- start 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66f4fb640 1 -- start start 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66f4fb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680675c0 0x7fa668195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66f4fb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6681963e0 0x7fa66819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66f4fb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6681969e0 con 0x7fa6681963e0 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66f4fb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668196b50 con 0x7fa6680675c0 2024-04-02T05:00:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66ca96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6681963e0 0x7fa66819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66ca96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6681963e0 0x7fa66819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56674/0 (socket says 172.21.15.70:56674) 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66ca96640 1 -- 172.21.15.70:0/28755970 learned_addr learned my addr 172.21.15.70:0/28755970 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.522+0000 7fa66d297640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680675c0 0x7fa668195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa66ca96640 1 -- 172.21.15.70:0/28755970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680675c0 msgr2=0x7fa668195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa66ca96640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680675c0 0x7fa668195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa66ca96640 1 -- 172.21.15.70:0/28755970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa660009900 con 0x7fa6681963e0 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa66ca96640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6681963e0 0x7fa66819a840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa65800b400 tx=0x7fa65800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa65e7fc640 1 -- 172.21.15.70:0/28755970 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa658011800 con 0x7fa6681963e0 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66819ae40 con 0x7fa6681963e0 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa65e7fc640 1 -- 172.21.15.70:0/28755970 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa658011e20 con 0x7fa6681963e0 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa65e7fc640 1 -- 172.21.15.70:0/28755970 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa65800f2e0 con 0x7fa6681963e0 2024-04-02T05:00:42.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.523+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66819b3a0 con 0x7fa6681963e0 2024-04-02T05:00:42.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.524+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66804e040 con 0x7fa6681963e0 2024-04-02T05:00:42.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.524+0000 7fa65e7fc640 1 -- 172.21.15.70:0/28755970 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa658010380 con 0x7fa6681963e0 2024-04-02T05:00:42.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.525+0000 7fa65e7fc640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dcd0 0x7fa654070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.525+0000 7fa65e7fc640 1 -- 172.21.15.70:0/28755970 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fa65808d990 con 0x7fa6681963e0 2024-04-02T05:00:42.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.525+0000 7fa66d297640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dcd0 0x7fa654070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:42.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.525+0000 7fa66d297640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dcd0 0x7fa654070190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa660033000 tx=0x7fa660009840 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:42.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.529+0000 7fa65e7fc640 1 -- 172.21.15.70:0/28755970 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa658056d70 con 0x7fa6681963e0 2024-04-02T05:00:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:42 smithi179 ceph-mon[84288]: pgmap v304: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:42 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/4104614610' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-02T05:00:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:42 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/168310195' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-02T05:00:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:42 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/733763848' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-02T05:00:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:42 smithi070 ceph-mon[113635]: pgmap v304: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:42 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/4104614610' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-02T05:00:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:42 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/168310195' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-02T05:00:42.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:42 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/733763848' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-02T05:00:42.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.821+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fa668197240 con 0x7fa6681963e0 2024-04-02T05:00:42.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.821+0000 7fa65e7fc640 1 -- 172.21.15.70:0/28755970 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v40) v1 ==== 107+0+4357 (secure 0 0 0) 0x7fa65805a430 con 0x7fa6681963e0 2024-04-02T05:00:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:42.825 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:58:12.180686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":34,"state":"up:active","state_seq":23,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34388,"qdb_cluster":[34388]},"id":1}]} 2024-04-02T05:00:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 37 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dcd0 msgr2=0x7fa654070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dcd0 0x7fa654070190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa660033000 tx=0x7fa660009840 comp rx=0 tx=0).stop 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6681963e0 msgr2=0x7fa66819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6681963e0 0x7fa66819a840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa65800b400 tx=0x7fa65800b8c0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 shutdown_connections 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa65406dcd0 0x7fa654070190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6681963e0 0x7fa66819a840 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 --2- 172.21.15.70:0/28755970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6680675c0 0x7fa668195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 >> 172.21.15.70:0/28755970 conn(0x7fa668074a50 msgr2=0x7fa6680fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 shutdown_connections 2024-04-02T05:00:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.823+0000 7fa66f4fb640 1 -- 172.21.15.70:0/28755970 wait complete. 2024-04-02T05:00:42.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-04-02T05:00:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.936+0000 7fd78bd7e640 1 -- 172.21.15.70:0/3382847502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 msgr2=0x7fd7840ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.936+0000 7fd78bd7e640 1 --2- 172.21.15.70:0/3382847502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 0x7fd7840ff640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd774009880 tx=0x7fd77402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.936+0000 7fd78bd7e640 1 -- 172.21.15.70:0/3382847502 shutdown_connections 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 --2- 172.21.15.70:0/3382847502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7840ffc10 0x7fd784103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 --2- 172.21.15.70:0/3382847502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 0x7fd7840ff640 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 -- 172.21.15.70:0/3382847502 >> 172.21.15.70:0/3382847502 conn(0x7fd784074a50 msgr2=0x7fd784074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 -- 172.21.15.70:0/3382847502 shutdown_connections 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 -- 172.21.15.70:0/3382847502 wait complete. 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 Processor -- start 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 -- start start 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 0x7fd784197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7841984e0 0x7fd78419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784198ae0 con 0x7fd7840ff260 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd78bd7e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784198c50 con 0x7fd7841984e0 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd789b1a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 0x7fd784197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd789b1a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 0x7fd784197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56700/0 (socket says 172.21.15.70:56700) 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.937+0000 7fd789b1a640 1 -- 172.21.15.70:0/2901880650 learned_addr learned my addr 172.21.15.70:0/2901880650 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd789b1a640 1 -- 172.21.15.70:0/2901880650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7841984e0 msgr2=0x7fd78419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd789b1a640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7841984e0 0x7fd78419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd789b1a640 1 -- 172.21.15.70:0/2901880650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774009560 con 0x7fd7840ff260 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd789b1a640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 0x7fd784197fa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd774009430 tx=0x7fd774004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd77affd640 1 -- 172.21.15.70:0/2901880650 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7740035f0 con 0x7fd7840ff260 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78419cee0 con 0x7fd7840ff260 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd77affd640 1 -- 172.21.15.70:0/2901880650 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd774003c10 con 0x7fd7840ff260 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd77affd640 1 -- 172.21.15.70:0/2901880650 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd774034440 con 0x7fd7840ff260 2024-04-02T05:00:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.938+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78419d440 con 0x7fd7840ff260 2024-04-02T05:00:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.939+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78404e040 con 0x7fd7840ff260 2024-04-02T05:00:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.939+0000 7fd77affd640 1 -- 172.21.15.70:0/2901880650 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd774004fd0 con 0x7fd7840ff260 2024-04-02T05:00:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.940+0000 7fd77affd640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd77006dcd0 0x7fd770070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.940+0000 7fd77affd640 1 -- 172.21.15.70:0/2901880650 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fd7740b0110 con 0x7fd7840ff260 2024-04-02T05:00:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.940+0000 7fd789319640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd77006dcd0 0x7fd770070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.940+0000 7fd789319640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd77006dcd0 0x7fd770070190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd784072e40 tx=0x7fd780005bf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:42.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:42.945+0000 7fd77affd640 1 -- 172.21.15.70:0/2901880650 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd7740794f0 con 0x7fd7840ff260 2024-04-02T05:00:43.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.157+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7fd7840ffc10 con 0x7fd7840ff260 2024-04-02T05:00:43.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.158+0000 7fd77affd640 1 -- 172.21.15.70:0/2901880650 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v40) v1 ==== 107+0+5207 (secure 0 0 0) 0x7fd77407cbb0 con 0x7fd7840ff260 2024-04-02T05:00:43.161 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:43.161 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44315,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6827/1653234055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1653234055},{"type":"v1","addr":"172.21.15.179:6827","nonce":1653234055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:58:12.180686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":34,"state":"up:active","state_seq":23,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34388,"qdb_cluster":[34388]},"id":1}]} 2024-04-02T05:00:43.161 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 38 2024-04-02T05:00:43.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd77006dcd0 msgr2=0x7fd770070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd77006dcd0 0x7fd770070190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd784072e40 tx=0x7fd780005bf0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 msgr2=0x7fd784197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 0x7fd784197fa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd774009430 tx=0x7fd774004040 comp rx=0 tx=0).stop 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 shutdown_connections 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7841984e0 0x7fd78419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fd77006dcd0 0x7fd770070190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 --2- 172.21.15.70:0/2901880650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7840ff260 0x7fd784197fa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 >> 172.21.15.70:0/2901880650 conn(0x7fd784074a50 msgr2=0x7fd784067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 shutdown_connections 2024-04-02T05:00:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.160+0000 7fd78bd7e640 1 -- 172.21.15.70:0/2901880650 wait complete. 2024-04-02T05:00:43.174 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.274+0000 7fb5f0f99640 1 -- 172.21.15.70:0/3981742476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec0ff260 msgr2=0x7fb5ec0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.274+0000 7fb5f0f99640 1 --2- 172.21.15.70:0/3981742476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec0ff260 0x7fb5ec0ff640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009880 tx=0x7fb5dc02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 -- 172.21.15.70:0/3981742476 shutdown_connections 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 --2- 172.21.15.70:0/3981742476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5ec0ffc10 0x7fb5ec103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 --2- 172.21.15.70:0/3981742476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec0ff260 0x7fb5ec0ff640 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 -- 172.21.15.70:0/3981742476 >> 172.21.15.70:0/3981742476 conn(0x7fb5ec074a50 msgr2=0x7fb5ec074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 -- 172.21.15.70:0/3981742476 shutdown_connections 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 -- 172.21.15.70:0/3981742476 wait complete. 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 Processor -- start 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 -- start start 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5ec0ff260 0x7fb5ec196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:43.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec197150 0x7fb5ec1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec19fc80 con 0x7fb5ec197150 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.275+0000 7fb5f0f99640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec19fdc0 con 0x7fb5ec0ff260 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec197150 0x7fb5ec1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec197150 0x7fb5ec1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56714/0 (socket says 172.21.15.70:56714) 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e3fff640 1 -- 172.21.15.70:0/2219858027 learned_addr learned my addr 172.21.15.70:0/2219858027 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5ea59c640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5ec0ff260 0x7fb5ec196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e3fff640 1 -- 172.21.15.70:0/2219858027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5ec0ff260 msgr2=0x7fb5ec196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e3fff640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5ec0ff260 0x7fb5ec196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e3fff640 1 -- 172.21.15.70:0/2219858027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc009560 con 0x7fb5ec197150 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e3fff640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec197150 0x7fb5ec1975b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400d6f0 tx=0x7fb5d400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e37fe640 1 -- 172.21.15.70:0/2219858027 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5d40051f0 con 0x7fb5ec197150 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e37fe640 1 -- 172.21.15.70:0/2219858027 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5d4009ce0 con 0x7fb5ec197150 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec1a0020 con 0x7fb5ec197150 2024-04-02T05:00:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5e37fe640 1 -- 172.21.15.70:0/2219858027 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5d4004e80 con 0x7fb5ec197150 2024-04-02T05:00:43.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.276+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec1a0500 con 0x7fb5ec197150 2024-04-02T05:00:43.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.278+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec04e040 con 0x7fb5ec197150 2024-04-02T05:00:43.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.278+0000 7fb5e37fe640 1 -- 172.21.15.70:0/2219858027 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb5d40046c0 con 0x7fb5ec197150 2024-04-02T05:00:43.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.278+0000 7fb5e37fe640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb5d806dc80 0x7fb5d8070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:43.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.278+0000 7fb5e37fe640 1 -- 172.21.15.70:0/2219858027 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fb5d408d6b0 con 0x7fb5ec197150 2024-04-02T05:00:43.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.278+0000 7fb5ea59c640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb5d806dc80 0x7fb5d8070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:43.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.279+0000 7fb5ea59c640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb5d806dc80 0x7fb5d8070140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc006010 tx=0x7fb5dc005dd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:43.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.283+0000 7fb5e37fe640 1 -- 172.21.15.70:0/2219858027 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb5d4056b40 con 0x7fb5ec197150 2024-04-02T05:00:43.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.523+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7fb5ec107480 con 0x7fb5ec197150 2024-04-02T05:00:43.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.524+0000 7fb5e37fe640 1 -- 172.21.15.70:0/2219858027 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v40) v1 ==== 107+0+5207 (secure 0 0 0) 0x7fb5d405a200 con 0x7fb5ec197150 2024-04-02T05:00:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:43.527 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.zvraae","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/3294303398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3294303398},{"type":"v1","addr":"172.21.15.70:6829","nonce":3294303398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44287,"name":"cephfs.smithi179.mzfimf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6825/1866330674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1866330674},{"type":"v1","addr":"172.21.15.179:6825","nonce":1866330674}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44315,"name":"cephfs.smithi179.atrmwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6827/1653234055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1653234055},{"type":"v1","addr":"172.21.15.179:6827","nonce":1653234055}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:44:52.788932+0000","modified":"2024-04-02T04:58:20.330302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi070.mnzrwt","rank":0,"incarnation":34,"state":"up:active","state_seq":23,"addr":"172.21.15.70:6827/17804049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":17804049},{"type":"v1","addr":"172.21.15.70:6827","nonce":17804049}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34388,"qdb_cluster":[34388]},"id":1}]} 2024-04-02T05:00:43.527 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 39 2024-04-02T05:00:43.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb5d806dc80 msgr2=0x7fb5d8070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb5d806dc80 0x7fb5d8070140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc006010 tx=0x7fb5dc005dd0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec197150 msgr2=0x7fb5ec1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec197150 0x7fb5ec1975b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400d6f0 tx=0x7fb5d400dbb0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 shutdown_connections 2024-04-02T05:00:43.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb5d806dc80 0x7fb5d8070140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5ec197150 0x7fb5ec1975b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 --2- 172.21.15.70:0/2219858027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5ec0ff260 0x7fb5ec196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.530 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 >> 172.21.15.70:0/2219858027 conn(0x7fb5ec074a50 msgr2=0x7fb5ec0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 shutdown_connections 2024-04-02T05:00:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.526+0000 7fb5f0f99640 1 -- 172.21.15.70:0/2219858027 wait complete. 2024-04-02T05:00:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:43 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1381048334' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-02T05:00:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:43 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/28755970' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-02T05:00:43.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:43 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2901880650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-02T05:00:43.541 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-02T05:00:43.552 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-02T05:00:43.552 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T05:00:43.552 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T05:00:43.570 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T05:00:43.570 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T05:00:43.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:43 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1381048334' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-02T05:00:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:43 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/28755970' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-02T05:00:43.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:43 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2901880650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-02T05:00:43.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T05:00:43.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 -- 172.21.15.70:0/218515466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae4067350 msgr2=0x7f8ae40677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 --2- 172.21.15.70:0/218515466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae4067350 0x7f8ae40677b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009880 tx=0x7f8acc02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 -- 172.21.15.70:0/218515466 shutdown_connections 2024-04-02T05:00:43.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 --2- 172.21.15.70:0/218515466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae4067350 0x7f8ae40677b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 --2- 172.21.15.70:0/218515466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae41061b0 0x7f8ae4106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 -- 172.21.15.70:0/218515466 >> 172.21.15.70:0/218515466 conn(0x7f8ae4075b40 msgr2=0x7f8ae4075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 -- 172.21.15.70:0/218515466 shutdown_connections 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 -- 172.21.15.70:0/218515466 wait complete. 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 Processor -- start 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 -- start start 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.767+0000 7f8ae96a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae4067350 0x7f8ae4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae96a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae41061b0 0x7f8ae4196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae96a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4196920 con 0x7f8ae41061b0 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae96a4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae419a6f0 con 0x7f8ae4067350 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae41061b0 0x7f8ae4196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8adbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae41061b0 0x7f8ae4196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56738/0 (socket says 172.21.15.70:56738) 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8adbfff640 1 -- 172.21.15.70:0/721388098 learned_addr learned my addr 172.21.15.70:0/721388098 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:43.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae2ffd640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae4067350 0x7f8ae4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8adbfff640 1 -- 172.21.15.70:0/721388098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae4067350 msgr2=0x7f8ae4195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8adbfff640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae4067350 0x7f8ae4195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8adbfff640 1 -- 172.21.15.70:0/721388098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc009560 con 0x7f8ae41061b0 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae2ffd640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae4067350 0x7f8ae4195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8adbfff640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae41061b0 0x7f8ae4196290 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009230 tx=0x7f8acc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae0ff9640 1 -- 172.21.15.70:0/721388098 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8acc0035f0 con 0x7f8ae41061b0 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae0ff9640 1 -- 172.21.15.70:0/721388098 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8acc003c10 con 0x7f8ae41061b0 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae419a970 con 0x7f8ae41061b0 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae0ff9640 1 -- 172.21.15.70:0/721388098 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8acc034440 con 0x7f8ae41061b0 2024-04-02T05:00:43.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.768+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae419ae70 con 0x7f8ae41061b0 2024-04-02T05:00:43.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.770+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae404e040 con 0x7f8ae41061b0 2024-04-02T05:00:43.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.770+0000 7f8ae0ff9640 1 -- 172.21.15.70:0/721388098 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f8acc004fd0 con 0x7f8ae41061b0 2024-04-02T05:00:43.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.770+0000 7f8ae0ff9640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8ad006dd90 0x7f8ad0070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:43.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.770+0000 7f8ae0ff9640 1 -- 172.21.15.70:0/721388098 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f8acc0b00f0 con 0x7f8ae41061b0 2024-04-02T05:00:43.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.770+0000 7f8ae2ffd640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8ad006dd90 0x7f8ad0070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:43.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.771+0000 7f8ae2ffd640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8ad006dd90 0x7f8ad0070250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4006fd0 tx=0x7f8ad4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:43.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.777+0000 7f8ae0ff9640 1 -- 172.21.15.70:0/721388098 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8acc079550 con 0x7f8ae41061b0 2024-04-02T05:00:43.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.967+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ae410d390 con 0x7f8ae41061b0 2024-04-02T05:00:43.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.968+0000 7f8ae0ff9640 1 -- 172.21.15.70:0/721388098 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17334 (secure 0 0 0) 0x7f8acc07cc10 con 0x7f8ae41061b0 2024-04-02T05:00:43.971 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:43.971 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":95,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:37:49.993595+0000","modified":"2024-04-02T04:58:01.605043+0000","last_up_change":"2024-04-02T04:56:39.019903+0000","last_in_change":"2024-04-02T04:43:12.453452+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:42:18.484513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-02T04:44:51.427296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-02T04:44:52.024713+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b5e5510-c4db-4073-b470-c777b557d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6803","nonce":4011619577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6805","nonce":4011619577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6809","nonce":4011619577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6807","nonce":4011619577}]},"public_addr":"172.21.15.70:6803/4011619577","cluster_addr":"172.21.15.70:6805/4011619577","heartbeat_back_addr":"172.21.15.70:6809/4011619577","heartbeat_front_addr":"172.21.15.70:6807/4011619577","state":["exists","up"]},{"osd":1,"uuid":"b751ceaf-b734-4280-ba22-82683b9776ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":64,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6811","nonce":2318524655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6813","nonce":2318524655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6817","nonce":2318524655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6815","nonce":2318524655}]},"public_addr":"172.21.15.70:6811/2318524655","cluster_addr":"172.21.15.70:6813/2318524655","heartbeat_back_addr":"172.21.15.70:6817/2318524655","heartbeat_front_addr":"172.21.15.70:6815/2318524655","state":["exists","up"]},{"osd":2,"uuid":"32080ab6-7809-4523-aab7-ca7a6d263ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6819","nonce":347493648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6821","nonce":347493648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6825","nonce":347493648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6823","nonce":347493648}]},"public_addr":"172.21.15.70:6819/347493648","cluster_addr":"172.21.15.70:6821/347493648","heartbeat_back_addr":"172.21.15.70:6825/347493648","heartbeat_front_addr":"172.21.15.70:6823/347493648","state":["exists","up"]},{"osd":3,"uuid":"0d71a0bb-5e7a-4339-afe7-06174dea4986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6801","nonce":657192449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6803","nonce":657192449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6807","nonce":657192449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6805","nonce":657192449}]},"public_addr":"172.21.15.179:6801/657192449","cluster_addr":"172.21.15.179:6803/657192449","heartbeat_back_addr":"172.21.15.179:6807/657192449","heartbeat_front_addr":"172.21.15.179:6805/657192449","state":["exists","up"]},{"osd":4,"uuid":"754e6f09-2a8e-42e2-a998-5f5a1addcb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6809","nonce":229347770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6811","nonce":229347770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6815","nonce":229347770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6813","nonce":229347770}]},"public_addr":"172.21.15.179:6809/229347770","cluster_addr":"172.21.15.179:6811/229347770","heartbeat_back_addr":"172.21.15.179:6815/229347770","heartbeat_front_addr":"172.21.15.179:6813/229347770","state":["exists","up"]},{"osd":5,"uuid":"46a73805-a770-4ed1-96f4-e64cf88810eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6817","nonce":2533703533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6819","nonce":2533703533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6823","nonce":2533703533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6821","nonce":2533703533}]},"public_addr":"172.21.15.179:6817/2533703533","cluster_addr":"172.21.15.179:6819/2533703533","heartbeat_back_addr":"172.21.15.179:6823/2533703533","heartbeat_front_addr":"172.21.15.179:6821/2533703533","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:17.145441+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:44.622049+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:14.717560+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:38.493463+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:04.319117+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:30.171864+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.15","mappings":[{"from":0,"to":5}]},{"pgid":"2.16","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1f","mappings":[{"from":0,"to":1},{"from":3,"to":2}]},{"pgid":"3.3","mappings":[{"from":4,"to":1}]},{"pgid":"3.11","mappings":[{"from":3,"to":1}]},{"pgid":"3.13","mappings":[{"from":3,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6827/3541837059":"2024-04-03T04:58:01.605007+0000","172.21.15.179:6824/511108019":"2024-04-03T04:57:35.242205+0000","172.21.15.179:6826/3541837059":"2024-04-03T04:58:01.605007+0000","172.21.15.70:6829/2586361462":"2024-04-03T04:57:09.380867+0000","172.21.15.70:6800/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1721127786":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1700662291":"2024-04-03T04:47:27.805482+0000","172.21.15.179:0/1433338238":"2024-04-03T04:48:41.679801+0000","172.21.15.179:0/1900906037":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6801/146104126":"2024-04-03T04:52:11.062737+0000","172.21.15.179:6825/511108019":"2024-04-03T04:57:35.242205+0000","172.21.15.70:6800/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/2463094374":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/940716355":"2024-04-03T04:38:18.397033+0000","172.21.15.179:0/1600665743":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6828/2586361462":"2024-04-03T04:57:09.380867+0000","172.21.15.70:0/3512109322":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.179:0/4148120600":"2024-04-03T04:48:41.679801+0000","172.21.15.179:6828/1862820759":"2024-04-03T04:48:41.679801+0000","172.21.15.70:0/2446527261":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/1353385001":"2024-04-03T04:47:27.805482+0000","172.21.15.179:0/1702319761":"2024-04-03T04:48:41.679801+0000","172.21.15.179:6829/1862820759":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6827/1351191990":"2024-04-03T04:56:43.851170+0000","172.21.15.70:6801/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/4149690148":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/39315361":"2024-04-03T04:47:27.805482+0000","172.21.15.70:6801/2814127860":"2024-04-03T04:47:27.805482+0000","172.21.15.70:0/110321404":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/4005781191":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/3239652190":"2024-04-03T04:52:11.062737+0000","172.21.15.70:0/2716917447":"2024-04-03T04:47:27.805482+0000","172.21.15.70:0/2937797857":"2024-04-03T04:39:58.163164+0000","172.21.15.179:0/546971533":"2024-04-03T04:48:41.679801+0000","172.21.15.70:0/2778101694":"2024-04-03T04:52:11.062737+0000","172.21.15.70:6826/1351191990":"2024-04-03T04:56:43.851170+0000","172.21.15.70:6800/146104126":"2024-04-03T04:52:11.062737+0000","172.21.15.70:6801/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2814127860":"2024-04-03T04:47:27.805482+0000","172.21.15.70:6801/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/2760078492":"2024-04-03T04:52:11.062737+0000","172.21.15.70:0/3184091580":"2024-04-03T04:52:11.062737+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T05:00:43.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8ad006dd90 msgr2=0x7f8ad0070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8ad006dd90 0x7f8ad0070250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4006fd0 tx=0x7f8ad4008040 comp rx=0 tx=0).stop 2024-04-02T05:00:43.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae41061b0 msgr2=0x7f8ae4196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:43.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae41061b0 0x7f8ae4196290 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009230 tx=0x7f8acc004040 comp rx=0 tx=0).stop 2024-04-02T05:00:43.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 shutdown_connections 2024-04-02T05:00:43.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f8ad006dd90 0x7f8ad0070250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae41061b0 0x7f8ae4196290 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 --2- 172.21.15.70:0/721388098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ae4067350 0x7f8ae4195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:43.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 >> 172.21.15.70:0/721388098 conn(0x7f8ae4075b40 msgr2=0x7f8ae40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:43.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 shutdown_connections 2024-04-02T05:00:43.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:43.970+0000 7f8ae96a4640 1 -- 172.21.15.70:0/721388098 wait complete. 2024-04-02T05:00:43.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:42:18.484513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-02T04:44:51.427296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-02T04:44:52.024713+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T05:00:43.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T05:00:44.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.086+0000 7fae07381640 1 -- 172.21.15.70:0/1857272181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae000675c0 msgr2=0x7fae000679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.086+0000 7fae07381640 1 --2- 172.21.15.70:0/1857272181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae000675c0 0x7fae000679a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009880 tx=0x7fadfc02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.086+0000 7fae07381640 1 -- 172.21.15.70:0/1857272181 shutdown_connections 2024-04-02T05:00:44.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.086+0000 7fae07381640 1 --2- 172.21.15.70:0/1857272181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae00067f70 0x7fae00104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.086+0000 7fae07381640 1 --2- 172.21.15.70:0/1857272181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae000675c0 0x7fae000679a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.086+0000 7fae07381640 1 -- 172.21.15.70:0/1857272181 >> 172.21.15.70:0/1857272181 conn(0x7fae00074a50 msgr2=0x7fae00074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.086+0000 7fae07381640 1 -- 172.21.15.70:0/1857272181 shutdown_connections 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.086+0000 7fae07381640 1 -- 172.21.15.70:0/1857272181 wait complete. 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae07381640 1 Processor -- start 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae07381640 1 -- start start 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae07381640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae000675c0 0x7fae00195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae07381640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae001963e0 0x7fae0019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae07381640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae001969e0 con 0x7fae001963e0 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae07381640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae00196b50 con 0x7fae000675c0 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae0491c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae001963e0 0x7fae0019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae0491c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae001963e0 0x7fae0019a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56746/0 (socket says 172.21.15.70:56746) 2024-04-02T05:00:44.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae0491c640 1 -- 172.21.15.70:0/2715436518 learned_addr learned my addr 172.21.15.70:0/2715436518 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.087+0000 7fae0511d640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae000675c0 0x7fae00195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fae0491c640 1 -- 172.21.15.70:0/2715436518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae000675c0 msgr2=0x7fae00195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fae0491c640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae000675c0 0x7fae00195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fae0491c640 1 -- 172.21.15.70:0/2715436518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc009560 con 0x7fae001963e0 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fae0491c640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae001963e0 0x7fae0019a840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b400 tx=0x7fadf000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fadf67fc640 1 -- 172.21.15.70:0/2715436518 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf0011800 con 0x7fae001963e0 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fadf67fc640 1 -- 172.21.15.70:0/2715436518 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadf0011e20 con 0x7fae001963e0 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0019ae40 con 0x7fae001963e0 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fadf67fc640 1 -- 172.21.15.70:0/2715436518 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf000f2e0 con 0x7fae001963e0 2024-04-02T05:00:44.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.088+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0019b3a0 con 0x7fae001963e0 2024-04-02T05:00:44.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.089+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0004e040 con 0x7fae001963e0 2024-04-02T05:00:44.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.089+0000 7fadf67fc640 1 -- 172.21.15.70:0/2715436518 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fadf0010380 con 0x7fae001963e0 2024-04-02T05:00:44.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.090+0000 7fadf67fc640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fadec06dcd0 0x7fadec070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.090+0000 7fadf67fc640 1 -- 172.21.15.70:0/2715436518 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fadf008da20 con 0x7fae001963e0 2024-04-02T05:00:44.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.090+0000 7fae0511d640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fadec06dcd0 0x7fadec070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.090+0000 7fae0511d640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fadec06dcd0 0x7fadec070190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fadfc006010 tx=0x7fadfc000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:44.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.095+0000 7fadf67fc640 1 -- 172.21.15.70:0/2715436518 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fadf0056e00 con 0x7fae001963e0 2024-04-02T05:00:44.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.277+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae001972b0 con 0x7fae001963e0 2024-04-02T05:00:44.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.278+0000 7fadf67fc640 1 -- 172.21.15.70:0/2715436518 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadf005a4c0 con 0x7fae001963e0 2024-04-02T05:00:44.281 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-04-02T05:00:44.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fadec06dcd0 msgr2=0x7fadec070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fadec06dcd0 0x7fadec070190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fadfc006010 tx=0x7fadfc000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae001963e0 msgr2=0x7fae0019a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae001963e0 0x7fae0019a840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b400 tx=0x7fadf000b8c0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 shutdown_connections 2024-04-02T05:00:44.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fadec06dcd0 0x7fadec070190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae001963e0 0x7fae0019a840 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 --2- 172.21.15.70:0/2715436518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae000675c0 0x7fae00195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 >> 172.21.15.70:0/2715436518 conn(0x7fae00074a50 msgr2=0x7fae000fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:44.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 shutdown_connections 2024-04-02T05:00:44.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.280+0000 7fae07381640 1 -- 172.21.15.70:0/2715436518 wait complete. 2024-04-02T05:00:44.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-02T05:00:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.412+0000 7fb6bbbca640 1 -- 172.21.15.70:0/1620561619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 msgr2=0x7fb6b40679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.412+0000 7fb6bbbca640 1 --2- 172.21.15.70:0/1620561619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 0x7fb6b40679a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009880 tx=0x7fb6b002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.412+0000 7fb6bbbca640 1 -- 172.21.15.70:0/1620561619 shutdown_connections 2024-04-02T05:00:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.412+0000 7fb6bbbca640 1 --2- 172.21.15.70:0/1620561619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b4067f70 0x7fb6b4104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.412+0000 7fb6bbbca640 1 --2- 172.21.15.70:0/1620561619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 0x7fb6b40679a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.412+0000 7fb6bbbca640 1 -- 172.21.15.70:0/1620561619 >> 172.21.15.70:0/1620561619 conn(0x7fb6b4074a50 msgr2=0x7fb6b4074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.412+0000 7fb6bbbca640 1 -- 172.21.15.70:0/1620561619 shutdown_connections 2024-04-02T05:00:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.412+0000 7fb6bbbca640 1 -- 172.21.15.70:0/1620561619 wait complete. 2024-04-02T05:00:44.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.413+0000 7fb6bbbca640 1 Processor -- start 2024-04-02T05:00:44.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.413+0000 7fb6bbbca640 1 -- start start 2024-04-02T05:00:44.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.413+0000 7fb6bbbca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 0x7fb6b4195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6bbbca640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b41962f0 0x7fb6b419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6bbbca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b41968f0 con 0x7fb6b40675c0 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6bbbca640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b4196a60 con 0x7fb6b41962f0 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6b9966640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 0x7fb6b4195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6b9966640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 0x7fb6b4195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56764/0 (socket says 172.21.15.70:56764) 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6b9966640 1 -- 172.21.15.70:0/3614054296 learned_addr learned my addr 172.21.15.70:0/3614054296 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6b9165640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b41962f0 0x7fb6b419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6b9966640 1 -- 172.21.15.70:0/3614054296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b41962f0 msgr2=0x7fb6b419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6b9966640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b41962f0 0x7fb6b419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6b9966640 1 -- 172.21.15.70:0/3614054296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0009560 con 0x7fb6b40675c0 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6b9966640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 0x7fb6b4195db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009430 tx=0x7fb6b0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6aaffd640 1 -- 172.21.15.70:0/3614054296 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6b00044a0 con 0x7fb6b40675c0 2024-04-02T05:00:44.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b419acf0 con 0x7fb6b40675c0 2024-04-02T05:00:44.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6aaffd640 1 -- 172.21.15.70:0/3614054296 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6b0003ab0 con 0x7fb6b40675c0 2024-04-02T05:00:44.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6aaffd640 1 -- 172.21.15.70:0/3614054296 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6b00055f0 con 0x7fb6b40675c0 2024-04-02T05:00:44.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.414+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b419b250 con 0x7fb6b40675c0 2024-04-02T05:00:44.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.415+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b404e040 con 0x7fb6b40675c0 2024-04-02T05:00:44.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.416+0000 7fb6aaffd640 1 -- 172.21.15.70:0/3614054296 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb6b0004e60 con 0x7fb6b40675c0 2024-04-02T05:00:44.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.416+0000 7fb6aaffd640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb6a006dcd0 0x7fb6a0070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.417+0000 7fb6b9165640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb6a006dcd0 0x7fb6a0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.417+0000 7fb6aaffd640 1 -- 172.21.15.70:0/3614054296 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fb6b00b0000 con 0x7fb6b40675c0 2024-04-02T05:00:44.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.417+0000 7fb6b9165640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb6a006dcd0 0x7fb6a0070190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4072e40 tx=0x7fb6a4005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.421+0000 7fb6aaffd640 1 -- 172.21.15.70:0/3614054296 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb6b0079490 con 0x7fb6b40675c0 2024-04-02T05:00:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:44 smithi179 ceph-mon[84288]: pgmap v305: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:44 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2219858027' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-02T05:00:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:44 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/721388098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T05:00:44.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:44 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2715436518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T05:00:44.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.603+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fb6b4108580 con 0x7fb6b40675c0 2024-04-02T05:00:44.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.603+0000 7fb6aaffd640 1 -- 172.21.15.70:0/3614054296 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb6b007cb50 con 0x7fb6b40675c0 2024-04-02T05:00:44.606 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-04-02T05:00:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb6a006dcd0 msgr2=0x7fb6a0070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb6a006dcd0 0x7fb6a0070190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4072e40 tx=0x7fb6a4005f90 comp rx=0 tx=0).stop 2024-04-02T05:00:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 msgr2=0x7fb6b4195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 0x7fb6b4195db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009430 tx=0x7fb6b0005e50 comp rx=0 tx=0).stop 2024-04-02T05:00:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 shutdown_connections 2024-04-02T05:00:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b41962f0 0x7fb6b419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb6a006dcd0 0x7fb6a0070190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 --2- 172.21.15.70:0/3614054296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6b40675c0 0x7fb6b4195db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 >> 172.21.15.70:0/3614054296 conn(0x7fb6b4074a50 msgr2=0x7fb6b40fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:44.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 shutdown_connections 2024-04-02T05:00:44.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.606+0000 7fb6bbbca640 1 -- 172.21.15.70:0/3614054296 wait complete. 2024-04-02T05:00:44.619 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:44 smithi070 ceph-mon[113635]: pgmap v305: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:44.619 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:44 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2219858027' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-02T05:00:44.619 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:44 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/721388098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T05:00:44.619 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:44 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2715436518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T05:00:44.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-02T05:00:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.726+0000 7ff739573640 1 -- 172.21.15.70:0/4132619506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7340ff170 msgr2=0x7ff7340ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.726+0000 7ff739573640 1 --2- 172.21.15.70:0/4132619506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7340ff170 0x7ff7340ff550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff724009880 tx=0x7ff72402bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.726+0000 7ff739573640 1 -- 172.21.15.70:0/4132619506 shutdown_connections 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.726+0000 7ff739573640 1 --2- 172.21.15.70:0/4132619506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7340ffb20 0x7ff734103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.726+0000 7ff739573640 1 --2- 172.21.15.70:0/4132619506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7340ff170 0x7ff7340ff550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.726+0000 7ff739573640 1 -- 172.21.15.70:0/4132619506 >> 172.21.15.70:0/4132619506 conn(0x7ff7340fa9f0 msgr2=0x7ff7340fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.726+0000 7ff739573640 1 -- 172.21.15.70:0/4132619506 shutdown_connections 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.726+0000 7ff739573640 1 -- 172.21.15.70:0/4132619506 wait complete. 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff739573640 1 Processor -- start 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff739573640 1 -- start start 2024-04-02T05:00:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff739573640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7340ff170 0x7ff734196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff739573640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff734197170 0x7ff7341975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff739573640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73419fca0 con 0x7ff734197170 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff739573640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73419fde0 con 0x7ff7340ff170 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff7327fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff734197170 0x7ff7341975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff7327fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff734197170 0x7ff7341975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56788/0 (socket says 172.21.15.70:56788) 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff7327fc640 1 -- 172.21.15.70:0/3058213844 learned_addr learned my addr 172.21.15.70:0/3058213844 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff732ffd640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7340ff170 0x7ff734196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff7327fc640 1 -- 172.21.15.70:0/3058213844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7340ff170 msgr2=0x7ff734196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff7327fc640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7340ff170 0x7ff734196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.727+0000 7ff7327fc640 1 -- 172.21.15.70:0/3058213844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724009560 con 0x7ff734197170 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.728+0000 7ff7327fc640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff734197170 0x7ff7341975d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff72800e9a0 tx=0x7ff72800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.728+0000 7ff71bfff640 1 -- 172.21.15.70:0/3058213844 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72800cc90 con 0x7ff734197170 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.728+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7341a0040 con 0x7ff734197170 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.728+0000 7ff71bfff640 1 -- 172.21.15.70:0/3058213844 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff72800cdf0 con 0x7ff734197170 2024-04-02T05:00:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.728+0000 7ff71bfff640 1 -- 172.21.15.70:0/3058213844 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff728010430 con 0x7ff734197170 2024-04-02T05:00:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.728+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7341a0520 con 0x7ff734197170 2024-04-02T05:00:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.729+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73404e040 con 0x7ff734197170 2024-04-02T05:00:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.730+0000 7ff71bfff640 1 -- 172.21.15.70:0/3058213844 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff7280106a0 con 0x7ff734197170 2024-04-02T05:00:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.730+0000 7ff71bfff640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff71c06dcd0 0x7ff71c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.730+0000 7ff71bfff640 1 -- 172.21.15.70:0/3058213844 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7ff728014070 con 0x7ff734197170 2024-04-02T05:00:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.730+0000 7ff732ffd640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff71c06dcd0 0x7ff71c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.730+0000 7ff732ffd640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff71c06dcd0 0x7ff71c070190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff724006010 tx=0x7ff724005dd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.739+0000 7ff71bfff640 1 -- 172.21.15.70:0/3058213844 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff7280585c0 con 0x7ff734197170 2024-04-02T05:00:44.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.969+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff734107470 con 0x7ff734197170 2024-04-02T05:00:44.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.969+0000 7ff71bfff640 1 -- 172.21.15.70:0/3058213844 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff7280585c0 con 0x7ff734197170 2024-04-02T05:00:44.972 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-04-02T05:00:44.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff71c06dcd0 msgr2=0x7ff71c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff71c06dcd0 0x7ff71c070190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff724006010 tx=0x7ff724005dd0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff734197170 msgr2=0x7ff7341975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff734197170 0x7ff7341975d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff72800e9a0 tx=0x7ff72800ee60 comp rx=0 tx=0).stop 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 shutdown_connections 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff71c06dcd0 0x7ff71c070190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff734197170 0x7ff7341975d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 --2- 172.21.15.70:0/3058213844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7340ff170 0x7ff734196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 >> 172.21.15.70:0/3058213844 conn(0x7ff7340fa9f0 msgr2=0x7ff7340fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 shutdown_connections 2024-04-02T05:00:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:44.971+0000 7ff739573640 1 -- 172.21.15.70:0/3058213844 wait complete. 2024-04-02T05:00:44.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.088+0000 7fa4c5b67640 1 -- 172.21.15.70:0/3816959274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c00fefe0 msgr2=0x7fa4c00ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.088+0000 7fa4c5b67640 1 --2- 172.21.15.70:0/3816959274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c00fefe0 0x7fa4c00ff440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009880 tx=0x7fa4a802bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.088+0000 7fa4c5b67640 1 -- 172.21.15.70:0/3816959274 shutdown_connections 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.088+0000 7fa4c5b67640 1 --2- 172.21.15.70:0/3816959274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c00fefe0 0x7fa4c00ff440 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.088+0000 7fa4c5b67640 1 --2- 172.21.15.70:0/3816959274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4c0103f20 0x7fa4c0104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.088+0000 7fa4c5b67640 1 -- 172.21.15.70:0/3816959274 >> 172.21.15.70:0/3816959274 conn(0x7fa4c0075b40 msgr2=0x7fa4c0075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.088+0000 7fa4c5b67640 1 -- 172.21.15.70:0/3816959274 shutdown_connections 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.088+0000 7fa4c5b67640 1 -- 172.21.15.70:0/3816959274 wait complete. 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4c5b67640 1 Processor -- start 2024-04-02T05:00:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4c5b67640 1 -- start start 2024-04-02T05:00:45.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4c5b67640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4c00fefe0 0x7fa4c0196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4c5b67640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c0103f20 0x7fa4c0197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4c5b67640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c0197640 con 0x7fa4c0103f20 2024-04-02T05:00:45.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4c5b67640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01977b0 con 0x7fa4c00fefe0 2024-04-02T05:00:45.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c0103f20 0x7fa4c0197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c0103f20 0x7fa4c0197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56800/0 (socket says 172.21.15.70:56800) 2024-04-02T05:00:45.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4b6ffd640 1 -- 172.21.15.70:0/1555704954 learned_addr learned my addr 172.21.15.70:0/1555704954 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.089+0000 7fa4bf7fe640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4c00fefe0 0x7fa4c0196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4b6ffd640 1 -- 172.21.15.70:0/1555704954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4c00fefe0 msgr2=0x7fa4c0196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4b6ffd640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4c00fefe0 0x7fa4c0196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4b6ffd640 1 -- 172.21.15.70:0/1555704954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a8009560 con 0x7fa4c0103f20 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4b6ffd640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c0103f20 0x7fa4c0197070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009230 tx=0x7fa4a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4bd7fa640 1 -- 172.21.15.70:0/1555704954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a80035f0 con 0x7fa4c0103f20 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c019fba0 con 0x7fa4c0103f20 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4bd7fa640 1 -- 172.21.15.70:0/1555704954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4a8003c10 con 0x7fa4c0103f20 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4bd7fa640 1 -- 172.21.15.70:0/1555704954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a8005670 con 0x7fa4c0103f20 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.090+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c019ff60 con 0x7fa4c0103f20 2024-04-02T05:00:45.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.091+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c004e040 con 0x7fa4c0103f20 2024-04-02T05:00:45.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.091+0000 7fa4bd7fa640 1 -- 172.21.15.70:0/1555704954 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa4a8004dd0 con 0x7fa4c0103f20 2024-04-02T05:00:45.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.092+0000 7fa4bd7fa640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa4ac06dd30 0x7fa4ac0701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.092+0000 7fa4bf7fe640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa4ac06dd30 0x7fa4ac0701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.092+0000 7fa4bd7fa640 1 -- 172.21.15.70:0/1555704954 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fa4a80b0070 con 0x7fa4c0103f20 2024-04-02T05:00:45.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.092+0000 7fa4bf7fe640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa4ac06dd30 0x7fa4ac0701f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0005fd0 tx=0x7fa4b000b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:45.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.096+0000 7fa4bd7fa640 1 -- 172.21.15.70:0/1555704954 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa4a8079500 con 0x7fa4c0103f20 2024-04-02T05:00:45.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.286+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa4c010c520 con 0x7fa4c0103f20 2024-04-02T05:00:45.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.286+0000 7fa4bd7fa640 1 -- 172.21.15.70:0/1555704954 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v95) v1 ==== 81+0+2557 (secure 0 0 0) 0x7fa4a807cbc0 con 0x7fa4c0103f20 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6827/3541837059 2024-04-03T04:58:01.605007+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6824/511108019 2024-04-03T04:57:35.242205+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6826/3541837059 2024-04-03T04:58:01.605007+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6829/2586361462 2024-04-03T04:57:09.380867+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2939436653 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1721127786 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1700662291 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/1433338238 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/1900906037 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/146104126 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6825/511108019 2024-04-03T04:57:35.242205+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/1197360681 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2463094374 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/940716355 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/1600665743 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6828/2586361462 2024-04-03T04:57:09.380867+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3512109322 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2440490727 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/4148120600 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6828/1862820759 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2446527261 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1353385001 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/1702319761 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6829/1862820759 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6827/1351191990 2024-04-03T04:56:43.851170+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2939436653 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4149690148 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/39315361 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2814127860 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/110321404 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4005781191 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3239652190 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2716917447 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2937797857 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/546971533 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2778101694 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6826/1351191990 2024-04-03T04:56:43.851170+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/146104126 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2440490727 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2814127860 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/1197360681 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2760078492 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3184091580 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:45.294 INFO:teuthology.orchestra.run.smithi070.stderr:listed 43 entries 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa4ac06dd30 msgr2=0x7fa4ac0701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa4ac06dd30 0x7fa4ac0701f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0005fd0 tx=0x7fa4b000b390 comp rx=0 tx=0).stop 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c0103f20 msgr2=0x7fa4c0197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c0103f20 0x7fa4c0197070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009230 tx=0x7fa4a8004040 comp rx=0 tx=0).stop 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 shutdown_connections 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fa4ac06dd30 0x7fa4ac0701f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c0103f20 0x7fa4c0197070 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 --2- 172.21.15.70:0/1555704954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4c00fefe0 0x7fa4c0196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 >> 172.21.15.70:0/1555704954 conn(0x7fa4c0075b40 msgr2=0x7fa4c00fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 shutdown_connections 2024-04-02T05:00:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.288+0000 7fa4c5b67640 1 -- 172.21.15.70:0/1555704954 wait complete. 2024-04-02T05:00:45.303 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-02T05:00:45.303 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T05:00:45.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T05:00:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.460+0000 7fb082abd640 1 -- 172.21.15.70:0/490291400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c100280 msgr2=0x7fb07c100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.460+0000 7fb082abd640 1 --2- 172.21.15.70:0/490291400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c100280 0x7fb07c100660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb070009880 tx=0x7fb07002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.460+0000 7fb082abd640 1 -- 172.21.15.70:0/490291400 shutdown_connections 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.460+0000 7fb082abd640 1 --2- 172.21.15.70:0/490291400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb07c100c30 0x7fb07c104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.460+0000 7fb082abd640 1 --2- 172.21.15.70:0/490291400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c100280 0x7fb07c100660 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.460+0000 7fb082abd640 1 -- 172.21.15.70:0/490291400 >> 172.21.15.70:0/490291400 conn(0x7fb07c0fbb20 msgr2=0x7fb07c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.460+0000 7fb082abd640 1 -- 172.21.15.70:0/490291400 shutdown_connections 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.460+0000 7fb082abd640 1 -- 172.21.15.70:0/490291400 wait complete. 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb082abd640 1 Processor -- start 2024-04-02T05:00:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb082abd640 1 -- start start 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb082abd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb07c100280 0x7fb07c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb082abd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c198400 0x7fb07c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb082abd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c198a00 con 0x7fb07c198400 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb082abd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c198b70 con 0x7fb07c100280 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb07bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c198400 0x7fb07c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb07bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c198400 0x7fb07c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56818/0 (socket says 172.21.15.70:56818) 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb07bfff640 1 -- 172.21.15.70:0/3148223957 learned_addr learned my addr 172.21.15.70:0/3148223957 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb080859640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb07c100280 0x7fb07c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb07bfff640 1 -- 172.21.15.70:0/3148223957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb07c100280 msgr2=0x7fb07c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb07bfff640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb07c100280 0x7fb07c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.461+0000 7fb07bfff640 1 -- 172.21.15.70:0/3148223957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb070009560 con 0x7fb07c198400 2024-04-02T05:00:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.462+0000 7fb07bfff640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c198400 0x7fb07c19c860 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb06800e9a0 tx=0x7fb06800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.462+0000 7fb079ffb640 1 -- 172.21.15.70:0/3148223957 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06800cc90 con 0x7fb07c198400 2024-04-02T05:00:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.462+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c19ce60 con 0x7fb07c198400 2024-04-02T05:00:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.462+0000 7fb079ffb640 1 -- 172.21.15.70:0/3148223957 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb06800cdf0 con 0x7fb07c198400 2024-04-02T05:00:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.462+0000 7fb079ffb640 1 -- 172.21.15.70:0/3148223957 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb068010430 con 0x7fb07c198400 2024-04-02T05:00:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.462+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c19d3c0 con 0x7fb07c198400 2024-04-02T05:00:45.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.463+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c19d6a0 con 0x7fb07c198400 2024-04-02T05:00:45.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.463+0000 7fb079ffb640 1 -- 172.21.15.70:0/3148223957 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb068005020 con 0x7fb07c198400 2024-04-02T05:00:45.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.464+0000 7fb079ffb640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb06c06dd90 0x7fb06c070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.464+0000 7fb079ffb640 1 -- 172.21.15.70:0/3148223957 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fb068014070 con 0x7fb07c198400 2024-04-02T05:00:45.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.464+0000 7fb080859640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb06c06dd90 0x7fb06c070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.464+0000 7fb080859640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb06c06dd90 0x7fb06c070250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb070006010 tx=0x7fb070000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:45.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.470+0000 7fb079ffb640 1 -- 172.21.15.70:0/3148223957 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb068057670 con 0x7fb07c198400 2024-04-02T05:00:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:45 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:00:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:45 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3614054296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T05:00:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:45 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:00:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:45 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:00:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:45 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3058213844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T05:00:45.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:45 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1555704954' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T05:00:45.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.681+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb07c04e040 con 0x7fb07c198400 2024-04-02T05:00:45.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.683+0000 7fb079ffb640 1 -- 172.21.15.70:0/3148223957 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17334 (secure 0 0 0) 0x7fb06805ad30 con 0x7fb07c198400 2024-04-02T05:00:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-02T05:00:45.687 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":95,"fsid":"a9031cd8-f0aa-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:37:49.993595+0000","modified":"2024-04-02T04:58:01.605043+0000","last_up_change":"2024-04-02T04:56:39.019903+0000","last_in_change":"2024-04-02T04:43:12.453452+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:42:18.484513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-02T04:44:51.427296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-02T04:44:52.024713+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b5e5510-c4db-4073-b470-c777b557d77e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":90,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6803","nonce":4011619577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6805","nonce":4011619577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6809","nonce":4011619577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":4011619577},{"type":"v1","addr":"172.21.15.70:6807","nonce":4011619577}]},"public_addr":"172.21.15.70:6803/4011619577","cluster_addr":"172.21.15.70:6805/4011619577","heartbeat_back_addr":"172.21.15.70:6809/4011619577","heartbeat_front_addr":"172.21.15.70:6807/4011619577","state":["exists","up"]},{"osd":1,"uuid":"b751ceaf-b734-4280-ba22-82683b9776ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":64,"up_thru":90,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6811","nonce":2318524655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6813","nonce":2318524655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6817","nonce":2318524655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2318524655},{"type":"v1","addr":"172.21.15.70:6815","nonce":2318524655}]},"public_addr":"172.21.15.70:6811/2318524655","cluster_addr":"172.21.15.70:6813/2318524655","heartbeat_back_addr":"172.21.15.70:6817/2318524655","heartbeat_front_addr":"172.21.15.70:6815/2318524655","state":["exists","up"]},{"osd":2,"uuid":"32080ab6-7809-4523-aab7-ca7a6d263ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6819","nonce":347493648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6821","nonce":347493648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6825","nonce":347493648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":347493648},{"type":"v1","addr":"172.21.15.70:6823","nonce":347493648}]},"public_addr":"172.21.15.70:6819/347493648","cluster_addr":"172.21.15.70:6821/347493648","heartbeat_back_addr":"172.21.15.70:6825/347493648","heartbeat_front_addr":"172.21.15.70:6823/347493648","state":["exists","up"]},{"osd":3,"uuid":"0d71a0bb-5e7a-4339-afe7-06174dea4986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6801","nonce":657192449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6803","nonce":657192449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6807","nonce":657192449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":657192449},{"type":"v1","addr":"172.21.15.179:6805","nonce":657192449}]},"public_addr":"172.21.15.179:6801/657192449","cluster_addr":"172.21.15.179:6803/657192449","heartbeat_back_addr":"172.21.15.179:6807/657192449","heartbeat_front_addr":"172.21.15.179:6805/657192449","state":["exists","up"]},{"osd":4,"uuid":"754e6f09-2a8e-42e2-a998-5f5a1addcb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6809","nonce":229347770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6811","nonce":229347770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6815","nonce":229347770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":229347770},{"type":"v1","addr":"172.21.15.179:6813","nonce":229347770}]},"public_addr":"172.21.15.179:6809/229347770","cluster_addr":"172.21.15.179:6811/229347770","heartbeat_back_addr":"172.21.15.179:6815/229347770","heartbeat_front_addr":"172.21.15.179:6813/229347770","state":["exists","up"]},{"osd":5,"uuid":"46a73805-a770-4ed1-96f4-e64cf88810eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6817","nonce":2533703533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6819","nonce":2533703533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6823","nonce":2533703533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2533703533},{"type":"v1","addr":"172.21.15.179:6821","nonce":2533703533}]},"public_addr":"172.21.15.179:6817/2533703533","cluster_addr":"172.21.15.179:6819/2533703533","heartbeat_back_addr":"172.21.15.179:6823/2533703533","heartbeat_front_addr":"172.21.15.179:6821/2533703533","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:17.145441+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:41:44.622049+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:14.717560+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:42:38.493463+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:04.319117+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:43:30.171864+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.15","mappings":[{"from":0,"to":5}]},{"pgid":"2.16","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1f","mappings":[{"from":0,"to":1},{"from":3,"to":2}]},{"pgid":"3.3","mappings":[{"from":4,"to":1}]},{"pgid":"3.11","mappings":[{"from":3,"to":1}]},{"pgid":"3.13","mappings":[{"from":3,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.179:6827/3541837059":"2024-04-03T04:58:01.605007+0000","172.21.15.179:6824/511108019":"2024-04-03T04:57:35.242205+0000","172.21.15.179:6826/3541837059":"2024-04-03T04:58:01.605007+0000","172.21.15.70:6829/2586361462":"2024-04-03T04:57:09.380867+0000","172.21.15.70:6800/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1721127786":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/1700662291":"2024-04-03T04:47:27.805482+0000","172.21.15.179:0/1433338238":"2024-04-03T04:48:41.679801+0000","172.21.15.179:0/1900906037":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6801/146104126":"2024-04-03T04:52:11.062737+0000","172.21.15.179:6825/511108019":"2024-04-03T04:57:35.242205+0000","172.21.15.70:6800/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/2463094374":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/940716355":"2024-04-03T04:38:18.397033+0000","172.21.15.179:0/1600665743":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6828/2586361462":"2024-04-03T04:57:09.380867+0000","172.21.15.70:0/3512109322":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.179:0/4148120600":"2024-04-03T04:48:41.679801+0000","172.21.15.179:6828/1862820759":"2024-04-03T04:48:41.679801+0000","172.21.15.70:0/2446527261":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/1353385001":"2024-04-03T04:47:27.805482+0000","172.21.15.179:0/1702319761":"2024-04-03T04:48:41.679801+0000","172.21.15.179:6829/1862820759":"2024-04-03T04:48:41.679801+0000","172.21.15.70:6827/1351191990":"2024-04-03T04:56:43.851170+0000","172.21.15.70:6801/2939436653":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/4149690148":"2024-04-03T04:38:51.244832+0000","172.21.15.70:0/39315361":"2024-04-03T04:47:27.805482+0000","172.21.15.70:6801/2814127860":"2024-04-03T04:47:27.805482+0000","172.21.15.70:0/110321404":"2024-04-03T04:39:58.163164+0000","172.21.15.70:0/4005781191":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/3239652190":"2024-04-03T04:52:11.062737+0000","172.21.15.70:0/2716917447":"2024-04-03T04:47:27.805482+0000","172.21.15.70:0/2937797857":"2024-04-03T04:39:58.163164+0000","172.21.15.179:0/546971533":"2024-04-03T04:48:41.679801+0000","172.21.15.70:0/2778101694":"2024-04-03T04:52:11.062737+0000","172.21.15.70:6826/1351191990":"2024-04-03T04:56:43.851170+0000","172.21.15.70:6800/146104126":"2024-04-03T04:52:11.062737+0000","172.21.15.70:6801/2440490727":"2024-04-03T04:39:58.163164+0000","172.21.15.70:6800/2814127860":"2024-04-03T04:47:27.805482+0000","172.21.15.70:6801/1197360681":"2024-04-03T04:38:18.397033+0000","172.21.15.70:0/2760078492":"2024-04-03T04:52:11.062737+0000","172.21.15.70:0/3184091580":"2024-04-03T04:52:11.062737+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T05:00:45.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb06c06dd90 msgr2=0x7fb06c070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb06c06dd90 0x7fb06c070250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb070006010 tx=0x7fb070000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c198400 msgr2=0x7fb07c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c198400 0x7fb07c19c860 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb06800e9a0 tx=0x7fb06800ee60 comp rx=0 tx=0).stop 2024-04-02T05:00:45.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 shutdown_connections 2024-04-02T05:00:45.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fb06c06dd90 0x7fb06c070250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb07c198400 0x7fb07c19c860 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 --2- 172.21.15.70:0/3148223957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb07c100280 0x7fb07c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 >> 172.21.15.70:0/3148223957 conn(0x7fb07c0fbb20 msgr2=0x7fb07c0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:45.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 shutdown_connections 2024-04-02T05:00:45.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.685+0000 7fb082abd640 1 -- 172.21.15.70:0/3148223957 wait complete. 2024-04-02T05:00:45.699 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:45 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:00:45.699 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:45 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3614054296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T05:00:45.699 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:45 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:00:45.699 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:45 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:00:45.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:45 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3058213844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T05:00:45.700 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:45 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1555704954' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T05:00:45.701 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:42:18.484513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-02T04:44:51.427296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-02T04:44:52.024713+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T05:00:45.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T05:00:45.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.804+0000 7f76b89b6640 1 -- 172.21.15.70:0/1463689565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 msgr2=0x7f76b40ff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.804+0000 7f76b89b6640 1 --2- 172.21.15.70:0/1463689565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 0x7f76b40ff590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009c20 tx=0x7f76a402c1b0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.804+0000 7f76b89b6640 1 -- 172.21.15.70:0/1463689565 shutdown_connections 2024-04-02T05:00:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.804+0000 7f76b89b6640 1 --2- 172.21.15.70:0/1463689565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76b40ffb60 0x7f76b4103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.804+0000 7f76b89b6640 1 --2- 172.21.15.70:0/1463689565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 0x7f76b40ff590 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.804+0000 7f76b89b6640 1 -- 172.21.15.70:0/1463689565 >> 172.21.15.70:0/1463689565 conn(0x7f76b40faa30 msgr2=0x7f76b40fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.804+0000 7f76b89b6640 1 -- 172.21.15.70:0/1463689565 shutdown_connections 2024-04-02T05:00:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.804+0000 7f76b89b6640 1 -- 172.21.15.70:0/1463689565 wait complete. 2024-04-02T05:00:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b89b6640 1 Processor -- start 2024-04-02T05:00:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b89b6640 1 -- start start 2024-04-02T05:00:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b89b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 0x7f76b4194d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b89b6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76b41952a0 0x7f76b4199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b89b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b41958a0 con 0x7f76b40ff1b0 2024-04-02T05:00:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b89b6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b4195a10 con 0x7f76b41952a0 2024-04-02T05:00:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 0x7f76b4194d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 0x7f76b4194d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56836/0 (socket says 172.21.15.70:56836) 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.805+0000 7f76b259c640 1 -- 172.21.15.70:0/1951064532 learned_addr learned my addr 172.21.15.70:0/1951064532 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76b1d9b640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76b41952a0 0x7f76b4199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76b259c640 1 -- 172.21.15.70:0/1951064532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76b41952a0 msgr2=0x7f76b4199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76b259c640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76b41952a0 0x7f76b4199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76b259c640 1 -- 172.21.15.70:0/1951064532 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a4009900 con 0x7f76b40ff1b0 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76b259c640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 0x7f76b4194d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f76a4006010 tx=0x7f76a4032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76a37fe640 1 -- 172.21.15.70:0/1951064532 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76a4034820 con 0x7f76b40ff1b0 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76a37fe640 1 -- 172.21.15.70:0/1951064532 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76a4003e30 con 0x7f76b40ff1b0 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b4199ca0 con 0x7f76b40ff1b0 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76a37fe640 1 -- 172.21.15.70:0/1951064532 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76a4004830 con 0x7f76b40ff1b0 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.806+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b419a200 con 0x7f76b40ff1b0 2024-04-02T05:00:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.807+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b404e040 con 0x7f76b40ff1b0 2024-04-02T05:00:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.808+0000 7f76a37fe640 1 -- 172.21.15.70:0/1951064532 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f76a403b040 con 0x7f76b40ff1b0 2024-04-02T05:00:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.808+0000 7f76a37fe640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f769c06dcd0 0x7f769c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.808+0000 7f76a37fe640 1 -- 172.21.15.70:0/1951064532 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f76a40b0aa0 con 0x7f76b40ff1b0 2024-04-02T05:00:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.808+0000 7f76b1d9b640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f769c06dcd0 0x7f769c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.809+0000 7f76b1d9b640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f769c06dcd0 0x7f769c070190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005fd0 tx=0x7f76a8005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:45.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.813+0000 7f76a37fe640 1 -- 172.21.15.70:0/1951064532 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f76a4079e80 con 0x7f76b40ff1b0 2024-04-02T05:00:45.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.990+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76b419a420 con 0x7f76b40ff1b0 2024-04-02T05:00:45.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.990+0000 7f76a37fe640 1 -- 172.21.15.70:0/1951064532 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76a407d540 con 0x7f76b40ff1b0 2024-04-02T05:00:45.995 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-04-02T05:00:45.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f769c06dcd0 msgr2=0x7f769c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f769c06dcd0 0x7f769c070190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005fd0 tx=0x7f76a8005850 comp rx=0 tx=0).stop 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 msgr2=0x7f76b4194d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 0x7f76b4194d60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f76a4006010 tx=0x7f76a4032b60 comp rx=0 tx=0).stop 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 shutdown_connections 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76b41952a0 0x7f76b4199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f769c06dcd0 0x7f769c070190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 --2- 172.21.15.70:0/1951064532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b40ff1b0 0x7f76b4194d60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 >> 172.21.15.70:0/1951064532 conn(0x7f76b40faa30 msgr2=0x7f76b40fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 shutdown_connections 2024-04-02T05:00:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:45.992+0000 7f76b89b6640 1 -- 172.21.15.70:0/1951064532 wait complete. 2024-04-02T05:00:46.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-02T05:00:46.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 -- 172.21.15.70:0/2676251878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ffc10 msgr2=0x7ff358103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 --2- 172.21.15.70:0/2676251878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ffc10 0x7ff358103b70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009880 tx=0x7ff34c02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 -- 172.21.15.70:0/2676251878 shutdown_connections 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 --2- 172.21.15.70:0/2676251878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ffc10 0x7ff358103b70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 --2- 172.21.15.70:0/2676251878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3580ff260 0x7ff3580ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 -- 172.21.15.70:0/2676251878 >> 172.21.15.70:0/2676251878 conn(0x7ff358074a50 msgr2=0x7ff358074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 -- 172.21.15.70:0/2676251878 shutdown_connections 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 -- 172.21.15.70:0/2676251878 wait complete. 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 Processor -- start 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 -- start start 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3580ff260 0x7ff358194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff358194ec0 0x7ff358195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580682f0 con 0x7ff358194ec0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff35f20e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3581a1ff0 con 0x7ff3580ff260 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.112+0000 7ff357fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff358194ec0 0x7ff358195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff357fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff358194ec0 0x7ff358195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56848/0 (socket says 172.21.15.70:56848) 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff357fff640 1 -- 172.21.15.70:0/2758307009 learned_addr learned my addr 172.21.15.70:0/2758307009 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff35cfaa640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3580ff260 0x7ff358194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff357fff640 1 -- 172.21.15.70:0/2758307009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3580ff260 msgr2=0x7ff358194980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff357fff640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3580ff260 0x7ff358194980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff357fff640 1 -- 172.21.15.70:0/2758307009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c009560 con 0x7ff358194ec0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff35cfaa640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3580ff260 0x7ff358194980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff357fff640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff358194ec0 0x7ff358195340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009230 tx=0x7ff34c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff355ffb640 1 -- 172.21.15.70:0/2758307009 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34c0037c0 con 0x7ff358194ec0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3581a2190 con 0x7ff358194ec0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff355ffb640 1 -- 172.21.15.70:0/2758307009 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff34c003de0 con 0x7ff358194ec0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff355ffb640 1 -- 172.21.15.70:0/2758307009 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34c034440 con 0x7ff358194ec0 2024-04-02T05:00:46.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.113+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3581a26a0 con 0x7ff358194ec0 2024-04-02T05:00:46.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.114+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35804e040 con 0x7ff358194ec0 2024-04-02T05:00:46.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.115+0000 7ff355ffb640 1 -- 172.21.15.70:0/2758307009 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff34c004fb0 con 0x7ff358194ec0 2024-04-02T05:00:46.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.115+0000 7ff355ffb640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff34006dcd0 0x7ff340070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.115+0000 7ff355ffb640 1 -- 172.21.15.70:0/2758307009 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7ff34c0b01c0 con 0x7ff358194ec0 2024-04-02T05:00:46.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.115+0000 7ff35cfaa640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff34006dcd0 0x7ff340070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.116+0000 7ff35cfaa640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff34006dcd0 0x7ff340070190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff350005c40 tx=0x7ff35000a430 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:46.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.120+0000 7ff355ffb640 1 -- 172.21.15.70:0/2758307009 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff34c0795a0 con 0x7ff358194ec0 2024-04-02T05:00:46.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.364+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff358067940 con 0x7ff358194ec0 2024-04-02T05:00:46.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.365+0000 7ff355ffb640 1 -- 172.21.15.70:0/2758307009 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff34c07cc60 con 0x7ff358194ec0 2024-04-02T05:00:46.369 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff34006dcd0 msgr2=0x7ff340070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff34006dcd0 0x7ff340070190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff350005c40 tx=0x7ff35000a430 comp rx=0 tx=0).stop 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff358194ec0 msgr2=0x7ff358195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff358194ec0 0x7ff358195340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009230 tx=0x7ff34c004270 comp rx=0 tx=0).stop 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 shutdown_connections 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7ff34006dcd0 0x7ff340070190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff358194ec0 0x7ff358195340 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 --2- 172.21.15.70:0/2758307009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3580ff260 0x7ff358194980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 >> 172.21.15.70:0/2758307009 conn(0x7ff358074a50 msgr2=0x7ff3580fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 shutdown_connections 2024-04-02T05:00:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.367+0000 7ff35f20e640 1 -- 172.21.15.70:0/2758307009 wait complete. 2024-04-02T05:00:46.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-02T05:00:46.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.483+0000 7fdaa7a66640 1 -- 172.21.15.70:0/2742551727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa0067f70 msgr2=0x7fdaa0104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.483+0000 7fdaa7a66640 1 --2- 172.21.15.70:0/2742551727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa0067f70 0x7fdaa0104c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fda90009880 tx=0x7fda9002bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.483+0000 7fdaa7a66640 1 -- 172.21.15.70:0/2742551727 shutdown_connections 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.483+0000 7fdaa7a66640 1 --2- 172.21.15.70:0/2742551727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa0067f70 0x7fdaa0104c70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.483+0000 7fdaa7a66640 1 --2- 172.21.15.70:0/2742551727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa00675c0 0x7fdaa00679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.483+0000 7fdaa7a66640 1 -- 172.21.15.70:0/2742551727 >> 172.21.15.70:0/2742551727 conn(0x7fdaa0074a50 msgr2=0x7fdaa0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.483+0000 7fdaa7a66640 1 -- 172.21.15.70:0/2742551727 shutdown_connections 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.483+0000 7fdaa7a66640 1 -- 172.21.15.70:0/2742551727 wait complete. 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa7a66640 1 Processor -- start 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa7a66640 1 -- start start 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa7a66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa00675c0 0x7fdaa0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa7a66640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa01962a0 0x7fdaa019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa7a66640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01968a0 con 0x7fdaa00675c0 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa7a66640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa0196a10 con 0x7fdaa01962a0 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa5802640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa00675c0 0x7fdaa0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa5802640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa00675c0 0x7fdaa0195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56868/0 (socket says 172.21.15.70:56868) 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa5802640 1 -- 172.21.15.70:0/3828421912 learned_addr learned my addr 172.21.15.70:0/3828421912 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa5001640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa01962a0 0x7fdaa019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.484+0000 7fdaa5802640 1 -- 172.21.15.70:0/3828421912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa01962a0 msgr2=0x7fdaa019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.485+0000 7fdaa5802640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa01962a0 0x7fdaa019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.485+0000 7fdaa5802640 1 -- 172.21.15.70:0/3828421912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90009560 con 0x7fdaa00675c0 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.485+0000 7fdaa5802640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa00675c0 0x7fdaa0195d60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00d900 tx=0x7fda9c00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.485+0000 7fda96ffd640 1 -- 172.21.15.70:0/3828421912 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9c009c00 con 0x7fdaa00675c0 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.485+0000 7fda96ffd640 1 -- 172.21.15.70:0/3828421912 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda9c009d60 con 0x7fdaa00675c0 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.485+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa019ad00 con 0x7fdaa00675c0 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.485+0000 7fda96ffd640 1 -- 172.21.15.70:0/3828421912 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9c0049d0 con 0x7fdaa00675c0 2024-04-02T05:00:46.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.485+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa019b260 con 0x7fdaa00675c0 2024-04-02T05:00:46.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.486+0000 7fda96ffd640 1 -- 172.21.15.70:0/3828421912 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fda9c004020 con 0x7fdaa00675c0 2024-04-02T05:00:46.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.486+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa004e040 con 0x7fdaa00675c0 2024-04-02T05:00:46.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.487+0000 7fda96ffd640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fda8c06dcd0 0x7fda8c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.487+0000 7fda96ffd640 1 -- 172.21.15.70:0/3828421912 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7fda9c08d980 con 0x7fdaa00675c0 2024-04-02T05:00:46.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.487+0000 7fdaa5001640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fda8c06dcd0 0x7fda8c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.487+0000 7fdaa5001640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fda8c06dcd0 0x7fda8c070190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda90006010 tx=0x7fda90000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:46.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.491+0000 7fda96ffd640 1 -- 172.21.15.70:0/3828421912 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fda9c056de0 con 0x7fdaa00675c0 2024-04-02T05:00:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:46 smithi179 ceph-mon[84288]: pgmap v306: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:00:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:00:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:00:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:46 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3148223957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T05:00:46.573 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:46 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/1951064532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T05:00:46.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.665+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fdaa0197110 con 0x7fdaa00675c0 2024-04-02T05:00:46.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.665+0000 7fda96ffd640 1 -- 172.21.15.70:0/3828421912 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v95) v1 ==== 107+0+11 (secure 0 0 0) 0x7fda9c05a4a0 con 0x7fdaa00675c0 2024-04-02T05:00:46.670 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-04-02T05:00:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fda8c06dcd0 msgr2=0x7fda8c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fda8c06dcd0 0x7fda8c070190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda90006010 tx=0x7fda90000bc0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa00675c0 msgr2=0x7fdaa0195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa00675c0 0x7fdaa0195d60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00d900 tx=0x7fda9c00ddc0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 shutdown_connections 2024-04-02T05:00:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa01962a0 0x7fdaa019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7fda8c06dcd0 0x7fda8c070190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 --2- 172.21.15.70:0/3828421912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdaa00675c0 0x7fdaa0195d60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 >> 172.21.15.70:0/3828421912 conn(0x7fdaa0074a50 msgr2=0x7fdaa00fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 shutdown_connections 2024-04-02T05:00:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.667+0000 7fdaa7a66640 1 -- 172.21.15.70:0/3828421912 wait complete. 2024-04-02T05:00:46.684 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:46 smithi070 ceph-mon[113635]: pgmap v306: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:46.684 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:00:46.684 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:00:46.684 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:00:46.684 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:46 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3148223957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T05:00:46.684 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:46 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/1951064532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T05:00:46.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T05:00:46.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.787+0000 7f31a3d47640 1 -- 172.21.15.70:0/2213302220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ffb20 msgr2=0x7f319c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.787+0000 7f31a3d47640 1 --2- 172.21.15.70:0/2213302220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ffb20 0x7f319c103b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f318c009880 tx=0x7f318c02bce0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.787+0000 7f31a3d47640 1 -- 172.21.15.70:0/2213302220 shutdown_connections 2024-04-02T05:00:46.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.787+0000 7f31a3d47640 1 --2- 172.21.15.70:0/2213302220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ffb20 0x7f319c103b60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.787+0000 7f31a3d47640 1 --2- 172.21.15.70:0/2213302220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c0ff170 0x7f319c0ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.787+0000 7f31a3d47640 1 -- 172.21.15.70:0/2213302220 >> 172.21.15.70:0/2213302220 conn(0x7f319c0fa9d0 msgr2=0x7f319c0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.787+0000 7f31a3d47640 1 -- 172.21.15.70:0/2213302220 shutdown_connections 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.787+0000 7f31a3d47640 1 -- 172.21.15.70:0/2213302220 wait complete. 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a3d47640 1 Processor -- start 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a3d47640 1 -- start start 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a3d47640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ff170 0x7f319c196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a3d47640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c197070 0x7f319c1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a3d47640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c19fba0 con 0x7f319c0ff170 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a3d47640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c19fce0 con 0x7f319c197070 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a1ae3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ff170 0x7f319c196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a1ae3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ff170 0x7f319c196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53328/0 (socket says 172.21.15.70:53328) 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a1ae3640 1 -- 172.21.15.70:0/3177433570 learned_addr learned my addr 172.21.15.70:0/3177433570 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-02T05:00:46.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a12e2640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c197070 0x7f319c1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a1ae3640 1 -- 172.21.15.70:0/3177433570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c197070 msgr2=0x7f319c1974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a1ae3640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c197070 0x7f319c1974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.788+0000 7f31a1ae3640 1 -- 172.21.15.70:0/3177433570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c009560 con 0x7f319c0ff170 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.789+0000 7f31a1ae3640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ff170 0x7f319c196b30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f319800d720 tx=0x7f319800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.789+0000 7f3192ffd640 1 -- 172.21.15.70:0/3177433570 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31980051f0 con 0x7f319c0ff170 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.789+0000 7f3192ffd640 1 -- 172.21.15.70:0/3177433570 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3198009ce0 con 0x7f319c0ff170 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.789+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c19ff40 con 0x7f319c0ff170 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.789+0000 7f3192ffd640 1 -- 172.21.15.70:0/3177433570 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3198010460 con 0x7f319c0ff170 2024-04-02T05:00:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.789+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c1a03c0 con 0x7f319c0ff170 2024-04-02T05:00:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.790+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319c04e040 con 0x7f319c0ff170 2024-04-02T05:00:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.790+0000 7f3192ffd640 1 -- 172.21.15.70:0/3177433570 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f3198005350 con 0x7f319c0ff170 2024-04-02T05:00:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.791+0000 7f3192ffd640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f318806dd90 0x7f3188070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T05:00:46.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.791+0000 7f31a12e2640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f318806dd90 0x7f3188070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T05:00:46.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.791+0000 7f3192ffd640 1 -- 172.21.15.70:0/3177433570 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6941+0+0 (secure 0 0 0) 0x7f319808d730 con 0x7f319c0ff170 2024-04-02T05:00:46.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.791+0000 7f31a12e2640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f318806dd90 0x7f3188070250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T05:00:46.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.799+0000 7f3192ffd640 1 -- 172.21.15.70:0/3177433570 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3198056c40 con 0x7f319c0ff170 2024-04-02T05:00:46.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.991+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f319c068a00 con 0x7f319c0ff170 2024-04-02T05:00:46.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.991+0000 7f3192ffd640 1 -- 172.21.15.70:0/3177433570 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v95) v1 ==== 81+0+2557 (secure 0 0 0) 0x7f3198017a10 con 0x7f319c0ff170 2024-04-02T05:00:46.996 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6827/3541837059 2024-04-03T04:58:01.605007+0000 2024-04-02T05:00:46.996 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6824/511108019 2024-04-03T04:57:35.242205+0000 2024-04-02T05:00:46.996 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6826/3541837059 2024-04-03T04:58:01.605007+0000 2024-04-02T05:00:46.996 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6829/2586361462 2024-04-03T04:57:09.380867+0000 2024-04-02T05:00:46.996 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2939436653 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:46.996 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1721127786 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:46.996 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1700662291 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:46.996 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/1433338238 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/1900906037 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/146104126 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6825/511108019 2024-04-03T04:57:35.242205+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/1197360681 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2463094374 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/940716355 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/1600665743 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6828/2586361462 2024-04-03T04:57:09.380867+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3512109322 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2440490727 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/4148120600 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6828/1862820759 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2446527261 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1353385001 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:46.997 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/1702319761 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:6829/1862820759 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6827/1351191990 2024-04-03T04:56:43.851170+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2939436653 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4149690148 2024-04-03T04:38:51.244832+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/39315361 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2814127860 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/110321404 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/4005781191 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3239652190 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2716917447 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2937797857 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:46.998 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.179:0/546971533 2024-04-03T04:48:41.679801+0000 2024-04-02T05:00:46.999 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2778101694 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:46.999 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6826/1351191990 2024-04-03T04:56:43.851170+0000 2024-04-02T05:00:46.999 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/146104126 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:46.999 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2440490727 2024-04-03T04:39:58.163164+0000 2024-04-02T05:00:46.999 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2814127860 2024-04-03T04:47:27.805482+0000 2024-04-02T05:00:46.999 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/1197360681 2024-04-03T04:38:18.397033+0000 2024-04-02T05:00:46.999 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2760078492 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:46.999 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3184091580 2024-04-03T04:52:11.062737+0000 2024-04-02T05:00:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:listed 43 entries 2024-04-02T05:00:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f318806dd90 msgr2=0x7f3188070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f318806dd90 0x7f3188070250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c005fd0 comp rx=0 tx=0).stop 2024-04-02T05:00:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ff170 msgr2=0x7f319c196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T05:00:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ff170 0x7f319c196b30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f319800d720 tx=0x7f319800dbe0 comp rx=0 tx=0).stop 2024-04-02T05:00:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 shutdown_connections 2024-04-02T05:00:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c197070 0x7f319c1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:6800/2021202469,v1:172.21.15.70:6801/2021202469] conn(0x7f318806dd90 0x7f3188070250 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:47.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 --2- 172.21.15.70:0/3177433570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319c0ff170 0x7f319c196b30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T05:00:47.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.993+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 >> 172.21.15.70:0/3177433570 conn(0x7f319c0fa9d0 msgr2=0x7f319c0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T05:00:47.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.994+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 shutdown_connections 2024-04-02T05:00:47.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-02T05:00:46.994+0000 7f31a3d47640 1 -- 172.21.15.70:0/3177433570 wait complete. 2024-04-02T05:00:47.009 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2024-04-02T05:00:47.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T05:00:47.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-02T05:00:47.036 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T05:00:47.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:47 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/2758307009' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T05:00:47.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:47 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3828421912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T05:00:47.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:47 smithi070 ceph-mon[113635]: from='client.? 172.21.15.70:0/3177433570' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T05:00:47.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:47 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/2758307009' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T05:00:47.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:47 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3828421912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T05:00:47.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:47 smithi179 ceph-mon[84288]: from='client.? 172.21.15.70:0/3177433570' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T05:00:48.608 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:48 smithi179 ceph-mon[84288]: pgmap v307: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:48 smithi070 ceph-mon[113635]: pgmap v307: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:50.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:50 smithi070 ceph-mon[113635]: pgmap v308: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:50.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:50 smithi179 ceph-mon[84288]: pgmap v308: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:52.604 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:52 smithi070 ceph-mon[113635]: pgmap v309: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:52.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:52 smithi179 ceph-mon[84288]: pgmap v309: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:54.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:54 smithi070 ceph-mon[113635]: pgmap v310: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:54.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:54 smithi179 ceph-mon[84288]: pgmap v310: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:56.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:56 smithi070 ceph-mon[113635]: pgmap v311: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:56.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:00:56.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:56 smithi179 ceph-mon[84288]: pgmap v311: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:56.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:00:58.706 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:00:58 smithi179 ceph-mon[84288]: pgmap v312: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:00:58.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:00:58 smithi070 ceph-mon[113635]: pgmap v312: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:00 smithi070 ceph-mon[113635]: pgmap v313: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:00.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:00 smithi179 ceph-mon[84288]: pgmap v313: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:02.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:02 smithi070 ceph-mon[113635]: pgmap v314: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:02.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:02 smithi179 ceph-mon[84288]: pgmap v314: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:04.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:04 smithi070 ceph-mon[113635]: pgmap v315: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:04.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:04 smithi179 ceph-mon[84288]: pgmap v315: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:06.626 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:06 smithi179 ceph-mon[84288]: pgmap v316: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:06.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:06 smithi070 ceph-mon[113635]: pgmap v316: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:08.682 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:08 smithi070 ceph-mon[113635]: pgmap v317: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:08.804 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:08 smithi179 ceph-mon[84288]: pgmap v317: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:10 smithi070 ceph-mon[113635]: pgmap v318: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:10.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:10 smithi179 ceph-mon[84288]: pgmap v318: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:11.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:01:11.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:01:12.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:12 smithi070 ceph-mon[113635]: pgmap v319: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:12.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:12 smithi179 ceph-mon[84288]: pgmap v319: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:13.731 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:13 smithi070 ceph-mon[113635]: pgmap v320: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:13.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:13 smithi179 ceph-mon[84288]: pgmap v320: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:16.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:16 smithi070 ceph-mon[113635]: pgmap v321: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:16.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:16 smithi179 ceph-mon[84288]: pgmap v321: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:18.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:18 smithi070 ceph-mon[113635]: pgmap v322: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:18.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:18 smithi179 ceph-mon[84288]: pgmap v322: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:20.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:20 smithi070 ceph-mon[113635]: pgmap v323: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:20 smithi179 ceph-mon[84288]: pgmap v323: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:22.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:22 smithi070 ceph-mon[113635]: pgmap v324: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:22.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:22 smithi179 ceph-mon[84288]: pgmap v324: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:24.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:24 smithi070 ceph-mon[113635]: pgmap v325: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:24.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:24 smithi179 ceph-mon[84288]: pgmap v325: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:26.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:26 smithi070 ceph-mon[113635]: pgmap v326: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:26.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:01:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:26 smithi179 ceph-mon[84288]: pgmap v326: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:01:28.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:28 smithi070 ceph-mon[113635]: pgmap v327: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:28.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:28 smithi179 ceph-mon[84288]: pgmap v327: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:30.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:30 smithi070 ceph-mon[113635]: pgmap v328: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:30.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:30 smithi179 ceph-mon[84288]: pgmap v328: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:32.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:32 smithi070 ceph-mon[113635]: pgmap v329: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:32 smithi179 ceph-mon[84288]: pgmap v329: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:34.322 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:34 smithi179 ceph-mon[84288]: pgmap v330: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:34.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:34 smithi070 ceph-mon[113635]: pgmap v330: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:36.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:36 smithi070 ceph-mon[113635]: pgmap v331: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:36.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:36 smithi179 ceph-mon[84288]: pgmap v331: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:38.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:38 smithi070 ceph-mon[113635]: pgmap v332: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:38.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:38 smithi179 ceph-mon[84288]: pgmap v332: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:40.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:40 smithi070 ceph-mon[113635]: pgmap v333: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:40.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:40 smithi179 ceph-mon[84288]: pgmap v333: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:41.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:01:41.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:01:42.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:42 smithi070 ceph-mon[113635]: pgmap v334: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:42.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:42 smithi179 ceph-mon[84288]: pgmap v334: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:44.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:44 smithi070 ceph-mon[113635]: pgmap v335: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:44.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:44 smithi179 ceph-mon[84288]: pgmap v335: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:46 smithi179 ceph-mon[84288]: pgmap v336: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:01:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:01:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:01:46.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:46 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:01:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:46 smithi070 ceph-mon[113635]: pgmap v336: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:01:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:01:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:01:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:46 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:01:48.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:48 smithi179 ceph-mon[84288]: pgmap v337: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:48 smithi070 ceph-mon[113635]: pgmap v337: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:50.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:50 smithi179 ceph-mon[84288]: pgmap v338: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:50.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:50 smithi070 ceph-mon[113635]: pgmap v338: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:52.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:52 smithi179 ceph-mon[84288]: pgmap v339: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:52.581 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:52 smithi070 ceph-mon[113635]: pgmap v339: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:54.152 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:53 smithi070 ceph-mon[113635]: pgmap v340: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:54.245 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:53 smithi179 ceph-mon[84288]: pgmap v340: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:56.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:56 smithi070 ceph-mon[113635]: pgmap v341: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:56.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:01:56.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:56 smithi179 ceph-mon[84288]: pgmap v341: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:01:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:01:58 smithi070 ceph-mon[113635]: pgmap v342: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:01:58.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:01:58 smithi179 ceph-mon[84288]: pgmap v342: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:00.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:00 smithi070 ceph-mon[113635]: pgmap v343: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:00.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:00 smithi179 ceph-mon[84288]: pgmap v343: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:02.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:02 smithi070 ceph-mon[113635]: pgmap v344: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:02.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:02 smithi179 ceph-mon[84288]: pgmap v344: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:04 smithi070 ceph-mon[113635]: pgmap v345: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:04.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:04 smithi179 ceph-mon[84288]: pgmap v345: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:06.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:06 smithi070 ceph-mon[113635]: pgmap v346: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:06.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:06 smithi179 ceph-mon[84288]: pgmap v346: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:08.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:08 smithi070 ceph-mon[113635]: pgmap v347: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:08.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:08 smithi179 ceph-mon[84288]: pgmap v347: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:10.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:10 smithi070 ceph-mon[113635]: pgmap v348: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:10.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:10 smithi179 ceph-mon[84288]: pgmap v348: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:11.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:02:11.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:02:12.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:12 smithi070 ceph-mon[113635]: pgmap v349: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:12.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:12 smithi179 ceph-mon[84288]: pgmap v349: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:14.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:14 smithi070 ceph-mon[113635]: pgmap v350: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:14.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:14 smithi179 ceph-mon[84288]: pgmap v350: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:16.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:16 smithi179 ceph-mon[84288]: pgmap v351: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:16.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:16 smithi070 ceph-mon[113635]: pgmap v351: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:18.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:18 smithi179 ceph-mon[84288]: pgmap v352: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:18 smithi070 ceph-mon[113635]: pgmap v352: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:20.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:20 smithi179 ceph-mon[84288]: pgmap v353: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:20.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:20 smithi070 ceph-mon[113635]: pgmap v353: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:22 smithi179 ceph-mon[84288]: pgmap v354: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:22.580 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:22 smithi070 ceph-mon[113635]: pgmap v354: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:24.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:24 smithi179 ceph-mon[84288]: pgmap v355: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:24.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:24 smithi070 ceph-mon[113635]: pgmap v355: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:26 smithi179 ceph-mon[84288]: pgmap v356: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:02:26.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:26 smithi070 ceph-mon[113635]: pgmap v356: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:26.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:02:28.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:28 smithi179 ceph-mon[84288]: pgmap v357: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:28.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:28 smithi070 ceph-mon[113635]: pgmap v357: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:30.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:30 smithi179 ceph-mon[84288]: pgmap v358: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:30.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:30 smithi070 ceph-mon[113635]: pgmap v358: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:32 smithi179 ceph-mon[84288]: pgmap v359: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:32.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:32 smithi070 ceph-mon[113635]: pgmap v359: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:34.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:34 smithi179 ceph-mon[84288]: pgmap v360: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:34.601 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:34 smithi070 ceph-mon[113635]: pgmap v360: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:36.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:36 smithi179 ceph-mon[84288]: pgmap v361: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:36.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:36 smithi070 ceph-mon[113635]: pgmap v361: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:38.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:38 smithi179 ceph-mon[84288]: pgmap v362: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:38.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:38 smithi070 ceph-mon[113635]: pgmap v362: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:40.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:40 smithi179 ceph-mon[84288]: pgmap v363: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:40 smithi070 ceph-mon[113635]: pgmap v363: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:41.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:02:41.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:02:42.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:42 smithi179 ceph-mon[84288]: pgmap v364: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:42 smithi070 ceph-mon[113635]: pgmap v364: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:44.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:43 smithi070 ceph-mon[113635]: pgmap v365: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:44.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:43 smithi179 ceph-mon[84288]: pgmap v365: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:46.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:46 smithi070 ceph-mon[113635]: pgmap v366: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:46.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:46 smithi179 ceph-mon[84288]: pgmap v366: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:02:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:02:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:02:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:02:47.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:02:47.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:02:47.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:02:47.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:02:48.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:48 smithi070 ceph-mon[113635]: pgmap v367: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:48.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:48 smithi179 ceph-mon[84288]: pgmap v367: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:50.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:50 smithi070 ceph-mon[113635]: pgmap v368: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:50.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:50 smithi179 ceph-mon[84288]: pgmap v368: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:52.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:52 smithi070 ceph-mon[113635]: pgmap v369: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:52.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:52 smithi179 ceph-mon[84288]: pgmap v369: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:54.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:54 smithi070 ceph-mon[113635]: pgmap v370: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:54.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:54 smithi179 ceph-mon[84288]: pgmap v370: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:56.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:56 smithi070 ceph-mon[113635]: pgmap v371: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:56.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:02:56.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:56 smithi179 ceph-mon[84288]: pgmap v371: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:56.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:02:58.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:02:58 smithi070 ceph-mon[113635]: pgmap v372: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:02:58.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:02:58 smithi179 ceph-mon[84288]: pgmap v372: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:00.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:00 smithi179 ceph-mon[84288]: pgmap v373: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:00 smithi070 ceph-mon[113635]: pgmap v373: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:02.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:02 smithi179 ceph-mon[84288]: pgmap v374: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:02.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:02 smithi070 ceph-mon[113635]: pgmap v374: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:04.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:04 smithi070 ceph-mon[113635]: pgmap v375: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:04.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:04 smithi179 ceph-mon[84288]: pgmap v375: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:06.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:06 smithi179 ceph-mon[84288]: pgmap v376: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:06.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:06 smithi070 ceph-mon[113635]: pgmap v376: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:08.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:08 smithi179 ceph-mon[84288]: pgmap v377: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:08.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:08 smithi070 ceph-mon[113635]: pgmap v377: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:10.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:10 smithi179 ceph-mon[84288]: pgmap v378: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:10 smithi070 ceph-mon[113635]: pgmap v378: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:11.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:03:11.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:03:12.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:12 smithi179 ceph-mon[84288]: pgmap v379: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:12.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:12 smithi070 ceph-mon[113635]: pgmap v379: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:14.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:14 smithi179 ceph-mon[84288]: pgmap v380: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:14.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:14 smithi070 ceph-mon[113635]: pgmap v380: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:16.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:16 smithi179 ceph-mon[84288]: pgmap v381: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:16.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:16 smithi070 ceph-mon[113635]: pgmap v381: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:18.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:18 smithi179 ceph-mon[84288]: pgmap v382: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:18 smithi070 ceph-mon[113635]: pgmap v382: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:20.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:20 smithi070 ceph-mon[113635]: pgmap v383: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:20.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:20 smithi179 ceph-mon[84288]: pgmap v383: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:22 smithi179 ceph-mon[84288]: pgmap v384: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:22 smithi070 ceph-mon[113635]: pgmap v384: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:24.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:24 smithi179 ceph-mon[84288]: pgmap v385: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:24.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:24 smithi070 ceph-mon[113635]: pgmap v385: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:26.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:26 smithi179 ceph-mon[84288]: pgmap v386: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:26.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:03:26.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:26 smithi070 ceph-mon[113635]: pgmap v386: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:26.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:03:28.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:28 smithi179 ceph-mon[84288]: pgmap v387: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:28.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:28 smithi070 ceph-mon[113635]: pgmap v387: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:30.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:30 smithi070 ceph-mon[113635]: pgmap v388: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:30.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:30 smithi179 ceph-mon[84288]: pgmap v388: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:32.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:32 smithi070 ceph-mon[113635]: pgmap v389: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:32.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:32 smithi179 ceph-mon[84288]: pgmap v389: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:34.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:34 smithi070 ceph-mon[113635]: pgmap v390: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:34.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:34 smithi179 ceph-mon[84288]: pgmap v390: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:36.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:36 smithi070 ceph-mon[113635]: pgmap v391: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:36.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:36 smithi179 ceph-mon[84288]: pgmap v391: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:38.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:38 smithi070 ceph-mon[113635]: pgmap v392: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:38.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:38 smithi179 ceph-mon[84288]: pgmap v392: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:40.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:40 smithi179 ceph-mon[84288]: pgmap v393: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:40 smithi070 ceph-mon[113635]: pgmap v393: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:41.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:03:41.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:03:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:42 smithi070 ceph-mon[113635]: pgmap v394: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:42.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:42 smithi179 ceph-mon[84288]: pgmap v394: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:44 smithi070 ceph-mon[113635]: pgmap v395: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:44.822 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:44 smithi179 ceph-mon[84288]: pgmap v395: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:46 smithi070 ceph-mon[113635]: pgmap v396: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:46.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:46 smithi179 ceph-mon[84288]: pgmap v396: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:47.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:47 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:03:47.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:47 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:03:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:48 smithi070 ceph-mon[113635]: pgmap v397: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:03:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:03:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:03:48.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:48 smithi179 ceph-mon[84288]: pgmap v397: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:48.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:03:48.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:03:48.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:03:50.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:50 smithi070 ceph-mon[113635]: pgmap v398: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:50.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:50 smithi179 ceph-mon[84288]: pgmap v398: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:52.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:52 smithi070 ceph-mon[113635]: pgmap v399: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:52 smithi179 ceph-mon[84288]: pgmap v399: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:54.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:54 smithi070 ceph-mon[113635]: pgmap v400: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:54.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:54 smithi179 ceph-mon[84288]: pgmap v400: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:56.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:56 smithi070 ceph-mon[113635]: pgmap v401: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:56.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:03:56.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:56 smithi179 ceph-mon[84288]: pgmap v401: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:56.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:03:58.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:03:58 smithi070 ceph-mon[113635]: pgmap v402: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:03:58.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:03:58 smithi179 ceph-mon[84288]: pgmap v402: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:00 smithi070 ceph-mon[113635]: pgmap v403: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:00.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:00 smithi179 ceph-mon[84288]: pgmap v403: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:02.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:02 smithi070 ceph-mon[113635]: pgmap v404: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:02.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:02 smithi179 ceph-mon[84288]: pgmap v404: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:04.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:04 smithi070 ceph-mon[113635]: pgmap v405: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:04.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:04 smithi179 ceph-mon[84288]: pgmap v405: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:06.723 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:06 smithi179 ceph-mon[84288]: pgmap v406: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:06.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:06 smithi070 ceph-mon[113635]: pgmap v406: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:08.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:08 smithi070 ceph-mon[113635]: pgmap v407: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:08.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:08 smithi179 ceph-mon[84288]: pgmap v407: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:10.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:10 smithi070 ceph-mon[113635]: pgmap v408: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:10.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:10 smithi179 ceph-mon[84288]: pgmap v408: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:11.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:04:11.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:04:12.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:12 smithi070 ceph-mon[113635]: pgmap v409: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:12.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:12 smithi179 ceph-mon[84288]: pgmap v409: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:13.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:13 smithi070 ceph-mon[113635]: pgmap v410: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:13.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:13 smithi179 ceph-mon[84288]: pgmap v410: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:16.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:16 smithi179 ceph-mon[84288]: pgmap v411: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:16.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:16 smithi070 ceph-mon[113635]: pgmap v411: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:18.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:18 smithi179 ceph-mon[84288]: pgmap v412: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:18 smithi070 ceph-mon[113635]: pgmap v412: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:20.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:20 smithi179 ceph-mon[84288]: pgmap v413: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:20.742 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:20 smithi070 ceph-mon[113635]: pgmap v413: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:22 smithi179 ceph-mon[84288]: pgmap v414: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:22.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:22 smithi070 ceph-mon[113635]: pgmap v414: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:24.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:24 smithi179 ceph-mon[84288]: pgmap v415: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:24.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:24 smithi070 ceph-mon[113635]: pgmap v415: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:26.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:26 smithi179 ceph-mon[84288]: pgmap v416: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:26.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:04:26.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:26 smithi070 ceph-mon[113635]: pgmap v416: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:26.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:04:28.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:28 smithi179 ceph-mon[84288]: pgmap v417: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:28.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:28 smithi070 ceph-mon[113635]: pgmap v417: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:30.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:30 smithi179 ceph-mon[84288]: pgmap v418: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:30.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:30 smithi070 ceph-mon[113635]: pgmap v418: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:32.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:32 smithi179 ceph-mon[84288]: pgmap v419: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:32.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:32 smithi070 ceph-mon[113635]: pgmap v419: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:34.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:34 smithi179 ceph-mon[84288]: pgmap v420: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:34.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:34 smithi070 ceph-mon[113635]: pgmap v420: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:36.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:36 smithi179 ceph-mon[84288]: pgmap v421: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:36.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:36 smithi070 ceph-mon[113635]: pgmap v421: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:38.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:38 smithi179 ceph-mon[84288]: pgmap v422: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:38.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:38 smithi070 ceph-mon[113635]: pgmap v422: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:40.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:40 smithi179 ceph-mon[84288]: pgmap v423: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:40.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:40 smithi070 ceph-mon[113635]: pgmap v423: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:41.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:04:41.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:04:42.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:42 smithi179 ceph-mon[84288]: pgmap v424: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:42.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:42 smithi070 ceph-mon[113635]: pgmap v424: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:44.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:44 smithi179 ceph-mon[84288]: pgmap v425: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:44 smithi070 ceph-mon[113635]: pgmap v425: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:46.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:46 smithi179 ceph-mon[84288]: pgmap v426: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:46.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:46 smithi070 ceph-mon[113635]: pgmap v426: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:48.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:48 smithi179 ceph-mon[84288]: pgmap v427: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:48.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:48 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:04:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:48 smithi070 ceph-mon[113635]: pgmap v427: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:48.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:48 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T05:04:49.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:49 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:04:49.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:49 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:04:49.572 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:49 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:04:49.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:49 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T05:04:49.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:49 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T05:04:49.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:49 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' 2024-04-02T05:04:50.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:50 smithi179 ceph-mon[84288]: pgmap v428: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:50.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:50 smithi070 ceph-mon[113635]: pgmap v428: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:52.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:52 smithi070 ceph-mon[113635]: pgmap v429: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:52.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:52 smithi179 ceph-mon[84288]: pgmap v429: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:54.279 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:53 smithi070 ceph-mon[113635]: pgmap v430: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:54.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:53 smithi179 ceph-mon[84288]: pgmap v430: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:56.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:56 smithi070 ceph-mon[113635]: pgmap v431: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:56.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:56 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:04:56.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:56 smithi179 ceph-mon[84288]: pgmap v431: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:56.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:56 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:04:58.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:04:58 smithi179 ceph-mon[84288]: pgmap v432: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:04:58.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:04:58 smithi070 ceph-mon[113635]: pgmap v432: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:00.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:00 smithi179 ceph-mon[84288]: pgmap v433: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:00.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:00 smithi070 ceph-mon[113635]: pgmap v433: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:02.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:02 smithi179 ceph-mon[84288]: pgmap v434: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:02.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:02 smithi070 ceph-mon[113635]: pgmap v434: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:04.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:04 smithi179 ceph-mon[84288]: pgmap v435: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:04.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:04 smithi070 ceph-mon[113635]: pgmap v435: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:06.373 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:06 smithi179 ceph-mon[84288]: pgmap v436: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:06.528 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:06 smithi070 ceph-mon[113635]: pgmap v436: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:08.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:08 smithi179 ceph-mon[84288]: pgmap v437: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:08.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:08 smithi070 ceph-mon[113635]: pgmap v437: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:10.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:10 smithi179 ceph-mon[84288]: pgmap v438: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:10.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:10 smithi070 ceph-mon[113635]: pgmap v438: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:11.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:11 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:05:11.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:11 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:05:12.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:12 smithi179 ceph-mon[84288]: pgmap v439: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:12.778 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:12 smithi070 ceph-mon[113635]: pgmap v439: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:14.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:14 smithi179 ceph-mon[84288]: pgmap v440: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:14.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:14 smithi070 ceph-mon[113635]: pgmap v440: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:16.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:16 smithi179 ceph-mon[84288]: pgmap v441: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:16.778 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:16 smithi070 ceph-mon[113635]: pgmap v441: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:18.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:18 smithi179 ceph-mon[84288]: pgmap v442: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:18.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:18 smithi070 ceph-mon[113635]: pgmap v442: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:20.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:20 smithi179 ceph-mon[84288]: pgmap v443: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:20.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:20 smithi070 ceph-mon[113635]: pgmap v443: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:22.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:22 smithi179 ceph-mon[84288]: pgmap v444: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:22.680 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:22 smithi070 ceph-mon[113635]: pgmap v444: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:24.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:24 smithi179 ceph-mon[84288]: pgmap v445: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:24.778 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:24 smithi070 ceph-mon[113635]: pgmap v445: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:26.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:26 smithi179 ceph-mon[84288]: pgmap v446: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:26.571 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:26 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:05:26.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:26 smithi070 ceph-mon[113635]: pgmap v446: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:26.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:26 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:05:28.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:28 smithi070 ceph-mon[113635]: pgmap v447: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:28.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:28 smithi179 ceph-mon[84288]: pgmap v447: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:30.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:30 smithi070 ceph-mon[113635]: pgmap v448: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:30.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:30 smithi179 ceph-mon[84288]: pgmap v448: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:32.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:32 smithi070 ceph-mon[113635]: pgmap v449: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:32.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:32 smithi179 ceph-mon[84288]: pgmap v449: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:34.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:34 smithi070 ceph-mon[113635]: pgmap v450: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:34.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:34 smithi179 ceph-mon[84288]: pgmap v450: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:36.623 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:36 smithi179 ceph-mon[84288]: pgmap v451: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:36.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:36 smithi070 ceph-mon[113635]: pgmap v451: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:38.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:38 smithi070 ceph-mon[113635]: pgmap v452: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:38.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:38 smithi179 ceph-mon[84288]: pgmap v452: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:40.778 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:40 smithi070 ceph-mon[113635]: pgmap v453: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:40.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:40 smithi179 ceph-mon[84288]: pgmap v453: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:41.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:41 smithi070 ceph-mon[113635]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:05:41.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:41 smithi179 ceph-mon[84288]: from='mgr.34102 172.21.15.70:0/2936769306' entity='mgr.smithi070.igidvr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T05:05:42.778 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:42 smithi070 ceph-mon[113635]: pgmap v454: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:42.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:42 smithi179 ceph-mon[84288]: pgmap v454: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:44.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:44 smithi070 ceph-mon[113635]: pgmap v455: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:44.821 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:44 smithi179 ceph-mon[84288]: pgmap v455: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:46.114 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-02T05:05:46.115 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-02T05:05:46.116 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-02T05:05:46.129 INFO:tasks.cephadm:Teardown begin 2024-04-02T05:05:46.129 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-02T05:05:46.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T05:05:46.166 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T05:05:46.201 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-02T05:05:46.201 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T05:05:46.221 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T05:05:46.259 INFO:tasks.cephadm:Stopping all daemons... 2024-04-02T05:05:46.259 INFO:tasks.cephadm.mon.smithi070:Stopping mon.smithi070... 2024-04-02T05:05:46.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070 2024-04-02T05:05:46.491 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:46 smithi179 ceph-mon[84288]: pgmap v456: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:46.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:46 smithi070 systemd[1]: Stopping Ceph mon.smithi070 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T05:05:46.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:46 smithi070 ceph-mon[113635]: pgmap v456: 65 pgs: 65 active+clean; 205 MiB data, 864 MiB used, 536 GiB / 536 GiB avail 2024-04-02T05:05:46.817 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:46 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[113631]: 2024-04-02T05:05:46.549+0000 7f6fd21e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi070 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T05:05:46.817 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:46 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070[113631]: 2024-04-02T05:05:46.549+0000 7f6fd21e5700 -1 mon.smithi070@0(leader) e3 *** Got Signal Terminated *** 2024-04-02T05:05:47.109 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:46 smithi070 podman[160161]: 2024-04-02 05:05:46.814482954 +0000 UTC m=+0.368100013 container died e6a7191c4a7f10e452b2625b6d3ecc0e4bb9d72152f0878614ff608d02c1fd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T05:05:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:47 smithi070 podman[160161]: 2024-04-02 05:05:47.305485225 +0000 UTC m=+0.859102275 container cleanup e6a7191c4a7f10e452b2625b6d3ecc0e4bb9d72152f0878614ff608d02c1fd14 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-02T05:05:47.529 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Apr 02 05:05:47 smithi070 bash[160161]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi070 2024-04-02T05:05:47.749 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi070.service' 2024-04-02T05:05:47.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T05:05:47.804 INFO:tasks.cephadm.mon.smithi070:Stopped mon.smithi070 2024-04-02T05:05:47.805 INFO:tasks.cephadm.mon.smithi179:Stopping mon.smithi179... 2024-04-02T05:05:47.805 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi179 2024-04-02T05:05:48.321 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:47 smithi179 systemd[1]: Stopping Ceph mon.smithi179 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T05:05:48.705 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:48 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179[84284]: 2024-04-02T05:05:48.342+0000 7fdc7129b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi179 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T05:05:48.705 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:48 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179[84284]: 2024-04-02T05:05:48.342+0000 7fdc7129b700 -1 mon.smithi179@1(peon) e3 *** Got Signal Terminated *** 2024-04-02T05:05:48.705 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:48 smithi179 podman[121704]: 2024-04-02 05:05:48.415192837 +0000 UTC m=+0.416481063 container died 1cbe8f43e3c6580d1fcb86827be37da2443e614db854edce97e8bc203368bcc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-02T05:05:49.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:48 smithi179 podman[121704]: 2024-04-02 05:05:48.910828429 +0000 UTC m=+0.912116645 container cleanup 1cbe8f43e3c6580d1fcb86827be37da2443e614db854edce97e8bc203368bcc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793) 2024-04-02T05:05:49.071 INFO:journalctl@ceph.mon.smithi179.smithi179.stdout:Apr 02 05:05:48 smithi179 bash[121704]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-mon-smithi179 2024-04-02T05:05:49.284 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@mon.smithi179.service' 2024-04-02T05:05:49.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T05:05:49.363 INFO:tasks.cephadm.mon.smithi179:Stopped mon.smithi179 2024-04-02T05:05:49.363 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-02T05:05:49.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.0 2024-04-02T05:05:49.688 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:49 smithi070 systemd[1]: Stopping Ceph osd.0 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T05:05:50.029 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:49 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[125695]: 2024-04-02T05:05:49.684+0000 7fa2f3ea9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T05:05:50.029 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:49 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[125695]: 2024-04-02T05:05:49.684+0000 7fa2f3ea9700 -1 osd.0 95 *** Got signal Terminated *** 2024-04-02T05:05:50.029 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:49 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0[125695]: 2024-04-02T05:05:49.684+0000 7fa2f3ea9700 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T05:05:55.389 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:55 smithi070 podman[160293]: 2024-04-02 05:05:55.138115423 +0000 UTC m=+5.582647119 container died 0dd09ba7bb691e68d699d3782f2c8b86ff46d17aa5dfe17dd11d10273f876587 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, ceph=True) 2024-04-02T05:05:55.697 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:55 smithi070 podman[160293]: 2024-04-02 05:05:55.476533716 +0000 UTC m=+5.921065397 container cleanup 0dd09ba7bb691e68d699d3782f2c8b86ff46d17aa5dfe17dd11d10273f876587 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T05:05:55.697 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:55 smithi070 bash[160293]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T05:05:56.029 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:55 smithi070 podman[160322]: 2024-04-02 05:05:55.694595626 +0000 UTC m=+0.553847463 container remove 0dd09ba7bb691e68d699d3782f2c8b86ff46d17aa5dfe17dd11d10273f876587 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-02T05:05:56.425 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:56 smithi070 podman[160417]: 2024-04-02 05:05:56.002188708 +0000 UTC m=+0.017998866 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T05:05:56.426 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:56 smithi070 podman[160417]: 2024-04-02 05:05:56.120434461 +0000 UTC m=+0.136244611 container create bf486f97a94d925b47497ae28a6fc75fc5723d3322ddee3df5dda84125bbbf2e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T05:05:56.426 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:56 smithi070 podman[160417]: 2024-04-02 05:05:56.339903335 +0000 UTC m=+0.355713491 container init bf486f97a94d925b47497ae28a6fc75fc5723d3322ddee3df5dda84125bbbf2e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.build-date=20240326, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T05:05:56.426 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:56 smithi070 podman[160417]: 2024-04-02 05:05:56.343867687 +0000 UTC m=+0.359677839 container start bf486f97a94d925b47497ae28a6fc75fc5723d3322ddee3df5dda84125bbbf2e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T05:05:56.426 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:56 smithi070 podman[160417]: 2024-04-02 05:05:56.42321178 +0000 UTC m=+0.439021934 container attach bf486f97a94d925b47497ae28a6fc75fc5723d3322ddee3df5dda84125bbbf2e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T05:05:56.737 INFO:journalctl@ceph.osd.0.smithi070.stdout:Apr 02 05:05:56 smithi070 podman[160417]: 2024-04-02 05:05:56.635710051 +0000 UTC m=+0.651520194 container died bf486f97a94d925b47497ae28a6fc75fc5723d3322ddee3df5dda84125bbbf2e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T05:05:57.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.0.service' 2024-04-02T05:05:57.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T05:05:57.331 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-02T05:05:57.331 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-02T05:05:57.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.1 2024-04-02T05:05:57.529 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:05:57 smithi070 systemd[1]: Stopping Ceph osd.1 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T05:05:58.032 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:05:57 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[131141]: 2024-04-02T05:05:57.646+0000 7f4f0850f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T05:05:58.032 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:05:57 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[131141]: 2024-04-02T05:05:57.646+0000 7f4f0850f700 -1 osd.1 95 *** Got signal Terminated *** 2024-04-02T05:05:58.032 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:05:57 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1[131141]: 2024-04-02T05:05:57.646+0000 7f4f0850f700 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T05:06:03.441 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:03 smithi070 podman[160524]: 2024-04-02 05:06:03.179445148 +0000 UTC m=+5.629491266 container died 10ef3364d9403246f75c580a8c30970d6d520b36b1c6cadee2af0fd73ea6d27c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, RELEASE=squid-6cec793, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-04-02T05:06:03.779 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:03 smithi070 podman[160524]: 2024-04-02 05:06:03.546014022 +0000 UTC m=+5.996060137 container cleanup 10ef3364d9403246f75c580a8c30970d6d520b36b1c6cadee2af0fd73ea6d27c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T05:06:03.779 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:03 smithi070 bash[160524]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T05:06:04.131 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:03 smithi070 podman[160545]: 2024-04-02 05:06:03.8139195 +0000 UTC m=+0.632391338 container remove 10ef3364d9403246f75c580a8c30970d6d520b36b1c6cadee2af0fd73ea6d27c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T05:06:04.461 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:04 smithi070 podman[160632]: 2024-04-02 05:06:04.031559422 +0000 UTC m=+0.018355953 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T05:06:04.461 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:04 smithi070 podman[160632]: 2024-04-02 05:06:04.139721681 +0000 UTC m=+0.126518195 container create e6f9363609685555f20b895feda79978ec01b6fdf60b893876c15350f9285cf9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_CLEAN=True, RELEASE=squid-6cec793, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T05:06:04.461 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:04 smithi070 podman[160632]: 2024-04-02 05:06:04.375432385 +0000 UTC m=+0.362228893 container init e6f9363609685555f20b895feda79978ec01b6fdf60b893876c15350f9285cf9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326) 2024-04-02T05:06:04.461 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:04 smithi070 podman[160632]: 2024-04-02 05:06:04.378596098 +0000 UTC m=+0.365392614 container start e6f9363609685555f20b895feda79978ec01b6fdf60b893876c15350f9285cf9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-02T05:06:04.462 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:04 smithi070 podman[160632]: 2024-04-02 05:06:04.458760618 +0000 UTC m=+0.445557141 container attach e6f9363609685555f20b895feda79978ec01b6fdf60b893876c15350f9285cf9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_CLEAN=True, RELEASE=squid-6cec793, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T05:06:04.779 INFO:journalctl@ceph.osd.1.smithi070.stdout:Apr 02 05:06:04 smithi070 podman[160632]: 2024-04-02 05:06:04.69190433 +0000 UTC m=+0.678700839 container died e6f9363609685555f20b895feda79978ec01b6fdf60b893876c15350f9285cf9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793) 2024-04-02T05:06:05.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.1.service' 2024-04-02T05:06:05.317 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T05:06:05.318 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-02T05:06:05.318 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-02T05:06:05.318 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.2 2024-04-02T05:06:05.529 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:05 smithi070 systemd[1]: Stopping Ceph osd.2 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T05:06:06.029 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:05 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[136723]: 2024-04-02T05:06:05.647+0000 7f342e96f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T05:06:06.029 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:05 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[136723]: 2024-04-02T05:06:05.647+0000 7f342e96f700 -1 osd.2 95 *** Got signal Terminated *** 2024-04-02T05:06:06.029 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:05 smithi070 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2[136723]: 2024-04-02T05:06:05.647+0000 7f342e96f700 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T05:06:11.493 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:11 smithi070 podman[160739]: 2024-04-02 05:06:11.198358968 +0000 UTC m=+5.663432832 container died 87cd16d907e6098ce8795c506b3ac88c5f8b6ab8ef32daee6cd204c81674e796 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T05:06:11.779 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:11 smithi070 podman[160739]: 2024-04-02 05:06:11.564667923 +0000 UTC m=+6.029741770 container cleanup 87cd16d907e6098ce8795c506b3ac88c5f8b6ab8ef32daee6cd204c81674e796 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T05:06:11.779 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:11 smithi070 bash[160739]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T05:06:12.197 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:11 smithi070 podman[160757]: 2024-04-02 05:06:11.857595172 +0000 UTC m=+0.654494822 container remove 87cd16d907e6098ce8795c506b3ac88c5f8b6ab8ef32daee6cd204c81674e796 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T05:06:12.522 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:12 smithi070 podman[160846]: 2024-04-02 05:06:12.097546806 +0000 UTC m=+0.024080930 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T05:06:12.522 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:12 smithi070 podman[160846]: 2024-04-02 05:06:12.216736803 +0000 UTC m=+0.143270908 container create 3ff6a9fdb9e6417fdcd0f8a788483fa4b0bbed8d5088408c913fb224403c9c9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-04-02T05:06:12.522 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:12 smithi070 podman[160846]: 2024-04-02 05:06:12.43622946 +0000 UTC m=+0.362763577 container init 3ff6a9fdb9e6417fdcd0f8a788483fa4b0bbed8d5088408c913fb224403c9c9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-02T05:06:12.522 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:12 smithi070 podman[160846]: 2024-04-02 05:06:12.439735736 +0000 UTC m=+0.366269848 container start 3ff6a9fdb9e6417fdcd0f8a788483fa4b0bbed8d5088408c913fb224403c9c9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-02T05:06:12.522 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:12 smithi070 podman[160846]: 2024-04-02 05:06:12.519399046 +0000 UTC m=+0.445933175 container attach 3ff6a9fdb9e6417fdcd0f8a788483fa4b0bbed8d5088408c913fb224403c9c9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T05:06:12.779 INFO:journalctl@ceph.osd.2.smithi070.stdout:Apr 02 05:06:12 smithi070 podman[160846]: 2024-04-02 05:06:12.747040992 +0000 UTC m=+0.673575110 container died 3ff6a9fdb9e6417fdcd0f8a788483fa4b0bbed8d5088408c913fb224403c9c9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True) 2024-04-02T05:06:13.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.2.service' 2024-04-02T05:06:13.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T05:06:13.389 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-02T05:06:13.389 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-02T05:06:13.389 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.3 2024-04-02T05:06:13.760 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:13 smithi179 systemd[1]: Stopping Ceph osd.3 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T05:06:14.071 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:13 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T05:06:13.757+0000 7fdc340d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T05:06:14.071 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:13 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T05:06:13.757+0000 7fdc340d8700 -1 osd.3 95 *** Got signal Terminated *** 2024-04-02T05:06:14.071 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:13 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T05:06:13.757+0000 7fdc340d8700 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T05:06:16.571 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:16 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T05:06:16.383+0000 7fdc30ecd700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.270805+0000 front 2024-04-02T05:05:52.270865+0000 (oldest deadline 2024-04-02T05:06:15.770962+0000) 2024-04-02T05:06:16.572 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:16 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:16.143+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:17.571 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:17 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T05:06:17.352+0000 7fdc30ecd700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.270805+0000 front 2024-04-02T05:05:52.270865+0000 (oldest deadline 2024-04-02T05:06:15.770962+0000) 2024-04-02T05:06:17.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:17 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:17.185+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:18.506 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:18 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3[94081]: 2024-04-02T05:06:18.367+0000 7fdc30ecd700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.270805+0000 front 2024-04-02T05:05:52.270865+0000 (oldest deadline 2024-04-02T05:06:15.770962+0000) 2024-04-02T05:06:18.506 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:18 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:18.214+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:18.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:18 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:18.503+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:19.495 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:19 smithi179 podman[121838]: 2024-04-02 05:06:19.22987848 +0000 UTC m=+5.643940612 container died f717fd6c83061d4da3cf9e26f8d49bae2ab603cfb1353eb8cfb8a50fc8dc676c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T05:06:19.495 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:19 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:19.212+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:19.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:19 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:19.522+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:19.821 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:19 smithi179 podman[121838]: 2024-04-02 05:06:19.591074107 +0000 UTC m=+6.005136229 container cleanup f717fd6c83061d4da3cf9e26f8d49bae2ab603cfb1353eb8cfb8a50fc8dc676c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T05:06:19.821 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:19 smithi179 bash[121838]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T05:06:20.191 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:19 smithi179 podman[121859]: 2024-04-02 05:06:19.876292416 +0000 UTC m=+0.643014106 container remove f717fd6c83061d4da3cf9e26f8d49bae2ab603cfb1353eb8cfb8a50fc8dc676c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326) 2024-04-02T05:06:20.441 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:20 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:20.188+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:20.441 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:20 smithi179 podman[121954]: 2024-04-02 05:06:20.202158261 +0000 UTC m=+0.107847392 container create 6ba38a2614249f9b7b9a1f5590b52de30a778e80d27e43ae7e744fbacd2aed0f (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-02T05:06:20.441 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:20 smithi179 podman[121954]: 2024-04-02 05:06:20.11234826 +0000 UTC m=+0.018037406 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T05:06:20.441 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:20 smithi179 podman[121954]: 2024-04-02 05:06:20.438968198 +0000 UTC m=+0.344657336 container init 6ba38a2614249f9b7b9a1f5590b52de30a778e80d27e43ae7e744fbacd2aed0f (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T05:06:20.737 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:20 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:20.559+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:20.737 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:20 smithi179 podman[121954]: 2024-04-02 05:06:20.442433297 +0000 UTC m=+0.348122431 container start 6ba38a2614249f9b7b9a1f5590b52de30a778e80d27e43ae7e744fbacd2aed0f (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, ceph=True, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T05:06:20.737 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:20 smithi179 podman[121954]: 2024-04-02 05:06:20.530197788 +0000 UTC m=+0.435886921 container attach 6ba38a2614249f9b7b9a1f5590b52de30a778e80d27e43ae7e744fbacd2aed0f (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True) 2024-04-02T05:06:21.035 INFO:journalctl@ceph.osd.3.smithi179.stdout:Apr 02 05:06:20 smithi179 podman[121954]: 2024-04-02 05:06:20.73524864 +0000 UTC m=+0.640937782 container died 6ba38a2614249f9b7b9a1f5590b52de30a778e80d27e43ae7e744fbacd2aed0f (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-6cec793) 2024-04-02T05:06:21.320 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:21 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:21.229+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:21.342 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.3.service' 2024-04-02T05:06:21.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T05:06:21.385 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-02T05:06:21.385 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-02T05:06:21.385 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.4 2024-04-02T05:06:21.571 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:21 smithi179 systemd[1]: Stopping Ceph osd.4 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T05:06:21.571 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:21 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:21.538+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:22.071 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:21 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:21.738+0000 7fd13eecd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T05:06:22.071 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:21 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:21.738+0000 7fd13eecd700 -1 osd.4 95 *** Got signal Terminated *** 2024-04-02T05:06:22.071 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:21 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:21.738+0000 7fd13eecd700 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T05:06:22.516 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:22 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:22.244+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:22.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:22 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:22.514+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:23.544 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:23 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:23.282+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:23.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:23 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:23.541+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:24.561 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:24.285+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:24.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:24 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:24.558+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:25.519 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:25 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:25.251+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:25.520 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:25 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:25.251+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:25.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:25 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:25.517+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:26.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:26.257+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:26.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:26.257+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:26.571 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:26.493+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:52.551776+0000 front 2024-04-02T05:05:52.551833+0000 (oldest deadline 2024-04-02T05:06:18.451933+0000) 2024-04-02T05:06:26.571 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:26 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4[99465]: 2024-04-02T05:06:26.493+0000 7fd137cba700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.952794+0000 front 2024-04-02T05:06:01.952825+0000 (oldest deadline 2024-04-02T05:06:26.052733+0000) 2024-04-02T05:06:27.492 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:27 smithi179 podman[122060]: 2024-04-02 05:06:27.206058525 +0000 UTC m=+5.581307993 container died 2e5bbf3235ff1eae6e2b736bdd3055d96e699eb8b70113f77acf331680b85666 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T05:06:27.492 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:27 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:27.302+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:27.492 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:27 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:27.302+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:27.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:27 smithi179 podman[122060]: 2024-04-02 05:06:27.596937133 +0000 UTC m=+5.972186591 container cleanup 2e5bbf3235ff1eae6e2b736bdd3055d96e699eb8b70113f77acf331680b85666 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T05:06:27.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:27 smithi179 bash[122060]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T05:06:28.162 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:27 smithi179 podman[122089]: 2024-04-02 05:06:27.841108112 +0000 UTC m=+0.628959818 container remove 2e5bbf3235ff1eae6e2b736bdd3055d96e699eb8b70113f77acf331680b85666 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux ) 2024-04-02T05:06:28.430 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:28 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:28.311+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:28.430 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:28 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:28.311+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:28.430 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:28 smithi179 podman[122184]: 2024-04-02 05:06:28.062704053 +0000 UTC m=+0.018191725 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T05:06:28.431 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:28 smithi179 podman[122184]: 2024-04-02 05:06:28.183069288 +0000 UTC m=+0.138556941 container create 4e57b1478cfd89952cc89ffc00ddfc2152485280c8d77704fbba174e65f025ad (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-02T05:06:28.431 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:28 smithi179 podman[122184]: 2024-04-02 05:06:28.42812407 +0000 UTC m=+0.383611741 container init 4e57b1478cfd89952cc89ffc00ddfc2152485280c8d77704fbba174e65f025ad (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-02T05:06:28.727 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:28 smithi179 podman[122184]: 2024-04-02 05:06:28.431599485 +0000 UTC m=+0.387087149 container start 4e57b1478cfd89952cc89ffc00ddfc2152485280c8d77704fbba174e65f025ad (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793) 2024-04-02T05:06:28.727 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:28 smithi179 podman[122184]: 2024-04-02 05:06:28.511042115 +0000 UTC m=+0.466529773 container attach 4e57b1478cfd89952cc89ffc00ddfc2152485280c8d77704fbba174e65f025ad (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T05:06:29.008 INFO:journalctl@ceph.osd.4.smithi179.stdout:Apr 02 05:06:28 smithi179 podman[122184]: 2024-04-02 05:06:28.724854896 +0000 UTC m=+0.680342550 container died 4e57b1478cfd89952cc89ffc00ddfc2152485280c8d77704fbba174e65f025ad (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-4-deactivate, RELEASE=squid-6cec793, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240326) 2024-04-02T05:06:29.311 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.4.service' 2024-04-02T05:06:29.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T05:06:29.352 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-02T05:06:29.353 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-02T05:06:29.353 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.5 2024-04-02T05:06:29.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:29 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:29.269+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:29.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:29 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:29.269+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:29.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:29 smithi179 systemd[1]: Stopping Ceph osd.5 for a9031cd8-f0aa-11ee-b647-cb9ed24678a4... 2024-04-02T05:06:30.071 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:29 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:29.668+0000 7f0bb3250700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T05:06:30.071 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:29 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:29.668+0000 7f0bb3250700 -1 osd.5 95 *** Got signal Terminated *** 2024-04-02T05:06:30.071 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:29 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:29.668+0000 7f0bb3250700 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T05:06:30.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:30.303+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:30.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:30 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:30.303+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:31.821 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:31 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:31.347+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:31.821 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:31 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:31.347+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:32.821 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:32 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:32.377+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:32.821 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:32 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:32.377+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:33.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:33 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:33.413+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:33.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:33 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:33.413+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:34.821 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:34 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:34.373+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-04-02T05:05:51.708909+0000 front 2024-04-02T05:05:51.709133+0000 (oldest deadline 2024-04-02T05:06:15.808907+0000) 2024-04-02T05:06:34.821 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:34 smithi179 ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5[104422]: 2024-04-02T05:06:34.373+0000 7f0bb0045700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-04-02T05:06:01.709640+0000 front 2024-04-02T05:06:01.709730+0000 (oldest deadline 2024-04-02T05:06:25.209616+0000) 2024-04-02T05:06:35.280 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:35 smithi179 podman[122290]: 2024-04-02 05:06:35.019394855 +0000 UTC m=+5.447614012 container died 71351459ac7073c4bdf8f97116cfac721b73aa45863a15a87542877d57ab8229 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T05:06:35.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:35 smithi179 podman[122290]: 2024-04-02 05:06:35.368190327 +0000 UTC m=+5.796409475 container cleanup 71351459ac7073c4bdf8f97116cfac721b73aa45863a15a87542877d57ab8229 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T05:06:35.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:35 smithi179 bash[122290]: ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T05:06:35.989 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:35 smithi179 podman[122308]: 2024-04-02 05:06:35.661491489 +0000 UTC m=+0.638620908 container remove 71351459ac7073c4bdf8f97116cfac721b73aa45863a15a87542877d57ab8229 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793) 2024-04-02T05:06:36.309 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:35 smithi179 podman[122400]: 2024-04-02 05:06:35.987291238 +0000 UTC m=+0.112268890 container create 2f83e4ccdeddb0b3efb95ec737e0b01024728a6cecbe557ce96a09a4b26d787b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T05:06:36.309 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:35 smithi179 podman[122400]: 2024-04-02 05:06:35.892894087 +0000 UTC m=+0.017871751 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T05:06:36.309 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:36 smithi179 podman[122400]: 2024-04-02 05:06:36.181985547 +0000 UTC m=+0.306963206 container init 2f83e4ccdeddb0b3efb95ec737e0b01024728a6cecbe557ce96a09a4b26d787b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True) 2024-04-02T05:06:36.309 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:36 smithi179 podman[122400]: 2024-04-02 05:06:36.185034754 +0000 UTC m=+0.310012401 container start 2f83e4ccdeddb0b3efb95ec737e0b01024728a6cecbe557ce96a09a4b26d787b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-02T05:06:36.309 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:36 smithi179 podman[122400]: 2024-04-02 05:06:36.306996094 +0000 UTC m=+0.431973750 container attach 2f83e4ccdeddb0b3efb95ec737e0b01024728a6cecbe557ce96a09a4b26d787b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T05:06:36.571 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 02 05:06:36 smithi179 podman[122400]: 2024-04-02 05:06:36.467763319 +0000 UTC m=+0.592740967 container died 2f83e4ccdeddb0b3efb95ec737e0b01024728a6cecbe557ce96a09a4b26d787b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-02T05:06:37.120 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a9031cd8-f0aa-11ee-b647-cb9ed24678a4@osd.5.service' 2024-04-02T05:06:37.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T05:06:37.162 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-02T05:06:37.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T05:06:37.323 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T05:06:41.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[77817]: fuse finished with error 0 and tester_r 0 2024-04-02T05:07:02.342 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T05:07:02.502 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T05:07:14.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T05:07:14.246 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T05:07:14.276 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-02T05:07:14.278 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848/remote/smithi070/crash 2024-04-02T05:07:14.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T05:07:14.315 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T05:07:14.315 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now 2024-04-02T05:07:14.317 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848/remote/smithi179/crash 2024-04-02T05:07:14.317 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T05:07:14.344 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T05:07:14.344 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-04-02T05:07:14.345 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-02T05:07:14.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-04-02T05:07:14.390 INFO:tasks.cephadm:Compressing logs... 2024-04-02T05:07:14.390 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T05:07:14.433 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T05:07:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi070.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mgr.smithi070.igidvr.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi070.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mgr.smithi070.igidvr.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.1.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi070.mnzrwt.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi070.zvraae.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi070.mnzrwt.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.0.log 2024-04-02T05:07:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi070.zvraae.log: 94.3% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi070.log.gz 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:: No such file or directory 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi179.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.4.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi179.log: gzip/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.audit.log: -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mon.smithi179.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.5.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mon.smithi179.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi179.mzfimf.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.5.log: 92.5%gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.3.log 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T05:07:14.468 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi179.mzfimf.log: 94.2% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi179.log.gz 2024-04-02T05:07:14.469 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mgr.smithi179.labmvs.log 2024-04-02T05:07:14.469 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T05:07:14.469 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mgr.smithi179.labmvs.log: 91.6% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T05:07:14.470 INFO:teuthology.orchestra.run.smithi070.stderr: 91.5% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T05:07:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T05:07:14.471 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T05:07:14.471 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 86.1% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T05:07:14.471 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.2.log 2024-04-02T05:07:14.472 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.log: 88.3% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T05:07:14.473 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T05:07:14.476 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T05:07:14.477 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 86.0% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T05:07:14.487 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.2.log: 90.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T05:07:14.487 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mon.smithi070.log 2024-04-02T05:07:14.489 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T05:07:14.493 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.log: 88.4% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T05:07:14.493 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi179.atrmwz.log 2024-04-02T05:07:14.511 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T05:07:14.515 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi179.atrmwz.log: 89.4% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mgr.smithi179.labmvs.log.gz 2024-04-02T05:07:14.522 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mon.smithi070.log: 91.1% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T05:07:14.895 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mgr.smithi070.igidvr.log.gz 2024-04-02T05:07:14.934 INFO:teuthology.orchestra.run.smithi179.stderr: 92.6% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mon.smithi179.log.gz 2024-04-02T05:07:15.806 INFO:teuthology.orchestra.run.smithi070.stderr: 90.5% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mon.smithi070.log.gz 2024-04-02T05:07:19.251 INFO:teuthology.orchestra.run.smithi179.stderr: 93.2% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-02T05:07:20.238 INFO:teuthology.orchestra.run.smithi070.stderr: 93.3% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-02T05:07:20.400 INFO:teuthology.orchestra.run.smithi179.stderr: 93.5% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-02T05:07:21.648 INFO:teuthology.orchestra.run.smithi070.stderr: 93.3% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-02T05:07:21.801 INFO:teuthology.orchestra.run.smithi179.stderr: 94.8% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi179.mzfimf.log.gz 2024-04-02T05:07:22.108 INFO:teuthology.orchestra.run.smithi070.stderr: 94.8% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi070.zvraae.log.gz 2024-04-02T05:07:22.161 INFO:teuthology.orchestra.run.smithi179.stderr: 94.8% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi179.atrmwz.log.gz 2024-04-02T05:07:22.458 INFO:teuthology.orchestra.run.smithi070.stderr: 93.6% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-02T05:07:22.715 INFO:teuthology.orchestra.run.smithi179.stderr: 93.6% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-02T05:07:29.945 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-02T05:07:29.945 INFO:teuthology.orchestra.run.smithi179.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-02T05:07:29.946 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-02T05:07:29.946 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m15.500s 2024-04-02T05:07:29.947 INFO:teuthology.orchestra.run.smithi179.stderr:user 0m48.520s 2024-04-02T05:07:29.947 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m1.424s 2024-04-02T05:07:30.271 INFO:teuthology.orchestra.run.smithi070.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-02T05:08:44.703 INFO:teuthology.orchestra.run.smithi070.stderr: 92.8% -- replaced with /var/log/ceph/a9031cd8-f0aa-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi070.mnzrwt.log.gz 2024-04-02T05:08:44.704 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-04-02T05:08:44.705 INFO:teuthology.orchestra.run.smithi070.stderr:real 1m30.258s 2024-04-02T05:08:44.705 INFO:teuthology.orchestra.run.smithi070.stderr:user 2m11.912s 2024-04-02T05:08:44.705 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m4.282s 2024-04-02T05:08:44.705 INFO:tasks.cephadm:Archiving logs... 2024-04-02T05:08:44.705 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848/remote/smithi070/log 2024-04-02T05:08:44.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T05:09:05.164 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848/remote/smithi179/log 2024-04-02T05:09:05.165 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T05:09:11.330 INFO:tasks.cephadm:Removing cluster... 2024-04-02T05:09:11.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 --force 2024-04-02T05:09:11.490 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T05:09:11.899 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a9031cd8-f0aa-11ee-b647-cb9ed24678a4 --force 2024-04-02T05:09:12.056 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: a9031cd8-f0aa-11ee-b647-cb9ed24678a4 2024-04-02T05:09:12.354 INFO:tasks.cephadm:Removing cephadm ... 2024-04-02T05:09:12.354 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T05:09:12.369 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T05:09:12.385 INFO:tasks.cephadm:Teardown complete 2024-04-02T05:09:12.385 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T05:09:12.396 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-02T05:09:12.396 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T05:09:12.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T05:09:12.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T05:09:12.504 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T05:09:12.514 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T05:09:12.515 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T05:09:12.517 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T05:09:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-04-02T05:09:12.530 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-04-02T05:09:12.533 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T05:09:12.533 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-04-02T05:09:12.533 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 63 -1809us[-1302us] +/- 76ms 2024-04-02T05:09:12.533 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 60 -3106us[-2598us] +/- 72ms 2024-04-02T05:09:12.533 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +4585us[+5092us] +/- 64ms 2024-04-02T05:09:12.533 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T05:09:12.534 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T05:09:12.534 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-04-02T05:09:12.534 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 -1821us[-1821us] +/- 76ms 2024-04-02T05:09:12.535 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 -3120us[-2601us] +/- 72ms 2024-04-02T05:09:12.535 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +4496us[+4496us] +/- 65ms 2024-04-02T05:09:12.535 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T05:09:12.535 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T05:09:12.546 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T05:09:12.547 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T05:09:12.562 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T05:09:12.658 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T05:09:12.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T05:09:12.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:09:12.769 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T05:09:12.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:09:12.799 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T05:09:12.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T05:09:12.837 INFO:teuthology.task.internal:Duration was 2328.631650 seconds 2024-04-02T05:09:12.837 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T05:09:12.862 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T05:09:12.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T05:09:12.865 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T05:09:12.906 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T05:09:12.908 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T05:09:13.111 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T05:09:13.111 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-04-02T05:09:13.111 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T05:09:13.130 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-04-02T05:09:13.130 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T05:09:13.157 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T05:09:13.157 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T05:09:13.172 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T05:09:13.226 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T05:09:13.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T05:09:13.240 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T05:09:13.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T05:09:13.900 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T05:09:13.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T05:09:13.927 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T05:09:13.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T05:09:13.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T05:09:13.969 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T05:09:13.992 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-04-02T05:09:14.021 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-04-02T05:09:14.077 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T05:09:14.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:09:14.091 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T05:09:14.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:09:14.131 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T05:09:14.143 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T05:09:14.144 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848/remote/smithi070 2024-04-02T05:09:14.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T05:09:14.197 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848/remote/smithi179 2024-04-02T05:09:14.197 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T05:09:14.246 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T05:09:14.247 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T05:09:14.250 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T05:09:14.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T05:09:14.307 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T05:09:14.307 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T05:09:14.341 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T05:09:14.342 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T05:09:14.345 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T05:09:14.357 INFO:teuthology.orchestra.run.smithi070.stdout: 265305 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 2 05:09 /home/ubuntu/cephtest 2024-04-02T05:09:14.357 INFO:teuthology.orchestra.run.smithi070.stdout: 528538 4 d--------- 2 ubuntu ubuntu 4096 Apr 2 04:45 /home/ubuntu/cephtest/mnt.0 2024-04-02T05:09:14.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:09:14.359 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-02T05:09:14.359 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-02T05:09:14.359 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi070 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-02T05:09:14.360 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T05:09:14.383 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T05:09:14.404 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-02T05:09:14.404 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T05:09:14.431 DEBUG:teuthology.nuke:shortname: smithi070 2024-04-02T05:09:14.431 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T05:09:14.455 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 04:21:44.465608', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTkL4ZL+yC5FNyyWTjrdfuZYVoGk7EaJZA39/WFvdE+eqGSYhicJI2UtPOePseT8KykcS58+rzjW2bN71aEYT0='} 2024-04-02T05:09:14.458 DEBUG:teuthology.nuke:shortname: smithi179 2024-04-02T05:09:14.458 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T05:09:14.482 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 04:21:44.467300', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlWYit8l5EfwMwMS/3uqYosTV4uwy7+LKuVzpWajwMvjfgo8DnqOoHUNERPi0WhRTLuiQ+pBYy7eicOYYRc8NM='} 2024-04-02T05:09:14.509 INFO:teuthology.orchestra.console.smithi070:Power off 2024-04-02T05:09:14.509 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T05:09:14.524 INFO:teuthology.orchestra.console.smithi179:Power off 2024-04-02T05:09:14.525 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T05:09:14.534 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2024-04-02T05:09:14.534 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:09:14.548 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off 2024-04-02T05:09:14.548 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:09:14.558 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-04-02T05:09:14.571 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-04-02T05:09:18.559 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:09:18.573 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:09:18.670 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-04-02T05:09:18.682 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-04-02T05:09:22.675 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:09:22.683 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:09:22.785 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2024-04-02T05:09:22.791 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-04-02T05:09:22.887 INFO:teuthology.orchestra.console.smithi070:Power off completed 2024-04-02T05:09:26.795 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:09:26.904 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off 2024-04-02T05:09:27.005 INFO:teuthology.orchestra.console.smithi179:Power off completed 2024-04-02T05:09:27.109 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2328.631650209427 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-02T05:09:27.110 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T05:09:27.198 INFO:teuthology.run:FAIL